c# Iterative reflection

I want to set values using reflection, but making the setting function able to access several subobjects. I'm having no problems when subobjects are classes, but with structs it's not working.

In example, I have the following class and structs

class MyConfig
{
    Gravity gravity;
}
struct Gravity
{
    Vector2 direction;
}
struct Vector2
{
    float X,Y;
}

I'd like to be able to set values like this:

MyConfig cfg=new MyConfig();
setValueViaReflection (cfg,"gravity.direction.X",76.5f);

which should obviously set cfg.gravity.direction.X to 76.5f

My code right now is this:

void setValueViaReflection (object obj,string fieldName,object value)
{
int i;
TypeInfo baseType=null;
FieldInfo field;

    string []split=fieldName.Split ('.');

    // get one subobject in each iteration
    for (i=0;i<split.Count()-1;i++)
    {
        string fname=split[i];
        baseType=obj.GetType().GetTypeInfo();

        field=baseType.GetDeclaredField (fname);
        if (field==null) return;

        obj=field.GetValue (obj);
        if (obj==null) return;

    }

    // finally you've got the final type, set value
    baseType=obj.GetType().GetTypeInfo();

    field=baseType.GetDeclaredField (split[split.Count()-1]);
    if (field==null) return;

    field.SetValue (obj,value);
}

I'm aware that I should use SetValueDirect, but using it makes no difference (the value is modified in "obj", but seem to be a value type, thus it's not changing the original object.

I think the problem is in field.GetValue which creates a value type, making the final SetValueDirect useless.

The code works well if Gravity and Vector2 are set as classes.