Dynamically set INT value to Nullable enum property without explicit type cast

I have a method that populates a DataTable to simple DTO object. To simplify I'll use this example:

public enum Gender : int
{
    Male = 1,
    Female = 2
}

public class Person
{
    //...
    public Gender? MyGender { get; set; }
}

static void Main(string[] args)
{
    int intValue = 2; // value from DB

    var o = new Person();
    var prop = o.GetType().GetProperty("MyGender");    
    prop.SetValue(o, intValue , null); // <- Exception
}

The above throws:

Object of type 'System.Int32' cannot be converted to type 'System.Nullable`1[Test.Program+Gender]'.

If I declare MyGender as Gender (not Nullable) everything works fine.

It also works if I use an explicit Cast prop.SetValue(o, (Gender)intValue, null);

BUT, I don't want to (and can't) use the explicit cast: (Gender)intValue because I have no knowledge of the underlying "hard" type when I create the DTO object . I was hoping for something like (which dose not compile):

var propType = prop.PropertyType;
prop.SetValue(o, (propType)intValue, null);

I also tried:

public static dynamic Cast(dynamic obj, Type castTo)
{
    return Convert.ChangeType(obj, castTo);
}    
var propType = prop.PropertyType;
prop.SetValue(o, Cast(intValue, propType), null);

Which throws:

Invalid cast from 'System.Int32' to 'System.Nullable`1[[Test.Program+Gender...]

I am at dead end. what are my options?