Casting a Delegate into an Action

I'm trying to improve my reflection code by creating Delegates for the Getter and Setter methods.

My code looks like this:

MyObject obj = new MyObject();
var prop = obj.GetType().GetProperty("Prop");
var getType = typeof(Func<>).MakeGenericType(prop.PropertyType);
var setType = typeof(Action<>).MakeGenericType(prop.PropertyType);

var getMethod = prop.GetGetMethod().CreateDelegate(getType, obj);
var setMethod = prop.GetSetMethod().CreateDelegate(setType, obj);

// I'd like to change this section and not to use a dynamic!!
dynamic castedGet = Convert.ChangeType(getMethod, getType);
dynamic castedSet = Convert.ChangeType(setMethod, setType);

CreateDelegate returns a Delegate and using DynamicInvoke isn't performance wise.

I casted (hardcoded) the Delegate into Action<T> \ Func<T> and saw a huge increase in my performance.

I then tried to cast the Delegate into Action<T> \ Func<T> in runtime (using Convert.ChangeType and dynamic) and my performance got hurt - probably due to the fact that I'm using a dynamic type.

I'm pretty sure that I can do this without dynamic.

I guess the solution has something to do with expression trees, but I'm not really sure how to code something like this. If someone has a good solution that doesn't use expression trees than it will be interesting to hear about it as well.