Getting a runtimeFieldHandle from an IntPtr

You can easily go from a runtimeFieldHandle to an IntPtr. But how could you get back, e.g. given an IntPtr get a runtimeFieldHandle? I've tried modifying the field directly using an expression tree, but I can't get the fieldinfo for the field which holds the value in a runtimeFieldHandle (maybe it's not called "m_value"?)

General problem is that I want to get a FieldInfo from an IntPtr.

FieldInfo pm_ptr = typeof(RuntimeFieldHandle).GetField("m_value", (BindingFlags)int.MaxValue);

ParameterExpression pParamHandle = Expression.Parameter(typeof(RuntimeFieldHandle), "pHandle");
ParameterExpression pParamPtr = Expression.Parameter(typeof(IntPtr), "ptr");
Expression<Action<RuntimeFieldHandle, IntPtr>> pexChange = (Expression<Action<RuntimeFieldHandle, IntPtr>>)Expression.Lambda(Expression.Block(typeof(void), Expression.Assign(Expression.Field(pParamHandle, pm_ptr), pParamPtr)), pParamHandle, pParamPtr);
    Action<RuntimeFieldHandle, IntPtr> pChange = pexChange.Compile();