Casting a int to an Enum without knowing the enum type

I need to cast an int variable to an enum of a type which is stored as a System.Type variable and set that enum to a struct.

The definition of my struct and the enum are roughly like this:

public struct MyStruct
{
    public MyEnum TheEnum { get; set; }
}

public enum MyEnum
{
    Value1 = 1,
    Value2 = 2,
}

The definition of the setting method is about this (note that it may not make sense, but I had to greatly reduce my code's actual functionality):

public void Set(MyStruct structure, Type enumType)
{
    int valueToSet = 2;
    structure.TheEnum = (MyEnum) Enum.ToObject(enumType, (int) valueToSet);
//                      ^^^^^^^^
//                    can't use this
}

which works, if the Set method knows that the enum is of type MyEnum.

The problem is, due to my code's structure, the casting method does not know the enum type and there is no way of passing it. I get the TheEnum property using reflection and did not yet find a way to set it from an int, and I would gladly appreciate any help.

EDIT: the actual code used to set the enum is

genericStruct.GetType().GetField("TheEnum").SetValueDirect(__makeref(genericStruct), (object) castedEnumValue);

but I don't know how to make castedEnumValue be the actual enum representation. So far it is int, and it does not work.