C#: How set an enum flag for a generic enum type in .NET Standard?

I want to implement the following method using .NET Standard:

public static void SetFlag<TEnum>(ref TEnum value, TEnum flag)
    where TEnum : Enum

I spend hours in trying to achieve this:

• Getting the | operator via reflection appears to be impossible for primitive types as enums are.
• Using dynamic requires referencing an extra package (Microsoft.CSharp.RuntimeBinder), but I would like my library to stay pure .NET Standard conform.

My latest idea was to manually compare TEnum to each valid enum type of {byte, sbyte, short, ushort, int, uint, long, ulong}. But this feels really weird and dirty:

try
{
    var v = (byte)(object)value | (byte)(object)flag;
    value = (TEnum)(object)v;
    return;
}
catch (InvalidCastException) { }

try
{
    var v = (int)(object)value | (int)(object)flag;
    value = (TEnum)(object)v;
    return;
}
catch (InvalidCastException) { }

// ...

throw new NotSupportException($"Unknown enum type {typeof(TEnum)}");

So is this really the only option .NET (Standard) provides here or what I am missing? Looking forward to your hints!