F# - How to create an async function dynamically based on return type

I am trying to create a function dynamically that can return different types based on it's input in F#. These types of functions are like proxies, to illustrate what I am trying to do, here follows an example that is not working correctly:

open FSharp.Reflection
open System

let functionThatReturnsAsync (irrelevantArgs: obj list) (returnType: Type) = 
    if returnType.GUID = typeof<string>.GUID
    then async { return box "some text" } 
    elif returnType.GUID = typeof<int>.GUID
    then async { return box 42 }
    elif returnType.GUID = typeof<bool>.GUID
    then async { return box true }
    else async { return box null }

// this works fine
let func = FSharpValue.MakeFunction(typeof<string -> Async<int>>, fun x -> box (functionThatReturnsAsync [x] typeof<int>))

// unboxing to that type works as well
let fn = unbox<string -> Async<int>> func 

async {
    // HERE THE ERROR
    let! output = fn "hello"
    printfn "%d" output
}
|> Async.StartImmediate

When I invoke fn it seem to be trying to cast FSharpFunc<string, FSharpAsync<obj>> to FSharpFunc<string, FSharpAsync<int>> but the cast is invalid. Even without the async CE, just invoking fn to get the async value fails:

System.InvalidCastException: Specified cast is not valid.
  at (wrapper castclass) System.Object.__castclass_with_cache(object,intptr,intptr)
  at Microsoft.FSharp.Core.LanguagePrimitives+IntrinsicFunctions.UnboxGeneric[T] (System.Object source) [0x00018] in<5ac785a3dff9fae1a7450383a385c75a>:0
  at <StartupCode$FSharp-Core>.$Reflect+Invoke@820-4[T1,T2].Invoke (T1 inp) [0x00011] in <5ac785a3dff9fae1a7450383a385c75a>:0
  at FSI_0019+it@182-10.Invoke (Microsoft.FSharp.Core.Unit unitVar) [0x0000a] in <a19bbccfdeb3402381709b6f2e8ef105>:0
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5ac785a3dff9fae1a7450383a385c75a>:0
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <9bbab8f8a2a246e98480e70b0839fd67>:0
  at <StartupCode$FSharp-Core>.$Control+StartImmediate@1223-1.Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x00000] in <5ac785a3dff9fae1a7450383a385c75a>:0
  at Microsoft.FSharp.Control.CancellationTokenOps+StartWithContinuations@964-1.Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo x) [0x00000] in <5ac785a3dff9fae1a7450383a385c75a>:0
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00103] in <5ac785a3dff9fae1a7450383a385c75a>:0
  at Microsoft.FSharp.Control.AsyncBuilderImpl+startAsync@430[a].Invoke (Microsoft.FSharp.Core.Unit unitVar0) [0x00033] in <5ac785a3dff9fae1a7450383a385c75a>:0
  at <StartupCode$FSharp-Core>.$Control.loop@124-50 (Microsoft.FSharp.Control.Trampoline this, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] action) [0x00000] in <5ac785a3dff9fae1a7450383a385c75a>:0
  at Microsoft.FSharp.Control.Trampoline.ExecuteAction (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] firstAction) [0x00017] in <5ac785a3dff9fae1a7450383a385c75a>:0
  at Microsoft.FSharp.Control.TrampolineHolder.Protect (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] firstAction) [0x00031] in <5ac785a3dff9fae1a7450383a385c75a>:0
  at Microsoft.FSharp.Control.AsyncBuilderImpl.startAsync[a] (System.Threading.CancellationToken cancellationToken, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] cont, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] econt, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] ccont, Microsoft.FSharp.Control.FSharpAsync`1[T] p) [0x00013] in <5ac785a3dff9fae1a7450383a385c75a>:0
  at Microsoft.FSharp.Control.CancellationTokenOps.StartWithContinuations[T] (System.Threading.CancellationToken token, Microsoft.FSharp.Control.FSharpAsync`1[T] a, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] cont, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] econt, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] ccont) [0x00014] in <5ac785a3dff9fae1a7450383a385c75a>:0
  at Microsoft.FSharp.Control.FSharpAsync.StartImmediate (Microsoft.FSharp.Control.FSharpAsync`1[T] computation, Microsoft.FSharp.Core.FSharpOption`1[T] cancellationToken) [0x0002b] in <5ac785a3dff9fae1a7450383a385c75a>:0
  at <StartupCode$FSI_0019>.$FSI_0019.main@ () [0x00019] in <a19bbccfdeb3402381709b6f2e8ef105>:0
  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00032] in <9bbab8f8a2a246e98480e70b0839fd67>:0
Stopped due to error

Is this even possible and to make the example work? I don't mind even fiddling with IL emits to get this working but I am not sure how. If something is unclear about the question, let me know and I will update.