How do we set a dictionary using reflection if we don't know the type arguments during compile-time?

We have a dictionary in some unknown class, Dictionary<TKey, TValue> MyDictionary and we have the FieldInfo object obtained using reflection. We also have the type arguments Type keyType = typeof(TKey) and Type valueType = typeof(TValue) which are obtained during runtime.

The key-value pairs are present in two separate lists List<object> keys and List<object> values.

How do we generate the dictionary and set its value using FieldInfo.SetValue()?

So far, we tried this:

var dictionary = Activator.CreateInstance(fieldInfo.FieldType);
var keys = new List<object>();
var values = new List<object>();
// Code to set keys and values
for (int i = 0; i < keys.Count; i++)
   (dictionary as IDictionary).Add(keys[i], values[i]);

fieldInfo.SetValue(mainObj, dictionary);

In specific case where TValue is List<int>, the add statement throws an exception

System.ArgumentException: 'The value "System.Collections.Generic.List1[System.Object]" is not of type "System.Collections.Generic.List1[System.Int32]" and cannot be used in this generic collection.