Sum a homogeneous List

I am working on a C# application. I have multiple lists of objects i.e. each list contains values of different types like double, int etc stored in the form of object in the list.

List<object> l1 = {1,2,3,4,5,6};

List<object> l2 = {1.1, 2.1, 3.2, 4.2, 5.6, 6.1, 7.5, 8.6, 9.2};

There can be many other lists of different types stored in the form of object. My issue is that i have to recognize the type of the object of the list from first value and add all the elements of the list. I am using the following code to get the type .

Type typeOfCurrentField = l1[0].GetType();

This gives me the type of the object. Now i have to get all the values of the list converted to this type. Currently, i am trying this:

dynamic sum = null;
Type typeOfCurrentField = null;
for(int i = 0; i < l1.Count; i++)
{
    if (typeOfCurrentField == null)
    {
        typeOfCurrentField = l1[i].GetType();
        sum = Convert.ChangeType(l1[i], typeOfCurrentField);
    }
    else
    {
        sum = sum + Convert.ChangeType(l1[i], typeOfCurrentField);;
    }
    
}

This gives me the following exception:

Operator '+' cannot be applied to operands of type 'double' and 'object' Exception of type ==> Microsoft.CSharp.RuntimeBinder.RuntimeBinderException

How can i overcome this problem ?