Object must implement IConvertible (InvalidCastException) while casting to interface

I'm trying to cast an object of a certain type to an interface it implements using Convert.ChangeType(), however an InvalidCastException gets thrown because the object must implement IConvertible.

The types:

public IDocumentSet : IQueryable {}

public IDocumentSet<TDocument> : IDocumentSet, IQueryable<TDocument> {}

public XmlDocumentSet<TDocument> : IDocumentSet<TDocument> {}

Excerpt from code where the error happens:

private readonly ConcurrentDictionary<Type, IDocumentSet> _openDocumentSets = new ConcurrentDictionary<Type, IDocumentSet>();

public void Commit()
{
    if (_isDisposed)
        throw new ObjectDisposedException(nameof(IDocumentStore));

    if (!_openDocumentSets.Any())
        return;

    foreach (var openDocumentSet in _openDocumentSets)
    {
        var documentType    = openDocumentSet.Key;
        var documentSet     = openDocumentSet.Value;

        var fileName        = GetDocumentSetFileName(documentType);
        var documentSetPath = Path.Combine(FolderPath, fileName);

        using (var stream = new FileStream(documentSetPath, FileMode.Create, FileAccess.Write))
        using (var writer = new StreamWriter(stream))
        {
            var documentSetType     = typeof (IDocumentSet<>).MakeGenericType(documentType);
            var writeMethod         = typeof (FileSystemDocumentStoreBase)
                                        .GetMethod(nameof(WriteDocumentSet), BindingFlags.Instance | BindingFlags.NonPublic)
                                        .MakeGenericMethod(documentSetType);
            var genericDocumentSet  = Convert.ChangeType(documentSet, documentSetType); <-------

            writeMethod.Invoke(this, new[] {writer, genericDocumentSet});
        }
    }
}

Now, I'm failing to understand why exactly this happens (as XmlDocumentSet is not a value type) and XmlDocumentSet<'1> implements IDocumentSet<'1>. Am I missing something? Or is there an easier way to achieve what I'm doing?