Reflection - SetValue from deep context

I am facing an issue, surely due to my lack of knowledge in the reflection process, while trying to set a "complex" class hierarchy based on Json files.

Here are my main model :

public class Names
{
    public Weapons Weapons { get; set; }
    public Armors Armors { get; set; }
    public Utilities Utilities { get; set; }
    public Names()
    {
        Weapons = new Weapons();
        Armors = new Armors();
        Utilities = new Utilities();
    }
}

Each of them having a list of sub-model like this:

public class Weapons
{
    public BattleAxe BattleAxe { get; set; } = new BattleAxe();
    public Bomb_Missile Bomb_Missile { get; set; } = new Bomb_Missile();
    // etc... Around 20 to 25
}

And finally the ended model which is the exact equivalent of each json files but may have very different properties :

public class BattleAxe
{
    public string[] Normal { get; set; } = new string[0];
    public string[] DescriptiveAdjective { get; set; } = new string[0];
    public string[] Material { get; set; } = new string[0];
    public string[] Type { get; set; } = new string[0];
    public string[] Title { get; set; } = new string[0];
    public string[] Of { get; set; } = new string[0];
    public string[] NormalForTitle { get; set; } = new string[0];
}

Since the MS Json deserializer does not support the conversion to a $type as Newtonsoft before, I tried to populate the values using reflection too like this (I've removed all the null-check for code readability) :

public static void Load()
{
    Names = new Names();
    foreach (var category in Names.GetType().GetProperties())
    {
        if (category is not null && !(category.GetGetMethod()?.IsStatic ?? false))
        {
            var categoryType = category.PropertyType;
            foreach (var item in category.PropertyType.GetProperties())
            {
                var itemType = item.PropertyType;
                var subTypeData = JsonSerializer.Deserialize<Dictionary<string, JsonElement>>(File.ReadAllText($"./Assets/Names/{categoryType.Name}/{itemType.Name}.json"));
                var concreteObj = Activator.CreateInstance(itemType);
                foreach (var key in subTypeData.Keys)
                {
                    if (itemType.GetProperty(key) is not null && concreteObj is not null)
                    {
                        var prop = concreteObj.GetType().GetProperty(key);
                        var convertedValue = ConvertJsonType(subTypeData[key], subTypeData[key].ValueKind, out var isReferenceType);
                        // It fails here
                        prop.SetValue(
                            isReferenceType ? convertedValue : null,
                            !isReferenceType ? convertedValue : null
                        );
                    }
                }
                item.SetValue(concreteObj, null);
            }
        }
    }
}

So it fails at the prop.SetValue(...) of the deepest object in the hierarchy with a different error depending on the type of value to set. If it is a reference, it throws a System.Reflection.TargetException : 'Object does not match target type' Exception And if it is value, it throw a System.Reflection.TargetException : 'Non-static method requires a target.' Knowing that I do not have problems around the deserialization as shown here, only the fact that I use a dynamic type (and my instinct tells me it is actually the problem...)

I do not add the ConvertJsonType(...) body as it is functional and really simple

I am more interested in the 'why' than the 'how' so if you can explain me the 'theory' behind the problem, that would help quite a lot :)

Thank you!

PS: I know I can simplify the things in a more readable/performant way but I must achieve it with reflection for personal learning :) Same for the System.Text.Json namespace, I do not intend to switch back to Newtonsoft for that