Adding items to a list without knowing the element type

I need to fill a list with specific data. This list is a property of another object. The elements of that List have the following rules:

• They contain a int-Property named "Id" which is writable
• They have a constructor without parameters

This is the code so far: (I did not add any plausiblity checks or error handling to keep this example simple)

private void SetListProperty(PropertyInfo property, ref object target, int[] ids) {
   Type propertyType=property.PropertyType();
   Type elementType=propertyType.GetElementType();
   PropertyInfo elementId=elementType.GetProperty("Id");

   var targetList=new List<>(elementType); // PseudoCode. Does not work

   foreach (int id in ids) {
     var element=Activator.CreateInstance(elementType);
     elementId.SetValue(element, id);
     targetList.Add(element);  // PseudoCode. Does not work
   }
   property.SetValue(target, targetList);
}

Example for calling that method:

public class User {
  public int Id {get;set;}
  public string Name {get;set;}
}

public class Group {
  public string Name {get;set;}
  public List<User> Users {get;set;}
}

var group=new Group { Name="nerds"};  
SetListProperty(typeof(Group).GetProperty(Users), ref group, new int[] {1,2,3,4,5});

So after calling that method, group should contain a property Users that has 5 elements each with the Id set.

I've seen similar questions here about creating a List of an unknown Type, but not how to actually add single items of an unknown type to that list.