Set property type based on reflection of another object

Lets say I have two classes:

public class IntKeyObject{
     public int Id {get;set;}
     [ForeignKey]
     public int ForeignKey {get;set;}
     public string SomeProperty {get;set;}
     ...
}
public class StringKeyObject{
     public string Id {get;set;}
     [ForeignKey]
     public string ForeignKey {get;set;}
     public string SomeOtherProperty {get;set;}
}

I would like to be able to make an container class as such:

public class ObjectViewModel<T>{
     public [the type of the objects key] ForeignKey {get;set;}
     public IEnumerable<T> MyList {get;set;}
}

where I initialize it like below:

new ObjectViewModel<IntKeyObject>();

Is there a way to use reflection on the object type to set my ForeignKey property type? Right now the only way I know how to do it is by explicitly passing in the Key type like so:

new ObjectViewModel<IntKeyObject, int>();
new ObjectViewModel<StringKeyObject, string>();

I would prefer to set that type by inference. I know how to find the correct property type from a class by getting the attribute, what I'm not sure about is how to set up my container class.