checking if any class has a custom attribute defined in assemby where both custom attribute and class are loaded dynamically

I am working in wcf services where services are host on random ports dynamically. Service contract and service behavior assembly are loaded dynamically and all types are scanned to match service name and its version. Same service can be running on different versions.To distinguish the versions of service we have created a custom ServiceIdentifierAttribute attribute.

    public class ServiceIdentifierAttribute : Attribute
    {
        private string _name;
        public string Name
        {
            get { return _name; }
            set { _name = value; }
        }

        private string _version;
        public string Version
        {
            get { return _version; }
            set { _version = value; }
        }
    }  

Service contract and its behavior class are decorated with SerivceIdentifierAttribute.

    [ServiceContract(Name = "ABCServicesV0.0.0.0")]
    [ServiceIdentifierAttribute(Name = "ABCServicesV0.0.0.0", Version = "V0.0.0.0")]
    public interface IABCService
    {
    }

[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall, ConcurrencyMode = ConcurrencyMode.Multiple, Name = "ABCServicesV0.0.0.0")]
    public class ABCService : IABCService
    {}

Service contract,attribute are defined in one assembly and Service implementation in another. We have a GenericSericeHost console application which dynamically host services by loading both assemblies. We need to search all types and get Service contract type from assembly.

private static bool SeachForServiceContractAttribute(Type type, String serviceIdentifier)
        {
            if (type.IsDefined(typeof(ServiceContractAttribute), false))
            {
                var attributeTypes = type.GetCustomAttributes();
                foreach (var attributeType in attributeTypes)
                {
                    try
                    {
                        ServiceContractAttribute attribute =  (ServiceContractAttribute)attributeType;
                        if (attribute != null && !string.IsNullOrEmpty(attribute.Name) && attribute.Name.Equals(serviceIdentifier))
                            return true;
                    }
                    catch (Exception ex)
                    {
                        Console.Write(ex.Message);
                    }
                }
            }
         return false;
        }

GenericServiceHost has a reference to ServiceContract assembly. At runtime ServiceContractAttribute attribute = (ServiceContractAttribute)attributeType; is throwing error Invalid cast exception. As two versions of the ServiceContractAttribute are loaded at runtime. One is loaded dynamically and another one by GenerciServiceHost reference. We can't remove the service reference as it will result into ServiceContractAttribute not defined complication error.

All different service implementations will have a different assembly and we don't want to add reference to all assembly from genereicservicehost as it will lead to rebuilding genericservicehost when any of the service behavior changes. We want GenericServiceHost to run all the time.

How we can make this to work by cast from assembly loaded type to assembly loaded type

ServiceContractAttribute attribute =  (ServiceContractAttribute)attributeType;

Any pointer ?