Remove a delegate from an event

I have an event as follows:

public event EventHandler<CustomEventArgs> RaiseCustomEvent;

Here is some code to remove a delegate from the above event that I have working:

public void RemoveDelegate(Delegate del)
{
    RaiseCustomEvent -= (EventHandler<CustomEventArgs>)del;
}

Here is some code that I have written to remove a delegate where I know the event name:

public void RemoveRaiseCustomEventDelegate(Delegate del, string eventName)
{
    var field = this.GetType().GetField(eventName, BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.GetField);

    var fieldValue = field.GetValue(this) as EventHandler<CustomEventArgs>;
    fieldValue -= (EventHandler<CustomEventArgs>)del;
}

In the above code, the fieldValue is set to null, however, when the event is raised, there is still a subscriber.

Can I please have some help to remove a delegate from an event, where I only have the event name?