Assembly.LoadFrom fails to load DLL

I have a program that loads plugins at runtime. I iterate through each directory in the "Plugins" directory and attempt to load the DLL found. Each plugin has its own directory with the plugin DLL file and another directory called "Dependencies". If the plugin's DLL invokes the AssemblyResolve event I do the following:

• Check if assembly is already in current AppDomain and return it if it is
• Check the plugin's Dependencies directory for the DLL and attempt Assembly.LoadFrom
• If no condition was met or last LoadFrom call fails just return null

Plugins that have no 3rd party dependencies or simple ones like Newtonsoft.Json or NHibernate load just fine. I have one plugin that depends on a DLL we'll just call "custom_library.dll." When the AssemblyResolve event is fired looking for this dependency I can confirm the file is in the Dependencies directory like it should be and even a File.Exists() call returns true.

Unfortunately the Assembly.LoadFrom() call for this DLL throws a FileLoadException with the message "Could not load file or assembly 'custom_library.dll, Version=5.3.136.0, Culture=neutral, PublicKeyToken=null'. Nothing else is provided in the exception beyond HRESULT -2146233079. I can also reference this DLL through Visual Studio without problem.

Why am I getting this exception when trying to load the DLL at runtime instead of through Visual Studio's Add Reference feature?