I have a Case Class, which for simplicity I will call MyUpperClass. This class contains a reference to another Case Class within it. I have called this MyAddressClass. I can reflect MyAddressClass as shown below, however when I attempt to cast an AnyRef returned from reading an object from HDFS via an ObjectStream using asInstanceOf I'm greeted with a Class not found exception for the MyAddressClass.
Why can the class not be found? It's definitely present on the Class Path as i created an uber Jar and I can even reflect it via name!
case class MyAddressClass(...)
case class MyUpperClass(... , address:MyAddressClass)
val mirror = universe.runtimeMirror(this.getClass.getClassLoader)
val classSymbol = mirror.classSymbol(Class.forName("com.MyObject$MyAddressClass"))
val classApply = classSymbol.companion.typeSignature.member(TermName("apply")).asMethod
val members = classApply.paramLists.flatten.map(x => (x.asTerm.name,x.typeSignature))
logger.info(members.map(_.toString()).mkString("\r\n"))
val anyRef = readObjectFromHDFS(addressMapLoc)
val instance = anyRef.asInstanceOf[MyUpperClass]
This then runs and produces:
17/07/09 10:34:08 INFO LoggerName$: (address_display,String)
(flatno_houseno_housename,scala.Option[String])
(houseno_housename,scala.Option[String])
(flat_number,scala.Option[String])
(house_name,scala.Option[String])
(house_number,scala.Option[String])
(po_box,scala.Option[String])
(road,scala.Option[String])
(suburb,scala.Option[String])
(city_district,scala.Option[String])
(city,scala.Option[String])
(state,scala.Option[String])
(state_district,scala.Option[String])
(postcode,scala.Option[String])
(country,scala.Option[String])
(country_code,scala.Option[String])
Exception in thread "main" java.lang.ClassNotFoundException: com.MyObject$MyAddressClass
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:677)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1819)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1986)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at scala.collection.immutable.HashMap$SerializationProxy.readObject(HashMap.scala:582)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2122)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2013)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at com.utils.SparkUtils$.readObjectFromHDFS(SparkUtils.scala:86)
Aucun commentaire:
Enregistrer un commentaire