dimanche 9 juillet 2017

Why is Scala Class not Found when class is present proven by reflection

I have a Case Class, which for simplicity I will call MyUpperClass. This class contains a reference to another Case Class within it. I have called this MyAddressClass. I can reflect MyAddressClass as shown below, however when I attempt to cast an AnyRef returned from reading an object from HDFS via an ObjectStream using asInstanceOf I'm greeted with a Class not found exception for the MyAddressClass.

Why can the class not be found? It's definitely present on the Class Path as i created an uber Jar and I can even reflect it via name!

case class MyAddressClass(...)
case class MyUpperClass(... , address:MyAddressClass)

val mirror = universe.runtimeMirror(this.getClass.getClassLoader)
val classSymbol = mirror.classSymbol(Class.forName("com.MyObject$MyAddressClass"))
val classApply = classSymbol.companion.typeSignature.member(TermName("apply")).asMethod

val members = classApply.paramLists.flatten.map(x => (x.asTerm.name,x.typeSignature))
logger.info(members.map(_.toString()).mkString("\r\n"))

val anyRef = readObjectFromHDFS(addressMapLoc)
val instance = anyRef.asInstanceOf[MyUpperClass]

This then runs and produces:

17/07/09 10:34:08 INFO LoggerName$: (address_display,String)
(flatno_houseno_housename,scala.Option[String])
(houseno_housename,scala.Option[String])
(flat_number,scala.Option[String])
(house_name,scala.Option[String])
(house_number,scala.Option[String])
(po_box,scala.Option[String])
(road,scala.Option[String])
(suburb,scala.Option[String])
(city_district,scala.Option[String])
(city,scala.Option[String])
(state,scala.Option[String])
(state_district,scala.Option[String])
(postcode,scala.Option[String])
(country,scala.Option[String])
(country_code,scala.Option[String])
Exception in thread "main" java.lang.ClassNotFoundException: com.MyObject$MyAddressClass
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:677)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1819)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1986)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
    at scala.collection.immutable.HashMap$SerializationProxy.readObject(HashMap.scala:582)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2122)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2013)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
    at com.utils.SparkUtils$.readObjectFromHDFS(SparkUtils.scala:86)





Aucun commentaire:

Enregistrer un commentaire