I have a function that looks like this:
private def readDS[T <: Product : TypeTag](path: String): Dataset[T] = {
spark
.read
.csv(path)
.as[T]
}
T is always a case class
I am trying to get T using
CC = Class.forName("SomeCaseClass")
The issue I have is then using this class as a type parameter in readDS function.
I had an idea to do something like this:
CC match {
case VALIDCC // check if VALIDCC is a subtype of Product and TypeTag[VALIDCC] is available // => {
readDS[VALIDCC]("/path/to/file")
}
case _ => logger.error("provided class can't be used to cast a DF")
But I have no idea how to match types in scala and also how to check if a type is subtype of Product and if implicit val tt = TypeTag[CC] is available in the scope to use CC in the readDS function.
Aucun commentaire:
Enregistrer un commentaire