I have a function that looks like this:
private def readDS[T <: Product : TypeTag](path: String): Dataset[T] = {
    spark
       .read
       .csv(path)
       .as[T]
}
T is always a case class
I am trying to get T using
CC = Class.forName("SomeCaseClass")
The issue I have is then using this class as a type parameter in readDS function.
I had an idea to do something like this:
CC match {
    case VALIDCC //  check if VALIDCC is a subtype of Product and TypeTag[VALIDCC] is available // => {
        readDS[VALIDCC]("/path/to/file")
    }
    case _ => logger.error("provided class can't be used to cast a DF")
But I have no idea how to match types in scala and also how to check if a type is subtype of Product and if implicit val tt = TypeTag[CC] is available in the scope to use CC in the readDS function.
Aucun commentaire:
Enregistrer un commentaire