I have a requirement where I need to develop Hive Custom UDFs that uses Java reflection API to make calls to external classes.
Since I'm new to Java Reflection, I have dedicated some time learning it and was able to do a basic implementation.
But I'm facing issues while writing unit test cases for this implementation as I'm facing some challenges in mocking Reflection APIs.
Below is the example for Hive Custom UDF.
ReverseString.java
package com.hive.udf;
import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.udf.generic.GenericUDF;
import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
import org.apache.hadoop.hive.serde2.objectinspector.primitive.StringObjectInspector;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
public class ReverseString extends GenericUDF {
private StringObjectInspector input;
Class<?> c = null;
Object ob = null;
@Override
public ObjectInspector initialize(ObjectInspector[] arg0) throws UDFArgumentException {
// check to make sure the input has 1 argument
if (arg0.length != 1) {
throw new UDFArgumentException("input must have length 1");
}
// create an ObjectInspector for the input
ObjectInspector input = arg0[0];
// check to make sure the input is a string
if (!(input instanceof StringObjectInspector)) {
throw new UDFArgumentException("input must be a string");
}
this.input = (StringObjectInspector) input;
System.out.println("Success. Input formatted correctly");
return init(arg0);
}
public ObjectInspector init(ObjectInspector[] arg0) throws UDFArgumentException {
try {
c = Class.forName("com.hive.inherit.DummyUdf");
ob = c.getConstructor().newInstance();
Method method = c.getMethod("print", String.class);
String res = (String) method.invoke(ob, "Test");
System.out.println("RES: "+res);
} catch (NoSuchMethodException | ClassNotFoundException | IllegalAccessException | InvocationTargetException | InstantiationException e) {
e.printStackTrace();
}
return PrimitiveObjectInspectorFactory.javaStringObjectInspector;
}
@Override
public Object evaluate(DeferredObject[] arg0) throws HiveException {
if (input == null || arg0.length != 1 || arg0[0].get() == null) {
return null;
}
String forwards = input.getPrimitiveJavaObject(arg0[0].get()).toString();
return reverse(forwards);
}
public String reverse(String in) {
String res = null ;
try {
Method method = this.c.getMethod("reverse",String.class);
res = (String) method.invoke(this.ob, in);
} catch ( NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
e.printStackTrace();
}
return res;
}
@Override
public String getDisplayString(String[] strings) {
return "reverses string";
}
}
Below is the class that is being called by Reflection.
DummyUdf.java
package com.hive.inherit;
public class DummyUdf {
public DummyUdf(){
System.out.println("DummyUdf");
}
public String print(String str){
System.out.println("DummyUdf-str:"+str);
return str;
}
public String reverse(String in) {
int l = in.length();
StringBuilder sb = new StringBuilder();
for (int i = 0; i < l; i++) {
sb.append(in.charAt(l - i - 1));
}
return sb.toString();
}
}
The unit test cases that I'm trying to implement,
ReverseStringTest.class
@RunWith(MockitoJUnitRunner.class)
public class ReverseStringTest {
@Test
public void testSimpleString() throws HiveException {
//ReverseString r = Mockito.spy(new ReverseString());
ReverseString r = mock(ReverseString.class);
ObjectInspector input = PrimitiveObjectInspectorFactory.javaStringObjectInspector;
when(r.init(Mockito.any())).thenReturn(input);
JavaStringObjectInspector resultInspector = (JavaStringObjectInspector) r.initialize(
new ObjectInspector[] { input });
Text forwards = new Text("hello");
when(r.reverse(Mockito.any())).thenReturn("olleh");
Object result = r.evaluate(new GenericUDF.DeferredObject[] { new GenericUDF.DeferredJavaObject(forwards) });
System.out.println(result);
assertEquals("olleh", resultInspector.getPrimitiveJavaObject(result));
}
}
The test case is failing with NullPointerException.
java.lang.NullPointerException at com.hive.udf.ReverseStringTest.testSimpleString(ReverseStringTest.java:34) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
Can someone please suggest on how to mock the reflection calls appropriately?
Thanks in advance.
Aucun commentaire:
Enregistrer un commentaire