jeudi 7 septembre 2023

Unit Test Cases for Hive Custom UDFs by mocking Java Reflection calls

I have a requirement where I need to develop Hive Custom UDFs that uses Java reflection API to make calls to external classes.

Since I'm new to Java Reflection, I have dedicated some time learning it and was able to do a basic implementation.

But I'm facing issues while writing unit test cases for this implementation as I'm facing some challenges in mocking Reflection APIs.

Below is the example for Hive Custom UDF.

ReverseString.java

package com.hive.udf;

import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.udf.generic.GenericUDF;
import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
import org.apache.hadoop.hive.serde2.objectinspector.primitive.StringObjectInspector;

import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;

public class ReverseString extends GenericUDF {

    private StringObjectInspector input;
    Class<?> c = null;
    Object ob = null;


    @Override
    public ObjectInspector initialize(ObjectInspector[] arg0) throws UDFArgumentException {

        // check to make sure the input has 1 argument
        if (arg0.length != 1) {
            throw new UDFArgumentException("input must have length 1");
        }

        // create an ObjectInspector for the input
        ObjectInspector input = arg0[0];

        // check to make sure the input is a string
        if (!(input instanceof StringObjectInspector)) {
            throw new UDFArgumentException("input must be a string");
        }


        this.input = (StringObjectInspector) input;
        System.out.println("Success. Input formatted correctly");

        return init(arg0);
    }

    public ObjectInspector init(ObjectInspector[] arg0) throws UDFArgumentException {

        try {
            c = Class.forName("com.hive.inherit.DummyUdf");
            ob = c.getConstructor().newInstance();
            Method method = c.getMethod("print", String.class);
            String res = (String) method.invoke(ob, "Test");
            System.out.println("RES: "+res);

        } catch (NoSuchMethodException | ClassNotFoundException | IllegalAccessException | InvocationTargetException | InstantiationException e) {
            e.printStackTrace();
        }

        return PrimitiveObjectInspectorFactory.javaStringObjectInspector;
    }


    @Override
    public Object evaluate(DeferredObject[] arg0) throws HiveException {

        if (input == null || arg0.length != 1 || arg0[0].get() == null) {
            return null;
        }
        String forwards = input.getPrimitiveJavaObject(arg0[0].get()).toString();

        return reverse(forwards);
    }


    public  String reverse(String in) {
        String res = null ;
        try {
            Method method = this.c.getMethod("reverse",String.class);
            res = (String) method.invoke(this.ob, in);
        } catch (  NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
            e.printStackTrace();
        }
        return res;
    }

    @Override
    public String getDisplayString(String[] strings) {
        return "reverses string";
    }
}

Below is the class that is being called by Reflection.

DummyUdf.java

package com.hive.inherit;

public class DummyUdf {
    

    public DummyUdf(){
        System.out.println("DummyUdf");
    }

    public String print(String str){
        System.out.println("DummyUdf-str:"+str);
        return str;
    }

    public String reverse(String in) {

        int l = in.length();
        StringBuilder sb = new StringBuilder();

        for (int i = 0; i < l; i++) {
            sb.append(in.charAt(l - i - 1));
        }
        return sb.toString();
    }
}

The unit test cases that I'm trying to implement,

ReverseStringTest.class



@RunWith(MockitoJUnitRunner.class)
public class ReverseStringTest {

    @Test
    public void testSimpleString() throws HiveException {

        //ReverseString r = Mockito.spy(new ReverseString());
        ReverseString r = mock(ReverseString.class);
        ObjectInspector input = PrimitiveObjectInspectorFactory.javaStringObjectInspector;
        when(r.init(Mockito.any())).thenReturn(input);
        JavaStringObjectInspector resultInspector = (JavaStringObjectInspector) r.initialize(
                new ObjectInspector[] { input });
        Text forwards = new Text("hello");
        when(r.reverse(Mockito.any())).thenReturn("olleh");
        Object result = r.evaluate(new GenericUDF.DeferredObject[] { new GenericUDF.DeferredJavaObject(forwards) });
        System.out.println(result);
        assertEquals("olleh", resultInspector.getPrimitiveJavaObject(result));
    }
}

The test case is failing with NullPointerException.

java.lang.NullPointerException at com.hive.udf.ReverseStringTest.testSimpleString(ReverseStringTest.java:34) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)

Can someone please suggest on how to mock the reflection calls appropriately?

Thanks in advance.





Aucun commentaire:

Enregistrer un commentaire