Annotation not getting copied to getter and setter

So, I have this annotation class

@Target(AnnotationTarget.PROPERTY_GETTER, AnnotationTarget.PROPERTY_SETTER, AnnotationTarget.PROPERTY)
@Retention(AnnotationRetention.RUNTIME)
annotation class MyTestAnnotation(val text: String)

And I'm using this like this

interface MyTestInterface {

    @MyTestAnnotation("temp")
    var tempString: String
}

This is how I'm instantiating my interface using reflection.

fun <T> with(myInterface: Class<T>): T {
    return Proxy.newProxyInstance(
        myInterface.classLoader,
        arrayOf<Class<*>>(myInterface),
        invocationHandler
    ) as T
}

private val invocationHandler = InvocationHandler { _, method, args ->
    Log.e("Called method:", method.name) // setTempString
    Log.e("declaredAnnotations", method.declaredAnnotations.size.toString()) // 0
    Log.e("annotations", method.annotations.size.toString()) // 0
    Log.e("args", args.size.toString()) // 1
}

I'm calling it like this

val myInterface = with(MyTestInterface::class.java)
myInterface.tempString = "123"

I'm not able to access the member text of the annotation class because in my invocationHandler I'm not getting the annotation(as you can see both the arrays are of length zero).

My question: Is there any way I can copy the annotation to the getter and setter so I can get access to the data which I put in the annotation?