Set struct field which is a pointer using reflection

My question is like this one but with a slight difference that my value is an interface{}.

Let assume we have a struct like this:

type User struct {
    ID         int
    Email      string
    ResourceID *int
}

and a function that sets values of input struct.

func setFieldValue(s interface{}, fieldName string, value interface{}) {
    v := reflect.ValueOf(s).Elem()
    fieldVal := v.FieldByName(fieldName)

    if fieldVal.Kind() == reflect.Ptr {
        // Here is the problem
        fieldVal.Set(reflect.ValueOf(&value))
    } else {
        fieldVal.Set(reflect.ValueOf(value))
    }
}

It is obvious that if I run the code I get the following error:

panic: reflect.Set: value of type *interface {} is not assignable to type *int

What should I do to fix this?