Should I be worried about the caveats of using unsafe?

The Go unsafe module comes with caveats that I am not sure whether should concern me.

One particular note that I don't really understand is:

Packages that import unsafe may be non-portable..

• If my package is doing what I want it to do should I be worried by it being non-portable?
• How does Go code become non-portable?
• And what are the other key 'got yas' I should be looking out for when using this package?

The Project (for those interested - https://github.com/littlecluster/struct_dot_notation)

I've started a small project to allow me to update structs using dot notation, for what I want to do it is working and my needs don't really get very complex. I'm basically splitting a string, reflecting on a struct and iterating through the fields until I find the field I want to update and then updating it.