Removing caps glued to a vinyl fencepost?

I have a vinyl railing system on my deck and the posts have caps glued onto them.

I’d like to remove the caps to put small solar lights on them.

Cutting the caps off isn’t really an option. Any ides on ungluing/removing the caps without destroying the posts?