screen to ray is acting strange

Hello,
Don't know what happens, but here it is:
I have a FPS camera and added some box colliders to it as 3d buttons. When I am on the ground I can use screen2Ray in order to trigger action. So this works, but as soon as my FPS lift ofs ( it has a jetpack) the button actions don't work.
I have checked everything. The colliders are real, they work on ground leve but not in the air. If feels like the colliders are staying on the ground, but they don't I can actually see them in the air as well.
Hope this makes a little bit of sense. But really want this working. Never seen this super strange behaviors before.
Regards! Tom