Bug VR button not working correctly

The sequence is this one:
-I press the primary button on my VR controller and it activates the ray
-I point the ray on the UI button and than i press the trigger button -> the UI button is clicked and works like it has to do
-When i remove the ray from the UI button and i press the primary button it activates the UI button and it MUST NOT happen (this is my problem)
-Than if i remove the ray away from the UI button and i click the trigger button this error goes away and restarts if i aim the UI button and than i press the trigger button

this is a video of the error.

It is like it stores the button also if i have not the ray that aim to the button and i am not pressing the trigger button but the primary button.

I also linked the images of the script and what i call when i press the primary button.


Can somebody help me?