Button not always clickable

My canvas is attached to a player and I'm trying to click the buttons for it to do its function but only by chance it can be clicked, I tried checking whether it is being highlighted and the color keeps flickering.

I checked their World position and world normal on the event system and the position changes constantly even when my mouse is not moving.

There are times when I have it hovered but the highlighted color does not change, but on the event system, it shows that it is hovering over the button. I checked that the button is interactable, it has graphic raycaster component. It has the highest order in layer.

 

I thought I would mention that in current development I am trying incorporate the Logitech HOTAS and have noticed that the button starts 'ON' at start of play. I have to press it turn to OFF but the system takes that as a trigger. Kind of annoying and it is forcing me to put up a menu to get the initial click out of the way.
I saw your post and realized these two could be tied together if you are using the new Unity Input API.

 

I'm not using the new unity input api for the buttons