Unity UI UI Elements taking focus rather randomly

The issue
There are, roughly speaking, 2 issues I have:

1. Randomly writing into text fields while trying to walk with WASD
2. Random ScrollView scrolling without mouseclick or the mouse even being near the element

I believe it might be caused by the same thing: UI elements randomly becoming focused. I say "randomly" because sometimes it's fine, sometimes it happens right after app start, sometimes it happens after some time. I never managed to see a pattern.

This is the most annoying for the random scrolling. It often happens that I want to click on something in a list, so I scroll down via mouse wheel, until the UI block with the desired button is visible. Then I move the mouse towards the button to click on it, but the scroll view moves with the cursor, as if it "runs away" from it.

I have been working on this project for about 2 years now and up until now, new features were more important than fixing UI quirks. I'm spending more time on polishing the app now, but this is one issue I just can't fix. Now that the (rest of the) app is more polished, those quirks are the only thing left making it appear cheap and buggy.

What I tried
I'm afraid I was unable to find any other information about this issue online.

The only time I am accessing the .isFocused property in my code is for switching to the next TextField. The whole thing is inside an if-clause checking for the tab key. I am pretty sure this does not happen through code.

I suspect the culprit being the EventSystem getting confused about mouse vs. touch input. When it randomly scrolls, the ScrollView moves with the mouse/pointer, like it should when you drag it with your finger. A mouse click fixes that behaviour, so it's as if the EventSystem thinks the mouse button is still being pressed from some click before.

Another thing is that my UI consists of approximately 20 "screens", which are all on top of each other. They cover the whole screen while their sub elements contain the actual graphics and dimensions. Those sub elements get (de-)activated as needed, so the screen GameObjects are always active. They don't have any graphics, so they shouldn't react to any input events, but maybe the EventSystem gets confused with those layers.

I even created a script that, when the mouse button is released, goes through all scrollbars and shortly sets them to not interactable. I am still using that script as it feels as if it's a bit better, but it didn't fix it.

The question
Does anybody know a fix for this or know why this happens? Or is there maybe a way to get more debug info from a scrollbar? I'm really looking for anything that could help here.

I am using version 5.3.4p6. Changing the version would take quite some effort, so I have not yet tried another version. I post this here instead of at Unity Answers as it seems to be way too specific for that.


Updates

It is not related to the TouchInputModule.
I wanted to test that theory, but while writing a script for that, my IDE showed that TouchInputModule was deprecated as it was included in the StandaloneInputModule now. The TouchInputModule was added to the EventSystem automatically in the beginning. However, I removed it and nothing changed

It is not an old bug
I tested with Unity versions 5.4.1p4 and 2017.1.0f3 - exactly the same behaviour.

It is not related to the mouse button "getting stuck" in some way
When the ScrollView follows the mouse even though the mouse button is released and I move the mouse over to another scroll view, THAT ScrollView starts following the mouse. If the app would wrongly think the mouse button is still pressed, the old ScrollView would move instead.
With this info, it rather seems as if the mouse y axis is temporarily registering as a scroll wheel.

 

This forum doesn't seem tooo active, but in case somebody finds this through google etc:
I switched off "Send Navigation Events" in the EventSystem and I was unable to reproduce the behaviour ever since. I don't need to navigate the UI with a keyboard or joystick/gamepad, so this is a solution for me.