Strange UI Automatic Navigation behaviour, potentially bug. Can anyone help?

So, we're using the Unity UI 'automatic' navigation for all menus because we populate the menus by code. I can't manually link stuff because it's not even there yet in editor mode.

Usually this has worked so far, but when i manually deactivate gameobjects containing selectables (hierarchies of them to be precise) to switch menu branches on/off, the system sometimes gets bugged out.

This seems to be a unity bug but i can't find anything by googling the issue apart from one 5 year old thread with 0 replies https://forum.unity.com/threads/forcing-refresh-of-auto-navigation.325093/

This is what it looks like before I toggle the gameobject hierarchy off/on:



This is what it looks like after:



The system for some reason thinks there is "something" between the credits and exit game button after the toggle. It wants to flow/navigate there and I cant get rid of this "ghost" object. But I also can't navigate there at all, it just doesn't do anything when I try to navigate to it.

Can anyone help?