Quest VR and Hybrid ECS

So I have found and interesting dilemma for my VR RTS-like game, as some post I have read stated:

1.The Hybrid Renderer does not work with the URP/LWRP

2.The Quest does not work with the HWRP

So I have thought of a work around and would like some feedback/advice. I would like to handle the unit moving/combat logic in a ECS based system for maximum performance but clearly the rendering aspect wont cant be done there. So my thought was to have a Unit Visual Manager that on some loop will check the state of the units through the entity manager and set the visual game objects at the appropriate positions and animations.

Now while I feel like that could work, would I just be losing that optimization gain by having the Unit Visual Manager do that every render loop?