Question Optimizing HDRP for VR is a pain: a thread

Hello, fellow Uniters.

I am working in a VR simulation of a concert for a College investigation. Having a very high end target hardware (GTX 2080) and aspiring to top graphics, I moved the project to HDRP (version 2019.3.f03, using HTC vive).

Moreover, after see this Unite presentation (especially with the 24:30 demo), I was excited about how the graphics could improve with this new technology.

This is a glimpse of the simulation I am building

Althogh being satisfied with the graphic aspect of it, the more I work on the Pipeline, the more lost I am.
Specific features, like real-time shadows and VFX graph particles, are very demanding even with minimal setup. Even if I only have two or three very low-res realtime shadows (with no extra features that could be too heavy) or a very humble smoke effect, performance goes from rock-solid 90 fps, to oscillating 45-60. I know that VR has very hard limitations, but its difficult to me to believe that it can't handle a simple VFX graph, knowing that my hardware is very powerful.

Usually, I would just shrug and try to optimize the scene as usual, but the HDRP puts some issues onto the table:

-Most optimizations I did (like GPU instancing, static batching, baking materials in all, using simple shaders, etc), although reducing Draw Calls, have no visible effect at improving framerate.
-Profiling tools (Like frame debugger, or GPU profiling) are very cryptic to me
-Documentation, tutorials and demos are very scarce, and makes information gathering very harsh.

This is what my profiler looks ike


Seems obvious that I am GPU bounded, but I have troubles isolating the cause, since is very erratic. Things that, sometimes, drop performance, start to work propery with no apparent reason. I don't know what I can disable in the HDRP options, since I am unable to find explanations about how nothing works and impacts on rendering.

Another option I have been toying with is modify the Render Pipeline by myself to elliminate all the aspect I don't need in the simulation, but, again, lack of documentation, tutorials and user experiences pushed me back from this.

Right now, I am in a uncertain situation. I am unsure if this is my fault for throwing expensive things into the scene, if things like VFX graph are in fact not intended to work in VR, what other optimizations I can implement, what is causing the overhead, if this is just due to the young nature of the HDRP, and what improving margin I can hope for by just waiting for Unity Technologies to optimize and improve the pipeline.

I will gladly accept any suggestion about what could be the way to go: dropping HDRP (although I will regret to lose all these cool graphics updates) and using something more low-to-the-ground, or just waiting for the pipeline to mature (mind that my project is not in a hurry, and I can work without the more demanding features by now).

I will be infinitelty grateful, of course, if you could provide any insight/documentation about HDRP features, since I am 70% convinced that I am making a huge setup mistake that causes the performance loss.

So, thank you in advance.

EDIT: if you want more context about my insecurities about HDRP optimization, check this thread I posted yesterday