Advise on best lighting solution/settings

I've tried a few partial questions to my lighting issues, but I thought it might be better to ask you all about the entire setup. Hopefully you can give me a hand at determining the best lighting settings/solution for this scenario.

Setup details
Unity 2019.3.15, HDRP. Night time. HDRI sky. Directional light (moon) at low intensity, baked.
Small neighborhood (see images) with several houses, street, some cars, some trees and grass. Everything is Static (Contribute GI) except the trees as they are supposed to move a little on the wind to create some "life" in the scene.
The white circles are lamp posts. Each one consists of four spotlights, pointing N, E, S and W (see shaded area around one of the posts in the image). One additional thing is that the light cones of the lights overlap as they would in real life. This however, means an additional issue with baking lightmaps as with Shadowmask baking you can only have 4 overlapping lights (per light? per view? per scene?).
Note: we'll be running on high-end gaming computers with HTC Vive Pro attached

What needs to happen
1. realtime shadows of the trees moving in the wind
2. realtime shadow of the player
3. the intensity of the lights on these lamp posts need to be controllable - ie need to increase a light from defautl to high when a player is close (got that part working with realtime lights) - it's okay if any baked light/shadows do not match perfectly with the changes in intensity.
4. Obviously the least draw calls coming from the shadows (see remark below)!

I've got it working fine using all realtime light (no surprises there). However, the shadow calculations are way too heavy. Looking in the Game view (see image for FP view) the number of draw calls with shadows ON is about 15.000 (!), with shadows OFF it's about 4.500. Using RenderDoc it's clear the shadows are the main culprit.

So the question is: what would be the smartest lighting solution the will give me a more or less realistic lightning/shadow scene without taking the huge shadow calc performance hit?

Thanks for any suggestions!!

Marco

Image: Overview top. White circles are lamp posts with 4 lights each (triangle shades show example of light zones).

Image: First person view showing a 4-light lamp post and some scenery. Tree with shadow in the background.