Lights are showing shadows from the wrong lights (HDRP 8.3.1)

So, I've got several realtime lights and one of the lights is in front of a spinning fan to get that fan shadow look. However, another completely different light in the scene is showing the casted fan shadow when it shouldn't be. If the lights shadowmap is set to "Every Frame" i don't have this issue. However, if I set the update mode to "on demand" and do an update request through code (HDAdditionalLightData.RequestShadowMapRendering) on the lights, the problem occurs.

(The reason why I have lights shadows to OnDemand is to save a huge amount of ms/fps on the lights. By updating the shadowmaps on select lights that don't require everyframe shadows, I'm saving like 30fps. )