Shadow mask serious artefacts in deferred rendering

I have a scene baked using shadowmasks by the progressive GPU light mapper and when I switch to the deferred render pipeline it does not render correctly. What appears to happen is the shadowmask will appear overlayed on dynamically loaded objects in the scene. An example is below where the rock is dynamically loaded and you can see the shadowmask of the buildings in the distance through it, almost like the rock is transparent. It seems like the depth buffer is messed up for the shadowmask.

I just don't understand why this is happening - is this some limitation of deferred rendering or is the unity deferred render just broken when working with static and dynamic objects in a scene that was baked using shadowmask?
Using Unity 2020.3.8.