Cameras, render textures, and culling masks: How could I correctly mask this element?

Hello,

I am trying to have 2D pixel art sprites into a low resolution 3D environment.

What I have now is 2 player cameras, set to 2 render textures, and a main camera on those renders textures. One of those two player cameras has a culling mask on everything except 2D sprites, and the other is only filming 2D sprites. The 2 render textures are then stacked on a canvas, and voila.

It all works nicely except one issue that I can't seem to solve. The 2D sprite, in my 3D world, has to be masked by the 3D objects; which works by default when both 2D sprites and 3D objects layers are rendered on the same camera. But by rendering 2D sprites only on my second camera, it seems I don't get the information of when my sprite is masked by 3D objects... How could I solve that?

How can I get my 2D sprites correctly masked by elements in front of them that aren't drawn by the camera rendering the sprites?

Here is the masking working correctly when I turn on the 3D objects layer on my 2D sprite camera (I don't want that)


Here is what I'm trying to solve; which is getting the same depth/masking but without seeing the 3D objects on my 2D sprite camera.



My camera/textures setup


The Culling Mask of my Environment camera


The Culling Mask of my 2DSprite camera


The rendered textures are drawn on a canvas on top of each other. The ones with the sprites is just transparent everywhere there is not sprite to draw.

I hope my question was clear enough.. thanks for your help!