How to create a shader that behaves like the sprite mask component ( but with extra features )

I need to use a sprite that masks another sprite that is then used to mask a 3rd sprite( by using the combination of the 2 masks)

Here is a picture of what Im trying to do:


I understand that this is not possible using sprite masks, because it would need to use a combination of both "visible inside mask" and "visible outside mask", which is not a valid option on sprite renderers.

So it seems I need a shader to accomplish this?

How would I go about creating a shader that makes this possible?

Here are the requirements:

• the shader needs to be able to receive the inputs of both mask 1 and mask 2(which are opaque color with transparent pixels on borders) and the sprite subject to be masked
  
• the transform.position of mask 1 and mask 2 and the sprite must be considered for the masks to work correctly
• the alpha=1 pixels of the masks must render as transparent even though they are a solid color( like the sprite mask component)
• there will be multiple instances of this character, so there must be a way for the masks to apply only locally to the object they are attached to. ( this is achieved with the sprite mask component by creating a parent with a sorting group component, essentially to avoid masks from different characters to interfere with eachother )

I'm currently using the sprites/default material, so maybe I should start from there?

Should I use shader graph? Is there a tutorial to something that fits the situation that you could point me to?

Any advice would be greatly appreciated. Thanks in advance!

 

Stencil Buffer. Learn how to do boolean logic. You can use cg frag shader, screen shader, or shader grapth. Its all the same logic
See this for best example
Logic Gates as Explained by Pumpkins « Adafruit Industries – Makers, hackers, artists, designers and engineers!
ElGY4auUUAMS6f4.png (1723×854) (adafruit.com)

Stencil Buffers (ronja-tutorials.com)

 

Hello I know this is very old post. I am making a custom 2d shader for my sprite in which the sprite is not drawn where it overlaps another game object with same shader. currently I am achieving this by using the stencil buffer
Stencil {
Ref 0
Comp Equal
Pass IncrWrap
}
This is working the way I want it to. But now the issue I am facing is that if another sprite renderer game object with default mat having mask interaction visible inside mask gets seen if it overlaps that gameobject with custom shader