Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Have a single-channel texture I want to use ask a cutout mask to not render 3d in those areas

Discussion in 'General Graphics' started by adamdsteel, May 1, 2021.

  1. adamdsteel

    adamdsteel

    Joined:
    Oct 1, 2018
    Posts:
    5
    Hey all.

    I'm doing an AR project, and I have a webcamtexture playing in the background to see the real world. Then there are normal Unity 3d objects on top of that.

    I have some unmanaged native (c++ code) that is generating a mask (currently its an Alpha8 texture) that I want to use to show only the real world (camera view) in parts of the screen.

    I can already display the texture on an object with the 0 parts as transparent and the 255 parts as opaque black, but I want to then apply that to the webcamtexture in such a way that the mask defines whether we render 3d objects on that pixel, or mask them out and just render the camera view background.

    - Not render (some) 3d objects in that area, but still render the WebCamTexture there somehow, or:

    - Re-render the WebCamTexture a second time, over the top (closer than any 3d objects), in just those areas where the mask is opaque. Use my Alpha8 texture as a cutout mask. This would be less efficient but would give the same effect and I'm happy to use this hack for now.

    As far as I'm aware I probably have to write a shader and need to somehow apply my texture to the stencil buffer.

    Can anyone give me some suggestions on the easiest way to achieve this? I don't expect you to do it for me, just some ideas on where to start. I don't speak a word of Unity's shader language, but I'm happy to learn, it's just way over my head right now and I want to make sure I'm learning how to do the right thing.

    I've found some open source cutout shaders like this:
    https://gist.github.com/ewandennis/84167e6083940e6dc1401adb876a802d

    But there's no documentation and I don't actually know how to apply it to real world objects to achieve this effect.