Question how to cull sections of a mesh based on lighting?

I am currently working on an HDRP project that involves revealing objects based on the color of light that is illuminating it.

For instance, if an object requires red light, then the object will be completely transparent until a red light source touches it. I achieve this through assigning the mesh renderer to a specific light layer (ie Red Layer), and the light source to the layer that represents its color.

The problem I am currently facing is that when an object’s material is opaque, if there is no light hitting it from its respective light layer, then it will be completely black. If I change the material to transparent, then the object will be transparent when not lit, however when lit it will be very dim, see-through, and the diffuse will not mix with the light.

My goal is to have the object have the visual effect of being transparent when not lit, and then whatever pixels are illuminated to act as if the object is opaque. In other words, I want black pixels on a mesh to be transparent/culled, and all other pixels to act opaque, or perhaps have the brightness of each pixel determine the alpha level.

Here is a visual example of my problem:

Also I’m not sure if this information is helpful, but I also have a Sphere Collider Trigger on the point light that matches the radius of the light. If there is a solution that requires a collider based on what the light is touching, I already have it set up.

Thank you for your help.

 

The only way to solve this is with a custom shader. If you don't need to worry about things like the revealing light getting shadowed, you could get away with passing a position and radius to the shader with a c# script and hiding the object when too far away. See the Sphere Mask node in a Shader Graph shader. Then use an opaque material with that sphere mask output as the alpha and an alpha threshold value set so the material is clipped. If you want some kind of fade, look at using the dither node with the sphere mask.

If you need shadows to matter ... that's harder. A lot harder. Especially with the HDRP. It's possible, but that'll require a much more custom shader. One that honestly if you're asking this question is going to be something you're unlikely to be capable of writing yourself. And I know few shader authors willing to touch the HDRP right now.

 

Thank you for your response. I have yet to delve into custom shader scripting, so I guess now is the time haha. I appreciate your solution, however I failed to mention that there can be multiple lights hitting multiple objects using the shader, so I would imagine it would be more optimal to simply cull black pixels through a custom shader.

I would assume the best way to go about this is switch to the built-in render pipeline, and then write a custom surface shader that clips pixels based on their color (black in this case) after the lighting model renders the object. I did a bit of research and finalcolor seems like the proper function to do the job with a clip() inside, however I'm not sure if I'm on the right track.

 

You can’t do what you want to do with the built in pipeline.

Or, rather you can, but it would still have to be done exactly the same way as I described above with a distance to a world position test. The built in renderer uses multiple passes to render multiple lights, and the base pass is always only a directional light and baked ambient lighting. A dynamic point light is rendered as a separate second pass that assumes the first pass already rendered the object fully opaque. If you tried to clip on final color, you’ll either get a nearly black object that’s always visible or always see through... basically one of the two options you’re already able to get.

If you need to support multiple lights, the URP & HDRP calculates all lights that affect an object in a single pass, if you’re using the forward rendering path. But Unity’s Shader Graph doesn’t expose that information easily. All the shading is all calculated after the graph part of the Shader Graph, though people have hacked in ways to access the lighting data for toon shading and similar effects. Generally they just get the main light and ignore the rest though.

If you want to use the real lighting, you need to use the URP or HDRP and write essentially a fully custom vertex fragment shader, either standalone or embedded in a Custom Function node.

My suggestion for you is, if you need to support multiple lights, have multiple positions you send to the shader. Figure out how many lights you need to support and have that many Vector4 properties w/ world position and radius.