Search Unity

How to cull gameobject from camera without using layers?

Discussion in 'General Graphics' started by a436t4ataf, Dec 27, 2019.

  1. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    431
    Layers are no use - the 32 limit is too small even for a single project, let alone for code that I'm distributing to different teams (I can't steal their layers, Unity has too few to start with).

    Given that, what's the best way to cull gameobjects during rendering?

    Things I've tried:

    1. Look for something in OnWillRenderObject() that lets you tell Unity "don't render it; cull it" - I expected at least a bool return, but seems no such method exists
    2. Looking for some form of "ShouldCull()" method, as I've had in other game engines. I can't find anything in the docs
    3. Replace rendering with manual call to Render(), embedded inside OnWillRenderObject(), and disable the object using SetActive(false) immediately before rendering, then restore it during
      OnRenderObject() - this works in Play mode, but fails in Editor mode
    4. Using shader-replacement - this doesn't work because you cannot say "render everything normally EXCEPT FOR object X", instead you have to manually rewrite every single shader that exists on every object everywhere in your scene. Which is effectively impossible.
    5. Nuking the mesh bounds, and then restoring it. This feels like a really bad idea from a performance POV, and you have to be a bit careful about choosing bounds that are DEFINITELY outside of Unity's frustum culling (which has some bugs - it will crash on some values - and has a margin-of-error, so simply sticking it a long way away won't work)

    This feels like I'm missing something obvious - surely Unity allows games to do their own culling? It's such a huge performance win!
     
    Last edited: Dec 27, 2019
  2. BattleAngelAlita

    BattleAngelAlita

    Joined:
    Nov 20, 2016
    Posts:
    122
    In SRP there is additional "rendering layer mask". But afaik its no exposed in standard rendering, nor URP/HDRP. And you only can use it if you write custom render pipeline.
     
  3. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    431
    I could understand if it's something they decided not to fix for the last few years "because SRP is coming soon and will let people fix this", but it's even more strange if SRP doesn't make this available.

    Render performance with the mesh.bounds trick seems to be "not terrible", but it feels very hacky (and I wouldn't be surprised if it costs significant performance in a few places - especially because it's messing-up any caches they have for the PVS stuff)..
     
  4. yzRambler

    yzRambler

    Joined:
    Jan 24, 2019
    Posts:
    11
    Hi, you can try stencil buffer in shader.
     
  5. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    431
    What do you mean?
     
  6. yzRambler

    yzRambler

    Joined:
    Jan 24, 2019
    Posts:
    11
    Let me try to explain it.
    Using stencil buffer, you can mask some area in the screen.
    So you need two shader files, one for camera (whole scene area. You can create a transparent plane gameObject in front of the camera covering whole scene) another for the culled gameObject.
    The shader for culled gameobject will set the mask value of the area in the screen, but not repaired stencil buffer value.
    The area masked by culled gameObject will be ignored in rendering with the shader for camera.
     
  7. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    431
    I've used stencial buffers lots for masking, but what you descrive is not culling, it's rendering twice and masking. I guess it's a possibility, but none of the other options increase draw calls, so I would expect this to be worse than them?
     
  8. yzRambler

    yzRambler

    Joined:
    Jan 24, 2019
    Posts:
    11
    increase draw call? I don't understand what your mean.
    You can try this:
    1. In the shader of culled gameobject, you set the stencil buffer value.(e.g. 1)
    2. In the sahder of transparent plane, you set a stencil test value and condition to ignore the area which value equal 1.
     
  9. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    431
    What happens when the culled object is transparent (for instance)? What happens when the culled object's shader is higher in the rendering queue? What happens when you don't want to (or can't!) change the culled object's shader?

    I don't think stencilling improves on any of the alternatives here.

    Culling is about doing CPU-side changes that affect the whole pipeline, where stencilling is GPU-side (by which time it's too late to do some things, and even if it's not too late, you pay the cost of having pushed stuff all the way through to the GPU when you could have dropped it before it even got there).
     
  10. joshcamas

    joshcamas

    Joined:
    Jun 16, 2017
    Posts:
    968
    I've used LODGroups with a single LOD for this. Sounds hacky because it is, but it works.

    Alternatively you can disable the renderers when the camera is at a certain distance.
     
  11. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    431
    Share more info on how you're (ab)using LODgroups for this :) ? I'm guessing something around: placing items in the last LODGroup, which tricks Unity into culling it?

    Also ... Have you compared it vs. null mesh-bounds?
     
  12. joshcamas

    joshcamas

    Joined:
    Jun 16, 2017
    Posts:
    968
    I have a component that grabs all child renderers, and put it in the first group. The last group has no renderers. This allows for culling. The upside is that it also handles fading, if you want that. (I did).

    I've never tried null mesh bounds.
     
  13. yzRambler

    yzRambler

    Joined:
    Jan 24, 2019
    Posts:
    11
    Cull gameObject in shader will be some restrictions. It's certainly.
    Do you want perform culling algorithm in script?
    That are some limitations too. I think so.
     
  14. Elvar_Orn

    Elvar_Orn

    Unity Technologies

    Joined:
    Dec 9, 2019
    Posts:
    32
  15. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    431
    As far as I know, CullingGroup doesn't provide any culling at all - I believe it was accidentally given the wrong name (it's really a "QueryVisibilityGroup", but it happens to be calculated during / as a side-effect of the Culling pass, so it got the C word in its name instead).

    In most engines, the equivalent to the CullingGroup API would have 2x as many methods, because it would have the methods + callbacks for ShouldCull? WillCull, PreCull, PostCull, etc.

    The Unity docs are weak here, but reading between the lines, they expect you to use:
    • either: GetComponent<Renderer>().enabled = false
    • or: GetComponent<Renderer>().gameObject.setVisible( false )
    to do the actual culling. Technically that's a kind of culling - but very coarse-grained and often inefficient, and doesn't work for most interesting/custom situations.
     
  16. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    431
    PS: the official docs are, as far as I can tell, absolutely wrong in the opening sentence: "CullingGroup offers a way to integrate your own systems into Unity’s culling and LOD pipeline." - it does not do that! Instead, it does the opposite: offers you a way to passively react to Unity's culling pipeline (you don't get to "integrate" with it at all :( ).

    NB: I'd be delighted to be proven wrong here! CullingGroup was the first place I looked when I wanted to cull, but quickly realised that there's nothing in the API that does any culling, and that no-one on the web seems to use it for culling, etc. i.e. it's not a culling API.

    EDIT: ... although it's a fine API! You can do some simplistic stuff with it, which is nice because otherwise you'd be implementing it all by hand (although not that hard to write, I think - all the info you need is already exposed publically?), but it doesn't seem to add much that wasn't already in Unity (I'm assuming it's higher performance than running your own visibility calcs per frame - but at this level of simplicity, those calcs are already likely to run so fast I wouldn't expect them to be an issue, and most of the performance cust is in the culling implementation, which CullingGroup gives you no access to / no integration with :( ).
     
  17. a436t4ataf

    a436t4ataf

    Joined:
    May 19, 2013
    Posts:
    431
    TL;DR: I've committed to using "Very large - but not too large!" mesh.bounds for this. It works (although, as noted, it exposes some obvious bugs inside Unity engine (I think one of them crashes the 2019.x editor, from memory). BUT I cannot think of a situation where someone would deliberately want to make mesh bounds of that size/shape, so I haven't bothered reporting them).

    If I identify a performance bottleneck that seems to be due to this hack later, I'll come back and cry for help, but for now it appears to be the unofficial(ly official?) way to do culling in Unity :).
     
unityunity