Search Unity

Unity Draw 5.4 introducing more draw calls?

Discussion in 'Editor & General Support' started by Revontuli, Aug 16, 2016.

  1. Revontuli

    Revontuli

    Joined:
    May 12, 2011
    Posts:
    29
    Upon upgrading to Unity 5.4, we've had some serious performance issues, especially on specific machines like the Macintosh. One thing we noticed were a bunch of draw calls added to our (admittedly complex) scene.

    Here is the frame debugger in 5.3.5:

    5.3Debugger2.png

    And here it is in 5.4:

    5.4Debugger2.png

    The latter screenshot selected while in the middle of drawing reflections, but they're both in the same place in the game. There's simply more draw calls being made across the board. The only change was upgrading to Unity 5.4.

    We're using the terrain and nature/trees/details system extensively, so I'm curious if the 5.4 update has made any changes to how that system works.

    As I noted in another thread (http://forum.unity3d.com/threads/5-4-lighting-changes.426181/#post-2754290) the lighting visibly changed (again) as well, and I don't know if one thing is affecting the other. The profiler has completely lost touch with how the game is running in the editor as well (http://forum.unity3d.com/threads/un...iler-fps-not-agreeing-with-actual-fps.425619/)

    Any insight as to what might be going on would be a a big help - otherwise this is becoming a Sisyphean task incorporating updates.
     
    Last edited: Aug 16, 2016
    Revontuli, Aug 16, 2016
    #1

  2. zombiegorilla

    zombiegorilla

    Moderator

    Joined:
    May 8, 2012
    Posts:
    9,051
    Edit - n/m: looking at wrong values.
     
    zombiegorilla, Aug 16, 2016
    #2

  3. Revontuli

    Revontuli

    Joined:
    May 12, 2011
    Posts:
    29
    EDIT: I'm still not sure why, but the doubling draw calls seem to be mainly applied to the Trees attached to the Terrain system. I'm not seeing another render pass as such, but it looks like each pass has twice as many elements in them (2000 calls to Render.OpaqueGeometry instead of 1000, etc.). The parameters for the Terrain in both versions seems to be identical - does anyone have an idea what might be causing this? I've been trying to roll back the 5.4 updated shaders, to 5.3.5 to see if there's a culprit there, but so far with little success.

    Here are two more screenshots showing the exact same place, side-by-side.

    In 5.3.5:

    Unity535StartingScreen.png

    And then after updating to 5.4:

    Unity540StartingScreen.png

    Nothing else was changed on our part. Looking at the frame debugger it looks like 5.4 revisions on how it renders the scene nearly doubled our draw calls. Any ideas on why the version change would mix up the render passes this much?
     
    Last edited: Aug 18, 2016
    Revontuli, Aug 17, 2016
    #3

  4. thebarryman

    thebarryman

    Joined:
    Nov 22, 2012
    Posts:
    130
    This. So much this.

    It's one thing to break the look and feel of scenes with an update to the engine. It's another to do so with no documentation or support provided.
     
    thebarryman, Aug 24, 2016
    #4

  5. Schubkraft

    Schubkraft

    Unity Technologies

    Joined:
    Dec 3, 2012
    Posts:
    1,073
    Schubkraft, Aug 25, 2016
    #5

  6. hoesterey

    hoesterey

    Joined:
    Mar 19, 2010
    Posts:
    659

    You shouldn't need a bug report or a repro project for this one. I'm seeing it with a new project and a blank scene with just unity spheres. I created a bug but you guys shouldn't need help to track this one down. Surprised it got through test.

    A blank scene, a single directional hard shadow casting light, and 1 sphere with the standard shader adds 11 draw calls.

    Every additional sphere will add 7 draw calls.

    So a scene with 6 spheres and a skybox is 95 batches.

    This is still bad but not as bad without vr supported. It goes down to about 4 batches per sphere.

    Scene:
    upload_2016-8-25_22-15-48.png

    Profiler
    upload_2016-8-25_22-16-27.png

    settings:
    upload_2016-8-25_22-16-56.png
     
    hoesterey, Aug 26, 2016
    #6
    richardkettlewell likes this.

  7. Schubkraft

    Schubkraft

    Unity Technologies

    Joined:
    Dec 3, 2012
    Posts:
    1,073
    @hoesterey We had a look at the sphere case.

    They don't get batched because they have too many vertex attributes to batch. If you repeat the same with cubes instead of spheres they batch.

    The standard shader is using more vertex attributes (position, normals, uv's, etc.) so multiplying that by the number of vertices, we probably go over the 900 limit for batching. See https://docs.unity3d.com/Manual/DrawCallBatching.html

    If you use the 'Unlit/Color' shader, the spheres will batch or if you just use a model with less vertices / shader using less vertex attributes.
     
    Schubkraft, Aug 26, 2016
    #7

  8. hoesterey

    hoesterey

    Joined:
    Mar 19, 2010
    Posts:
    659
    Regardless of batching 7 spheres should not be 74 draw calls. Without a shadow casting light I cant imagine 7 spheres being above 7 draw calls. With a shadow caster maybe 14. (an additional call per sphere). If your rendering into the depth buffer first maybe another 1 per sphere making it 21. I just don't see how we get to 7 per sphere.

    Can you explain why 7-11 draw calls for a single sphere is expected behavior? I could see a batch per material * number of shadow casting lights, but the sphere has one material and there is a single shadow caster in the scene. How are we arriving at so many batches per sphere?
     
    Last edited: Aug 26, 2016
    hoesterey, Aug 26, 2016
    #8

  9. hoesterey

    hoesterey

    Joined:
    Mar 19, 2010
    Posts:
    659
    My graphics engineer ran some tests this is what he came up with.

    Hey, so I ran some experiments to see if I could see the same thing. It's wacky. Shadows are hella expensive in Unity, and VR makes it way worse. I was getting +10 batches with a single-material plane object in VR. I was also seeing the batch count go up when shadows were on even when you added an object that was flagged to NOT cast shadows. Crazy.

    Here's what I was getting for a 100x100 plane with both 1 and 2 materials (standard shader) in mono and stereo mode. These numbers are worst-case for shadows because the object is so big, but it gives an idea of how bad it can be:

    Mono:
    base batch count is 1 (skybox)
    +0 base batch count per light shadow source

    Adding 1 object, 1 material:
    +1 per object
    +1 per object when shadow source is present, but 'Cast Shadows' is set to 'off'
    +5 per object when shadow caster is on
    1 (base) + 2 (scene) + 5 (shadows) = 8 batches total for single object

    Adding 1 object, 2 materials:
    +2 per object
    +2 per object when shadow source is present, but 'Cast Shadows' is set to 'off'
    +9 per object when shadow caster is on
    1 (base) + 2 (scene) + 11 (shadows) = 14 batches total for single object

    Stereo:
    base batch count is 5 (skybox, plus VR post-processing)
    +2 base batch count per light shadow source

    Adding 1 object, 1 material:
    +2 per object
    +2 per object when shadow source is present, but 'Cast Shadows' is set to 'off'
    +6 per object when shadow caster is on
    7 (base) + 2 (scene) + 8 (shadows) = 17 batches total for single object

    Adding 1 object, 2 materials:
    +4 per object
    +4 per object when shadow source is present, but 'Cast Shadows' is set to 'off'
    +10 per object when shadow caster is on
    7 (base) + 4 (scene) + 14 (shadows) = 25 batches total for single object

    The added batch count for shadows scales with the number of shadow sources you have in the scene. So for example a single shadow source in VR will add +14 batches for every 2-material object for shadows, but if you have two shadow sources it's +28, three it's +42, etc...

    Also interesting to note: there was a big leap when going from 1 to 2 materials on the object (+6 in mono, +8 in stereo), but then each additional material was only +2 batches in mono and +4 batches in stereo. BTW, this is all with forward rendering. I haven't tested deferred yet.

    So yeah, shadows are hella expensive- especially in VR. Batch count will go way down once Single-Pass Stereo Rendering becomes more of a thing, but I don't think we're quite there yet.
     
    hoesterey, Aug 26, 2016
    #9
    ilmario likes this.

  10. hoesterey

    hoesterey

    Joined:
    Mar 19, 2010
    Posts:
    659
    Any movement on this? From my understanding one of the major goals of the 5.4 upgrade was to take some of the load of the CPU. This issues negates any CPU gains and means that most VR games are not going to be able to have shadows and be in performance.
     
    hoesterey, Aug 30, 2016
    #10
    MrEsquire likes this.

  11. Matjio

    Matjio

    Unity Technologies

    Joined:
    Dec 1, 2014
    Posts:
    108
    @hoesterey Extra draw calls / batches come from Shadow cascade. So each extra shadow cascade is one extra drawcall in the worst case. Which makes for 1 sphere close to the camera in Forward not stereo: 1 for depth + 4 for each shadow cascade + 1 for main renderloop. And this is worst case where the first cascade has all objects. Normally, the first cascade has only a few objects, the second a bit more and so on until shadow distance. Shadow cascade repartition, distance, and number can be configured in QualitySettings.
     
    Matjio, Aug 30, 2016
    #11

  12. Matjio

    Matjio

    Unity Technologies

    Joined:
    Dec 1, 2014
    Posts:
    108
    @Revontuli I have tried to reproduce on a simple scene with SpeedTree trees and terrain, and have exactly the same drawcalls in 5.3 and 5.4. You can use the slider in the FrameDebugger panel while looking at the game view and go through the batches sequence which seem higher in 5.4 and do the same in 5.3 to try to guess what is rendered in extra. If it comes from shadows and trees, note that the configuration of shadows per LOD level is done in the SpeedTreeMaterial in your project (eg: billboard can cast shadows,...). Also check the shadow settings in your quality settings in case they would have been mixed up at upgrade time. Finally, in what has changed in 5.4 and that could have some potential impact: Graphics: In deferred rendering, lightmapped objects affected by mixed-mode lights fallback to forward rendering." towards bottom of https://unity3d.com/unity/whats-new/unity-5.4.0.
     
    Matjio, Aug 30, 2016
    #12

  13. hoesterey

    hoesterey

    Joined:
    Mar 19, 2010
    Posts:
    659
    Matjio,
    Some questions for you, hopefully you can help me understand the system a bit more.

    Why are draw calls doubling with VR supported? Isn't the point of single wide rendering to reduce these draw calls?

    Why do objects cost more draw calls if there is a shadow casting light in the scene, even if the added object has "Cast Shadows" set to off, and "Receive Shadows" set to false?

    To see what I'm talking about
    1. add a plane and a sphere to the scene. Now add a directional light that casts shadows. See the draw calls.
    2. Turn off shadow casting on the directional light, see the draw calls.
    3. Turn on shadow casting on the directional light, turn off recieve and cast shadows on all objects in the scene, see the draw calls. (they will be higher then if the shadow on the light is turned off, even though no shadows are cast in the scene. This is true of more complex scenes too, if objects don't cast or receive shadows they still cost more draw calls if a shadow casting light exists.


    Just to ensure I'm understanding, is it expected behavior for the scene below to be 41 draw calls? Its a sphere a plane and two shadow casting lights.
    upload_2016-8-30_8-47-2.png

    Also is it expected behavior for this scene that consists of 36 spheres, a plane, and two directional shadow casting lights to be 237 draw calls when every object is set to not cast or receive shadows? (482 draw calls when they are set to cast and receive shadows)
    upload_2016-8-30_8-51-37.png

    I just want to make sure I have the proper information to set my expectations around what kind of performance Unity is going to provide in these types of situations.
     
    Last edited: Aug 30, 2016
    hoesterey, Aug 30, 2016
    #13

  14. Matjio

    Matjio

    Unity Technologies

    Joined:
    Dec 1, 2014
    Posts:
    108
    @hoesterey If you look at the framedebugger in Unity you can see a lot explained, but I will go through the details:

    FIRST SCENARIO

    1. Add a plane and a sphere to the scene. Now add a directional light that casts shadows. See the draw calls.

    = 18 worst case (not VR)

    • Depth pass = 2 (plane and sphere)
    • Shadow pass = 8 = Nb shadow cascades x (plane+sphere) => here it is worst case because our scene only contains objects in the first cascade so there is no real need for cascade...
    • Main render loop = 2 (plane and sphere)
    • +some clear of buffers, skybox, applying shadows

    2. Turn off shadow casting on the directional light, see the draw calls.

    = 4 drawcalls
    • Main render loop = 2 (plane and sphere)
    • +clear + skybox
    3. Turn on shadow casting on the directional light, turn off recieve and cast shadows on all objects in the scene, see the draw calls

    = 7 drawcalls
    • Depth pass = 2 (plane and sphere)
    • Main render loop = 2 (plane and sphere)
    • +some clear of buffers, skybox
    Here there is extra cost because we have turned on depth pass because there is a directional shadow casting light. Depth pass is enabled if forward rendering and something needs it (like here or if a script explicitely needs it - thanks Aras for precision on this).

    SECOND SCENARIO: Its a sphere a plane and two shadow casting lights = 41 draw calls.

    In VR without single pass, the combinatory makes the numbers:
    • Depth pass = 4 = (plane and sphere) x 2 directional lights
    • Shadow pass = 16 = Nb shadow cascades x (plane+sphere) x 2 directional lights => note that we do the shadow only once in VR and apply the same shadow map for both eyes with a trick + here it is worst case again because our scene only contains objects in the first cascade
    • Main render loop = 8 = (plane and sphere) x nb pixel lights (we need an extra drawcall for the lighting from the second direction light) x nb eyes
    • +skybox + many clear of buffers and applying shadows (note that it represents about 20 drawcalls here because the scene is simple so the overhead of shadows + VR in terms of nb of "setup" (and quite cheap) drawcalls seems high.
    THIRD SCENARIO: 36 spheres, a plane, and two directional shadow casting lights to be 237 draw calls when every object is set to not cast or receive shadows? (482 draw calls when they are set to cast and receive shadows

    The same equation applies, and in the worst case (with all objects in the first cascade) it could be worst than 482!
    • Depth pass = 74 = (plane and 36 spheres) x 2 directional lights
    • Shadow pass = 296 = Nb shadow cascades x (plane+36 spheres) x 2 directional lights
    • Main render loop = 148 = (plane and 36 spheres) x nb pixel lights (we need an extra drawcall for the lighting from the second direction light) x nb eyes
    • +skybox + many clear of buffers and applying shadows (note that it still represents about 20 drawcalls here same as with sa simpler scene)
    If we have no caster or receivers then we save all the Shadow pass:
    • Depth pass = 74 = (plane and 36 spheres) x 2 directional lights
    • Main render loop = 148 = (plane and 36 spheres) x nb pixel lights (we need an extra drawcall for the lighting from the second direction light) x nb eyes
    • +skybox + many clear of buffers and applying shadows (note that it still represents about 20 drawcalls here same as with sa simpler scene)
    SINGLE PASS: Your numbers say that Single pass was not activated. Single pass is not activated by default in Unity 5.4 (because for example some of our ppost effects don't work with Single pass). To activate it:

    upload_2016-8-31_17-16-7.png

    In the 3rd worst case scenario it gives (74 less drawcalls):
    • Depth pass = 74 = (plane and 36 spheres) x 2 directional lights
    • Shadow pass = 296 = Nb shadow cascades x (plane+36 spheres) x 2 directional lights
    • Main render loop = 74 = (plane and 36 spheres) x nb pixel lights (we need an extra drawcall for the lighting from the second direction light) x nb eyes => only place where we save drawcalls but these 74 saved drawcalls are potentially the most expensive because they require the full execution of the shader.
    • +skybox + many clear of buffers and applying shadows (note that it still represents about 20 drawcalls here same as with sa simpler scene)
    CONCLUSION

    -DrawCalls dont have the same cost. For example shadow pass is only rendering in one color channel, and depth pass is rendering with cheaper pixel fill. Extra Clears are "cheap".
    -In real case scenario, all the shadow cascades are not filled with all the geometry.
    -Beware your Shadow cascade: Check the setup (distance, % of each cascade) in the quality settings how it expands and how you need it. Don't cast or receive shadows where you don't need it. Use cheaper shadows if you can (eg: blob shadows)
    -Beware lights in forward to not overlap too much because they require more rendering (or check max number of pixel lights and light priorities)
    -Check the free Valve's VR renderer for Unity which is a more constraint pipeline in terms of number of lights and type of shadows, but with cheaper cost (https://www.assetstore.unity3d.com/en/#!/content/63141)
     
    Matjio, Aug 31, 2016
    #14

  15. hoesterey

    hoesterey

    Joined:
    Mar 19, 2010
    Posts:
    659
    @Matjio
    Thank you very much for taking the time to run though this with me. So you know my numbers were posted with "Single pass rendering" enabled. I'm not 100% on the numbers that came from my graphics engineer, I assumed he had single pass enabled but I'll have to verify.

    Right now with only 1 shadow casting light our Camera.Render() is taking 6.5 out of my 11ms on the CPU and this cost is primarily in RenderForwardOpaque.RenderLoopJob. This makes me think the cost is due to batches. This is also much higher then 5.3, though maybe a project setting accounted for that change.

    We were trying to avoid the Valve renderer as the extra overhead of updating a 3rd party plugin to render worried me, but after testing it, the performance improvement is pretty amazing.

    Are there any future plans for optimizing how shadows render in the stock unity renderer?

    Thanks again for taking the time.
     
    hoesterey, Sep 1, 2016
    #15
    Matjio likes this.

  16. Matjio

    Matjio

    Unity Technologies

    Joined:
    Dec 1, 2014
    Posts:
    108
    @hoesterey
    When you say performances are worst than in 5.3, is it in VR, with Single pass?
    We have somebody working on Shadows improvement.
     
    Matjio, Sep 2, 2016
    #16

  17. hoesterey

    hoesterey

    Joined:
    Mar 19, 2010
    Posts:
    659
    Yes its in VR with single pass. My Camera.Render went from 5.2ms to 6.5ms. Also as a heads up I oddly have no difference in draw calls with single pass on or off.
     
    hoesterey, Sep 3, 2016
    #17