Improving Performance In HDRP (7.3.1)

I was wondering if anyone had any tips on improving performance in HDRP?

We're at a point where we've optimised as much of the game as is possible at this point, but still find it very difficult to get a solid 60fps in the editor (on decent machines, and occasionally scratch 60fps at the very start of the game in Build).
Our bottleneck seems to be HDRP as the render loop takes up at least 10-11ms per frame on the CPU.

I've attached a sample profile.

Any suggestions would be greatly appreciated



Link to Profile: https://we.tl/t-s3emzMVOJS

For reference, this particular machine has an i7-7700k (@4.2GHz) and a GTX 1070 8GB.


Edit: Added our HDRP asset for that Quality Setting but it makes little to no difference at other levels

 

I've got the same situation going for me, I have an i9-7900X with a RTX 2080ti 11GB gpu and 32gb of ram. I'm using HDRP 7.3.1 in Unity 2019.3.10f1. In profiler, RenderPipelineManager.DoRenderLoop_Internal() is taking like 30-40% of the main thread, 20-25% of that is Inl_HDRenderPipelineRenderCamera.

When I was on the built-in renderer I was seeing 110-120fps before moving to HDRP and adjusting everything to the new pipeline (now at about 50fps). Adjusting the quality settings doesn't really do that much to offset the difference any, like quality on volumetrics, etc. What's interesting, is that if I downgrade my GPU to a 1080ti 11GB, I get the same framerate. So there's something going on with the CPU utilization and HDRP and how the engine is using the CPU for some type of processing and not really taking advantage of the GPU in certain circumstances for some particular workload. So there's some kind of bottleneck throttle happened somewhere either in the engine or hardware platform set when HDRP is used.

I read some advice that you can try to disable the SRP Batcher to see if that helps, but there doesn't seem to be any way to disable it in HDRP 7.3.1 as it appears to have moved someplace in the maze of options.

I totally get that HDRP is for "next generation hardware" but it seems like the current production version has a crutch on the CPU for some of the workflow pipeline.

 

No, it's very smooth on an AMD card from late 2014, so you will both have to file bug reports or take a much closer look at where your CPU time is being spent.

I get you have slowdowns but I spent 2 years on HDRP solid (almost daily) and tested it with a bunch of GPUs including a 980, which is easily equivalent to the OP's card. And it flew, really flew. 200FPS+ on busy scenes.

So the assumption it's all next gen is totally wrong. It's designed for current gen consoles (the originals, not the refresh) and can scale to next gen with RT.

You guys are probably doing bad dev practises. It can be slow if used badly or you assume you can do the same practises as built in. For a start design out multiple cameras. These things do totally different things in HDRP. Each camera (by default) does a huge amount of up-front optimisation work to absorb larger workflows later.

You can't just stack em and leave em (if that's a thing being done).

I get I probably sound harsh, but how else will you stop assuming and start digging? Provide more info and better answers can be had.

 

I get what you're saying... Is this a totally anecdotal post on performance? Pretty much and sure, there's some new performance tools that Unity has released and I haven't tried those, but it's kind of like when most people have a problem, they're looking for the low-hanging fruit first.

That being said, if there's a shift in the approach towards development (other than ECS/DOTS) towards changes in HDRP, that would be good to review. I've only got one camera, but good to know anyways. If you have any recommendations towards some best practices after switching over to HDRP, that would be helpful. Am I the perfect Unity dev? no... am I instantiating 10k gameobjects in update() and do/while loops? no.

I'll keep on keeping on, thanks for the tips.

Update:
This is also in-editor framerate (outside of editor build is over 100fps)

 

The main thread cost is substantial and could be improved greatly. It's completely valid to pressure them on that point.

The main thread hit can be very significant even with everything done right, at settings that you need just to be competitive in your genre. Depends on the game.

Big picture having followed the bleeding edge for quite a while, I think the plan is to solve this via DOTS for the most part. Not by trying to solve it in SRP/HDRP core. The DOTS renderer in 2020.1 solves a piece of it. There are a number of other significant pieces that are a ways off.

So big picture if you are doing it all right and still under pressure. DOTS is what you need most likely. Not just for the rendering improvements but to move everything non rendering related that you can off the main thread.

 

So, may well be doing bad dev practices for HDRP. I don't know, there's just a lack of info on this. This is why I'm asking the question to begin with...

We aren't using multiple cameras (well we have multiple but only one is ever active at any given time). Well aware that things need to be done in a different way to built-in and we've followed as much advice as possible from across the forums.

HDRP on the CPU is using 10-11ms every frame pretty much regardless of settings changes we make, so I don't know what else to try, we don't really have any indication what is causing it.

 

You really need to start with knowing what stuff costs. I see you have skinned meshes there. What's the cost of that in an otherwise empty scene? You should know the cost of every major moving part really. Start with a baseline of nothing and then start enabling different parts of your game. And use the profiler don't rely on the stats window.

Just keep narrowing it down and you will eventually find what the issue is.

 

I also provided the HDRP asset we're using (in order to see if we're making any obvious settings mistakes) and a profiler dump in the hopes someone may spot something I'm missing.


To provide more info, it's a city builder.
We're using one Directional Light on Mixed
A 2048 x 2048 Unity Terrain
Baked Lighting and Reflection Probes where possible
Some Post Processing but it doesn't seem to alter frame render times
Baked Occlusion Map
Utilising GPU instancing as much as possible on our shaders/materials

We have Skinned Meshes but they have very little to no impact initially (later on yes, causes an impact when we have lots), disabling them has no impact in this scene.

Here's a screenshot from our profiler (just after launching the scene)
I would Deep Profile but that seems a bit useless in HDRP at the moment unfortunately.

 

Oh we never rely on the Stats window (other than as a general guide to Tri counts and FPS), all our debugging we do using the Profiler.

We have a firm grasp on the costs of every other part of the game, all our code that is.

You're right though, we will have to (and have been) tear it back to basics and build up with the rendering side and see what adds the heavy rendering costs.

If you guys have any advice or links to any good pieces on optimisation and good practices in HDRP I'd love to see them

 

I just went quickly over all the post in this thread.

From my experience: I already made two 3D games with HDRP.
My first game actually used HDRP when it was still in preview. It was very painful to work with it back then, but I managed it. The point is: Our scenario was photorealistic with a lot of realtime point lights casting realtime shadows. We also used Enlighten, means dynamic realtime GI. With my RTX 2060 and Intel i7-9650H, I get roughly 60 fps. But on our map we have some small areas where too much shadows overlapping, means when the player goes there, the fps is about 30.

My second game, which is in a more stylized appearence used the new HDRP without Enlighten. The setup is not as heavy as the other game. Again same specs. It runs with about 60 to 90 fps (depening what is going on). Very heavy of enemy animations and high quality physics. That's the actual bottleneck of the game.

General tips what I'm learned from HDRP, but actually it applies to like every other engine:

- Choose the quality of post-processing effects based on your fps budget. You can easily kill the games performence with Depth of Field sampling a 1:1 image. So use half or quarter size.
- High quality contact shadows are also very expensive.
- Decrease Screen Space Reflection sampling.
- More than one camera rendering the scene kills the performence complety.
- Realtime reflections are very heavy. I never can use reflection cubemaps. Only in rare situations I use one single planar reflection probe. But it also just works with heavily stripping out post processing and rendering layers.
- Avoid too many realtime shadows.
- If you also use Enlighten with HDRP: Well, the same old rules for Enlighten apply. But there is very long and good tutorial from Unity about how to deal with it.
- Avoid high quality volumetric lighting. High quality only slighty improves the image but will cost too much fps. It's not worth it. Instead apply the hidden option "Filter" under a light component. You have to click the cog icon to expand that option.
- Consider using GPU Skinning if you have a lot of animations and complex rigs. But from my experience GPU Skinning is almost always the better option.
- Reuse the materials per object and use texture atlas techniques where you can.
- Do you need always 4k textures?
- Do you need tesselation everywhere?
- Is your realtime shadow cascade range too high?
- Must every tiny and unimportant object cast shadows in your scene?
- Consider favour spot lights over point lights. Not every light bulb needs a realtime point light with realtime shadows.
- Are you using mutilthreading? There are animation C# jobs you could use. For example once I used a C# job to multithread the animation of a water surface (Gerstner waves).

One thing I never tried myself is DOTS and the hybrid rendering. With heavy mesh instancing you could also gain performence. But it is not production ready.

 

How many lights on scene did you limit yourself with to keep such performance?

 

I've got the same issue as OP. I've got a Unity terrain of size 1000 with the MicroSplat shader applied (that's the only geometry in the scene).
Now this scene gives me ~130 FPS with the built-in renderer (default graphic settings, in editor) and ~70 FPS when recreating the same terrain in a new HDRP project (Unity 2019.3.8f1 & HDRP 7.3.1) with MicroSplat HDRP.
I'm running this on my laptop which is i7 6700HQ, GTX 970M GDDR5 6GB and 32GB DDR4.
I was also assuming my laptop wasn't next-gen enough, apparently I'm wrong
I've played around with the HDRP render settings and the camera and there's no real difference in FPS.
Of course it could just be that MicroSplat is badly optimized for HDRP.
When looking in the profiler RenderPipelineManager.DoRenderLoop_Internal() uses 30-60% of the main thread when the game is running, with the one and only camera staying still and basically no custom scripts on.

 

I used deferred rendering. So it's performent to have a lot of light sources as long the radius is quite small.
I used a very short shadow distance like 20 to 30m. It worked because the map had a lot of rooms and L shaped turns. So you would not see many lights at a very huge distance.

 

I noticed this too.

Is there a way to increase draw distance of the lights ?

 

I’m having the same issues, feeling very frustrated.

 

Do we have an official best practice guide for HDRP. I would love to solve where this performance issues could be coming from

 

To OP - 25k batches is a lot of batching work, which I'm not 100% is the cause, but it's a high number. One of the most important things to do in general is merge meshes (either in runtime code or a modelling package), since more meshes = more batching work for the CPU. It is generally well-worth having big meshes that result in coarser occlusion culling, than many small meshes that all need to be set up for rendering by the CPU.

Some additional tips, and some gripes with HDRP:

> Very important to find the 'light distance' and 'shadow distance', and 'reflection probe' distance options hidden in the cog icons. By default they're set to effectively infinity. Since lights & probes don't get culled properly, this means unless you fix a distance (usually by gauging if the player will notice or be able to see it), every light & probe in your scene is rendering by default, as well as every shadow, killing performance.

> Planar / realtime reflection probes are not remotely performant and for tech demos only, or tiny rooms with a mirror (and not much else), and need a script killing the probe if they player leaves the area.

> In general (main gripes here); real time shadows are utter framerate killers. Realistically if you have an 'average' complex scene (say 1m tris), one or two real time shadow sources are your limit in any given frame, maybe 3-4 if you dial down the resolution to pixel-art levels. This results in a lot of awkwardness, since a light behind a wall that's not casting shadows will have it's volumetrics and light bleed through the wall. So you end up having to design the whole level to accomodate the renderer to a fairly extreme degree.

> This should mean you fall back to baked Shadowmasks, but these seem to have terrible performance overheads too, so the reality is you often need to fall back to baked lights with only 1-2 real time, shadow casting lights per area, and maybe 1-2 real time, non-shadow casting lights, carefully set up to cover the issues with baked, like the lack of specularity.

> The final big issue is that whilst the GPU version eased the pain, Unity's lightmapping is still awful; with a lethal combination of unpredictable, buggy, mediocre results that look nothing like the diffuse realtime render, and baffling sliders that need a lot of tuning against excruciating bake times (and when working/tuning creatively, even 10 mins is excruciating). This and related bugs like lights randomly switching off and on ignoring their mode. I'm sure there's a complex technical reason why the beautiful realtime shadows can't just be stuck into a lightmap, but it's annoying they can't (and changing them to 'on enable' from 'realtime' has little performance difference).

The conclusion I've reached with HDRP is if you're making a beautiful RT render of an object, or small locale, e.g. for a portfolio/archvis/tech demo, with minimal or controlled camera movement - it's great. If you're expecting to make an actual game (or reasonably scaled level), where things like handling a light on the far side of a wall without it bleeding-through and need to use and manage large numbers of lights/probes/shadows, it's very hard to stand HDRP next to Unreal and find reasons to argue in favor of HDRP.

 

So, multiple cameras...you can't just stack em and leave em....what do you do then?

 

Hi!!! I just answered in another forum thread related to this topic. It opened a long time ago and in order not to repeat myself I leave the link ...
https://forum.unity.com/threads/hd-...ormance-comparison.526509/page-6#post-5983316

I personally think the same as @tuinal and HDRP cannot currently be used for projects with somewhat large scenes ... unless they are quite empty ... needless to say if you have day / night cycles with scenes where there is a 300m ~ 500m field of view in front of you ... puff ...
There are a lot of things that kill performance too much with HDRP. But it seems to me that the problem comes from how Unity works the part of the cpu. If I am wrong, someone will correct me, but I think that almost all the processes that Unity does on the cpu do not take advantage of all the threads of the processor and that could be the bottleneck.

Someone with experience could provide information and data if I am wrong?

Cheers

 

Simply put, HDRP is a piece of SH#T , why bother using it if you need to change all these settings. I get Much higher fps on the standard pipeline. If you want volumetric lights just use Aura (free)

 

Comments like that doesn’t help anyone. Loads of devs are working hard on HDRP and have been very helpful trying to solve issues for people on this forum. Last thing they need are comment like that.

 

How could you improve performance when you cant even see real vertex count. How is it possible that when i load in a 1 000 000 vertex count mesh, i see in stats around 300 k vertices?

 

These are pretty much the limits i experience with HDRP. Except that instead of having 1m tris the scene has about 3M in the worst case. The biggest impact do in fact have shadow maps but in my opinion the impact is huge at least when compared to the build in renderer.
The scene consists mostly of 2 bigger point lights and few small point/spot ones and other lights that do not cast shadows. Im also using a directional light. The scene is indoor consisting of 3 rooms but also has windows. This setup was no problem at all with built in and i could have thrown at least double the amount of lights with rt shadows enabled until the main thread was slower than 16ms, but with HDRP i had to restrict to a maximum of 2-3 shadow casting point lights to get the same acceptable performance. Additionally im using less post processing and no more volumetric lights. I do not use any RT reflections (not with HDRP nor with the built in).
This results at least in nearly the same performance but worse visuals.
The most frustrating about this is that shadow settings do not seem to affect performance at all. 32x32 vs 2k x 2k yields the same performance but different visuals. Also changing the atlases does not help.
The problem is that some lights have to use shadows because the scene consists of lots objects that can be opened. If some lights do not cast shadows then light leaks through the objects and it looks weird.
Not quite sure why the renderer is not able to handle the scene if its said being able to process far more complex lighting setups.

Thats something i did not check. Maybe this helps.

For me changing to on enable helped a lot regarding performance. The problem with that setting is not that the shadow is somewhat "static" but that it seems there are projection issues when the camera moves. One example i noticed is with the directional light. I set it to onenable and as long as the camera is not moving everything looks fine.
The game starts with a camera direction that is somewhat parallel with the directional lights direction. Now when moving on specific spots and looking at steep angle down on an area that is illuminated by the directional light there appears a round shadow in the center of the screen.
So onenable seems to be a good option to use if you have a static camera. But for a real game scenario in which one can move freely this does not seem to be a good pick.

However does someone know why the shadow map has to be recomputed each frame to look correct, even if the light source is not moving? I thought shadow maps depent on the light position. Using the light matrix the fragments coordinates are transformed into light space. If im not misundertanding something then the shadow map should be independent of the camera view.

 

First things that veritably Leap at me is that you're spending three times more time decals than your Gbuffer so there's an obvious target for optimization, second for those decals you are not using GPU instancing but SRP Batching which is an Entirely different beast, with SRP Batcher you do not minimize material numbers but rather have as few shader variant passes as possible and use almost any number of materials your target VRAM budget allows.

This is propably too late to help you but fire up any follow ups if this is still relevant. Our project went from being CPU bottlenecked to having 50% free air on the CPU once we understood how the new systems actually work.

 

Hi JJ, definitely still relevant! If you're happy to consult a little please give me an email
oliver@orthrusstudios.com

Would be incredibly appreciated even just some high level advice.

 

Sent you an email as requested, have a nice evening!

 

Do you have some resources on how it actually works? It would be really nice to read up on some in depth optimization guides for HDRP. Really hoping Unity will provide one someday.

 

This Unity blog gives a good overview starting point, beoynd that you'll need someone on your team to actually spend several days learning/researching how the SRP render pipeline works, preferably the guy who does your shader needs and if you dont have such a person then the person responsible for optimization in general.
Would heartily recommend the same person spend another few days profiling and googling the results, nothing quite gives one an intuitive feel for what is right and wrong than particular experience within particular project.

Protip in the case that you're using Unity shaders, Unity materials toggle keywords on the shaders in response to what settings are on/off in them. If you have two materials that are Identical in every other way but one has a normal map and the other doesn't, it'll toggle normal maps off keyword and those two materials will be rendered with two different shaders, leading to a shader setup. As you can imagine, you don't need insane amounts of small asset setup mistakes to Really hurt your framerates, both on GPU and CPU. Ensure you do asset validation that everything is as unified as possible, at the latest near end of development but preferably when you start worrying about performance.

 

Practically the same problem our profiler shows us. The project is under threat of being canceled as we cant have a high performance due to HDRP

 

Maybe try building the project and than profile or upgrade to latest LTS (2020 LTS) version

 

Mega runtime performance tips for HDRP

 

That's not at all what I'm seeing. I don't have any extra cameras, reflection probes or anything. It's just that every effect is ludicrously heavy. Even bloom (an effect that has been used for several generations now) at medium quality easily takes up several milliseconds on an RTX 3060, and DoF is completely unusable in real time. Turning off every effect I can barely has framerate touching 90FPS.
Geometry is much heavier than in any other engine. As few as 2m vertices already push the framerate below 60.

I've turned off everything I'm not using. Use low settings for anything where that doesn't look like ass. Even disabled volumetrics entirely and the RP can't even render 2 million vertices at 60FPS.

This is at 1080P with DLSS enabled even.

I can turn off effects, get a couple ms here or there but I feel I'm missing something big. It just doesn't feel right. This card should be able to do so much more. All these effects have been in games for years, sometimes decades, and yet when you enable them in HDRP it suddenly costs several milliseconds on a modern card? But then I profile in Nsight and nothing really stands out. Everything is equally bad.

 

Almost 2 years since then, and now on URP but I did look back and find that HDRP is significantly slower than it was last time I used it. I wonder if this is because of Render Graph (which you can disable). Other changes made were to shadows and so on. I did have conservative settings including allow it to dynamically rescale to acceptable visual quality under load.

I was using it before they changed shadows and added DXR support and I have a feeling it's a lot slower than it used to be. Slow enough that I'm OK with remaining on URP for now. A lot of the visual effects can be emulated in URP at a fraction of the cost where accuracy is not a concern.

Thus, I believe HDRP should be used if raytracing is the main goal, with a fallback that's acceptable. That kind of target.

 

Render graph does seem kinda huge in Nsight. I just assumed it was a collective name for "the renderer". I'll see if I can disable it.

 

We are running VR (2160x2160 pr. eye), contact shadows, SSR, SSAO, DLSS, volumetric fog/light , 8 x terrains (4k heightmaps/splatmaps), 5000m vegetation, ocean system +++ at 50+ fps on a 3080 laptop GPU (130w) and 11th gen i7. Rendering 6.1 million tris / 8.7 million vertices, 478 shadow casters. We are GPU bound...

 

Yeah I was CPU bound. I've gotten so used to blaming everything on the GPU that I didn't even profile CPU usage.
How are you rendering foliage? Seems like that's my issue.

 

Vegetation Studio Pro... It is very performant (both on CPU/GPU)

 

CPU performance is a huge bottleneck in Unity and will often be the reason GPU performance looks less: simply because it can't be fed anywhere near fast enough.

Any profiler screenshots? try sorting by millisecs etc.

 

Yeah it seems to be doing everything on the main thread for some reason.
It looks okay now that I disabled occlusion culling. Will probably be even better with more efficient vegetation rendering. There's a lot of draw calls now - which I presume is 90% instances of the same 3 trees.

Anyway, it's back to being GPU bound now.

 

https://forum.unity.com/threads/new-batchrenderergroup-api-for-2022-1.1230669/#post-7845297

This likely would alleviate a lot of main thread drawing. I don't think it's obvious just how the numbers look when CPU bound for rendering (but it shows GPU bound because it's waiting for CPU).

There is a heck of a lot of allocations in that above shot and the profiler still seems to indicate to me it's spending most of it's time with CPU.

Hard to tell without seeing columns and sorting them.

 

It's spending a total of 10ms on the main thread, while the GPU time is about 14ms. That sounds GPU bound to me.
Allocations are mostly mine but don't really seem to have an impact on performance. There's no GC spikes.

 

Quick test - how much faster is it if the resolution rendered to is something trivial like 720p?

 

Rendering at 640x480 gives me about two milliseconds compared to 2560x1080. But that's in the editor where there's always a few ms of EditorLoop.
CPU should be much quicker in builds.

 

Am I correct in assuming that resolution affects more GPU than CPU performance?

 

Yes.

 

I don't think resolution affects CPU performance at all, so lowering res is a quick way to see where the bottleneck is.

This can be misleading though - as I said, CPU performance isn't representative in the editor. But if you see a performance increase it's definitely GPU bottlenecked.

 

BTW is unity using your dedicated graphics card ?

 

I don't know what other graphics card it would be using. There's only one in my PC

 

It sometimes uses the integrated ones like the intel HD graphics or Vega (AMD)... https://www.google.co.in/amp/s/pureinfotech.com/set-gpu-app-windows-10/amp/?espv=1 u can see here how to do it for windows (don't know for others)..... See if this helps in improving your performance

 

Yeah I don’t have integrated graphics. I know about all that - I'm not exactly a beginner
Just kinda new to this whole HDRP thing.