Bug Quest 2 Performance loss 2020.3 -> 2021.3

Both of my projects are affected in no particular scene or special case, they both have a larger overhead across the whole runtime. None of those projects are using Depth or Opaque, no post processing, and the one I mentioned before is fully unlit, no lights and no shadows.

I know Unity isn’t allowed to do anything with my code, but a policy is just a policy- been there, saw that, multiple times at my previous jobs.

 

I finally submitted a bug regarding the performance issue. I was able to narrow it down to something to do with URP rendering and not anything else running in my game.

The scene I submitted was just a simple map with all my game logic removed. Running on 2020, App T is 6000, and GPU L is 3. In 2021, App T is 7000 and GPU L hovers at 4 to 5.

The good news is that while trying to optimize for this issue, I ended up finding about 3 or 4 areas that definitely improved performance overall.

 

Awesome! What improvements did you find as well? Or are they really specific?

 

I do not think it will happen. Eventually Quest 3 will release and they will be even less motivated to track down performance issues like that.

 

@DevDunk

• Don't animate the scale of Canvas UIs is the biggest one. I went ahead and just nixed the Canvas UI altogether anywhere during gameplay. They were just meters after all so no biggie and easy to replace with non-Canvas versions.
• Restricted light intensity to a value larger than 0, otherwise it causes a dip in frame rate. I'm fairly certain it is optimized to run faster while it is set to 0, but it's not very fast switching between 0 to anything else.
• I found more places where I was calling GetComponent instead of caching, although they were in places where they are not called often, but every little bit helps.
• I primed the dictionary that I use to cache events, although I'm not exactly sure how much that helped.
• I set Depth Texture Mode to Force Prepass, although I'm not exactly sure how much that helped.
• Embarrassingly, I had left antialiasing set to 4x, so I brought that back down to 2x.
• I enabled Fast sRGB/Linear conversions, although I'm not exactly sure how much that helped.

 

Pretty sure 4x antialiasing is recommended by Meta.

 

https://developer.oculus.com/documentation/native/android/mobile-msaa-analysis/
Yup.

 

Strange, I went through some optimization list somewhere and it recommended 2x. 4x definitely looks better, but it really does hurt game performance.

 

2x is recommended for the Quest 1, Quest 2 is definitely 4x with only a minor perf loss (as evidenced by that link).

 

Here's another comparison using Unity 2020. At 2x antialiasing, GPU L is at 3, and App T is at 6000. If I bump up the antialiasing to 4x, GPU L goes to 5, and App T goes to 6800.

So my question is: is GPU level 5 to high to sustain? I believe the higher that number is, the quicker it drains the battery, and the more likely that thermal throttling will occur.

 

The classic "throw more power at the problem" move.
I really want to believe they will sort the issue out, there's no need to explain how much of a stupid move it would be to ignore it. Not to mention Quest 2 will be around for quite some time after Q3 releases, we still have Q1 players around after all.
But you might be right, after all game developers are not really the priority at Unity.

 

How is this issue progressing, Unity team? Can we at least know what is causing the performance regression beyond 2020.x (or even 2019.x if we want to be accurate) or is it a well kept secret?

 

Something tells me the “solution” will be to wait the new 2023.3 tech stream releasing April next year: https://blog.unity.com/engine-platform/2023-3-coming-april-2024-with-updates

 

Don't have faith in Unity's blog posts.

Whether you think they are outright dishonest and an attempt to gaslight us into thinking they care about the stuff we want them to do, or whether you think they are honest (and naive) declarations of actually wanting to improve things, they will at some point collide with reality / higher up managers and the end result will be the same:

Not enough things will change for the better and a bunch of things will change for the worse.

 

Having the 2023 LTS come out at the end of 2024 is pure insanity. Just skip a year for **** sake.

 

@DanjelRicci Just noticed this is marked as fixed in 2023.3 Unity Issue Tracker - [URP][XR] Performance degradation when comparing Android Quest 2 builds across 2020.3 and 2023.x (unity3d.com)

 

Saw this yesterday but didn’t follow up due to the Gamescom craze. This will be the first thing I’ll test once I’m back home (assuming it’s already available). Still absurd the fix didn’t arrive for a more stable version first.

EDIT: wait, isn’t 2023.3.x the version that is planned for 2024? Please tell me I’m wrong.

 

It says "Under Consideration for 2022.3.X" so there's that, also it's about a degradation in performance that's specific to 2022 and up, or at least that's what the bug report says now.

 

Looks like the saga continues, I’m ready to bet it’s never coming for any other version, and it’s not actually fixed on 2023.3.0a2.

Wondering where our license money goes. Maybe to fund their tattoo and barber booth right outside the Gamescom.

 

To be fair it does make sense to me for fixes to come to the new version first, and they're normally decent at backporting important stuff. 2023.3 beta is supposed to be in December if you're desperate for it.

 

Yeah no pressure, it’s not that this problem has been around for two years already after all.

 

I don't work for Unity, I can't make it come out any faster. Just trying to relay some information for you.

 

I’m not pointing the finger at you, sorry if it looked like so. I’m just extremely tired to watch the direction this issue is going. Meta SDK stopped supporting the only version where performance is still acceptable (of course, it’s 3 years old), and the fix doesn’t land on a LTS version, countrary of what the LTS name implies. It would be understandable for a fresh, non critical issue, but this is not the case.

EDIT: Meta is about to launch some extra controls over age groups early next year, and these require SDK v56: https://developer.oculus.com/blog/age-group-self-certification-apis-meta-quest-developers/. The current SDK already stopped supporting Unity 2020 so we need to be able to update Unity without losing performance before v56 is enforced for games that need to use platform features.

 

Are we going to get any communication as to what the fix does? Really curious.

 

It's disabling per light soft shadow settings on standalone XR devices.
Not really the fix I was hoping for, but I'll be testing performance again next week

 

That's not what the current fix should be about. They told it was about soft shadow quality back in June when they marked the issue as fixed, then they reopened it because soft shadows weren't related to the core problem. I mentioned it earlier in the thread, the performance regression happens even when using no lighting and shadows at all, so the fix they did in 2023.3.0a1 must be something new. A fellow VR developer found this change in the graphics pipeline which might be related, but I don't really know.

By the way, the issue on the bug tracker now says In Progress, and it's In Review for 2022.3, 2023.1 and 2023.2 which are actual versions. Here's hoping this story might end positively soon enough.

 

In the new 2023.3 alpha notes you can search for the big report number and see the proposed solution. They have been working on more performance fixes as well, so maybe it actually has improved next to that issue as well.
Either way, only way to confirm is to test it out

 

Just checked that, you're right. I don't really know what to think, I only know the problem is not about shadows because my latest published game doesn't have any lighting at all, and performance with 2021+ doesn't reach the minimum requirement by Meta.

EDIT: thinking further about this, it wouldn't surprise me too much if this was the actual fix even for games that are not using shadows at all- who knows how tangled the URP code could be right now. I will test this once the fix lands on 2022 LTS.

 

Game Info

• Runs on Quest and Quest 2
• Antialiasing is set to 2x
• Bloom is enabled and uses Intensity and Scatter
• There's 1 directional light, and 4 point lights in the scene used for benchmarking.
• OVRPlugins is set to OpenXR
• Graphics API set to OpenGLES3
• Color space is Linear

The latest benchmarks.

Unity 2020.3.48f1

• CPU U 21-40
• GPU U peak 70 hovers around 67
• APP T peak 6350 hovers around 5900
• CPU L always 4
• GPU L always 3
• Stale always 0
• Early always 0

Unity 2022.3.8f1

• CPU U 32-50
• GPU U peak 78 hovers around 77
• APP T peak 7380 hovers around 7000
• CPU L always 4
• GPU L mostly 3 jumps to 4 for half a sec
• Stale always 0
• Early always 0

Unity 2023.1.11f1

• It upgraded with the following issues
  • sound was delayed by half a second
  • lights in my scene were off when they were supposed to be on
  • animators were ignoring some script properties I'm animating

Synopsis:

• Previously when I upgraded to 2022, GPU L was always 5
• The game seems to stutter less in this version of 2022, but the weather is cooler, and so is my house
  • I have to mention that my headset has never triggered thermal throttling

Can't wait to try out the new fix coming our way. After reading above about soft shadows, I hope Danjel is correct and it's some URP thing that never really disabled it completely.

 

https://forum.unity.com/threads/2020-3-2023-3-urp-standalone-vr-performance.1487736/

My testing is done, results speak for themselves. Feel free to give feedback on the testing in that thread if needed

 

I think the issue page has been updated to "Fix In Review for 2022.3.X, 2023.1.X, 2023.2.X"

 

My bug regarding performance was flagged as a duplicate today, and they said that it is fixed in:
2023.1.14f1, 2023.2.0b10, 2023.3.0a2

I upgraded to 2023.1.14.f1 to test performance.

• It appears that 2023 performs at the same level as 2020.
  
• The audio in 2023 still plays a half a second after it is played while testing on the quest. I created a test project just to make sure it wasn't my game in some way, and it is a half a second behind there as well. There's a bug in the editor too, where the sound doesn't even play, or rather, it stops playing after running the game.
• Animating properties with an Animation now work.

Edit:
Looks like someone already submitted a bug for the audio delay:
https://issuetracker.unity3d.com/is...ng-to-play-it-in-the-android-slash-ios-player

 

Sounds good. Personally I am waiting for the fix to arrive on 2022.3.11, which should be out soon. Part of me fears the fix will not work this well, and that we've bascally lost part of the performance forever in the transition from 2020 to newer versions, but I’ll be extremely glad to be proven wrong.

 

I look forward to testing 2022 as well. Meta is taking a million years to approve my game (review began on March 31st and still waiting), so I'll probably have time to test and release on 2022 (unless of course they fix 2023 before then).

 

As someone who has just completed a massive project update, removing XR Interaction Toolkit in favour of the Oculus SDK and moving from 2019.4 to 2021.3 I'm glad the performance issues I'm seeing are not 100% down to our project.

 

I have just tested my three active VR projects on Quest 3, which on paper has 33% more CPU power and 2.5x GPU power compared to Quest 2. I've tested all three projects with the same identical graphics and quality settings I use on Quest 2.

The two projects using Unity 2020.3.48 ran at CPU level 2 and GPU 2/3 all the times, stuck at 90fps with lots of performance room to spare. I was able to increase resolution, remove FFR, enable anisotropic filtering on all textures and both projects still ran smoothly at 90fps.

The project using Unity 2022.3.10 instead runs like manure: CPU is always at level 4 and 100% usage, framerate is choppy, inconsistent and never hits 72fps. This project right now is running much better on Quest 2, and I hope you understand this doesn't make any sense. Remember I am using the exact same graphics and quality settings as Quest 2 right now.

I really, really hope the much awaited fix on 2022.3.11 is going to actually fix the problem in its entirety because, again, this makes absolutely no sense. I am incredibly tired of having to deal with this, it's been way too long and so far we got zero communication about the issue.

 

2022.3.11 is out now.

 

Version 2022.3.11 released yesterday and I took some time to try it with my current project, the one already on 2022.x.

I'm relieved to report that there is definitely an improvement here. I don't know exactly what changed and by how much (it would be great to have some official statement on this since we had zero so far), but all I can see is that now my current project is running with enough performance to spare on Q3 when using the same Q2 setting, enough to be able to switch from 72 to 90fps on Q3 and increase some other graphics settings. At the moment of writing this post, the changelog is empty.

I still doubt this update gave us back all the performance that was lost from 2020, but I'd need more accurate tests for this and at this point I am too tired to care, and happy just enough with what I have. I believe I can finally go back to work on the game rather than on workarounds to make it work, and will roll with whatever performance headroom I have available at the moment.

 

Well looks like I'll take back everything I've said until this is cleared: did 2022.3.11 force-disable soft shadows for Quest, or for Android in general? They only work in the editor.

I would like to mention that soft shadows on Quest never used to be an issue in Unity 2020: V-Speedway is running at 90fps with 2048 shadow res + 2 cascades + soft shadows on, plus 4x MSAA, mirrors and realtime reflections. Why isn't it possible to get the same identical result on Unity 2022? Can we get a single clue on what's happening here?

 

At one point Unity closed this bug report because the submitted project performance loss was caused by shadows being enabled.
They said the performance loss was due to default Shadow settings being more taxing in newer versions than old. It was only reopened after the original author pushed. Perhaps the fix was just to disable it altogether on quest and call it a day? Ha

 

Yes I remember that. Soft shadows have never been a problem but they continue to push on this without any actual clue. I've chatted with someone from the XR team and he too continued to mention soft shadows, and blaming it partially on Meta and their OpenXR implementation (which again, has never been a problem). I'm going to rollback to Unity 2020, my stomach hurts.

 

I think they disabled the per light shadow differences or something. It definitely shouldn't disable all soft shadows, as that's a big yikes. I am not fully sure what they exactly changed tbh.
The issue I originally reported was about unity 2022+ upgrading the 2021/2020 soft shadows to medium quality, while it was on low before. Changing the quality to low instead of medium fixed the issue in that case.
Either way, it's a bit of a mess right now by the sound of it. Also heard people having performance issues with quest 3 passthrough performance, especially when debugging

 

Yeah it's definitely a mess. Soft shadows worked on 2022.3.10 and completely disappeared now on 2022.3.11, which of course makes the game faster, but again, performance of soft shadows has never been an issue. I always set soft shadow to low on Quest if given the choice (i.e. Unity 2020 was always locked on low), but this didn't change the fact that any version after 2020.3 was slower regardless of using shadows at all.

EDIT: I've reported the bug here in another thread for now, apparently soft shadows were meant to work and not just disappear like this.
I also tested my current project both on Unity 2020.3.48 + sdk v50, and Unity 2022.3.11 + sdk v56, under identical conditions, and at least performance seems the same, so I guess that's a win. The only difference was that soft shadows were not visible in 2022.3.11.

 

After upgrading to Unity 2022.3.11 and updating the Oculus plugin to the latest, I made a new build but something is messed up with the rendering. The game starts since I hear audio and the interaction sounds, but the camera is just a mess of colours that fade in and out. Both on my new Quest 3 as well as Quest 2. Guess I'm going to have a great time this weekend figuring out what render setting is bugged this time.

 

Good news and bad news. The good news is that the performance is good. The bad news is that the lighting in one of my levels stopped working.

When you do your performance test, definitely make sure that you're not connected to the quest with Unity or the MQDH. When I first tested, I was connected to MQDH, and it was bad (but that's to be expected). After disconnecting, all was good, and the GPU U was at 54, which is ~16 lower than before. It could be all the lights in the scene were broken and off, we'll see. I'll try to play with it more over the weekend.

 

I haven't tested but I noticed a new warning on light objects:

"Per-light soft shadow quality level is not supported on untethered XR platforms. Use the Soft Shadow Quality settings in the URP Asset instead"

It seems a bit odd to me that there a different things going on under the hood for shadows on XR.

 

Edit: I disable the script that controls all my lights, and now they work. I've updated the numbers below to reflect that.

I've had to stay on 2020 still since I can't figure out why the lights in my scene have completely shut off after upgrading to 2022.3.11f1 - they work fine on my PC.

Anyway, here are the numbers:

Unity 2020.3.48f1 (debug)

• CPU U 21-40
• GPU U peak 70 hovers around 67
• APP T peak 6350 hovers around 5900
• CPU L always 4
• GPU L always 3
• Stale always 0
• Early always 0

Unity 2022.3.11f1 (release)

• CPU U 27-52
• GPU U peak 75 hovers around 70
• APP T peak 7200 hovers around 6500
• CPU L always 4
• GPU L mostly 3
• Stale always 0
• Early always 0

Dunno if I'm allowed to pimp here , but Meta just so happened to approve my game for App Lab last Monday, and I just made it public this morning! So if anyone is interested in playing the Unity 2020 version of my game, it's available here on meta:
https://www.meta.com/experiences/6025372824164562/

 

I want to say that I truly appreciate all the work you're doing in testing all these versions of Unity. Its amazing how much of a S*** show this engine is right now, and they wonder why people don't upgrade from old versions!