Bug Terrain Detail mesh & trees missing motion vectors leading to TAA issues

edit: The issue stems from the fact that anything using instancing/indirect instancing won't write motion vectors.
And terrain detail grass & trees are instanced, so they also don't write motion vectors.
This is a pretty signficant issue. Any vegetation that isn't simple has to use instancing
but because it has no motion vectors, it'll face serious issues with TAA and cause quite some ghosting & blurriness.


This isn't just HDRP, but all render pipelines as well


Case 1372542
And case 1372553

edit: the bug report was closed as won't fix

 

After some testing, the issue has to do with the terrain detail mesh option.
The shader is indeed writing motion vectors, and it works as expected when placed as a mesh, but when using the new (2021.2) terrain option, Detail mesh meant for grass, it's missing motion vectors.

I'm not sure if it's a terrain bug or if we're supposed to do something to make it work now.

 

These are the old reports
(Case 1372542 & Case 1372553)
I updated both and attached a project

Turns out terrain really is bugged. Even when using the new shadergraph wind meant for detail mesh motion vectors are still missing. But when placed as a mesh, it's there

 

Sorry for asking here but there's a chance that it's related:

1. Does the new terrain detail mesh feature use DrawMeshInstanced or DrawMeshInstancedIndirect?

2. Do motion vectors (especially when the shader graph is used) work with DrawMeshInstanced and DrawMeshInstancedIndirect?

 

I tried placing them as trees, they still don't work, and trees don't use GPU instancing.
I also tried using the shadergraph wind meant for detail mesh grass, still no motion vectors

wrong: as for DrawMeshInstanced and indirect, I believe they do. I remember using it a while ago with nature renderer and it worked as expected.

 

Actually, I'm not sure if they work with draw instanced or indirect anymore.
Tried rendering them as meshes but with indirect instancing
and motion vectors are gone... I only tested Indirect though.




How am i supposed to have grass that isn't tanking performance without insane ghosting from TAA? Do i really need to use other anti-aliasing because of this issue, maybe go back to the stone ages with FXAA or SMAA? I would seriously rather stick with HDRP's TAA.

 

HDRP Forward and MSAA is still a viable path you could try.

 

@PutridEx @francescoc_unity @SebLagarde

We are still on pretty old version of HDRP (7.4.3) but I've just tested it.

Motion vectors are not visible when using DrawMeshInstanced and DrawMeshInstancedIndirect but hey work with same material when it's rendered by standard GameObject with mesh renderer component. I'm using HDRP debug (Full Screen Debug mode for motion vectors) for testing.

I didn't noticed this before as we wasn't implementing proper wind yet so our vegetation is still not animated.

Now I'm worried that this is by design and not a bug and motion vectors just don't work with instanced rendering.

There is an entry called "Fixed support for instanced motion vector rendering" in HDRP 12 changelog but I don't know if it's related to this. Anyway looks like @PutridEx is using HDRP 12 already with same problems.

 

Forward isn't really an option for me due to performance issues
My FPS quite literally halves when using forward mode. And I don't have many lights, in the scene i'm testing just a directional light.

from 14.5ms to 27ms

I seriously hope this is not normal, or at least if it is, that it'll be fixed soon before or at the

 

That is incredibly odd if it's only one light, as deferred is only getting a boost from lots of lights! that means you're probably hitting some kind of HDRP bug if 2+ ms are added on, even if MSAA is used instead of other AA (or maybe you just enabled both so it was a little slower and vsync pushed it to half the framerate).

Back when I used to use HDRP, before all the raytracing shenanigans and changing shadows, performance was rock solid with 1 light forward vs deferred in all the scenes I ever tested.

 

Based on profiling, all the additional performance time is on the GPU.
In particular, ForwardOpaque.
I just tried disabling all grass rendering, and the difference is now more reasonable, around a 1.5ms difference (slower) for forward compared to deferred. That's the scene with just the terrain, which I have quite a few of and it's quite big,
and post processing.

It's possible that forward has trouble with instancing & indirect instancing? Alternatively, it could have something to do with how it processes objects in general for forward

 

Updated thread..
the issue isn't particular to the terrain.
The terrain faces the issue because it also instances all trees & grass (detail mesh)
the actual issue is that instancing and indirect instancing don't write motion vectors.

Even if your mesh & wind have motion vectors working fine, when instanced it won't have motion vectors.
This leads to significant ghosting and blurriness, since TAA doesn't have much to work with.
It's especially problematic on grass, and it's impossible to have fast wind of any type.
TAA makes vegetation look quite a bit nicer, gives it a smooth look.
and in general does a better job with everything, not just vegetation.
so i'd rather use it than say SMAA or FXAA.

 

@PutridEx

What beta version are you using for testing this? Release notes for 2021.2 beta 9 have this entry:

"- HDRP: Fixed support for instanced motion vector rendering"

Anyway I have no idea if it's related to this issue.

 

You had me hoping in the first half.
Unfortunately I'm using beta 15, issue still there

 

https://forum.unity.com/threads/enabling-motion-vector-pass-for-drawprocedural.939452/#post-7609909

So here's the answer I think. There is a new Graphics.Render API with RenderParams which supports motion vectors.

 

interesting, can this be used now in 2021.2 and it's working with motion vectors?
so if i'm understanding correctly, we can use it to render instanced or indirect instanced and get motion vectors?

 

QA got back to me and I think they got it confused, they accepted the bug, I was damn happy for a sec thinking instanced objects missing motion vectors was recognized, but it seems QA accepted the bug report as
"higher TAA quality should give better results"
expected: Higher = better quality
actual: higher != always better quality

Which is obviously not what I wanted to convey.
Pretty sure that's not a bug.

Case:1372553

 

Look what it did to my grass, my beautiful innocent grass....
And that's only light wind, anything stronger and I'd get a heart attack

 

Death date: 2021, dec10
cause: Excessive ghosting & blur leading to a loss of will to live/heart attack

 

Putrid, I had similar issues on my open world game.

To skip to the end, this motion blur issue is not going to get resolved in a few patch cycles. I had the same issue and consulted Unity and back then terrain vegetation was not even supported. To make things worse, when lighting is fully renderered with more contrast and higher spectrum of darker and brighter areas, this ghosting is going to get 1000 times worse. In my experience, it is best to slow down the movement of vegetation (not the amount but more of how fast it moves around pixel wise) and then turn on DLSS. It masks the issue 90% of the time.

As for how practical my suggestions are...welp.

But in general, I've found out unity TAA causes a lot of rendering issues and DLSS just auto fixes it (1. AI reconstruction at pixel level, and 2) TAA is turned off when DLSS is on)

For the same reasons I don't recommend FSR because it used Unity TAA and with it all the many years of TAA rendering issues.

Hope this helps.

 

Thanks,
unfortunately i had access to an RTX card for a bit but i'm now back to my laptop with a GTX 1070
And anyone playing my project will use TAA unless they have an RTX, so TAA is the main thing I have really.

Without TAA, grass aliasing is insane, and especially flickering. TAA makes it look a lot nicer, smoother.
But then we have ghosting, no motion vectors. I honestly won't give up on this, it's such a big deal -- pretty disappointing to see it ignored for so long. How am i supposed to have grass? Place them as meshes and reduce density by 80%?
Going back through unity versions, I tested a lot -- and it's always been like this. Maybe even on built-in.
The only alternative is to switch to FXAA but it doesn't do as good a job as TAA, and blurs the entire screen quite a bit.

 

Vote for the issue here please: https://issuetracker.unity3d.com/is...of-gameobjects-with-higher-quality-preset-set

 

found this: https://github.com/keijiro/InstancedMotionVector
it's very old, but seems like it demonstrates a way for motion vectors when using indirect instancing. Built-in.
I believe terrain is using instancing, not drawindirect instancing, but still.

 

This issue still exists and is quite annoying for thoae of us using assets like GPU Instancer.

 

Unity Issue Tracker - [HDRP] Motion Vectors are not applied to the GameObjects when they're rendered through Terrain (unity3d.com)
Closed as won't fix
I replied to the internal bug report thus the "closed as won't fix" status isn't there anymore

Even though there's a new API for exactly this added by unity recently. Nature renderer can now render terrain grass with motion vectors, but unity's own terrain can't

 

Like I said before. Many years of nightmare when it comes to TAA in Unity.

Just let it go mate. Unity isn't interested. Just look at their latest Terrrain demo....
Sometimes we just have to let it go...
let it go....

 

@francescoc_unity I guess u are the only unity guy who listens forum and solve all the pain points that community complains for years Unity definitely needs more developers at hdrp team...

 

This problem existed since at least Unity 2020 and still is in 2022.1.
Now I also have an issue that comes and goes with TAA's motion blending where the moment the camera moves everything in screen starts to flicker/ghost regardless of motion vectors enabled or not.
Unfortunately I wasn't able to reproduce it on a new proj so it is likely caused by a 3rd party system idk...

 

This affects me as well and prevents me from using TAA in my project. Hope for a fix.

 

Yes, still happening in latest alpha. Unfortunately the bug was closed with "no plans to fix this bug" There is an API for adding motion vectors to indirect instancing now, unity recently added it, hopefully unity adds it to terrain trees and detail objects soon.

Unity Issue Tracker - [HDRP] Motion Vectors are not applied to the GameObjects when they're rendered through Terrain (unity3d.com)

Fun stuff

 

Rip, so the only viable solution is to buy yet another 3rd party asset to fix the issue. (nature renderer)

 

This is just absurd.

 

This must hurt. Unity is weird this way - so close to greatness, but then you stumble into small but insurmountable issues that make an entire feature unusable.

-off topic-- Not meaning to go off topic, but this is what happened with local volumetric clouds causing terrible black spots on alpha clipped objects (like our custom grass). But to their credit, the bug was fixed within a few months of reporting. Now we're stuck on a similar issue where the order of drawing the local clouds cannot be controlled in relation to other transparent objects. --end off topic--

I feel that in this case, we're lucky because we're using a custom grass system (in-house). Unity's terrain features, although greatly upgraded in recent years, do still suffer from some issues and drawbacks. It just doesn't feel like a first-class citizen in the Unity eco system.

I wonder if it is worth re-reporting this bug from scratch, but being extremely careful in the wording and images so there is no more misunderstanding (I noticed the comment in the bug tracking, suspecting such a misunderstanding). This might also be a case of Terrain Team thinks it is HDRP Team's problem, and vice versa?

 

I cleared the misunderstanding back when I reported the bug, and they updated the bug title, so they understood the issue

To sum up the reply I got on the most recent Questions & answers done by unity when asked about this issue (1/27/23):

Spoiler: Unity dev response

We (URP/HDRP teams) are aware of both issues internally (lack of SpeedTree motion vectors, which impact TAA quality and terrain not rendering them for trees and details) and coincidentally, the person responsible for SpeedTree shaders has just transitioned into looking into this task as planned (We briefed him about the problems and possible solutions Yesterday)! We can’t give any hard estimates as he’s still in the discovery stage, but we are looking to have a solution for this for both URP and HDRP in the not too far future.

Now if unity says not too far in the future, this usually means a few years to never. Hopefully not.
The terrain team decided to close the bug as won't fix, I wonder what they think is worth fixing, seeing how far behind unity's terrain is.