Is this amount of TAA ghosting on a skinned renderer expected?

Case 1426984
Default blending TAA value:

I'm almost positive now there's a motion vectors issue. There's no way HDRP's TAA causes this much ghosting.
Not sure where the issue is though, skinned motion vectors is ticked. Maybe it's related to the camera controller somehow, or HDRP's motion vectors have issues with skinned MV.
Or it's related to the terrain? not sure..

 

Did you maybe turn off motion vectors somewhere?

 

I don't think, didn't touch any HDRP settings, and motion vectors show up when using the debugger and moving the camera

 

no it's just that bad

 

I guess you also switched "skinned motion vectors" on already on SMR? Some kind of ghosting will always be there, especially on high contrasts. I try to avoid this by using darker colors for scene objects which get often overlayed by moving objects.

 

Yeah, skinned motion vectors is ticked
I mean I understand some ghosting, but this is pretty severe. People hate TAA when it has that much ghosting, and honestly I can't blame them. And with HDRP you kind of need to use TAA, unlike URP/built-in.

If this is considered expected behavior, I'll be hoping for further improvements to TAA because this will be immediately noticed by players. It's not a tiny amount of ghosting that you need to zoom in or slow things down to notice. It's very noticeable at normal speed of turning the character or moving the mouse. I believe jumping as well.

And this is with the minimum amount of TAA blending, default values for blending would make it more noticeable.

 

default blending value:

I'm almost positive now there's a motion vectors issue. There's no way HDRP's TAA causes this much ghosting.
Not sure where the issue is though, skinned motion vectors is ticked and . Maybe it's related to the camera controller somehow, or HDRP's motion vectors have issues with skinned MV.

 

Try without the terrain. Does it still ghost? Use debug motion vector view so you can see the motion vectors.

 

Yeah considered that so I tried it with a cube, using the same soil texture, ghosting was still there with the cube in the background
I did use the motion vector debugger and motion vectors seem to be written, not sure if they're correct though

 

You sure? Because I don't see bad motion vectors over the sky or orange cube.

 

camera:







You can download the project from the bug # report

 

Are u using motion blur??? I think i also had this issue but after disabling motion blur it fixed it!?

 

I tried to add some seagull flock animation, but got issues with skinned mesh renderers and DLSS due to MV's I think.

 

I actually did play with blending/flickering settings, and originally I posted a gif with the minimum amount of blending possible, unfortunately the ghosting was still very apparent.

Nope, motion blur disabled in default global settings & scene volume

 

I've been saying this for a long time.

Unity's motion vector will break randomly, and not work.

This is only noticeable when TAA is on.

I've been saying this for like 5 years now.

It starts to paint the screen when the color contrast is high.

If the object is near camera, it smudges.

If the material uses triplanar, it smudges.

Vertex animated objects with Motion Vector? -> it smudges.

And then restart PC -> fixed.

I've given up on TAA long time ago.

Solution: Don't use TAA, turn on DLSS and let AI fix it.

Funny thing is, even if you disable motion vector for vertex animated materials, it will fix it to some degree.

Love nVidia for this.

 

Thing is you have to use TAA, not everyone has an RTX card.
I really don't want to go back to SMAA/FXAA, it's 2022 and we're doing PBR games. Flickering shouldn't be a thing.

Hopefully this is sorted. Maybe TAA will receive improvements in 2022.
And then, there's the issue of terrain grass and trees having no motion vectors.. (anything instanced loses MV)

 

Is this even possible to fix? Even UE has TAA ghosting everywhere(this is why i cant watch any UE related videos on youtube, my eyes start bleeding when i see how badly everything smearing in UE editor)

 

But it's definitely more better than unity!!

 

Very possible. Many games have amazing TAA implementation that even do well on small meshes (like wires/fences).
A good TAA implementation is amazing, and should be a top-priority for all 3D PBR engines.

It will literally be on-screen all the time, effecting everything. One of the most important things to get right.
UE4 TAA is also not that good, but it's fairly stable and works. Skinned charcters are fine with it, almost no ghosting.
I did many tests with it, and played many UE4 games that use TAA.
UE4 (don't know about 5) has the same grass issue as unity though, instanced grass loses motion vectors. Which leads to ghosting/blurriness.

In UE5 They introduced TSR, mainly for upscaling, but it also does anti-aliasing, it's basically better TAA compared to UE4 TAA, but a bit more expensive.

They also introduced TAA gen5 in 4.26 (or near that version), I believe it's similar to old TAA but with some adjustments, it does a slightly better job when it comes to flickering/small meshes and blur for most projects.

Unlike unity, unreal only has TAA and FXAA, so you're basically forced to use TAA.
Hence, almost every unreal game out there uses TAA. It does have it's fair share of issues though, prior to UE5 at least.
And, Unreal engine for whatever reason has really bad aliasing when TAA is disabled, worse than you'd normally expect, they seem to rely on TAA a lot.

Many HDRP titles so far only use SMAA, possibly because TAA wasn't in a good situation prior to 2021. (all games I checked are on 2020 or older)

Hopefully this bug gets looked at soon.

 

I had mentioned about the similar issue previously and @pierred_unity replied this https://forum.unity.com/threads/strange-smering-effect-when-objects-move.1245514/#post-7934146

 

I don't think this project fits his comment, it's a 3rd person and not really fast moving.
He was more talking about first person, very fast moving cameras.

In the gif, even a jump leads to noticeable ghosting.
And as you can see, the camera was moving very slowly at the start, unfortunately ghosting is still there. I believe it's possibly a motion vector issue, maybe something wrong with the character.

Also while exploring in this project, I found a (possibly, could be wrong) bug with speed rejection and sent it to unity, bug case # 426305

 

Actually, I think the issue is not that.

In my experience, objects near and on focus to the camera, seems to be having this issue. For example, in an FPS environment, the player hand/gun does that, but not much with close by environment.

Also, try the following, triplanar seems to ghost like crazy, I've tried literally same texture but one with normal repeat mapping and the other triplanar and ghosting occurs on triplanar only (probably why it happens with terrain as well in the first post)

But for some reason, as long as the object is moving in screen space, ghosting is not as severe.

So, my guess, if the object is moving in world space but not moving on screen space, (so, character on screen, FPS hands and guns) then this ghosting starts to go crazy. The same happens with racing games near tires.

I know that most racing games with TAA implement a mask layer to handle this, and perhaps Unity can look into a similar approach.

If you want to confirm my theory, then put a finger on the rotating character's head, and see where ghosting occurs. Ghosting is happening where the head hasn't been. Which means the motion vectors are generated correctly, but the TAA is not taking camera rotation/movement into account. Or if it is, then the code needs revision. TAA is notorious for ghosting but not like this.

Regardless, TAA needs work.

 

HDRP's TAA implementation increases history blending (decreases the current frame's effect on the final output) when there's high color variation in a given area and the previous frame's color at that pixel was dark. This is likely one of the bigger contributors to the smearing in the OP's video (you can see mostly dark colors being smeared there).

Here's the problem code: https://github.com/Unity-Technologi...g/Shaders/TemporalAntialiasing.hlsl#L699-L704

In my own project I've been experimenting with changing that function to decrease its impact when there's some motion in the pixel and to be less dependent on the history color's brightness (since that part also causes broken-looking brightness shifts when anything about the history blending % changes), with some success so far.

With improvements to that and to history rejection and a few other aspects like the area of influence used by TAA (and maybe some sort of material ID rejection system, even if it's limited), HDRP's TAA wouldn't be too far off from other implementations in terms of artifact issues. It's not there yet though.

 

sounds pretty reasonable.
regarding motionvectors: first video would not really need per object motion vectors as there isn't anything moving but the camera. so looking into the contrast thing absolutely makes sense here.

 

Was curious how built-in TAA would handle this, same texture:
(ignore the low FPS, that's just the video)

As you can see there's no ghosting, almost as if TAA isn't on.
Built-in TAA also offers a jitter paramater which is great to finetune the amount of TAA, for the video it wasn't changed though
The same at high speeds & camera movement

 

Since switching to 2021 the TAA is really broken
On 2020 TAA worked perfectly fine, now after our switch to latest and 2021 LTS its glitchy
There is heavy blurring, instability, never wanting to stabilize, outlines when particles are behind the object, and a general strange white fringe on everything if you compare with photoshop.

I can not see the ghosting effect of OP however on 2021 and this is definitely not normal.




Bug - 2021.3 - instability and varied bugs (TAA broken / Visual Environment breaks) - Unity Forum

Since there has been a lot of work on DLSS, FSR and Dynamic resolution + Camera tweaks,
some things must have broken there

 

have you looked into the fps for both hdrp and birp? i guess birp will run at 150-200fps, while hdrp might only run at 60-90?
assuming that taa will use the last 9 frames for reconstruction these 9 frames would mean way less time = change in birp resulting in less artifacts just do to fps.

 

tested with 60 FPS in built-in, same results
plus, I seriously doubt FPS would be the cause. I can see it happening if I was running at 20-40 FPS, but 60 and over there is very little difference.

It just needs improvements as chap stated, to be honest I didn't think HDRP's TAA would have such significant issues.
(Hopefully instanced grass gets motion vectors one day as well )

 

IDK but from my limited testing yesterday, i was not getting any smudging or blurriness because of TAA on grass(painted with GPU instancing).... I am using TAAU and there is an option in the last part of the materials properties of something called motion vectors for vertex animation which I have ticked... also i am using 2022.2 B13 version

 

Awesome! Is there a publicly available branch of the Graphics repository with these changes we can try, or will we have to wait for it to be merged into the main branch?

 

Thanks for the changes, much appreciated, but it is also alarming that the community has been saying this and it took years for someone at Unity to actually try it and see.

Also, I am concerned that there is no dedicated team for TAA as nowadays TAA or temporal related rendering is pretty much rendering quality itself. Shouldn't there be a dedicated team for this all the time? I mean, Unity isn't a powerhouse rendering engine like UE, and can only handle small scale scenes, but at least Unity should be trying harder to make that better.

I often get the feeling that Unity is not that much interested in rendering quality. Even if the market is moving on.

Just my 2 cents.

Also, I hope that the fix is actually released for 2021 and not for 2022. Please don't say that 2021 is in LTS and further features won't be added...I've had this so many times now and I am afraid...

My current project is on 2021 and I have been battling to keep TAA in line and now I see why...

 

Thank you for the honest feedback. Really appreciated. Also thank you for being honest about backports. If you need anything from us, please ask. I would love to help given the HDRP team size, now that I hear it. Although I think it is a managerial misstep to not give the HDRP team the resources they deserve. After all, HDRP is spearheading Unity.

 

Fully agree!
I coudnt care less about URP or built-in

 

Awesome, both for the improvements and for the backports (especially for the 21.3!). Can't wait to test it out, maybe I'll give a TAA another shot.

 

It's true but in the movies, a small group of heroes saves the day. I think you guys all saved my eyes at least <3

 

I don't want to be that guy that constantly asks "when", but since those are improvements that I am particularly excited about I have to ask it at least just once. Is there any eta about the backports landing (especially in 21.3)?

 

@francescoc_unity are these changes already in the latest 2022.2 beta and 2023 alpha?

 

https://unity3d.com/kr/unity/whats-new/2021.3.6

Is it Updated?

 

Thank you for your answer
I'll have to update it and test it

 

Awesome. Had to try it. It's been annoying me for quite some time:

Quite visible with the right wing, ie the one that has the wall in the background.

Great job, thank you!