Search Unity

  1. Engage, network and learn at Unite Austin 2017, Oct 3 - 5. Get your ticket today!
    Dismiss Notice
  2. Introducing the Unity Essentials Packs! Find out more.
    Dismiss Notice
  3. Check out all the fixes for 5.6 on the patch releases page.
    Dismiss Notice
  4. Unity 2017.1 is now released.
    Dismiss Notice
  5. Help us improve the editor usability and artist workflows. Join our discussion to provide your feedback.
    Dismiss Notice
  6. Unity 2017.2 beta is now available for download.
    Dismiss Notice

Temporal Anti-Aliasing

Discussion in 'Image Effects' started by Tim-C, Jun 21, 2016.

  1. Panokani

    Panokani

    Unity Technologies

    Joined:
    Jun 6, 2013
    Posts:
    12
    Hi there!

    I am afraid this is actually the case.

    It's very hard to tell what's going on in this scene from the video and the screenshots you have provided. Would it be possible for you to provide a simple, shaven repro for us to take a look at? Right now we have no idea what's your pipeline like and how is it set up.

    To the contrary, so far our empirical evidence shows that TAA is vastly superior in comparison to SMAA we have had. Your screenshots clearly show artifacts that are not expected behavior of TAA. I am thinking the devil is in the detail. However, without a repro case it's hard to tell what's going on.
     
  2. Panokani

    Panokani

    Unity Technologies

    Joined:
    Jun 6, 2013
    Posts:
    12
    Hey there,

    Unity's TAA is not based off of Playdead's implementation. In fact our current one on the stack is our own original research. That said we did make use of Playdead's AABB intersection function and their projection matrix calculation code, which we are very thankful for.

    Our implementation uses Unity's own motion buffer and doesn't require any sort of velocity buffer tagging. I am afraid your problem is quite irrelevant.
     
  3. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    963
    Ah, of course, I forgot about that. Then it is indeed not the same issue. (Unless the Unity motion buffer is based on the TAA velocity buffer from Playdead ;-) )

    I actually changed some of the projection matrix calculations to enable VR support. And the velocity shader for moving objects of course. Oh, and I added a simple tonemapper to do the blending more on the final result than on the original linear data.
     
  4. Panokani

    Panokani

    Unity Technologies

    Joined:
    Jun 6, 2013
    Posts:
    12
    It's not.
     
  5. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    150
    Hey, I'll try to prepare a unity project for you guys.
    I'll try to make it as small as possible while still maintaining the problem, I'll report back soon.
     
  6. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    150
  7. Panokani

    Panokani

    Unity Technologies

    Joined:
    Jun 6, 2013
    Posts:
    12
    Hey dadude123,

    I downloaded the scene, and I couldn't reproduce your problem.

    I noticed that the blending factors for TAA are both 1 and tweaked them a little, and dropped the spread quite a lot (we usually keep this one low). The result is that it looks much smoother than SMAA, however the super tiny UI element that moves rather fast loses quite some detail (same with SMAA as well). This happens since all AA algorithms try to smooth out jaggies, and ~1 pixel width text on a contrasting background is the perfect example for a jaggy.

    Usually in games the UI is composited on top of the render after post-processing has been performed, and is not part of the post-processed framebuffer.

    Have you tried tweaking the parameters to see what happens?
     
  8. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    150

    Here's what I get from the demo scene:
    TAA off: http://i.imgur.com/AgSXW5X.png
    TAA on (with settings like in the scene i uploaded): http://i.imgur.com/BgvUaZf.png


    TAA on (spread=0.38, stationary=0.75, motion=0.75, sharpen=0): http://i.imgur.com/Zt1vmTk.png

    That's a more than "a little detail" thats los imho :)
    I changed the settings until the top edge of the wall was almost smooth. But I couldn't get it any smoother than this - which is objectively worse than SMAA.

    Can you please show me some screenshots how this looks for you with different settings? Maybe there's some bug on my computer where it does the inverse (ignoring the edges)
    Since you coulnd't reproduce my problem, could 1. you show me how things look for you? and 2. what TAA settings you'd use for the best possible quality in that demo scene.

    The thing that really surprises me is that in my last screenshot you can clearly see that the text and grass are overly blurred, while the top of the wall (wall - sky edge) isn't antialiased at all. Is that not a problem with your settings?

    Do you at least agree that something in my screenshots seems to be wrong?
    Do you get different results??
     
    Last edited: Jan 26, 2017
  9. Panokani

    Panokani

    Unity Technologies

    Joined:
    Jun 6, 2013
    Posts:
    12
    Hey again!

    Yes, I get completely different results.
    Here's what I get with no antialiasing:
    [​IMG]
    Here's what I get with SMAA:
    [​IMG]
    And here's what I get with TAA:
    [​IMG]

    With SMAA, notice the missing aliased edges on the grass blades. It does a good job of antialiasing at sharp and contrasting edges. This problem isn't there with TAA; however with high frequency and noisy aliasing like with foliage, TAA tends to smudge and ghost things. Unfortunately, it's implicit to the nature of the algorithm. That said, I am not getting anything like your screenshots, and I could up the spread all the way to 1.
     
  10. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    150
    Ok, wow, that is surprising.
    The top edge of the wall is perfectly smooth in your TAA screenshot. Almost indistinguishable from the SMAA.
    Indeed TAA looks completely different for you...


    Now what could possibly be the reason that our results differ so much?
    I mean I have a hard time finding any TAA settings that could even come close to what your screenshots look like.

    Could it be some driver issue? ( GTX970 Driver: 376.33 )
    Maybe I have an outdated version of tha AA shader? This is the right link, isn't it: https://github.com/Unity-Technologies/PostProcessing ?

    And I assume you didn't change any other quality settings, so I honestly have no idea whats going on.
    Any guesses? Should I just use SMAA until the github repo is updated with whatever you guys are working on?
     
  11. Panokani

    Panokani

    Unity Technologies

    Joined:
    Jun 6, 2013
    Posts:
    12
    Try these settings for a change:
    Spread: 0.92
    Stationary blending: 0.88
    Motion blending: 0.58
    Sharpen: 0.3

    Also, I noticed you have MSAA on, please disable that as well.

    I doubt it, I used to develop on a GTX970 myself.

    I checked this and your version isn't that outdated, no major changes to TAA for quite some time. In fact I used your own project to produce the screenshots.

    Nothing changed in quality settings. We're not working on any major changes to TAA, the one in the repository is final. We'll try to figure out what you're experiencing and get back to you. In the meantime can you try again with MSAA disabled?
     
  12. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    150
    I disabled MSAA in the quality options but it didn't make any difference at all. After that I enabled it and then a warning appeared in the TAA settings "taa might not work correctly while msaa is active", which wasn't there before.
    Pretty sure that the camera being set to deferred rendering seems to have disabled it anyway so far...

    The settings you gave still suffer from the same problem: non-edges get too blurry; sharp edges only get smoothed a little bit. (And everything that gets smoothed still isn't as smooth as with SMAA)

    I don't know about you but the difference in the screenshots I made and the ones you made are too big to not be a bug (even when I compare against the ones I just made with your new settings).

    Maybe I can try on another computer today or tomorrow - I'll report back when I have more information.
    Hopefully a completely new/different setup will give better results.

    edit: In your screenshots the moving panel+text is not visible.
    Does TAA give you same terrible "ghosting" on it? (And if not the same, then how much? none at all?)
     
  13. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    150
    Just tested some more and figured something out.
    When you have a solid material, then a worldspace UI + Text the ghosting is not as bad as when you have either the skybox or something transparent (Grass or Tree-Billboards) in the background.

    Something about having transparent things and then UI ontop of it makes the TAA flip out when stationary blending is 0.9 or higher.

    Still doesn't explain the bad quality on the edges, but maybe its a start for you.
     
  14. Panokani

    Panokani

    Unity Technologies

    Joined:
    Jun 6, 2013
    Posts:
    12
    I think we're onto something here. Looks like it's an AO issue caused specifically on 5.5. I myself am running a later version internally from trunk, so that's why I never noticed it.

    Disabling and re-enabling AO should fix it for some weird reason.
    We're looking into the root cause, but it seems to be fixed in 5.6.
     
    elbows likes this.
  15. Chman

    Chman

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    718
    So it turns out it's not related to AO and whatever is causing this isn't fixed in 5.6. In the meantime, disabling/enabling TAA or AO or any object on the scene should fix it until the next run. Note that it only seems to happen in the editor.

    @dadude123 Could you please fill a bug about this issue so we can track it ? Thanks !
     
  16. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    150
    Created #79
     
  17. Flurgle

    Flurgle

    Joined:
    May 16, 2016
    Posts:
    217
    Tested it out, looks promising

    @Tim-C I'd love to see a few TXAA component settings screenshots (or have people share them) with environment label (like outdoors wilderness forest, grassland / mountain, cavernous area, Sci fi corridor, etc).
     
  18. DanTreble

    DanTreble

    Joined:
    Aug 31, 2010
    Posts:
    583
    We just updated to the latest commit (using 5.5.1p4) and are still getting really bad jittering on transparent objects unless removing [ImageEffectOpaque] from the effect. Are there any solutions to this at the moment?
     
  19. topofsteel

    topofsteel

    Joined:
    Dec 2, 2011
    Posts:
    873
    I'm testing this out again for a new project. It's come a long way since it was first released. I'm getting fantastic results at run time, but some odd artifacts in the editor. I have not spent a great deal of time debugging it, starting here. Thanks.

    temporal_01.png temporal_02.png
     
  20. Xel_Naga

    Xel_Naga

    Joined:
    Oct 21, 2015
    Posts:
    46
    For VR it's still not working?
     
  21. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    19,672
    Gonna hazard a guess that the team is busy researching solutions and trying to grow fruit upside down*.





    *you never know what Unity devs get up to really
     
  22. Kalata8662

    Kalata8662

    Joined:
    Feb 7, 2017
    Posts:
    54
    Hey, I tried the temporal antialiasing and it works perfect in the editor. I tried to export to web GL thought, and in the browser it is all black. Is support for Web GL planned at all and when it can be expected? Thank you.
     
  23. roberteker

    roberteker

    Joined:
    Feb 22, 2017
    Posts:
    121
    I would be interested in an update regarding this as well.
     
  24. Argenzio

    Argenzio

    Joined:
    Apr 27, 2014
    Posts:
    183
    Yes we really need TAA for VR.
    For now we are blocked in forward + MSAA... it would be wonderful to use Deferred + TAA.
     
    plmx and b4th like this.
  25. roberteker

    roberteker

    Joined:
    Feb 22, 2017
    Posts:
    121
  26. elbows

    elbows

    Joined:
    Nov 28, 2009
    Posts:
    1,266
    I was under the impression that TAA in general isnt very good for VR because the constant sub-pixel head tracking movement makes TAA results too blurry at the resolutions VR headsets currently support.
     
  27. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    3,249
    The issue with directional shadows can be worked around, and I think will be fixed properly once Unity's Scriptable Render Loop "HD" renderer gets officially released. I also think it might be solvable by disabling cascaded shadows in the graphics tier settings (for 5.6 on), but then you loose cascaded shadows ...

    The issue of specular causing issues with MSAA from over interpolation can be solved as well, as per my post in the second thread. I also found out that my original implementation of Valve's method was a bit off, and it works nearly flawlessly, but the same issue of requiring additional interpolants exists. However there are many causes of "fireflies" or sparkles that may or may not actually be from MSAA, and some issues are exacerbated by issues that can be reduced or solved by using the standard shader instead of the legacy specular shaders. Some of the mountain of posts complaining about fireflies are completely different issues, some of which happen with or without MSAA and people are simply expecting it to solve for them not realizing MSAA only works on polygon edges and thus won't solve mid-surface aliasing like that caused from normal maps, shaders, or mip mapping.

    For the most part it's an issue that isn't a problem most of the time, but adds some cost to the shader that is unnecessary, and can potentially even cause issues on platforms not using MSAA. It would be nice if Unity offered some kind of shader preprocessor define for MSAA to enable it, but it might not be worth it to them as it's doubling the shader variants for something that is only going to be a problem for a very small fraction of people. Also, as I mentioned before, the Standard shader is pretty good as suppressing most the cases with out needing the "real" fix.

    That little sub-pixel wiggle is what TAA already does to get good AA. If you read up on TAA you'll invariable run across the term "halton sequence", which is a bunch of psuedo random points with zero overlap which allows for excellent subpixel coverage if used to jitter the camera's projection matrix. There's even been some discussion on disabling the jitter for VR as humans naturally "jitter", but it only really works if the tracking is good enough (ie: Vive and Oculus could do this for most of the tracking range, PSVR isn't quite accurate enough though).

    Image stolen from the Wikipedia article on Halton Sequence:
    [​IMG]

    Also for deferred rendering with VR, TAA is kind of the only option, apart from super sampling, and the various issues with with TAA (thin lines being smudged out, aliasing on recently un-occluded regions, general blurring) means you kind of need both TAA and some amount of super sampling to get acceptable quality, which is what UE4 does when using its deferred rendering path for VR.

    Technically it is possible to use MSAA with a deferred rendering path, that's what the Frostbite engine did at the end of the Xbox 360 / PS3 generation, but it isn't something Unity supports, or most engines for that matter. It's also something that's way too expensive to use with VR, and TAA has entirely supplanted its use with deferred elsewhere.
     
    plmx, hippocoder and RomBinDaHouse like this.
  28. mephistonight

    mephistonight

    Joined:
    Dec 14, 2016
    Posts:
    11
    Hi,

    Like so many things in game development, it feels like TAA is a great addition to the toolbox and one with pro's and cons versus other anti-aliasing methods.

    I'm currently using the post processing stack from the Asset Store with TAA tuned to the settings Panokani mentioned above. I take some of the issues mentioned above as just a need to tweak depending on the scene TAA is being applied to but there's on issue I don't seem to be able to account for:

    If I switch between TAA and MSAA there is a massive impact on the intensity of the lighting. MSAA is really dark whereas TAA seems closer to what I would expect with linear colour space in use.

    Has anyone else experienced this? Is it something I should be expecting?

    **EDIT** - it seems the light intensity changes I was experiencing is actually to do with the 'Color Grading' effect that also comes as part of the post processing asset. If you enable TAA it appears to overwrite the Filmic (ACES) tonemapper. Is this because TAA does some tonemapping as part of its processing, so it take preference over the Color Grading section's tonemapper?
     
    Last edited: May 8, 2017
  29. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    963
    That's actually what I always do for VR. It's not really a matter of jittering or tracking accuracy, but more the fact that you can't stand still.

    TAA works as long as the camera moves. (In a "random subpixel" way, so the pixels don't align between frames.) The jitter comes in to solve the lack of temporal information when the camera doesn't move at all. With VR the camera never stops moving, so I always disable jitter then.

    The overall look of TAA is fairly blurry, but it can be sharpened in post or by increasing the aliasing on the input. (Sampling a texture at a higher mipmap level for example.)

    In the end TAA fits well with VR, because both require a high frame rate to work correctly. And in my opinion also because the camera never stands still in VR. That is perfect for TAA.
     
    Alverik likes this.
  30. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    3,249
    It is a matter of tracking accuracy, as many VR devices will go to sleep and stop updating tracking if the measured movement is below some threshold. The noise and accuracy of the PSVR is such that a human can keep their head stable enough to invoke that sleep state. Generally not a problem since it can wake itself up instantly as soon as you do move, but it can occasionally cause TAA to break.

    If it wasn't for the dis-occlusion issues, and thin line handling, I would say TAA is nearly perfect. My experience has been MSAA always beats TAA for performance, aliasing reducing, and clarity for geometry edges... assuming you don't need the features of deferred and aren't using a lot of shiny surfaces or otherwise having problems with shader aliasing. I build anti-aliasing into all of my shaders and have been making heavily stylized VR games, so it's not been a problem for me.
     
  31. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    963
    Ah, haven't worked with PSVR yet, so I'm assuming there is no sleep state.

    I totally agree that MSAA is a better option if possible, but I do like deferred rendering for lighting and additional reflection passes. (Architectural instead of games. A little less freedom when it comes to the level design or style.)
     
  32. yu-wan

    yu-wan

    Joined:
    Jan 4, 2016
    Posts:
    18
    I tried to put the temporal AA component on the demo of CorridorDemo, but it turns out that the transparent object(water on the floor) always jittering/jumping. My unity version is 5.4.4f1. While reducing the Spread to 0.5, the anti-aliasing effect is not that evident. Does anyone know what has happened?
     
  33. yu-wan

    yu-wan

    Joined:
    Jan 4, 2016
    Posts:
    18
    I've checked through the frame debugger and found that the TAA is executed before rendering the transparent objects. May I know how to deal with such case? The water of video in the first page doesn't look that jittering.
     
  34. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    963
    Yes, that is how TAA generally works I'm afraid. It requires the depth buffer as input, so it's executed right after the opaque materials are rendered.

    For transparent items, there are a few options:
    1. Render them after TAA, but with a camera without jitter.
    2. Render them before TAA as opaque geometry, but after real opaque geometry. Your additional choice here is to either overwrite the depth value or not.

    I've used option 2 up to now. There is something to say for option 1, but you'll miss the AA. So you'd want to apply MSAA to the transparent items. The complication there is that you can't directly reuse the opaque z-buffer without MSAA in the transparency pass.
     
  35. yu-wan

    yu-wan

    Joined:
    Jan 4, 2016
    Posts:
    18
    Many thanks.
    I'm not using MSAA since I'm in deferred mode. I'm curious that is it compulsory everything will write Zbuffer for a proper TAA work? Since the transparent object in opaque will still not write depth.
    BTW, would you like to tell me how to enforce transparent object being rendered in opaque after real opaque? Just modify the render queue?
    Is it the way how the demo video follow? I saw that the transparent water in corridor floor is not jumping at all.
    Again, thanks for your reply.
     
  36. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    963
    The transparent parts are always rendered in a forward way without using the g-buffer, so it should be possible to enable MSAA just on that pass. (RenderTexture with MSAA on a separate camera just for the transparent parts.)

    The depth values are indeed required for proper/correct TAA. So as with many things, it doesn't combine well with transparency. Just changing the render queue is enough to move the transparent objects before TAA.

    By adjusting the shader, you can also have it write to the z-buffer. This is a choice between two wrong situations. In reality you're looking at multiple depth values on the same pixel, but we have to choose one. In my experience TAA works best if you choose the value closest to the camera, with z-writing enabled in transparent shaders.
     
  37. SunnyChow

    SunnyChow

    Joined:
    Jun 6, 2013
    Posts:
    270
    i feel that when the TAA is on, it's very to get Z-fighting for transparent object that is close to some opaque object. Any suggestion to fix this problem?
     
  38. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    963
    You're missing a word there, but TAA doesn't really cause z-fighting. If anything, it reduces the effect of z-fighting, because of the blending. Close surfaces cause z-fighting and you'll need to solve that independent of the used AA technique.
     
  39. SunnyChow

    SunnyChow

    Joined:
    Jun 6, 2013
    Posts:
    270
    i think it does, by jittering the camera's projection matrix. When TAA is off, my case (a cube and a quad) doesn't have z-fight. When TAA is on, it does.
     
  40. Argenzio

    Argenzio

    Joined:
    Apr 27, 2014
    Posts:
    183
    Totally agree. TAA cause z fighting
     
  41. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    963
    Well, I still totally disagree. And I stay on the view that it only reduces the visual impact of z fighting by blending. Shifting the projection matrix doesn't increase z fighting, because the relative distance within the near to far range stays the same. It's no different than moving the camera around a little.
     
  42. Argenzio

    Argenzio

    Joined:
    Apr 27, 2014
    Posts:
    183
    I have this issue placing a plane with Alpha texture near a wall. Like a decal. So i think z-fight is visible only when using trasparent texture.
     
  43. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    3,249
    There's a terminology issue here.

    Z fighting is a visual artifact caused by having two overlapping polygons flicker as a result of the limits of depth precision.

    TAA won't cause this, strictly speaking. But the view projection jitter can cause the intersection of opaque and transparent objects to flicker in a way reminiscent of z fighting.
     
    TTTTTa, brn, arnoob and 4 others like this.