Search Unity

Will dlss 2.0 soon be available?

Discussion in 'General Graphics' started by matt1101, Apr 6, 2020.

  1. matt1101

    matt1101

    Joined:
    Mar 10, 2017
    Posts:
    37
    Seems like it takes forever
     
  2. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    28,165
    is dlss 1.0 available?
     
  3. matt1101

    matt1101

    Joined:
    Mar 10, 2017
    Posts:
    37
    What I've seen dlss is not available. I saw a video where it was in Unreal Engine, but haven't seen it in Unity.
     
  4. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    666
    I think for Unity to skip on DLSS 2.0 would be a big mistake.

    1) It gets rid of TAA's ghosting and texture blurring issues
    2) It brings 4K gaming to a realm of possibility.

    But, given that PS5 and Xbox SX are AMD based, there is a big chance that Unity will wait for AMD's DLSS which probably will be hardware agnostic. This probably will make Unity's rendering far behind Unreal once again (Unreal already has DLSS) until AMD's counterpart is released. (I expect 1 year for AMD to release it, another year for Unity to implement)...

    Let's just hope that my predictions are wrong and Unity implements DLSS 2.0 right away.
     
    Ryiah and matt1101 like this.
  5. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    666
    I was just wondering if the HDRP team is looking into DLSS 2.0.

    It looks like DLSS 2.0 is superior to TAA in every possible way.

    1) DLSS2.0 gets rid of ghosting
    2) 4K gaming performance
    3) transparent material rendering is stabilized.

    Here is a video for the interested.
     
  6. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    666
    As far as I understand, DLSS 2.0 no longer needs training and is a general-purpose tech. DLSS prior to 2.0 (so, 1.9 and earlier) used to be like you said as it needed training on a massive scale which was not feasible for us.

    Here is nVidia's main website with more info:
    https://www.nvidia.com/en-us/geforce/news/nvidia-dlss-2-0-a-big-leap-in-ai-rendering/

    and here is the exact exert:
    One Network For All Games - The original DLSS required training the AI network for each new game. DLSS 2.0 trains using non-game-specific content, delivering a generalized network that works across games. This means faster game integrations, and ultimately more DLSS games.
     
    AwakeMeerkat likes this.
  7. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    28,165
    And I'll merge these threads. The order of merged posts are chronological.
     
    AlanMattano likes this.
  8. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    28,165
    Yeah, does it work on AMD? I would doubt it. But ms and sony are AMD.

    I'd like to suggest to Unity that they have a backend like XR, where each manufacturer is responsible for specific implementations. So far Oculus, Valve etc are all maintaining their own XR implementations in Unity. Unity just provides an abstracted interface for many possible features.

    If applied to SRP, that would give us DLSS 2.0. It's also why UE4 can offer it, source and all.
     
    AlanMattano likes this.
  9. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,223
    Only on Nvidia RTX gpu's, so even majority of Nvidia gpu's don't support it. It needs the tensor cores hardware to work which only exist on RTX lineup.

    I looked at the UE4 program and you need to apply to access it fully. Basically this means they hand out access to it per project basis atm.
     
  10. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    28,165
    It's what I expected. So you'd probably need a branch of an SRP to make it work, and this would likely just be HDRP. And you'd need to be applying for access. I expect this restriction will lift though.
     
  11. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    666
    So is raytracing, but we are moving forward with the expectation that it will be widely implemented. Although I am not too sure how AMD's tensor core counterpart will come out. I just hope Unity is ready to implement AI-enhanced upscaling as soon as it is available.
     
  12. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,223
    Well, there's a raytracing support on some non RTX gpu's from nvidia, altho they are too slow to be used still. Also AMD's next gen cards will have DXR support (but I doubt we'll see dlss compatible tensor cores from amd as afaik it's a proprietary nvidia implementation).
     
  13. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    28,165
    AMD's simply waited a couple of years not chasing nvidia so that they might use general purpose compute with a few tweaks to achieve DXR support. Just a matter of having fast enough memory and enough of it for it to make sense.

    Unique hardware in GPUs can be a burden or a blessing depending on the time, I guess. In any case I tend to prefer gpu compute tech because even mobile can nibble at it :)
     
  14. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    666
    while dlss is nVidia's AI enhanced upsampling isn't. My guess is Microsoft would probably include it in their DX12 soon, or at least I hope. Given that both AMD and nVidia are trying to achieve 4K, it seems inevitable as the performance/quality is quite outstanding. The hardware part, tensor cores as you mentioned worries me a bit though
     
  15. superjayman

    superjayman

    Joined:
    May 31, 2013
    Posts:
    185
    This is a GAME CHANGING FEATURE!... Unity when will you have support for DLSS 2.0? Also, don't forget about proper Raytracing support?
     
  16. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    17,314
    AMD has had a solution available for almost an entire year known as Radeon Image Sharpening.

    https://www.techspot.com/article/1873-radeon-image-sharpening-vs-nvidia-dlss/
     
  17. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    666
    Ryiah, the two are vastly different. While AMD wants people to see it as their counter part solution, CAS is just upscaling done at its best at most. A dynamic pixel filter to be more exact.

    DLSS is a reconstruction of image through deep learning accelerated by RTX hardware. There are obvious performance benefits but there are huge image benefits when it comes to transparency and other related images.

    The two are different as finite state machines are to AI neural networks. They are not comparable in anyway other than the fact that AMD wants it to look like it.

    Also, as many of you have noticed DLSS is improving over time as the nvidia training data is getting added with servers, whilst CAS is just there not improving. Maybe a bit of an improvement but not like DLSS. It is literally learning to be better every second. A little scary to be honest.

    People laughed at DLSS 1, and rightfully so, then DLSS 2 just surprised everyone and with the upcoming DLSS 3, it is going to replace TAA as nVidia plans it to work for all games with TAA. My guess is that anyone with an RTX card will choose DLSS over TAA as it will be widely implemented (DLSS3), faster than TAA and better looking (no ghosting) and also superior texture sharpness.

    IMHO, the big question at the moment is when AMD will finally respond to nVidia's AI related stuff. It is definitely a losing game over time as nVidia's implementation improves every second like earning interest. When AMD finally implement their own DLSS, it will take as much and probably more time than nVidia. So, yes, DLSS is very significant, and will increase its influence very quickly. AI enhanced imaging is the future and this won't change regardless. I just hope AMD will come out with their solution ASAP.
     
    Last edited: Jul 31, 2020
    MaxPirat and Rich_A like this.
  18. hookmanuk

    hookmanuk

    Joined:
    May 28, 2013
    Posts:
    15
    DLSS is now available to use in VR as well.

    This article explicitly calls out unity as missing any sort for DLSS, support in VR would be a massive help given the extra processing power needed for VR titles.

    https://uploadvr.com/nvidia-dlss-ai-upscaling/
     
  19. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    11,171
    I've avoided this thread for a while, but I want to add a bit side commentary to this whole discussion.

    Nvidia's DLSS 2.0 is amazing, and very different from AMD's Radeon Image Sharpening. The later is more comparable to Nvidia's Image Sharpening feature that's been available for years, though RIS is superior to Nvidia's implementation. However most games that use TAA already have something like AMD's Radeon Image Sharpening implemented to counter the blurring artifacts of TAA, including Unity.

    People here seem to be under the impression that it's Unity's decision to not include DLSS. DLSS 1.0 required per-game training, so it was never going to be an engine only feature. DLSS 2.0 was added to Unreal, but originally only as a non-public branch that you have to ask Nvidia permission to access. Nvidia's clearly been working closely with Epic on this for a while, and it's the only publicly available engine it's been implemented in. It's far more likely that Nvidia themselves have chosen to limit themselves to working with Epic on this for the early public version of DLSS. Plus Unreal is mainly a deferred renderer with TAA enabled by default. The vast majority of users will be setup already to make use of DLSS.

    In contrast Unity is still primarily a forward renderer, with deferred being optional, and TAA being yet another option that people may or may not use. It makes less sense for Nvidia to spend effort supporting Unity when most titles using it won't be in a position to take advantage of the tech.

    However, I do expect to see Unity's HDRP get support for DLSS, probably some time after DLSS 3.0 has matured.

    Note: I have no insider info on this. I am not a Unity employee, nor have I asked anyone at Unity this (not that they'd be able to say anything). It just doesn't seem fair to toss Unity under the bus here without any kind of information either way. On the other hand they do tend to shy away from vendor specific anything.
     
    Last edited: Sep 8, 2020
  20. G-Mika

    G-Mika

    Joined:
    Jul 17, 2012
    Posts:
    94
    The current process to get DLSS is still tricky. You have to first get the SDK which exclusive to a selected number of unreal engine users ou custom engine users. If you are selected, you can use the SDK but you will get a watermark. Once your game is ready, you can ask nvidia to remove the watermark.

    I think nvidia want to control/limit the usage of DLSS because it's still early tech and they don't want the tech to have a bad reputation. It's super tricky to access this tech unless you are a well known developer.

    I guess it won't be a widely accessible SDK until they release DLSS 3.0 or something. For sure Unity has discussed this tech, it's just not ready to be used by wide range of users.

    AI upscaling is the future, Microsoft + AMD are working on it for consoles, the tech will become widely available, probably not until 1 or 2 years still.
     
  21. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    666
    I am just saying that Unity is always late to the party and when eventually they arrive, they come half baked.
     
  22. Rich_A

    Rich_A

    Joined:
    Nov 22, 2016
    Posts:
    338
  23. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,223
    Camarent and PutridEx like this.
  24. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    28,165
    VR DLSS-likes for ALL!*


    *can dream
     
    AlanMattano likes this.
  25. PutridEx

    PutridEx

    Joined:
    Feb 3, 2021
    Posts:
    505
    In a different engine, a different branch & approval isn't required anymore, it's a plugin.
     
  26. elJoel

    elJoel

    Joined:
    Sep 7, 2016
    Posts:
    120
    Any plans to bring this to URP?
     
  27. Shizola

    Shizola

    Joined:
    Jun 29, 2014
    Posts:
    293
    I'm hoping we get a plugin sooner than later.
     
  28. Onigiri

    Onigiri

    Joined:
    Aug 10, 2014
    Posts:
    247
    They are working on HDRP integration right now
     
  29. N1warhead

    N1warhead

    Joined:
    Mar 12, 2014
    Posts:
    3,874
    No they aren't. They are 'considering' it. Not sure why it's much to consider, DLSS and FidelityFX are no-brainers at this point, even if you don't want to use Ray Tracing the performance benefit is just worth it.

    I realize this thread isn't super new (last responded in Feb), but I just wanted to make sure people who run into this thread like me are aware that it's only under consideration, not actually being made. Unless under consideration means they have prototypes and such made, no idea.

    upload_2021-4-1_6-33-56.png
     
  30. Gokcan

    Gokcan

    Joined:
    Aug 15, 2013
    Posts:
    266
    https://github.com/Unity-Technologies/Graphics/tree/HDRP/DLSSTmp

    I am sure they try somethings about dlss...
     
    Shizola and N1warhead like this.
  31. Onigiri

    Onigiri

    Joined:
    Aug 10, 2014
    Posts:
    247
  32. Rich_A

    Rich_A

    Joined:
    Nov 22, 2016
    Posts:
    338
  33. elJoel

    elJoel

    Joined:
    Sep 7, 2016
    Posts:
    120
    Step 1: Tell people URP is the way to go for good performance and VR
    Step 2: Release DLSS only for HDRP
     
    DebugLogError and Rich_A like this.
  34. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    11,171
    URP = Great performance for VR.
    HDRP + DLSS = Still not as good performance as URP, but makes HDRP VR plausible.
     
  35. kecho_garcia

    kecho_garcia

    Unity Technologies

    Joined:
    May 5, 2021
    Posts:
    3
  36. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,223
    Been following that PR for a while, looking good :) Any estimates how long till we get access to the nvidia module so we can try it out? I'm assuming that will ship with future 2021.2 engine versions at some point.

    Also, is URP support totally off the table? Like others mentioned before here it would se really cool to be able to utilize DLSS in URP VR. HDRP still has quite much overhead for VR usage even with the help of DLSS.
     
    kecho_garcia likes this.
  37. kecho_garcia

    kecho_garcia

    Unity Technologies

    Joined:
    May 5, 2021
    Posts:
    3
    It is already in the trunk! so whenever that version gets promoted to the next release, it is out and about!
    Some extra testing in your projects would be absolutely lovely.

    Purely technically speaking, in order to support URP there is quite a bit of work, it will require:
    * Proper velocity pass
    * A more defined post process chain
    * Support using the DRS system for hardware / software modes
    * More intense detection code of some of the passes above
    * Maybe more things I haven't figured out yet?

    I dont have information of when or how this work would be done, since I am just an engineer. Hope this answer some of your questions! Feel free to ask anything technical :)
     
  38. Rich_A

    Rich_A

    Joined:
    Nov 22, 2016
    Posts:
    338
    Do you think that modifying URP to support these features and thus DLSS would actually decrease performance on other platforms? (or for people on PC not using DLSS)
     
    GliderGuy likes this.
  39. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,223
    There is actually work being done for URP to support movecs: https://github.com/Unity-Technologies/Graphics/pull/4408 but I have no idea how the rest of the mentioned things go there.

    It did, thanks for the quick response, much appreciated :)
     
    GliderGuy likes this.
  40. kecho_garcia

    kecho_garcia

    Unity Technologies

    Joined:
    May 5, 2021
    Posts:
    3
    Excellent question! The answer is no. We make absolutely sure that if we add a platform specific feature, the other platforms stay neutral. I cant think of any technical reason to cause this with DLSS so I am say 95% confident it will be fine for other platforms.
    It's an excellent question because as you know, sometimes rendering features do cause an overall expense in other platforms!
     
    GliderGuy and Rich_A like this.
  41. m0nsky

    m0nsky

    Joined:
    Dec 9, 2015
    Posts:
    153
    Hi @kecho_garcia ,

    I did some playing around with DLSS (2021.2.0a17 + a fresh master clone) today.
    I tested my own ray tracing heavy scene where I lose around 5% performance with DLSS enabled. I also tried the SpaceshipDemo project mentioned in the PR, where I lose around 30% performance with DLSS enabled.

    The nvidia plugin is enabled in the package manager, I've enabled DLSS in my HDRP asset and default volume profile with the following settings.



    Both DX11 & DX12, Windows 20H2 + RTX 3080, tested on both NVIDIA driver 466.11 & 466.27, in both editor & standalone.

    Is this a known "issue"? If not, do you have a plug & play project to find out if this is a hardware/driver issue?

    Edit
    Found the issue, my post processing presets includes physically based DOF which doesn't seem to play nicely with DLSS right now.

    I've created a boilerplate project and posted some performance metrics over here
     
    Last edited: May 14, 2021
    mgear likes this.
  42. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    666
    Wow, I am surprised that Unity actually pulled through...bunny hopping on my workstation right now.

    This rocks.
     
  43. petersx

    petersx

    Joined:
    Mar 5, 2015
    Posts:
    239
    Hi,
    Is there is a special trick to enable DLSS on 2021.2.0a17 version ? . I enabled NVIDIA in package manager, but I don't see a DLSS options nowhere ?
    Any advice ?
     
    olix4242 likes this.
  44. m0nsky

    m0nsky

    Joined:
    Dec 9, 2015
    Posts:
    153
    I am currently creating a plug & play project which has DLSS enabled for anyone wanting to try it out. I will edit this post with a link to the github page when I'm done.

    Edit
    I've just posted the project over here
     
    Last edited: May 14, 2021
    olix4242 likes this.
  45. petersx

    petersx

    Joined:
    Mar 5, 2015
    Posts:
    239
    Thanks, in your project DLSS is working :))
     
    m0nsky likes this.
  46. petersx

    petersx

    Joined:
    Mar 5, 2015
    Posts:
    239
    In your scene I have above 100fps with DLSS (QuadHD)
    I dont' know why, but in my scene I have 7fps with and without DLSS and sometimes with DLLS enabled it drops to 5fps occasionally (QuadHD)
    tested od RTX 2070 Super, i7-9700F, 64GB RAM
     
    Last edited: May 14, 2021
  47. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    848
    Same question, except I dont even see NVIDIA in package manager. How do I get it?
    (I'm on Unity 2021.2.0a20)
     
  48. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    354
    its a built in package (found under built in in package manager)
     
  49. Cascho01

    Cascho01

    Joined:
    Mar 19, 2010
    Posts:
    1,273
    Is it possible to enable/disable dlss at runtime?

    Code (CSharp):
    1.     public void SetDLSS(bool enable)
    2.     {
    3.         Camera.main.allowDynamicResolution = enable; // Has no effect!?
    4.     }
    5.  
     
  50. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    11,171
    That's not related to DLSS at all. That's Unity's dynamic resolution system they implemented before DLSS even existed, and which only works on some platforms. It doesn't actually do anything on it's own when enabled and has to be manually adjusted from script even on platforms it does work on. Specifically you need to have access to the accurate GPU frame timings, which Unity does not expose to c# on Windows making it mostly useless.
    https://docs.unity3d.com/Manual/DynamicResolution.html
     
unityunity