Unity and Nvidia RTX Raytracing and AI on GPU?

Will Unity be able to take advantage of the power of Nvidia's new RTX GPUs e.g. GPU Raytracing and ML AI processing?

I know Unity is platform agnostic and they did not adopt the Nvidia Physx hardware boosting APIs.

How much faster will games running on game engines that take advantage of this technology be?

Article on Nvidia RTX with demo videos - https://www.rockpapershotgun.com/2018/08/20/nvidia-turing-rtx-2080-specs-price-release-date/

 

I would imagine them to be slower but higher quality, or the same with marginal quality increases, depending on how the technology is used.

I think perhaps one possibility would be you would replace SSR with this, or if you wanted to use some fast denoising, maybe that would be a thing?

Really, people need a few years R&D tinkering with this to get the best from it, and by then the market will exist for it.

 

The consumer card was literally just announced today, so it's too early to tell

I'm not sure why you think anyone has an answer to this entirely impossible to predict outcome/

 

Well in an industry where high end games can take 2-3 years to make choosing which engine based on it's technology stack is a big factor for studios.

And having the best looking graphics and highest performance is a big wow factor.

 

You seem to believe there are games that take advantage of the very latest and greatest technology available. Last time we truly had a game like that was Crysis and to my knowledge there are no shipped titles that come close to the same heights that game aspired to. No, I don't count vaporware.

If the industry had a habit of building games around GTX 1080+ hardware it would be a different story but they don't and every indication is that only the top tier consumer cards will have dedicated raytracing hardware. If a company wants to make use of it they would either have to make it optional or they would have to lose the vast majority of their customers.

 

Overwatch doesn't particularly mind what the hardware is, running at 60fps on a toaster. Looks perfect according to its millions of currently active paying gamers.

 

By the way I'm not aware of any non-proprietary game engine that uses hardware acceleration for PhysX. You mentioned Unity but Unreal is another one that doesn't and CryEngine has its own physics solution. Lumberyard being a derivative of CryEngine will have its own solution too. Basically it's one of those things that never caught on in a big way.

 

There were AAA titles shown today during the Nvidia RTX presentation that had the option to turn hybrid ray tracing on or off. So things like shadows, lights, and reflections could look vastly better with "RTX ON" than with "RTX OFF". It looks like those AAA studios already had enough R&D time to integrate some of the new features. At a minimum, I am curious when Unity will add the option to enable or disable RTX (hybrid raytracing). I have to imagine that Nvidia already made this tech available for Unity to integrate, since Nvidia showed demos of RTX with other major engines.

 

It wouldn't surprise me if some of the engine developers at these companies built their own raytracer in their spare time.

 

Don't expect a lot of direct ML AI processing magic. What I expect to see is Unity promptly add RTX-ON/RTX-OFF options to the Unity API for enabling and disabling the new hybrid raytracing features for shadows, lights, and reflections.

 

Sure. I expect a lot of devs have. Raytracing itself is not hard. Doing any amount of raytracing in real time is the impressive part.

 

Or if it could be harnessed for lightmap baking in near real time!

 

Just finished digging around to see what developers have been working with up till this point. Microsoft has had a fallback layer available since they announced DirectX Raytracing. Complete benchmarks are in the link below but basically using it just for reflection the performance was 174 million rays per second for a GTX 1080.

I'd love to see how the performance of the new dedicated cards stacks up to that.

http://boostclock.com/show/000198/DXR-fallback-layer-perf-preview.html
https://github.com/Microsoft/DirectX-Graphics-Samples/tree/master/Libraries/D3D12RaytracingFallback

We already know AMD made their solution available to Unity. Back in March it was achieving 200 million rays per second.

https://blogs.unity3d.com/2018/03/2...ated-into-unitys-gpu-progressive-lightmapper/

 

But Unreal have had this for a long time already
https://www.unrealengine.com/en-US/...microsoft-s-directx-raytracing-and-nvidia-rtx

 

I have to imagine that was the fallback but it could have been engineering samples.

 

I am also very interested in this. I am currently considering buying a new Card, and if unity can use real time raytracing, I would definitely consider the new RTX 2080TI.

I'd love to hear something official on this, I don't remember seeing anything in the roadmap about raytracing, but maybe the scriptable render pipeline could use it?

Also, if real time raytracing was available in Unity, would it go some way to bridge the gap between "rendered" scenes in, say, Daz3D and Substance Painter, and the In-game graphics in Unity? I find that it is very difficult to get similar quality results in unity to the ones that I get with ray-traced renderers.

 

This is only for higher end cards, what dev studio aims for higher end cards only?? None.

 

I, too, don't think these cards will get cheaper and more common as time goes on. /s

 

There is a difference between aiming for them and supporting them. It isn't like the companies aren't gaining something valuable by supporting raytracing for a very minor feature. NVIDIA will basically be advertising their games for them. It's important to remember that there are a lot of gamers eager for new cards that will be watching those conferences.

 

Right, but something that is only available on high end cards today might be mainstream in 2-3 years. Given the dev cycle of a lot of games, it might be worth adding support for these features now.

 

Waiting for the first game created entirely out of mirrors...

 

Would be nice if it can be used to accalerate the upcoming unity gpu lightmapper

 

Still can't afford one.

I'm not really moved by this. Fully appreciate it. Like it, will use it for replacing SSR and better shadows, or better GI. It wouldn't greatly improve the quality of the visuals in my game if I'm honest and this generation simply isn't going to create enough of an improvement to remotely justify the cost. Hopefully it will be cheaper over time.

Of course, going to appreciate it if HDRP team add a button for it, and I've been looking forward to an announcement like this for a while, and it's come in a full 5 years sooner than I expected.

Unreal Engine has support for it
on the way, confirmed.

(Unity is glaring at me, let me hide)​

 

Well they partnered with nvidia to do a lot of the early demos, but this stuff wont be available for users of Unreal for some months yet (unless my knowledge is already out of date, eg I havent had a chance to look at Mondays announcement yet).

Unity were on the partners slide at the presentation nvidia did last week. So its reasonable to expect them to add this stuff eventually.

Best not to think of it as one thing though, in both these engines we might get certain raytracing-based effects before others.

 

I dont think in terms of promptly at all, not least because nvidia (and microsoft DX12 raytracing API) mostly provide the underlying capability and what Unity need to do goes well beyond providing an option to simply switch RTX on and off. Its not built like that and does not work like that at all, at least as far as I know. It requires much graphics engineering work by engine developers, so I expect it to take some time. I will be happy to be proved wrong about this if I have missed something.

 

Its the sort of headline feature that they are likely to make the most of via a snazzy official announcement at some point, rather than just popping it on the roadmap.

But there are clues in at least one area. Work done by Eric of Unitys research department into a raytracing-based approach to nice area light shadows for example.

I did start to go on about that in the scriptable render pipeline thread back in June, eg if you check out the following post and a few of the subsequent replies (and note that one of the Unity responses mentions DXR, with DXR being DirectX raytracing)....

#721

They arent confirming anything there but this stuff is all rather relevant anyway. And I was going on about 'it might take a long time' because back then we didnt know what sort of hardware would be necessary to really start doing this stuff (eg RTX cards were not announced back then and earlier GDC demo spoke of using multiple cards to get the demos working at realtime rates).

 

There were demos shown today using both UE4 and Frostbite engines where they could enable and disable RTX to show the difference.

 

https://blogs.msdn.microsoft.com/directx/2018/03/19/announcing-microsoft-directx-raytracing/
According to this, Unity is on the list of engines planning to support the DirectX Raytracing API.

 

Remember that Octane has an integration with Unity and version 4, which will be deployed to the masses with a fully-featured free license later this year, takes full advantage of DXR and RTX technologies.

 

That's going to be LIT.

 

Before or after the Founder's Edition tax?

https://old.reddit.com/r/nvidia/comments/98xzfb/geforce_rtx_information_megathread/

 

Unity RTX Confirmed.

 

Would like they make enlighten fast as GPU lightmapper than DirectX Raytracing API.

As it is, new technology probably won't be for production till 2020.

 

For people saying it's new technology and won't be ready for years, there are literally games coming this year with it implemented. Battlefield V, Metro and Shadow of the Tomb Raider are all coming with this technology enabled, along with 20+ games planning support including Final Fantasy XV. The tech is here, ready to use, that's why people are asking if/when Unity will implement it as UE4 will have it implemented before the end of the year if I'm not mistaken

 

Nested prefabs was confirmed back in 2012

 

Yes of course once Unity have written various clever and complex graphics things that use raytracing it will be possible to switch them off, just like currently we can switch various post-processing effects on and off. But that doesnt mean that all Unity have to do is provide an RTX on/offf switch. One of your posts makes it sound like the effects that use raytracing are built into the card and all Unity need to do is provide a switch, and this is not the case at all.

We dont know how much development work Unity might already have done on implementing some of this stuff, so its always possible something may arrive quicker than I imagine, but the idea that Unity can just promptly add RTX support without a lot of work is completely false.

 

Have nvidia said anything about stereoscopic support? Raytracing is done through casting a ray from the camera towards each individal pixel (Screen to world mapping) and trace the path back to the lightsource. I hope they can reuse that information for the second camera otherwise it means 100% overhead for the second camerra

 

Is Octane Real time though? Or will it be? They say that real time support is coming, but they never mentioned when.

 

Even something to improve baked lighting would be welcome...

 

PL is already in the works for improved baked lighting. But I think Unity needs faster GI computation than anything.

For example, look how fast is Godot's GI computation. Enlighten does look better, but if you need to pre-vis fast Godot's GI is much more economic.

 

Supposedly it can be used at runtime. In the announcement of their integration into UE4 last week they mention real time path tracing games as one of its features.

We'll find out soon enough.

 

Have you followed these real time GI steps?

https://unity3d.com/learn/tutorials/topics/graphics/introduction-precomputed-realtime-gi

 

Is something changed in Unity 2018? I don't see this additional options for GI

Compare to:

 

Unity's process for adding RTX will be VERY very hard.

Spoiler: NDA INFORMATION

 

I see what you mean. Yeah, Unity would need to do some work to integrate RTX support. What I am saying is they should promptly do that, and then add a simple RTX toggle in the API for game developers to use to enable or disable the RTX feature. I am not trying to suggest that it would be easy or trivial for Unity to do that, though.

 

There is probably some portion of the raytracing workflow that could be re-used for multiple cameras, but I would imagine that each camera would need to process its own rays in order to look right. Since the RTX stuff is hybrid raytracing (instead of full raytracing), it is likely there are VR centric optimizations available.

 

Yeah, maybe. The brain is really sensitive to differences between the two cameras, even floating point precision can be a problem

 

The VR optimizations will probably just come from the built in forveated rendering support and next gen VR headsets with eye tracking