Unity and Nvidia RTX Raytracing and AI on GPU?

DX12 was definitely unreliable in Unity 5.5, but it feels stable (at least in my experience) in Unity 2018.1. Which version of Unity are you seeing DX12 issue? What specific DX12 issues are you seeing?

 

Does not work for me in 2018.2.3, maybe it's single pass and VR that creates the problems though.

 

From a game quality and performance perspective, what benefits would RTX have, for instance, this article breaks down the performance trade-off of quality settings in PUBG (https://www.pcgamer.com/best-pubg-settings/).

Setting - Performance Impact - Low to Ultra
Effects - 25% (explosions and particles)
Shadows - 16%
Post-processing - 15%
Foliage - 2%
Motion Blur - 2%
Anti-aliasing - negligible

DLSS would be classed as Anti-aliasing so makes very little difference in this case.

Possibly the biggest impact for RTX would be in shadows and maybe post-processing for reflections, focus effects, god rays or ambient occlusion.

The thing is do we have a very high spec Unity game or Demo where we can benchmark the various quality settings that will be most impacted by RTX?

 

Book of the Dead

 

All of this speculation is meaningless for a few reasons:

1. We don't know anything about the RTX chart aside from the vague implication of "faster."
2. Because of the substantial changes to how the RTX cards work, we can't really use prior benchmarks as a frame of reference.
3. Many of the RTX tech gains haven't had relevant benchmarks shown at all.

 

As I understood RTX actually helps artists to get reatime vis and GI of the scene without baking or building lights.

For example, Unity dev said that Book of the dead demo is not build for day and night cycle. So RTX could help around this problem if integrated in Unity.
Too bad is not affordable card for small developers.

 

Is there any EXE of this yet? or is the youtube teaser the only thing that came out of all this hype?

 

https://assetstore.unity.com/packag...-projects/book-of-the-dead-environment-121175

 

Yes, I know there is the environment project. But I was expecting the actual full demo to go out one day.
It was supposed to be an actual interactive demo and the youtube video just a teaser.
It's been months from that announcement and the demo is nowhere to be found.

 

It's just a way of thinking about the potential of RTX as a way to displace GPU heavier tasks into raytracing hardware and therefore boosting the performance of games.

What will be interesting will be RTX benchmarks on existing games and then the first games that make use of RTX vs none RTX features.

I was hoping that we would have Unity Quality and performance benchmarks comparable to the PUBG breakdown of each Quality setting and its impact on performance.

 

I was hoping any Unity communication on the matter.

 

That is it. It might not be good enough for you but at least the media is licensed for use in your own games, and it demonstrates pretty much the same workload. What would be missing is an animated arm and the default character controller (which was also pretty crappy frankly, in the recorded demo).

In other news, Unity is hard at work on a brand new open character controller and animation system, so hopefully you could be satisfied by the right to use the media + wait a little bit for a really, really good character controller?



... all free of charge, of course.

 

I suppose you are talking about my comment on Book of the Dead. (correct me if I wrong)
I was simply saying Unity talked a lot about a super-hyped demo that never came out in the end and there is no official news on the matter. It would be as simple as saying: "it's not coming out".
Also, it's very nice to have all this high quality content free to use. Those things are always appreciated.
But sadly the shaders aren't Unity's standard, continuing the trend of making custom systems and shaders in demos for very specific scenarios instead of expanding the engine and integrating those developments in the core (managing compatibility and surface shader support for this)
This gives the false impression that standard Unity vegetation and terrains use this advanced technology, when they are in fact add-ons with limited compatibility, even if they are free.

That is indeed good news, even If I'm quite satisfied with Mecanim actually.
But please, don't interpret what I'm saying with being "unsatisfied" with Unity: I LOVE UNITY.
Unity is at the core of my work. I would hate to switch to another engine.
It's just that Unity (the company) seems to be progressively turning into Autodesk or Microsoft with all this corporate speech and tricks.

 

Ah but there is more to this. Book of the Dead is essentially a work in progress. The engine that powers it is HDRP, and that's very much work in progress. When the video was made there were no volumetrics, so they repurposed the blacksmith solution. Now there is a full volumetric system in HDRP, so that fixes some of the things they couldn't bring you properly.

The next problem is the terrain. The terrain in book of the dead is just a basic unity one and there's no special tech there in BOTD for it, so essentially that's WIP as well. As far as I guess the new terrain tooling comes in 2018.3 and 2019.1 but these versions are pure guesses on my part. I do know the terrain is coming along well so I can confirm it it's going to be cool.

Book of the Dead was really showcasing HDRP, nothing else. So everything else is just a bonus. When 2018.3 drops, I suggest grabbing HDRP from package manager or just using a HDRP template project then upgrading HDRP in package manager. This will let you have 3.x of HDRP and you can drop book of the dead assets into it and it should really get much closer to what you want.

Basically Unity felt they needed to show off HDRP there and then and a lot of that stuff was still being made at the time. The terrain stuff is the next thing that will complete the puzzle though.

I think the real problem here for you (sorry if I'm mistaken) is that you were promised something but the stuff that was promised is not arriving in the correct sequence.

 

I have rechecked the Unity roadmap and don't see any of this listed.
Maybe if there was a centralized place to know the state of development of each of those things globally (instead of having to check each github page of each single feature) it would be more simple for us to know were is this going.
Sincerely, I have lost track of all official github projects for Unity. It's like the company has externalized it's whole development repository and is expecting us to keep track.

The real problem is I'm a developer and don't like marketing speech by corporate suits.
HDRP is still in beta. It's been announced like it's done when it's not even cooked.

It's a bold enterprise, but it's still in development.
Not like a work in progress as "we will make it better over time" but more like "the workers are building the kitchen as you make your coffee". It's clearly stated as "not production ready" an prone to change completely like physically based shading was changing in it's beta stages.

My problem is NOT with it being late or still being in development.
I'm a developer, I understand development takes time. My problem is with the marketing stunt of announcing things that are still in development as if they were already done and implemented when it's clearly not the case.
The highlights of Unity 2018 were all features in development. They aren't even bundled with Unity 2018 and many must be even downloaded form github!

Those are not features of Unity 2018, those are the features of Unity 2019.5!
When Unity 4 launched, the highlighted feature was Mecanim. And it was DONE, and it was integrated and it worked out of the box. They exposed more internal bits to code as Unity progressed, but it was DONE.
When Unity 5 launched, the highlighted features were the new audio pipeline, physically based shaders, standard shaders and real time radiosity with Enlighten. And all of this was DONE, included and perfectly integrated with the engine even to the surface shader coding implementation.

Now we have plenty of official work in progress projects on GitHub that don't even seem to be done by the same company but by independent groups. And they announce this as if it was done and then you see the actual developers say it's NOT done.

I'm afraid this is feature creep on all it's glory and I suspect hiring an ex CEO of Electronic Arts has more to do with all of this than any of the developers at Unity.

 

Unity's roadmap is incomplete at best. If you must know the latest progress they're making on features you're far better off watching the latest keynote and roadmap videos they present at Unite and other conferences. They're both more in-depth and up-to-date with the actual progress being made. Unity's roadmap is neither of those.

 

Yes, I noticed that too. And I watch every keynote, but they seem more advertising than real information.
To the point that now, when they talk about a shiny new feature, I can't help but think: "yeah, maybe in a year and a half it will actually be ready".
It's more like watching one of those 90s TV shows about future technology.
I'm still waiting for my flying car.

 

Totally agree with all your sentiments but you know, I'm not bothered because while these are promises and so on, I find that my game takes up all my time before I can really be that worried about undelivered promise X.

I too, don't feel impressed by Unity's marketing and sales aggression, and frankly, I'll bet half of Unity's developers aren't impressed by it either because it does add a layer of stress that I try to mitigate with transparency like in this thread. But ultimately, I guess we should just ignore all that marketing and focus on the roadmap.

The roadmap is out of date, but that's the kind of out of date that doesn't result in the feeling of broken promises. It's "ok I have that to work with..." and it works.

I get why Unity is aggressive on sales and marketing, I mean all of this sweet stuff coming isn't cheap...

Still, HDRP isn't out. And that by default means any BOTD promise can't really be fulfilled and it's just early access.

Not helpful for the hardcore developer, but we are smart enough to see past that and work with what we have I guess. Unity could be a little clearer it seems, on the roadmap.

 

Gaijin Entertainment shows off it's Enlisted MMO running on GeForce RTX, using Vulkan to do a custom global illumination algorithm that's enhanced by Nvidia's hardware support for ray-tracing. It's important to point out that this is running with no precomputed lighting, which is part of why the non-ray-traced version looks so flat and bad. With ray-tracing, "pre-baking" the lighting and shadows is no longer required.

 

It's a nice video but obviously heavily biased toward demonstrating the GPU capabilities as there are already games out for years with similar graphical fidelity.

 

WOW. A moderator that doesn't censor me for not being a fanboy!
You have my respect now.

 

The thing is, it will be YEARS before realtime radiosity can compare to pre-baked radiosity like Enlighten.
There's a reason why baking lightmaps takes ages to compute, even on GPU-accelerated lightmappers.
I think realtime raytracing is the future, but right now you have better quality precomputing lighting into textures or using Enlighten for dynamic radiosity and GI.
The thing is, the raster rendering has evolved so much and is using so clever tricks that it will be better and faster than real raytracing.

An example: in the star wars demo, you could easily bake all the lights and use reflection probes, since you can't really see the stormtroopers reflecting on the captain armor in realtime. A raster version would be nearly identical and way faster without any RTX technology...

I have been thinking about upgrading my GTX1080 to a new RTX2080 just for programming the new raytracing processors. But I have dismissed the idea because by the time this technology will be stable and useful, a new card will have been released, making this first gen completely obsolete.

Also, we're indies here, we can't compete with corporations who can just phone Nvidia to make them change the drivers if they find something that doesn't work!

As much as I love the direction this is going, I think I will pass on this generation.

 

Can you remember this from a couple of years ago...

Imagination doing hardware accelerated ray tracing and I think Unity was talking about this technology if I'm not mistaken as a potential way to calculate GI or for high-end demos.

The thing is this was not being run on a high end gpu it was working on a mobile GPU...

 

Isn't DLSS an all-around performance boost for rasterization? Like 100%.

 

It is, but you "need" a new RTX to accelerate it. (I mean, until someone makes the same thing with regular compute shaders)

 

Well yes of course it takes RTX hardware. The question was what performance gains were to be found in supporting RTX, and at a base line it's a significant increase to performance if enabled (DLSS), in the way @Arowx was breaking things out, not a negligible percent like the contribution (or cost) of AA.

 

The RTX cards has tensor cores designed to do AI. DLSS is one possible use case of this. The effect seems to be good temporal stable AA that is running on these cores.

You would not get any speedup on the normal rendering than the time your AA would take to process. The benefit would of course depend on your current AA type from MSAA to TAA or super sampling AA that is much heavier.

 

The more I read about RTX, the more I think this is half-assed raytracing.
It reminds me of intel Optane sticks...
I mean if it's not FULL raytracing like Nvidia Optix + Deep Learning denoise, why even bother?
RTX is not ready for OptiX in realtime (they would have said it loud and clear), so not good enough to compete with raster right now.
(I would LOVE to be proved wrong though!)

 

No one said it is a full ray tracing. What they are promoting is a hybrid rasterization/raytracing system that for the first time could be done real-time.

You rasterize the scene as normal and then you apply realtime GI with the raytracing cores that will give you better lighting, shadows, automatic ambient occlusion and reflection.

The cards themself seems do to normal rendering a bit faster per core/Mhz than the last generation because of some stucture changes. It now does integer and floating point calculations in paralell.

For games not using any of the new raytracing cores this will give you better performace than on the 10xx series when GPU bound allowing for more games to run in 4k. Some estimates has been 50% better but benchmarks are not out yet.

when it comes to the realtime GI and shadows it will not accelerate your current games just make them look better and give some new capabilities. The normal cuda cores does not have to calculate shadows but they do have to process much more vertices. In order to do the proper GI and reflections 3d models in all directions of the camera has to be processed. A wall behind the camera can not reflect light if it is not processed by the vertex shader and in GPU memory in its correct location.

Using the new realtime GI will probably not be possible at 4k for the first generation but will give you a much better looking full HD game. Doing stuff you could never calculate real-time before and allowing for procedural and destructive scenes with good GI and shadows.

Lennart

 

I know.
What I'm saying is: this is not the raytracing revolution Nvidia would like us to believe. (just watch the video presentation from the CEO)
I'm as happy as anyone to have new and shiny GPU tech, but if it's not full raytracing like OptiX then it's just a gimmick right now.
Surely in 2 or 3 generations it will be awesome. But now it's just so-so.
It's the future, just not the present yet.
Again, I would LOVE to be wrong.

 

With any announcement or release, people have to unpick the wildest hyperbole in presentations from the technical reality. Both at GDC and with the recent GPU announcements, this has certainly been the case. For example some people totally missed the important detail that what has been unlocked here seems to be enough performance to make hybrid solutions a reality now, not helped by nvidias CEO making it sound like the era of 'faking it' is at an end, and that the era of realtime ray-tracing has finally arrived.

But just because too much hype has been employed, doesnt mean this stuff is just a gimmick. The specific effects they demonstrated at GDC and more recently are attainable. And thats still a real gain, and graphics developers are likely to do some other stuff with this tech once they have had more time to experiment with it. So, it really depends what qualifies as awesome to a person, and whether their expectations have been set way too high by misleading hype. Plenty of people that understand the actual capabilities are still excited about this stuff, and it is noteworthy in some ways that OptiX never was. Personally I am very excited about getting nice area lights with nice shadows, and having another solution for reflections.

 

Of course, those things are cool, but they are still just effects that can be turned on and off and won't have a dramatic impact on the game.
I'm a sucker for AO, but the fact is you can fake it very very well on screen space (try HBAO on the store or VAO ).
The same with smooth shadows: look at Next-Gen Soft Shadows on the store.
And this without needing specific nvidia hardware.
For me, it will be a game changer when we'll do full realtime raytracing, because it has the potential to change how we play. Until then, any evolution is welcomed, but it's still not THAT exciting.

 

A true revolution wouldn't be feasible because you would need to have hardware sufficient for an entire game and software already developed to take full advantage of it. That being said it's still very much a revolution because it opens the doors to techniques we previously couldn't handle due to performance constraints. It's definitely not just an evolution.

 

Ok, let's see how it goes.
I will be the first to celebrate this.

 

Full raytracing would be awesome, but the tech for doing that at an affordable price in a consumer product is simply not there yet. The RTX tech (hybrid raytracing) adds some impressive things for consumers now, and that is exciting.

Personally, the complaining about RTX reminds me of when consumers complained about the first DX11 cards. Most people just wanted faster DX9 cards, because their existing games were DX9. As such, DX11 hardware did not mean much for a lot of users other than the general speed improvements. Today, we all know now how much of a visual improvement DX11 offers over DX9. Micorosft DXR and Nividia RTX are getting the same initial reception and will probably be well liked later.

 

I don't really go for this RTX stuff if I'm honest, it's not something I think is a good step. It's one of those jump the gun moves with hardware that's been lurking in R&D a while and nvidia thinks there's mid-term money in it (there is).

Ultimately way stronger compute is the answer of course. But that'll be more expensive / longer term. Probably quite long indeed

 

Well, who knows! Maybe they are right and it'll give us new possibilities that we can't imagine right now.
I'm holding my money until then however.

 

To me, that's not how innovation happens. Nvidia is basically "the industry" here. If you are not moving forward with "the industry" you are being left behind by it. Even if the average consumer won't have the tech for a couple or few years, the savvy developer will be ready when they do. You don't want to be looking at an extended development cycle when it is already time to have the tech.

Then again, some developers are making good money off pixel art and 2D games, so not everyone has to be a leader.

You also said "For me, it will be a game changer when we'll do full realtime raytracing, because it has the potential to change how we play." which I think is incorrect as well. The tech demos explained some of how hybrid raytracing changes gameplay. Where and how objects and monsters can be hidden is the most obvious thing. And catching the subtle reflection or shadow of something behind the field of view in a realistic way. This is huge.

 

The savvy developer has to invest the money in what will generate a return of investment.
I'll let you lead the technological revolution.

 

NVIDIA's presentation was mostly marketing drivel. My understanding (after spending some time searching for actual information) is that the Tensor "cores" are basically just additional SIMD hardware to accelerate FP16 operations.

https://en.wikipedia.org/wiki/Volta_(microarchitecture)#Details
https://en.wikipedia.org/wiki/Multiply–accumulate_operation#Fused_multiply–add

 

I just want that nice raytracing tech so I can make stunningly lit anime hallway scenes.

You know the ones.

 

Yeah, like when the specialised Physx hardware was incorporated in some magical way. Generally, it's just more grunt and some driver stuff.

 

So this is all just Blast Processing? Got it!

 

I did this test few months ago when that demo was introduced - just to show that rasterization can give a similar fidelity. It's not something that was done in weeks, and my assets were pretty basic - but then, on the other side, mine demo was made in few hours.
https://olivr.info/2018/03/23/do-we-really-need-realtime-raytrace/

 

Video looks great.

The big advantage for the RTX approach is that you can do this on procedurally generated content or scenes that can be destroyed in gameplay. Where normal approaches would not work since there is no geometry to pre bake lightmaps from.

 

For those who don't know what's this Blast Processing thing: explanation.
To be fair, the genesis did have better all around capabilities than the SNES: every game was technically very different from the others while every SNES game used the same graphical effects over and over.
So Blast Processing was actually a real thing!

 

That's true if you are making a full procedural world like minecraft, but for destructible worlds you can use Unity's realtime GI, light probes and screen space ambient occlusion. It will do the trick just fine.

 

It uh... really won't.

 

That said even if you had a situation where you were able to bake lighting ahead of time being able to do it at runtime would result in a smaller download and potentially a smaller footprint on the device if you were willing to do it every time the game started.

 

And all this won't make better games. Gameplay died a long time ago, hastily buried under the parallax occlusion mapped patio.