Jove 2.0: DX11 Rendering System (Alpha Release)

If I use 1 cascade of the 4 in Unity's default renderer and make the distance really low (for that cascade), I do indeed get z-fighting issues.

The solution, is to render a separate shadow map just for the first person models. Games like CS:GO do this if you look closely. However, in Unity I need to use 2 cameras because there's no way to use a light or shadow map specifically for that purpose.

Another thing to consider is the fact that using a cascade just for the first person models would be a waste of performance. Every other mesh would have a lot of unnecessary instructions while we know for sure that cascade will never be selected for them.

 

Do you mean a 16 bit GI map? Ill be happy to do it, although i'm on my laptop so I dont know the amount of time such a thing would take to calculate. Mainly doing this so I can help my dad with his work (he's an architect) and having a realtime visualisation of his work would give him quite the upper hand in his work.

 

It should take exactly the same time to calculate, since the only difference with RGBM HDR is the output image type and therefore a way you cram the high range of calculated values into it in the end.

@Aieth, this is not directly related to Jove, but I figured you're interested in papers on subjects like deferred approaches to rendering in general, so I guess this one might be interesting to you: http://jcgt.org/published/0002/02/04/

 

When you say z-fighting, do you mean z-fighting as in the shadows are lacking precision or the actual geometry rendering is lacking precision? If you turn off shadows all together, do you still get z-fighting issues or is that just for the shadows? If its just the shadows, it should work without modification in Jove right now, shadows are handled differently than in standard Unity.

I was mostly wondering if Enlighten in Unity can be toggled to use 16 bit textures instead of 8. With the HDR going on in new Jove RGBM just doesn't cut it. I want to look into integrating Enlighten into Jove but I have not gotten to it yet. I have not spent more than two minutes looking at it, so not sure what it can do (the Unity integration that is, I am familiar with Enlighten as a standalone product).

I'm always interested in such things. This particular one seems less applicable to desktops and more applicable to mobile GPUs, though that might change with the arrival of more low level DX12/glnext and the reduction in draw call cost they bring with them.

 

Hmm, hopefully it doesnt become problematic when the time come. Ill be doing only 1 room so I dont know if that will solve anything.
Maybe a future thing to do (like after all the other stuff hehe)
If you think that there will be a lack of quality with the GI ill be happy to just move to my next project on the list (asset store items)

 

I can't really say before I have a chance to take a look at Enlighten I've been postponing it for a while to let it mature, but given that Unity are pushing RC's now I'm gonna take a look at it.


Anyway, I did say that the SSR was almost finished before. That turned out to be a lie. I got my hands on a better test scene, an archviz flat (thanks @lazygunn). That scene exposed a few "uh oh" artifacts. So after a few more rounds of ironing kinks there have been a few changes. Mostly technical, but the result is that its a lot more stable with less flickering now than before. I've also exposed a few tradeoff variables to the users (quality vs nonflickering etc), to allow users to tweak to their needs.
The most striking visual effect is that reflection edges are no longer as harsh as before, semi rough surfaces allow reflections to blur into their surroundings. Two images below, with SSR on and off.

 

Are you properly re-normalizing after failed SSR samples? There seems to be a dark hallow around stuff.... if that miraculously technical description helps at all.

 

Not sure if I understand I don't see any dark halos. What you might be seeing is that the cubemap reflections of the sky are a lot brighter than the SSR of the cubes, and its the transition you are talking about? If thats the case, the SSR is actually the more accurate of the two, as the sky reflections do not have any kind of occlusion.

 

It's amazing how subtle yet more immersive the SSR shot is by comparison. It gives more depth and weight to the shot. I'm sure that feeling is even more pronounced if moving through that scene in real time.

Looking forward to the update and Unity 5 compatibility.

 

Looking great! Can you tell us something about release date estimates too?

 

Yeah it really helps with contacts between objects. After using it I can't imagine not using it.

Well I'm in the wrapup phase. But I'm wrapping up so many things... I'm hoping to start beta testing the new patch soon at least


I need to rework how cubemap mip sampling works. The SSR is now far superior to the cubemaps and they don't look good next to each other. The below picture is SSR, no cubemap.

 

Damn, that reflection looks gorgeous. And to think that we finally get an SSR that's not just a looker but is properly integrated into the pipeline and performant, yum.

 

That is looking great, good work! On reflection (hah) the scene I gave you could be better prepared but i suppose im glad it helped troubleshoot!

It's taking a ridiculous amount of time but i'm preparing things for a nice long art drive, that i can integrate into several of my projects, so hopefully you'll have something from that to try out

 

In your with/without SSR the ground to the left of the cube in the indent, off in the distance, seems darker with SSR. Though now that I look at it that might be a downsampling artifact. You've seen in it motion though so I'll trust your judgment. Speaking of which, when will we get to see some video footage?

 

My only question is that purple wall to the left is pinkish in the reflection on the floor. Is that supposed to be that way?

 

The floor is not colorless, so yeah. The reflection will probably become less colored as you change view direction to observe the floor at more grazing angle, I would guess - albedo contribution to the color would then drop due to fresnel, with reflection shifting to more mirror-like appearance.

 

The reflections are not traced using the reflection ray, I am instead importance sampling a blinn-phong distribution. This basically means that rough surfaces can cover a whole lot more area, so its possible that what you are seeing is the surface reflecting backwards, since its rough. Also with really strong normal maps there is nothing to prevent a reflection ray from actually going into the surface that spawned it. Sounds worse than it is, its the price we pay for using normal maps instead of actual geometry (damn you limitations).

Yeah its colored by the copper floor. If you look closely you'll see the the blue is more greenish, and the yellow turned orange in the reflection Using a white floor it looks exactly like the geometry being reflected.

 

I love you Aieth. Looks amazing!

 

Very cool! It will be great to see how the SSR behaves on higher polygon count meshes (50K+) Like scans, cars or even used on interior scenes. Also how well it mixes with baked GI.

..also hopefully rendering correctly in stereo on the DK2

This looks very promising Aieth!

 

I'm looking to use Jove because it looks like the best rendering Unity can currently do. I do have a few questions.

1. How good is the anti aliasing solution, have you implemented anything or will you be implementing anything down the roadmap? One problem we are greatly experiencing which has been making us stick to using forward rendering purely for the MSAA are the aliasing problems on our ship rigging. The ships move so the aliasing visually becomes worse. Here is a comparison which you can flick between:




2. As I said we have ships that are pretty much levels, constantly moving as dynamic objects. Would Jove benefit dynamic type games or is it mostly for static environments?

 

The quality of the SSR is independent of the scene complexity. It does however suffer from the same drawbacks as all other screen space methods (works great in an empty room but there is missing data if you place objects in it). I'm also excited to see how well it blends with Enlighten. I hope Enlighten is more flexible than the Beast integration was, but I've gotten the impression that it is So I am hopeful.

I'm afraid to say that there is no anti aliasing solution as of yet I am looking to implement temporal AA, and a lot of the leg work has already been done as I have implemented temporal SSR. I am using something similar to Unreal Engine, based more on signal processing than typical rendering techniques. So it is coming at least, most probably in the patch after the one that I am working on finalizing now. Ship rigging really is a worst case scenario for aliasing isn't it ? If you want to see an example of how it is likely to look when finished, theres a few pictures in this blog post on Assassins Creeds temporal AA http://bartwronski.com/2014/03/15/temporal-supersampling-and-antialiasing/

As for dynamic environments, that is probably what Jove does best at the moment. I am going to implement support for lightmap style lighting, but currently the only ambient mode is IBL. For your case that should be ideal though, it is hard to bake ships moving around on an ocean The quality is significantly improving in the next patch, with several updates to how ambient lighting is handled.

 

WOW! My jaw dropped, you've gotten me seriously excited for Jove now! Especially since you specifically showed a rigging example.

If it happens that is seriously the best thing ever for our game and would greatly improve so many things. I'm still reading through all the posts in this thread, I'm a little concerned on the compatibility issues with some of our things.

1. Do you have a guesstimate month when you would implementing temporal AA?
2. An ocean ike suimono or the community shader I suppose wouldn't work with Jove. We would have to wait for one to be written for Jove?
3. Would you say that Jove is still and always will be miles ahead of Unity 5 rendering? Why hasn't Unity hired you yet

 

It should be easy to rewrite them, referencing Jove shaders. Basically, you just need to make them output GBuffer variables (rougness, metalness, occlusion and so on) instead of standard Unity outputs. If it's a forward shader with custom lighting, there are ways of adapting them too.

I'd say it is, and the reason for that is targeting DX11 and mid to high end hardware. Unity absolutely must play jack of all trades role in providing it's features, so it's extremely unlikely to focus on stuff like state of art deferred rendering anytime soon as such work won't benefit a big chunk of their customers.

 

1. I really don't like doing guesstimates as they never end up correct, but I am hoping to push patch 2.3 out in April. Take that with a huge grain of salt though.

2. If you are familiar with shaders, it should be trivial to replace the lighting model used in a water shader with the lighting model in Jove. Take a look at the shader code below, you could take that and insert whatever you want into the diffuseColor and specularColor component of the input struct, fill in the normal and the world position and then it just works. It is also possible to add in more complex behavior (the glass shaders do this, using refractive direct lights and ambient), but for an ocean shader that shouldn't be necessary.



3. You are gonna have to ask them about the employment And while you are at it, bug them for texture array support. As bac9-flcl said, Jove isn't better because I am better than Unitys programmers, we're targeting different hardware ranges and we have different factors to take into consideration. Unity is a jack of all trades, which is its strength but also its weakness.

 

Ahh adapting shaders is much easier than i expected, very useful!

 

Apologies in advance for this, but I'd like to make a general appeal regarding releases, previews etc of Jove 2.0.

Given that your existing customers at this stage were more than happy to explore the bleeding edge with Jove 2.0, please don't be afraid to release stuff earlier than your instincts might thats in a state that is not really 'finished' or fully useable in real projects just yet.

I mean I don't want to bully you, unfairly pressurise you or anything, but personally I love having a proper glimpse of stuff for myself, being able to play with stuff, even if it isn't right yet. I don't know if the way you've redeveloped Jove 2.0 offers any of those points along the way, but I'd certainly feel a lot more satisfied with my early purchase if there was something new to play with at some point in the next 4 or 5 weeks.

Anyway thats just my own selfish example timescale, and I won't get all angry if you completely ignore this appeal! The 'more finished' release that you already plan to happen is obviously what matters most after all, but I get these cravings for stuff to play with, especially when I know so much has changed from the version we currently have available to us.

Cheers.

 

@Aieth - For someone who has never really gotten into shader writing, are you going to have some sort of tutorial or step by step for shader adaptation to Jove...

I see by the responses of some that it's "simple" or "trivial" but please remember, "one man's ceiling is another man's floor".

 

I'm hoping ShaderForge will eventually support Jove, once Jove is in a more final state. It's a lot more intuitive for us non-coders than sifting through shader logic.

 

Thanks for the reply Aieth. If you ever need a 3D artist alongside bac9 to make some sort of scene for a Jove release trailer I'd love to help! I really think Jove could hit the Unity scene hard with some proper marketing.

 

It better be, I spent a week working out the new structure and converting everything to it I'm really happy with it now.

I agree with you completely, and you are not being selfish This patch has taken a while, I was exploring ambient lighting oct-nov and then had almost no time to work on Jove until middle of January. Things are looking up now and I've been working more than full time for a month now. I'm finalizing patch 2.2 now, polishing up stuff and I'm planning to go into a beta version really soon, which as soon as it passes the community stress test will get uploaded to the asset store. I'm gonna be collecting names of people interested in partaking in the beta, can I assume that I am to add your name to that list?

Yeah trivial is really relative. Let me rephrase that then, if you can code shaders adapting to Jove lighting is trivial. If you've never seen shader code before this is your Everest. I could do some shader programming tutorials later on for the community at large, but there's really no shortcut to fixing a shader for Jove. You either know shaders or you don't, I'm afraid I'm always happy to answer questions though, especially if its from someones who is working with Jove.

I am hoping for something similar as well. It is something I will investigate in the future when Jove reaches a 1.0 status.

I've intentionally been skimping on marketing until I reach a more complete state. I could've held back Jove another year before doing a full launch, but I really needed user input to help take it further I always need assets and especially full scenes. They don't have to be big or complex, but having a diverse set of test objects really helps with exposing flaws and/or potential improvements. I thought I was done with the SSR a week ago until I ran it on an archviz scene where it looked horrible



I'm officially calling the SSR implementation done. So that's it for new developments for patch 2.2. All that remains is cleaning up, fixing and more fixing. That's probably gonna take a while, but a beta version isn't far off now. Anyone interested in partaking in a beta when it becomes available please send me your in-voice number in a PM and I'll add you to the beta participation list. I'm hoping sometime next week!

EDIT: Almost forgot. I did some profiling to find out how expensive it ended up being. I can't really profile it properly, because when I go full screen the profiler goes blank (no idea why) and manually timing it won't work because it takes less than 1ms. So what I can say is this, on a GTX 560 ti the timings are:
0.5ms for 1200x900
<1ms for 1920x1200
Using the default settings (the same settings as in the screenshots)

 

Well, you know to put me down. I'll finish modelling/setting out my current test scene to the point where my current premade objects are all used, then get busy with uvws and dDo and then you should have some stuff to mess with

 

Add me too for the beta hehe

 

Thanks, and yes please Thats assuming its supposed to work with whatever Unity 5 RC is out at the time, as I too am no longer using Unity 4.

 

You know I'm all for it, don't you

And daym, those performance numbers for SSR. That's a huge step up from certain other solutions. Can you go into detail about it's implementation at some point? How you compensate for various artifacts, how it's using the resources, where in the pipeline it's happening and such. Would be an interesting read for sure.

 

I don't own a Unity 4 license anymore, so everything hence forth is going to run on either both Unity 4 and 5 or just 5 (sorry to say that, but everything that has to do with lightmaps has changed so much from 4 to 5 that it becomes quite hard to support both versions at once. That and Unity 5 finally supports floating point CPU textures).

 

Apologies if this has been answered, but how does performance stack up vs Unity5 equivalent shaders? We use linear/deferred. Is it working with consoles?

 

Your first question is trickier to answer than one might think I'll give it my best shot. Deferred in Unity and Jove are not as similar as they might seem at first glance.

The design behind Jove is based around two main ideas.
1. Lighting can be evaluated at any time at any point in the view frustum.
2. Lighting can be evaluated in a single pass.

It might seem simple (and perhaps like obvious choices) but this places demands on everything inside the pipeline. To accomplish this I'm using clustered shading. Which isn't really a shading technique, it's more of a "light lookup" data structure. Into this data structure I feed all the lights in the scene and all the reflections probes. For those of you familiar with tiled shading, the difference between that and clustered is that clustered also culls in the z direction, meaning its basically a 3d volume bent to fit the view frustum.

This is then uploaded to the GPU and then any shader, deferred, forward, particle or even the volumetric scattering, can access any lights. This also ensures design idea 2 is fulfilled. The advantage of this is that you reduce bandwidth requirements. I'm sure many have heard that bandwidth is the number one bottleneck today, and minimizing it goes a long way towards good performance.

For example, lets assume a 16 byte GBuffer (which is a fairly small GBuffer by todays standards, I think Unity is 20 bytes? Don't quite me on that though). Now, say that you have a directional light and a point light which covers half the on screen pixels (I know that you know this hippocoder, I'm being meticulous so that the less technical understand as well ). When calculating the lighting, the shader needs to access the GBuffer to get the pixel properties (color, normal, position etc). This means that per lit pixel the bandwidth cost ends up being 16 * affected pixels. Then the result has to be written to the lighting buffer, which is usually a HDR texture at 8 bytes. So thats another 8 bytes per pixel. So per light per pixel you pay 24 bytes. In Full HD this is 1920*1080*24 = 47mb. Say we're targeting 60fps, that is 4.8GB per second.

So for each full screen pass we pay 4.8GB. By eliminating light passes, and doing everything in a single pass, this cost is almost completely gone (you trade that for fetching light data, but that is a lot less data). So texture bandwidth wise, Joves approach is probably going to be faster. How much faster depends on the scene, if all you have is a global cubemap and a directional light there is little benefit to this system.

Not considering costs, there are other benefits to Joves approach. Design idea 1 means that you can light anything, there is no such thing as transparent materials looking awful because they can only access a single light or something like that.

There are also differences between the actual shading in Unity and Jove. The alu cost per light is higher in Jove, because it does more than Unity lights do. The soon coming patch 2.2 features completely redone lighting code, every light is now 100% treated as an area light with correct horizon handling (light wrapping around objects from large area lights). I am going to be implementing simpler light modes as well, in case you don't want to pay the cost for as correct as possible area lights.

When it comes to post processing that should also be a lot faster in Jove, given the combination of stuff. If allowed post processing explodes in bandwidth requirement (I wrote a lengthier comment on this here if anyone is interested http://forum.unity3d.com/threads/ssrr-request-from-community.297961/#post-1971462).

Basically, the more complex your scene becomes light and probe wise the better Jove should scale over Unity. A straight up comparison between a point light in Jove and a point light in Unity should have Unity winning out, but that is not taking into consideration that Jove uses more complex lighting calculations. Jove has a little overhead from its approach, but that also means it scales better.

As for consoles, I am sorry to say that I do not know Everything is done inside Unity, and there are no features in use that are not supported on the consoles, so given Unitys multi platformness it should work. However, I am involved in several game projects slated for launch this year on XBox One so sooner or later I'm going to be sitting down with a devkit and going through everything.

If you want to discuss this in depth hippocoder you are more than welcome to PM me. I have previously, and am currently, working with several companies on titles in development.

 

I see what you did there ;P

Good stuff as always Aieth, like the explanation of the comparisons.

 

Really quite an awesome post and a magical read into the inner workings of Jove. A lot of fun to read over a coffee - thanks

 

That topic never ends well, so lets please let it die after this

I'm glad you appreciated it and I hope it answered your questions.

 

yes, or sipping on green tea. The caffeine bump is even tricking my brain into thinking it actually understands everything you've explained here @Aieth . Whether my brain actually understands or not is up for debate, but it all sounds logical as I sip my tea and nod in agreement at everything you've written.

 

I think I actually understood it, which says a lot for the last year's work which was hugely motivated by Jove 2! Just goes to show. Started work on implementing a bunch of shaders in shader forge which i'm hoping will produce a consistent approach to hand-fixing for Jove, looking forwards to the new beta now, with just a few fillers for the scene i'm doing needed for now (i think, this may not be satisfactory, but then these scenes are desired for testing, not advertising) i should have a good place to try out not just the reg shaders but particles, animation, and if i can figure it out, a slightly refactored volumetric effect (prob beyond me). Need to cut the scene poly count down from 5 mil though aha.

A quick question, in terms of PBR workflow, i should be going for metalness over spec/gloss right? (before i hit dDo)

 

To answer your question lazygunn, initially when PBR was much newer in the art pipeline people were using the spec/gloss workflow. Now more and more people just use metalness since it has a lot more benefits and is faster to author textures.

 

The question is Jove-specific, but i think it is as you said, just wanted to make sure

 

Yes, Jove is using metalness (and using the free channel that comes with that approach for translucency - if you want materials with that property).

 

I found doing the arch viz scene that Aieth mentioned that when translucency is offered as an implicit option of the shader, it has a pretty radical effect on how you approach your materials, and this along with the integrated post effects made Jove so compelling - you don't realise how important translucency can be until it's always there (Worth adding now that the above 1:1 comparison Aieth made with Unity-regular does not, with humility, include the strength and plethora of integrated and high quality post effects that puts the joint cost in money and performance of such effects in unity assets to shame)

Just a bit of advice I have to give, i think, and maybe a hint to send in the direction of Knald, which will generate your translucency maps extremely quickly. You'll be surprised at how useful they are - Knald is great for generating a bunch of maps (although i tend to use substance designer for most maps), but it's worth it for translucency alone

 

When do you and @Dolkar see the contact hardening shadows being released?

 

this exam jove prova

https://scontent-lhr.xx.fbcdn.net/h...=65769152d6c7514be6d58c398efadf90&oe=554AD1DC

https://fbcdn-sphotos-c-a.akamaihd...._=1435110111_7cf158b7c6cacd124cc3175d710550c9

 

Any plans to add order independent transparency to the mix?

 

I can give you a quick rough explanation on how it works It's run after the GBuffer pass but before any shading occurs. It is rendered in half res, using a downsampled depth buffer (in 16 bit precision to save performance). When an intersection is found the motion vector buffer is sampled at that pixel and the lighting is then sampled from the previous frame.This means you basically get multiple bounce SSR.
The actual raymarching finding an intersection is the easy part, getting that up and running only took a few hours. Then it goes through downsampling and blurring (using a gaussian fit to the BRDF). These blurs are then composed into the final SSR texture based on the roughness of the pixel and the distance from the pixel to its intersection (causing contact hardening reflections).
After this the final SSR texture is running through a temporal super sampling filter, combining it with the previous SSR texture. The temporal blending borrows some ideas from presentations from Epic and Crytek, and is based more on signal processing rather than usual reprojection techniques. Basically, it clamps the history of a pixel (e.g the previous value) to the new neighborhood, weighting every sample according to its luma. How strong this filter is is tweakable. More strength means less flickering (or none) but also means slightly blurrier reflections.

I'm pretty happy with how the reworked translucency turned out for patch 2.2, but it is one of those things that really need testing. Can I count on you to give it a good whirl? As the others have said, Jove does metallic. New for patch 2.2 is a reflectance texture that controls the specular strength of dielectrics. At 127 it maps to the same value that was hardcoded before (0.04), and it should remain in the range of roughly 50-180 (only going below for cavity maps or above for gemstones).

I've considered it off and on but the performance implications are huge. I've basically got 90% of the code to build it, since I already have a GPU generated linked list shader, but I don't think I'm going to do it. It comes with a pretty big overhead (even if there are no transparent materials on screen), and it can potentially explode in memory usage without any way to predict it (suddenly you just get artifacts and the only way to stop them is to preallocate more memory, and it chugs a lot of memory!)

Looking good I've been meaning to add in a dual layered shader, allowing for better rendering of cars and say waxed floors. Hopefully I'll get to it soon enough.