Jove 2.0: DX11 Rendering System (Alpha Release)

I just find the current shaders to make the leaves look a bit too 'flat' to say, needs AO from vertex colours but there is no vertex colour shader for double sided, I think that will give it the 'pop' and depth it needs.
Ill continue messing around, I've left it a 0.2 all the way but I wanted to show what other values look like.

 

Leaves are actually surprisingly shiny,so if you haven't already I would add some specular,also maybe use a lower translucency. You definitely don't need a dedicated leaf shader as in blender cycles all I used was a translucency shader with diffuse and some rough specular and I got really nice results.

 

As i said i use 0.2 trans and I set gloss is at 60.
The issue i has is that the leaves look flat since I cant use vertex colour AO.
And as said before the shadows might need to be softer for the leaves (as in semi transparent)
Just being fussy thats all.

Edit: Also maybe some vertex movement in the shader based on UV's

Edit 2: Harnessing my lack of shader skills, I, Tiny Man, managed to create a shader so complex (well not really I just copied and pasted code across) that the leaves look even better (in my opinion).

Those leaves I feel have a lot more 'depth' to them now with the AO.
The shader I created multiplies the vertex colour to both AO, Albedo and Spec. It derives from being completely PBR to a better looking image. Will tinker it quite a bit to get it to look better (some areas are too dark in my opinion).

Edit 3: Just realized that the shadows of the leaves is a 1 way shadow, since they're double sided, the side that is usually culled in a normal shader isn't casting a shadow. (can be seen on the grown, shadows there aren't very thick.

 

I'll make sure to include an emissive particle shader the next patch, that was an oversight on my part.

I'm aware there are no good tree shaders, I just have not gotten to it yet I can explain how the translucency works. A value of 1 means there is no diffuse, all light passes straight through. A value of 0.5 means half the light scatters as diffuse and half the light goes through, so a value between 0.2 and 0.4 sounds about right for leaves. I'm hoping to improve the translucency part later, but we'll see how far I can take it.

Glad you managed to merge the shaders I'm proud of you! If you're shader does not have this tag:
"JoveShadowCaster"="Cutout"
add it and the leaves should *hopefully* cast shadows from both sides. If they still don't, well, I'll fix that when I make a tree shader


To everyone else, and especially @8Infinite8, I have an announcement. It could be considered both good and bad news. I've decided to postpone the ambient modes patch to later on. This is due to several reasons, among them compatibility problems sailing up on the horizon (both future Jove features and Unity 5 stuff like enlighten), so it really feels like to much of a leap of faith hoping it is gonna work out. That, and I've just gotten so tired of working on it and really need to work on something else for a while.

So, instead I am doing this. I am first off refactoring the entire shader code base (almost done).
I am then implementing more proper versions of area lights, ranked in order of likelihood that they will make it in: sphere, disk (spotlight), tube, rectangle.

I am adding reflectivity as an extra texture, removing the hardcoded reflectivity of non metallics at 0.04 and instead allowing it to vary between 0.0 - 0.16 (almost all materials are within 0.02-0.06, with only gems being brighter).

I am adding screenspace reflections in addition to the current IBL probe system, where the probes fill the holes that SSR could not find.

I am changing all cubemaps from RGBM to 16bit HDR (Unity doesn't natively support this, at least in 4.x).

I am adding support for DX10.1 similar to what Unreal Engine 4 has. This is probably gonna be rough for a while though as I don't have a rig I can test it on, and I am unclear to how compute shaders behave under Unitys "DX10.1 as DX11" mode.

I am also moving the entire lighting pipeline over to physical values, instead of arbitrary intensity it is going to be photometric measurements (luminosity). You might be questioning why this would be a good thing, you don't care if you can replicate your living room light in game. That was my first reaction as well, but if you think about it it is actually a brilliant idea. If all light sources are calculated using values that match real life, all lights have correct ratios per definition. The sun is as bright as it is supposed to be compared to an indoor light. This also means that, before tonemapping, the raw data collected in the texture is the same thing as the raw data collected by a digital camera when taking a picture. This means that all the tricks digital cameras use can be implemented in shaders and all of a sudden you have a rendered image that looks really realistic.

So, in addition to the previous paragraph, I am also finally implementing what is hinted at in the UI called "Virtual Camera"

And then I'm also going to add deferred decals.

Not everything is going to make it into the first patch, stuff is gonna be churned out as soon as it is ready.

EDIT: Oh yeah I almost forgot, with the SSR I'm going to be adding temporal Anti Aliasing. Note that temporal AA is in most cases not a all in one solution, it still benefits from having something like FXAA run as a post process.

 

So good news Thanks for the update.

 

Thanks for the reply Aieth .
I got the tag but no results :O

With the lights what unit will you use for measuring the strength?

 

I'll fix it when I make a tree shader later then, no idea why it doesn't work for you though : /

Lights will be using luminous flux and luminance. User choice on which to manipulate.

 

Super stuff Aieth
Will you add an anisotropic shader for things like brushed metal and hair?
Keep it up

 

Yeah I will, eventually. There are a lot of things I want to do and only so much time


Here's a sneak peak of what I'm currently working on

 

Area lights? Nice!

Will it be able to handle the stretched effect of lights reflecting on water?

 

Any light handles that included the ones already in Jove. You rarely see it though, as the range attenuation clips it. Getting that effect using lights is therefore pretty expensive, the only way to do it is to make the attenuation far larger than you actually want to. SSR should prove better at doing that since it doesn't suffer from the same range limitations and the effect only appears in grazing angles anyway.


Updated the reflection probe system. Added distance based roughness and remade the user interface/parameters. You can now also rotate the box volumes, and the overall system is slightly faster than before.
The volume offset allows the cubemap to be baked at any position and not just the center of the volume (e.g in this picture I baked it closer to the ground to avoid artifacts).
The fade distance replaces the previous outer and inner volumes, you now just specify over how long a distance you want the volume to fade.
Blending priority allows more predictable overlapping of volumes. If you place a volume with a high priority inside a volume of medium priority the high priority volume gets priority and then smoothly fades into the medium volume. Previously it was hard to predict what probe would take precedence as the system judged it based on distance from volume center. Now you have more flexibility when it comes to small probes inside large ones.
Finally the bounces is used for baking. It allows the user to specify how many bounces of light should be simulated (4 means the cubemap is rendered a total of 4 times etc). This replaces the old way of "just click the button a few times".

 

Super cool . I will use these lights with my GI proxy system, this would allow for much better result in the Global Illumination bounce light distribution than simple point lights i guess.

I may be moving to Jove at some later point in my project, now that Unity 5 will remove some of the DX11 limitations as far as target platforms are concerned, so it is very nice to see it upgraded with such cool features.

 

As a pre-warning, I haven't looked at the documentation for Jove so I'm not entirely sure how the rendering works for Jove, but I'd like to ask you a question.

Currently, in Unity 4.6.1, I'm trying to implement a scene where a character is sitting in the center of the camera with one directional light. I've setup the camera to use my own custom projection matrix, so that the final image is skewed. This combined with some head-tracking software gives the illusion that the character is popping out of the screen.

A big part of the illusion is using shadows, and because of my custom projection matrix Unity's shadows either don't show up properly or show up very poorly (aliased, etc).

So my question, before I purchase your plugin, is that would it be possible to use a custom projection matrix for my camera and still use your massively better lighting/shadow system?

 

Glad to hear it What limitation is it Unity 5 is removing? I have not been keeping up as of late.

I'm afraid the current version does not support custom projection matrices. The functionality is all there but it need a shader side update, which is coming next patch. I take it your matrix is still a perspective matrix? In that case it is going to work next patch. Orthogonal matrices would require different handling shader side so they are not supported. It is not impossible but it is extra work and I have yet to receive and requests for them.
I handle shadows differently than Unity so there *should* be no problems, with emphasis on should



While transferring Jove unto the new lighting system I had to revisit the dynamic sky to make sure the values rendered match real physical values. I figured I'd take the opportunity and implement some improvements while I was at it. For example, it now supports the camera leaving the surface and heading into space. Makes for great screenshots.

 

I am not entirely sure it is the case, but from posts here i gathered Unity 5 will be able to compile DX11 code to other than MS platforms, like PS4, which is one of my target systems.

If this is indeed true and working well, i will definitly be looking into DX11 additions and a possible replacement of the renderer with Jove.

 

@Aieth , thanks for the in-depth breakdown. It sounds like the next patch will do what I want. I am indeed using a perspective camera matrix. If you don't mind me asking do you know when the next patch will be released?

 

@Ben BearFish couldn't you just render with a traditional projection and correct it with a pixel shader after the render pipeline? If I understand what your saying the areas that would end up getting stretched would be farther from the viewer anyway so it wouldn't matter...

 

@LilWiebe , I suppose that's possible, but every demo video I've seen regarding projection modification seemed to mention in the comments they felt the effect didn't quite look right with shaders because of shadow and lighting issues with Unity.

 

Hopefully within a couple of weeks. There has been a lot of changes and I don't want to rush anything out just to realize there are embarrassing bugs in there


The area lights are done for now. Now onto shadows! I will do textured rectangular lights at some point, I feel shadows are more important though Single pass deferred area lights

 

cool

 

Rewrote the bloom algorithm. The new version has four advantages over the old.
1. Longer radius, making for smoother gradients from bright objects
2. It is faster as it uses less texture bandwidth
3. It hogs slightly less VRAM than the previous approach, leaving more memory for other stuff
4. It does not flicker at all, even with camera movement and super bright pixels (usually called fireflies)

 

I really appreciate that you're rigorous about going over code to make it both efficient and feature-filled.

 

On that note I also went over the clustered shading algorithm. There are now no limits at all on how many lights you can (well, almost, you can only have 2^14 point lights, 2^10 spot lights, 2^12 tube light and 2^12 rectangular lights). Basically, it scales with the requirements the scene presents. This means that if you only have a few lights it is going to be faster than before as well. It keeps track of and analyzes the recent history to find how much memory it needs to allocate at minimum, and adapts to that without spiking which causes memory fragmentation.

 

How are you planning to get shadows with the area lights? Plain old shadow maps or some more accurate area light shadow approximation?

 

Shadow maps are still the only way to do real time shadows in a general sense. Epic has their distance fields which is very cool but only works for static objects. There's a few tricks to make it fancier though

@Ben BearFish
Could you possibly share the projection matrix you want to use with me? I've got custom projection matrices working in my dev version and I could test if yours work

 

Epics technique doesn't work for objects that are animated, dynamic non-animating objects work as well, maybe it would be possible to use their technique for all non-animated objects then use a low res shadowmap for just animated characters?? It probably is better to work on a solution that everyone can avail of first though...

 

Dank stuff aieth, keep it up

 

@Aieth Here you go. I attach this script to my camera gameobject. The Corners array are 4 Transforms that make up the corners of the near plane, which allows me to change them in the scene view. lookTarget is the center of my near plane. As I mentioned earlier this script changes the projection matrix based on the camera'a view angle from the center of the near plane to give a proper skewing visual illusion effect.

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3. using System;
4.  
5. [ExecuteInEditMode()]
6. public class CameraProjectionMatrix : MonoBehaviour
7. {
8.     private Camera theCam;
9.     public Transform[] Corners;
10.     public Transform lookTarget;
11.  
12.     // Use this for initialization
13.     private void Start()
14.     {
15.         theCam = camera;
16.         UpdateProjectionMatrix();
17.     }
18.  
19.     public static Matrix4x4 GetOffCenterMatrix(float left, float right, float bottom, float top, float near, float far)
20.     {
21.         Matrix4x4 offCenterMatrix =  Matrix4x4.zero;
22.  
23.         offCenterMatrix[0, 0] = 2.0f * near / (right - left);
24.         offCenterMatrix[0, 2] = (right + left) / (right - left);
25.         offCenterMatrix[1, 1] = 2.0f * near / (top - bottom);
26.  
27.         offCenterMatrix[1, 2] = (top + bottom) / (top - bottom);
28.  
29.         offCenterMatrix[2, 2] = (-(far + near)) / (far - near);
30.         offCenterMatrix[2, 3] = (-2.0f * far * near) / (far - near);
31.         offCenterMatrix[3, 2] = -1.0f;
32.  
33.         return offCenterMatrix;
34.     }
35.  
36.  
37.  
38.     // Update is called once per frame
39.     private void LateUpdate()
40.     {
41.        UpdateProjectionMatrix();
42.     }
43.  
44.     public void UpdateProjectionMatrix()
45.     {
46.         if (theCam == null)
47.             return;
48.  
49.         if (Corners == null || (Corners != null && Corners.Length < 4) || (Corners != null && Corners[0] == null))
50.             return;
51.  
52.         Vector3 pa = Corners[0].position; //Bottom-Left
53.         Vector3 pb = Corners[1].position; //Bottom-Right
54.         Vector3 pc = Corners[2].position; //Top-Left
55.         Vector3 pd = Corners[3].position; //Top-Right
56.  
57.         Vector3 pe = theCam.transform.position; // eye position
58.  
59.         Vector3 vr = (pb - pa).normalized; // right axis of screen
60.         Vector3 vu = (pc - pa).normalized; // up axis of screen
61.         Vector3 vn = Vector3.Cross(vr, vu).normalized; // normal vector of screen
62.  
63.         Vector3 va = pa - pe; // from pe to pa (eye position to Bottom-Left)
64.         Vector3 vb = pb - pe; // from pe to pb (eye position to Bottom-Right)
65.         Vector3 vc = pc - pe; // from pe to pc (eye position to Top-Left)
66.         Vector3 vd = pd - pe; // from pe to pd (eye position to Top-Right)
67.  
68.         float near = -lookTarget.InverseTransformPoint(theCam.transform.position).z; // distance to the near clip plane (screen)
69.         float far = theCam.farClipPlane; // distance of far clipping plane
70.  
71.         float eyeToScreenDistance = Vector3.Dot(va, vn); // distance from eye to screen
72.  
73.         float left = Vector3.Dot(vr, va) * near / eyeToScreenDistance; // distance to left screen edge from the 'center'
74.         float right = Vector3.Dot(vr, vb) * near / eyeToScreenDistance; // distance to right screen edge from 'center'
75.         float bottom = Vector3.Dot(vu, va) * near / eyeToScreenDistance; // distance to bottom screen edge from 'center'
76.         float top = Vector3.Dot(vu, vc) * near / eyeToScreenDistance; // distance to top screen edge from 'center'
77.  
78.         // Projection matrix
79.         Matrix4x4 newProjectionMatrix = GetOffCenterMatrix(left, right, bottom, top, near, far);
80.  
81.         // Assign matrix to camera
82.         theCam.projectionMatrix = newProjectionMatrix;
83.  
84.     }
85. }

 

I have another question Aieth, I haven`t checked yet but, can you access the fresnel in the shader?
Reason being as I want to use it to make a cloth shader and I need the fresnel to modify the diffuse slightly, like this


I did this image here really easily in blender, and if i'm able to get the fresnel in the jove shader I can easily modify it slightly for my liking.

 

I would also like to know this, I can create wet surfaces quite easily in blender cycles by blending between diffuse and specular by a fresnel value of 1.33

 

Yeah I love blender cycle nodes, and shader forge allows me to access this powerful way of shader creation. Maybe it can be possible to author a Jove shader in shader forge through the custom lighting property, I'll do some experimenting.

 

I still can't believe unity hasn't implemented a node system yet, its really annoying having to open up Photoshop just to make tiny tweaks to textures, also even while I'm fairly proficient with shaders it's so much quicker for some effects to use nodes. Love the cushions BTW

 

Cheers, should be able to get an image like that in Jove once SSR is implemented.

 

Ok I can see why that matrix won't run in standard Unity. Unfortunately it currently does not run in Jove either I think Unity does this, and I know Jove does, but usually the w component of a transformed vector is linear depth. I'm fairly certain that matrix could be modified to output linear depth in w. It has been a while since I did manual matrix math but I'd be willing to give it a shot later on when I have time for it, if you can't figure it out.

@Tiny Man
That depends on what you mean by accessing the fresnel. You can overwrite lighting passes and do your own lighting. You could check out the Jove glass shader or the included example shaders to see how its done. You should know however that the entire shader structure is changing majorly in the next patch

 

@Aieth , I'm a bit rusty on my matrix math as well. That may be part of my issue. Are you saying that I could get the shadows to correctly render by modifying the w components in my projection matrix? I'm a little lost to why that affects the shadows in Unity, sorry.

Also, is there a certain place in Unity's pipeline I should be modifying this? Their documentation on this particular subject is a bit sparse.

 

@Ben BearFish
Now that I think of it I am not so sure it is fixable with regular Unity, I just have no idea how exactly their shadow mapping works. It is however fixable for Jove, as long the .w component outputs linear depth. *Or* if the shaders are modified in such a way that the depth value fed to the GBuffer is calculated in some other way. It is fixable, it just requires a little massaging to work I'm going to do it eventually, but unfortunately I have more pressing features that need to be done yesterday

 

Any updates aieth?

 

There's a few things going through the refinement mill at the time.
To start with I implemented some simple but effective post effects of my own, vignetting and grain.
Translucency as it exists in the current Jove version is gone, there are no more per light controls. I've instead cooked up a different way to handle it and the result is shown below. First screenshot has 100% translucency and the second has 0%. This generally means translucency is a lot easier to work with.





I also added a much improved version of specular occlusion. For reference compare this picture to the 100% translucent one, the only thing that has changed is that I have turned off specular occlusion.



And if you have not noticed it yet here is another hint of what is to come



Temporally supersampled in more ways then you can count.


Also, @Dolkar is working on this and its shaping up really good. Contact hardening shadows



What else? Well, I've ported it to Unity 5 (I no longer own a Unity 4 license) and I am soon to start a DX9/OpenGL compatibility port. The new light types are fully integrated into the pipeline. The way you write forward shaders has been simplified A LOT (and deferred shaders too, slightly. Only they were not that complicated to begin with).

For example, here is how the simple emissive shader looks now https://dl.dropboxusercontent.com/u/233290703/NewEmissiveShader.png
The main idea is that you fill in the ForwardInput struct with whatever values you desire, pass it in and Jove handles everything else by itself. If you need more control you can fill the entire FragmentData struct by yourself. Or if you need more control over how lighting works, you can redefine the default lighting functions using a macro. This is how the glass shader adds refractive specular lighting.

So yeah, there is a lot of stuff happening. I'm working 12 hour days to meet deadlines I have, all of a sudden I find myself involved in a lot of different projects with differing needs It is all very exciting.

EDIT: Oh yeah and the BRDF has been quite heavily updated as well. The energy conserving approximations used now are far superior to the previous ones and everything is just generally slightly sharper. Most of this stuff is based on the recent Frostbite presentation but I've also whipped out Matlab and done some cool stuff of my own.

 

The Unity 5 update is extremely awesome to hear, especially since they have started using the RC moniker with the most recent public beta version.

 

Awesome work! Is your SSR quality dependent on far/near planes? Also does it work with multiple cameras? (like a first-person camera overlayed on another, although that wouldn't be needed if I could assign a second directional light with shadows only for that first-person camera which in Unity is impossible)

 

It isn't dependent on near/far planes, unless you make the view frustum so big that you get precision issues but then its not just the SSR thats gonna start failing.
I don't think I understand what you mean with overlayed camera? And how can a camera be replaced with a shadow casting light?
It works for any amounts of cameras though, but it does require memory per camera (temporal reprojection needs the previous textures so they have to be stored).



I wish I had better textures to show these off, I need to look into purchasing some. If anyone reading this is familiar with a good source of PBR textures/models/whatever please tell me

Here's a screenshot of a bunch of spheres/cubes with varying parameters (color, roughness, metallic, translucency). I'm gonna get a comparison screenshot of how Jove looked before later, but the looks have changed quite a lot. It is subtle, but it makes a huge impact on proper models (the stuff that I have to test on I can't share publicly, sorry!)

 

What I meant is that I have a "normal" camera that has a directional light casting shadows (4 cascades) and then a second camera that uses another shadow map (1 cascade, lower quality) so that the first person model has nice shadows.

In the standard Unity renderers I can't just use 1 camera because if I use 2 lights to do the same thing, I can't use a culling mask for the first person models for the second light (can only exclude 4 layers and not use only 1)

 

I am on a Mac, so DX9 compatibility makes me very excited. Would it be possible to implement a DX9-compatible triplanar shader?

 

Awesome stuff Aieth, gonna start an archviz project soon so these features will really make it pop!

Everyone here appreciates the amount of time and effort you have put into this project.

 

@Aieth, everything is looking gorgeous! And If I'm not mistaken, specular occlusion is a special case of screenspace AO where it's used to subtract specular, right? Or is it full occlusion based on SSR?

As about great PBR textures, there is one place to get them:
http://www.quixel.se/dev/megascans

It's not open atm, but I'm pretty sure some people from this thread might have a beta access to that library. Their textures are 100% delicious. Low-res subset of those textures is used for base layers in their free DDO Photoshop plugin that is openly available, so you can, in theory, generate some procedural textures there too.

If that won't work and if using stuff like sample assets from Marmoset isn't to your liking, hit me with a PM, I'll come up with something.

 

awesome!!
will post my excava later.
and reflections on screenshot looking very good. if there wasnt problem with them, then it will solve my problems with reflections and this will get my life much much easier.

 

@Aieth, how useful would this be to you?
http://blogs.unity3d.com/2015/02/06/extending-unity-5-rendering-pipeline-command-buffers/

 

Please, Everyone, Stop asking for new features, so he can release the current ones already, then you can ask again for more awesome stuff.

 

NEVER!!!

 

Oh okay. Have you used Jove before? Because if I am understanding what you are asking correctly, then in Jove it should be possible to force a shadow cascade to pretty much land atop of your gun model, meaning two cameras are not needed. That said it is probably better if I implement support for manually specifying the extents of the shadow cascades, I'm adding it to my todo list. Or is the issue that you get z-fighting with a really close nearplane?

A triplanar shader is really easy to add. Please note though that when I say DX9/OpenGL compatibility it means compatibility, it isn't the whole feature set because that is actually impossible to do on those libraries.

And I really appreciate all the feedback I'm getting from the community
Archviz really does need high quality lightmaps to prosper. Now that I'm on Unity 5 which finally supports floating point CPU textures I'm gonna see if I can't get my lightmapper working properly. I also have to look closer at enlighten integration, although my main concern with that is workflow and precision, I hope you can force it to do 16 bit textures and not just 8 bit RGBM.

It is the first case. SSR inherently means occlusion since it doesn't separate visibility from lighting, however there is the problem that the pixel a ray is hitting isn't guaranteed to have raytraced lighting itself. Therefore its usually better to just apply the specular AO on SSR as well, which is what Im doing.

It is still limited in the way SSR always is. For those of you who have not worked with it, I'm afraid it isn't the miracle fix it often looks to be from screenshots. It requires a bit of manual work to hide artifacts and stuff. On the plus side, its the only good way to have real time reflections on todays hardware.

Possibly useful. I can use it to inject stuff before transparent rendering but after opaque, currently I can only differentiate between forward and deferred.

I'm stopping myself every day from spinning off and working on something else I thought of a new way to do completely dynamic GI (does not require static geometry) which I'm itching to test. That implementation would likely take me months though, so I'm saving it for when Jove is out of beta/alpha.
Speaking of which, I'm gonna call it beta as soon as local shadow maps and SSR are in. SSR is pretty much done now after 10 days of full time work (I thought it was gonna take ~3), so that just leaves local shadows to be fully implemented.