Search Unity

  1. New Unity Live Help updates. Check them out here!

    Dismiss Notice

Jove 2.0: DX11 Rendering System (Alpha Release)

Discussion in 'Assets and Asset Store' started by Aieth, Aug 17, 2014.

  1. Dolkar

    Dolkar

    Joined:
    Jun 8, 2013
    Posts:
    576
    To clear up confusion: Bent normals and SSDO are two different extensions of the base AO algorithm. They both try to make it more dependent on the lighting, but still, the way and the results aren't the same. SSDO provides contact shadows and small scale GI, but for direct lights only. That means that in the screenshots above, it would pretty much look like standard AO, since most of the room is illuminated by indirect light. Bent normals and cones, on the other hand, allow to approximate the incoming light from ALL sources and should nicely fill out all the missing details when using radiance volumes, like the shadows on the Jove logo.
     
    Last edited: Nov 23, 2014
    LilWiebe and blueivy like this.
  2. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    498
    So, if I understand correctly, bent normals is something that you guys are putting on top of the ao that's already in jove?
     
  3. Aieth

    Aieth

    Joined:
    Apr 13, 2013
    Posts:
    805
    As large as you would want to. Of course, the more detailed lighting the more memory but you can create several volumes for different areas of your game world and switch between them (only one can ever be active at the same time so they would have to overlap).
    The probe placement is mainly automatic. The general idea is that the algorithm handles 99% of itself, and then you as the user can define areas where you want more detail or correct any potential mistakes the algorithm does (there are always going to be edge cases where the system fails, so being able to manually fix those is really valuable).
    Yeah, if we do it that is. Bent normals are not currently in use.
     
    Tiny-Man likes this.
  4. ilya_ca

    ilya_ca

    Joined:
    Nov 19, 2011
    Posts:
    274
    Looks very nice!

    Does the system support PBR? Will it work with Alloy? What about Marmoset Skyshop?

    Will it work with existing forward rendering shaders?

    Thanks.
     
  5. ludiares

    ludiares

    Joined:
    Jul 14, 2011
    Posts:
    242
    Short answer: yes, no, no, no
     
  6. ilya_ca

    ilya_ca

    Joined:
    Nov 19, 2011
    Posts:
    274
    Thanks, as far as I understand Jove ships with a few core shaders that can be used almost for anything?
    How does one create custom shaders for Jove?

    What about image effects support?
     
  7. Aieth

    Aieth

    Joined:
    Apr 13, 2013
    Posts:
    805
    It only does PBR, although there is nothing stopping you from implementing your own lighting model if you possess the necessary knowledge. Jove does not work with Alloy or Skyshop, but it does support the same functionality (or at least will, there is no skin shader available at this time). It will work with existing forward shaders on the condition that they are unlit. Since Jove replaces Unitys lighting pipeline shader written for Unitys pipeline would not work in Jove.
    Yes, Jove has a bunch of included shaders (standard bumped, cutout, transparent, glass, emissive). Since it is using PBR the standard shader should cover 95% of all your use cases. You can create both custom forward and deferred shaders (all deferred shaders are locked to the main Jove lighting function though, so you only control the inputs (say vertex texture blending)). Deferred is fairly simple currently, forward a little more involved. It is being cleaned up next patch.
    I described custom deferred shaders in the next patch in this post (http://forum.unity3d.com/threads/jo...tem-alpha-release.262823/page-12#post-1823520). It is a little more verbose in the current version, but if you understand that post then the current version should be no problem either.

    As for image effects. Currently everything that does *not* use the depth texture, or relies on say the alpha channel containing the specular intensity, works. Basically everything that takes an image, does something to its colors and outputs work. So you can use color correction, but not depth of field (depth texture) or SSR (specular intensity in alpha). This changes slightly next patch though, you will be able to use depth texture reliant shaders. I realized today I can simply redefine Unitys depth macro, I can't believe I never thought of that before. So the only modification that would have to be done to existing shaders is add a "#include "JoveCG.cginc" command.
     
  8. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    5,070
    While realistic renders will mainly use PBR, I imagine games - which are not always entirely realistic - will want to have shaders for various features. Hair, skin, eyes, fur, force field, snow, car paint, toon / outline, hologram / night vision / see-through, dissolve ... and probably a few others.

    What about a jewel shader? Does Jove 2 natively handle that?
     
  9. Aieth

    Aieth

    Joined:
    Apr 13, 2013
    Posts:
    805
    Very true and this is what the forward part of Jove is for. Hair, skin and eyes are quite complex and they are on the list of things to do before final version. Fur falls under that as well.
    Snow should be doable with the standard shader.
    Force field should work with an unlit particle shader.
    Car paint I've got some ideas for, again it is on the list :)
    Non realistic rendering such as toon/outline you could technically implement on your own, I'm not sure where this would fit into Jove but it is one of the possibilities I want to look further into during a post final release patch (aka post 1.0).
    If you have any requests for dissolve and the like I'd be happy to implement it for the next patch. I take it you mean as a variation of the standard deferred shader, just with dissolve added?

    As for a jewel shader, I am fairly certain you could use the Jove glass shader for a convincing jewel.
    I've been meaning to add a forward shader that uses a traditional specular map instead of metallic work flow. The standard deferred shader would still be faster (so you would not want to use the forward shader for everything!) but it would mean you don't have to compromise the flexibility of the specular map, if you need it it is there as an option.
     
  10. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    5,070
    Hi Aieth ... I have no special requests right now. I'm just throwing things out there. I picked up Jove 2 mainly to help sponsor its development because I thought it was a cool idea, it's beautiful, and it is pushing the envelope. I'm not sure I will ever actually use it, but we'll see.

    Jove 2 may be re-inventing the wheel - and doing a great job of it - but I don't know how far I want to go in that direction. I kind of like the safety and comfort of being able to grab things off the Unity shelf (like TrueSky, etc.), and plug them in and go. If there was some kind of "Unity compatibility mode" with Jove, where Jove could render stuff on some layers while Unity rendered on other layers (or some odd thing like that), that would probably be ideal. But if something like that can be done, it will be up to you geniuses to figure out. I have no idea.
     
  11. Tiny-Man

    Tiny-Man

    Joined:
    Mar 22, 2014
    Posts:
    482
    I think jove is eventually going to be all in one kind of thing.
    Like eventually realtime clouds i believe will be created in the future.
     
  12. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    5,070
    Well, if that's the case ... here's hoping he gets far enough along that he adds a DX11 GPU-based hair strand simulation. :)
     
    blueivy likes this.
  13. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    498
    I think first a shader like this one since it has to do with the actual rendering of hair
    http://ir-ltd.net/real-time-hair-in-vr-part02/
    Since jove is a renderer first, hair simulation doesn't necessarily fall under that.
    Pair that with some nice skin, eye, and cloth shaders, and you'd have the full character pipeline.
     
    Last edited: Nov 28, 2014
    8Infinite8 and hopeful like this.
  14. 8Infinite8

    8Infinite8

    Joined:
    Jul 27, 2013
    Posts:
    112
    I would be happy to donate that shader for 'science'. We actually have a very nasty road block with our order independent transparency and DX11. It used to work fine in DX9 but it's hard to implement in DX11 as they made some stupid changes. I would imagine combined with Jove it woudl look very cool.
     
    blueivy and hopeful like this.
  15. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    498
    I would love if you donated the shader and so would a lot of other people! In the name of 'science' :p how's the skin shader Flynn going?
     
  16. 8Infinite8

    8Infinite8

    Joined:
    Jul 27, 2013
    Posts:
    112
    Not meaning to hijack this thread at all, sorry Aieth.

    By donate, I mean in the sense for Jove, or to anyone that can help solve the transparency issue.

    The progress on the skin shader is slow at the moment.
     
    blueivy likes this.
  17. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    498
    Yeah I know you meant for Jove, I don't want the hair shader to myself. :p Can't wait to see what it can produce :D
     
  18. Aieth

    Aieth

    Joined:
    Apr 13, 2013
    Posts:
    805
    Hey everyone!
    I have had a really busy past month. Despite all that, I have made some very significant progress. I can count the things left to do on one hand now ;)
    The performance is less than 1ms on my GTX 560ti. This isn't going to change when I add in point + spot light support. As for the quality of it, you be the judge. Below is a clean environment that should expose most artifacts. I'll post a Sponza scene eventually when I have time and I have improved the workflow of it.

    DirectionalSkyGI.png
     
  19. chiapet1021

    chiapet1021

    Joined:
    Jun 5, 2013
    Posts:
    604
    Awesome to see an update on this thread--which I visit almost daily just in case my alerts aren't working. :)
     
  20. Licarell

    Licarell

    Joined:
    Sep 5, 2012
    Posts:
    434
    Looks dy-no-mite!
     
  21. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,448
    Realtime??
     
  22. Aieth

    Aieth

    Joined:
    Apr 13, 2013
    Posts:
    805
    Oh right, probably shouldve mentioned that. Yes, realtime :)
     
    chiapet1021 likes this.
  23. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,448
    Wow okay then "daaaaamn guuuurl!!!"
     
    chiapet1021 likes this.
  24. chingwa

    chingwa

    Joined:
    Dec 4, 2009
    Posts:
    3,479
    It's beautifuuul. :D
     
  25. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    498
    How many bounces does it do?
     
  26. Licarell

    Licarell

    Joined:
    Sep 5, 2012
    Posts:
    434
    @chingwa Hey Chingwa... do you plan on supporting Jove2 with your suimono asset?
     
  27. eskovas

    eskovas

    Joined:
    Dec 2, 2009
    Posts:
    1,344
    Looks amazing there Aieth :D
    Really impressed with the <1ms realtime. I guess everyone will want to see more screens of that in action :p
     
  28. Aieth

    Aieth

    Joined:
    Apr 13, 2013
    Posts:
    805
    As it is precomputed it does as many bounces you want to ;)
    I'd recommend waiting for the next patch before doing that. I'm improving shader creation quite a bit, and hopefully it is going to be the final version (as in backwards compatible when new patches arrive).
    Great to hear! There is no way around it, GI is very demanding. The great part of this technique is that all the heavy calculations are precalculated, meaning the runtime performance is really good. This does mean that only static geometry contributes to the GI (but dynamic geometry gets it as well), but there are drawbacks to every technique :)
     
    hopeful likes this.
  29. Tiny-Man

    Tiny-Man

    Joined:
    Mar 22, 2014
    Posts:
    482
    Pretty damn cool
     
  30. Licarell

    Licarell

    Joined:
    Sep 5, 2012
    Posts:
    434
    Duly noted... there are a few on my list to hit up once that has completed like Suimono which I need still, Amplify texture 2, VertEXmotion, etc.

    The developers I've contacted seem interested but I'm sure they are waiting to see how much work it will be to port their assets over and if it will be worth the R&D investment to them...

    Rock their socks Aieth!
     
  31. chiapet1021

    chiapet1021

    Joined:
    Jun 5, 2013
    Posts:
    604
    @Licarell I hope ShaderForge is also on your list. :)
     
  32. TechnicalArtist

    TechnicalArtist

    Joined:
    Jul 9, 2012
    Posts:
    721
    Jove have skin shader?
     
    8Infinite8 likes this.
  33. zelmund

    zelmund

    Joined:
    Mar 2, 2012
    Posts:
    396
    awsome! keep goin like a train! =)
     
  34. Aieth

    Aieth

    Joined:
    Apr 13, 2013
    Posts:
    805
    Not at the moment no. Jove does support translucency so you can hack together something okayish, but a proper skin shader is coming later on
     
  35. Aieth

    Aieth

    Joined:
    Apr 13, 2013
    Posts:
    805
    Here's the dynamic GI on a "real" (programmer arty) scene. Hopefully going to have time to post a Sponza demo this weekend! Performance requirement is less than 1ms on a GTX 560ti. Only works with static geometry but allows completely dynamic lighting.

    DirectionalSkyGI2.png
     
    blueivy likes this.
  36. Licarell

    Licarell

    Joined:
    Sep 5, 2012
    Posts:
    434
    So with a character in this scene would we use probes to light it?

    Sorry if this is a stupid question...
     
  37. Frpmta

    Frpmta

    Joined:
    Nov 30, 2013
    Posts:
    461
    I have limited knowledge in rendering, but is it that approach for GI is just not compatible with dynamic objects or could there be a way to enable the use of dynamic objects with a requirement of let's say 3 or 5ms.
     
  38. chingwa

    chingwa

    Joined:
    Dec 4, 2009
    Posts:
    3,479
    I think Aieth mentioned that while dynamic objects don't contribute to the GI, they would still receive it...

    I'd love to, but I think it's a bit early from both ends to seriously look into this yet. From the way Jove seems to be shaping up though, I can see it eventually becoming a defacto requisite for those wanting to do high-end scene rendering with Unity. I've been following this project for a while, but I haven't jumped on board (mainly because I just don't have the time) so I don't know how involved it is to incorporate shaders into the Jove workflow...
     
  39. Licarell

    Licarell

    Joined:
    Sep 5, 2012
    Posts:
    434
    So receive light but not cast shadows... right?


    Hummm... this gives me an idea... @ Aieth would we be allowed to share your commented shader info with some of these 3rd party assets that require adding custom code to work with Jove 2... He who have not ask not...

    Thanks!
     
  40. TechnicalArtist

    TechnicalArtist

    Joined:
    Jul 9, 2012
    Posts:
    721
    thanks for reply
     
  41. Tiny-Man

    Tiny-Man

    Joined:
    Mar 22, 2014
    Posts:
    482
    Hope to see more!
     
  42. JecoGames

    JecoGames

    Joined:
    Jan 10, 2013
    Posts:
    135
    Dynamic meshes still cast and receive shadows, its the global illumination that they only receive.
     
  43. zelmund

    zelmund

    Joined:
    Mar 2, 2012
    Posts:
    396
    how much time need to bake data for dynamicGI in jove? for this simple room above for example.
     
  44. 8Infinite8

    8Infinite8

    Joined:
    Jul 27, 2013
    Posts:
    112
    These are very nice but we seem to be missing some hot spot GI bounce from the main direct light source that's casted onto the floor or walls. Is that a limitation of this method?
     
  45. Aieth

    Aieth

    Joined:
    Apr 13, 2013
    Posts:
    805
    It doesn't deal with specular reflections, only diffuse. You might be confused now since that wouldn't be picked up by a cubemap/SSR and you are correct!
    What I mean is that when light enters through the window and hits the wall, some of that light is scattered all around it (diffuse) and that is simulated. Another part of the light bounces off the wall (specular), hits the floor and is then scattered as diffuse light.
    That part of the radiance transfer isn't simulated, that is too complex to be handled by anything but a full blown integration (or an estimate of it like Enlighten is). Another downside to the dynamic GI is that, and I apologize for not being able to explain this better, is that it isn't represented by color values, it is represented by frequencies. This means it is "fuzzy", there's no super clear distinction between when the light hits the green wall and when it is just next to it, in a frequency representation the light is just kind of on the green wall.

    Downsides said, on the plus side the local hotspot you are speaking about could be done using SSDO, an approach I am considering (later on though). SSDO adds more local information, meaning you could possible even use less samples for the radiance volumes and still get a better quality.
    Another upside as previously hinted to is the performance. As far as I know no other GI method can compete with this technique.

    The dynamic GI is by far the slowest baking method, both the lightmap and the static radiance volumes are a lot faster. The bake time is dependent on amount of triangles and how dense they are. This room is 7000 triangles and it precomputes in just above two minutes on my computer. This is going to increase though if you want to support point/spotlights as well. For workflow I recommend using the static mode while working and updating the scene goemetry, and then doing a full dynamic GI bake when you are happy with it.
     
    8Infinite8 likes this.
  46. JecoGames

    JecoGames

    Joined:
    Jan 10, 2013
    Posts:
    135
    I heard it only took ubisoft 8 minutes to bake their probes for their similar technique for the entire Havana city in AC4, that said though they probably used 4 titans:)
     
  47. Licarell

    Licarell

    Joined:
    Sep 5, 2012
    Posts:
    434
    I hope the baking is through the GPU because I have 3 in my rig!
     
  48. Tiny-Man

    Tiny-Man

    Joined:
    Mar 22, 2014
    Posts:
    482
    I guess ill just sit here with my 660 :(
     
  49. Aieth

    Aieth

    Joined:
    Apr 13, 2013
    Posts:
    805
    They made some severe cuts in the technology though. They used rasterization but that eliminates everything but single bounce GI. I am considering doing the same for baking point and spot lights but as far as sky and sun lighting goes I felt it was too much of a drawback.

    @Licarell Dynamic radiance volumes is baked on the CPU. Static radiance volumes and lightmaps are on the GPU. That said, having more than one GPU will only boost the static radiance volumes as it is the only one based solely on rasterization :)
     
    8Infinite8 likes this.
  50. Tiny-Man

    Tiny-Man

    Joined:
    Mar 22, 2014
    Posts:
    482
    Now Aieth, ill just slip a 10 in your back pocket and you can maybe just hand that GI over...
    Just keep that between you and me.
     
    JecoGames likes this.
unityunity