Search Unity

SEGI (Fully Dynamic Global Illumination)

Discussion in 'Assets and Asset Store' started by sonicether, Jun 10, 2016.

  1. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    7,000
    You are right, it does sound like they're supporting a light mapper of some kind ... but he makes a mention about it "all being baked" and "the octane renderer handles all of that for you". The translation for that is it being baked into a format that their Octane renderer can use to playback an effectively prerendered scene on top of geometry. He makes no mention of this being something usable by the Unity renderer. I also suspect it only looks totally correct from a basically fixed or very narrow camera position. It's not unusual for a "lightmap" in an offline renderer to store specular highlights and reflections because the camera position is fixed.This is a terminology issue for people hearing words that mean one thing for real time and something very different for offline rendering. For Octane they're probably being a little smarter to let the camera move around more in the scene, but it's not going to be the general case you're thinking of.

    I mentioned this already, see that video at around 3 minutes to see what true real time currently looks like, and that's probably being demoed using an Nvidia Titan or two on a fairly small (ie: low resolution) viewport. To be fair it's also probably Octane 3. Brigade and Octane 4 look amazing, but who knows when the PowerVR hardware will materialize, if ever, especially on mobile. With Apple dropping PowerVR in less than 2 years, and Intel killing off Atom there aren't going to be a lot of devices running Wizards (the name of PowerVR's upcoming raytracing capable GPUs), and even if Apple ships with a Wizard in the interim there's no guarantee they'll expose those features. Makes it a hard sell for devs to spend the time to support as you can bet there won't be comparable hardware support in any of the Mali, Adreno, or Tegra parts.
     
    RB_lashman likes this.
  2. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    And Lightmapping will still be faster than any ray traced solution for similar results.
     
    RB_lashman likes this.
  3. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Hey everyone, I just wanted to let you know that I've come across a new idea for SEGI and I'm going to strike while the iron is hot. So, I'll be working on that for a while. The update with cascades will be delayed a bit because of this (I'm sure you guys understand the importance of exploring new ideas while they're exciting and fresh). I'm just taking the first baby steps, but I'll let you guys know either way whether it ends up being promising or not!
     
  4. Frpmta

    Frpmta

    Joined:
    Nov 30, 2013
    Posts:
    461
    It is not like cascades are going to run away :D
     
    RB_lashman and sonicether like this.
  5. IronDuke

    IronDuke

    Joined:
    May 13, 2014
    Posts:
    132
    Strike while the iron is hot? I'll bring a blowtorch if that'll help. :D Have at it, man!

    --IronDuke
     
  6. MariskaA

    MariskaA

    Joined:
    Feb 17, 2017
    Posts:
    11
    Hello sonicether!

    Thank you so much for SEGI, the possibilities are so great. However, I'm coming to you with the hope you could help me with some problems...
    I know it's for the outdoor scenes mostly, but I wanted to use it in this case for a room, dynamically generated by a user who can chose the size and where are doors and windows and so on... The problem is I can't get a setisfying and realistic result. Here on the photo, the floor is fully reflective. But the colors of the reflections are weird (blue below red??) and is not like "a mirror" it's all deformed... Another thing is that there is this weird shadow around the furniture. (on the exemple, the furniture is against the wall, but it looks like it's not).

    Do you have any proposition to make that can help me with that?

    Thank you so much for your time!

     
    RB_lashman likes this.
  7. BruceJRBarratt

    BruceJRBarratt

    Joined:
    Jun 20, 2015
    Posts:
    46
    Thanks for the update @sonicether. Best of luck and if it doesn't work out it's still an amazing asset. I'm using it on a slightly fried intel hd5500. I mostly use half again the low voxel resolution on a 100 x 100 area. forest3.jpg
     

    Attached Files:

  8. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,447
    That's very painterly Bruce, love it. Whats the performance like, especially on an integrated GPU?
     
    buttmatrix and RB_lashman like this.
  9. buttmatrix

    buttmatrix

    Joined:
    Mar 23, 2015
    Posts:
    609
    I don't want to derail the thread, but this is incredibly beautiful +1
     
    Last edited: Jun 27, 2017
    RB_lashman likes this.
  10. CWolf

    CWolf

    Joined:
    Oct 24, 2011
    Posts:
    106
    I can't help on the weird colour reflections you're seeing but I know the actual reflections aren't meant to be crystal clear or mirror like - it's a limitation / intended trade off with the techniques used in SEGI. I believe it's due to the voxelation technique but don't quote me on that.
     
  11. MariskaA

    MariskaA

    Joined:
    Feb 17, 2017
    Posts:
    11
    Thanks for the reply! I've finally chosen to completely desactivate reflections from SEGI and to use a script just for that.
     
    Last edited: Apr 12, 2017
    CWolf and RB_lashman like this.
  12. BruceJRBarratt

    BruceJRBarratt

    Joined:
    Jun 20, 2015
    Posts:
    46
    Thanks guys :),

    I'm getting about 10fps with all the image effects and SEGI with a very dense forest in scene view (850 * 450). I know that sounds horrific but I think it's impressive that I can even move around with all this on such low end graphics. There are lots of things not optimized as well. The trees are attached to the terrain so they're using their rendering LODs for voxelization.
     
    SteveB and RB_lashman like this.
  13. yohami

    yohami

    Joined:
    Apr 19, 2009
    Posts:
    124
    Thanks for the update @sonicether. Here's a very, very early stage of my game using SEGI, just a week of work into this scene, six weeks total since we started. The character needs a redo - but just look at that light!
     
    SteveB, Lex4art, sonicether and 4 others like this.
  14. boundingbox

    boundingbox

    Joined:
    Mar 31, 2013
    Posts:
    25
    I know this is from a while ago but do you remember any specs on the scene you were testing? How many triangles were there, what was the frame buffer resolution how many cone samples per pixel were you using? I ask because I have a project with simple scenes (compared to most games) and a lower resolution frame buffer
     
    Nobrega_ and RB_lashman like this.
  15. OnlyVR

    OnlyVR

    Joined:
    Oct 5, 2015
    Posts:
    50
    That is correct. My apologies I wrote OTOY will provide the light baking solution in Unity. I was wrong. OTOY will add the solutiotion for creating kind of 360 VR file for Unity scene.
     
    Flurgle and RB_lashman like this.
  16. BruceJRBarratt

    BruceJRBarratt

    Joined:
    Jun 20, 2015
    Posts:
    46
    Hi Sonic, Any progress with your new idea?
     
    Flurgle, arnoob, SteveB and 1 other person like this.
  17. vonpixel

    vonpixel

    Joined:
    Oct 2, 2012
    Posts:
    31
    Has anyone had any luck doing a picture in picture camera with SEGI? It seems to not like it very much. Would it be possible to have that camera use the same voxel volume as SEGI? Right now they are just the regular non-segi enabled cameras and they look terrible in comparison.

    Any ideas? Tricks? Thoughts?
     
    RB_lashman likes this.
  18. RoyArtorius

    RoyArtorius

    Joined:
    Jul 11, 2013
    Posts:
    69
    Honestly, SEGI already does things that I just can't imagine not having in a game. It has actual real time reflections, emissive materials generating light, and shadows being lit up by bounced lighting. All of it with a stable 60 fps on my not-so-old but not-so-powerful computer. I'm more than happy with SEGI as it currently is. Anything else @sonicether develops for it is just icing on a delicious cake :D
     
  19. SAOTA

    SAOTA

    Joined:
    Feb 9, 2015
    Posts:
    220
    Love SEGI and use it for so many different experiments and now finally an app.

    Just my luck getting used to DX11 and totally ignoring that fact when porting to MAC. SEGI is one of my features to be left behind, which means GI differs massively between windows and mac versions. - Gained some performance on slower specced macs, so there's that ;)

    Does anyone know if METAL will allow us to use SEGI or something similar in the future? I rely heavily on runtime mesh generation and on windows SEGI is absolute magic!
     
    Flurgle and RB_lashman like this.
  20. bngames

    bngames

    Joined:
    Jul 3, 2012
    Posts:
    57
    When will it be updated for 5.6 ?
     
    castor76 and RB_lashman like this.
  21. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    1,399
    It would be amazing if SEGI can be used to bake the lightmap in runtime. I think it will have more application today than actual realtime due to its current performance issue.

    Having said that, I am ( and probably a lot others ) looking for some lightmap baker in runtime. Does anyone know if such exist for Unity? How about UE4? Does UE4 support runtime lightmap baking? Sorry if I walked off from the topic a bit.
     
    RB_lashman likes this.
  22. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    4,190
    The runtime cost is still voxelization at a significant percentage, so maybe you can have both by baking the voxel (and streaming?).
     
    RB_lashman likes this.
  23. Mauri

    Mauri

    Joined:
    Dec 9, 2010
    Posts:
    1,493
    We really should stop demanding things. I'm sure sonicether has to worry about much more important things right now. We don't know, if his new approach will result in something good to work with... Generally said, I don't really see the point in SEGI having a lightmap baking feature. Would that not make the whole point of having actual working realtime GI obsolete?

    Nope.
     
    RB_lashman likes this.
  24. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    1,399
    Baking lightmap runtime is still useful because of performance reason and procedurally generated levels.

    I don't mind sonic working on a new approach or another. I just wish he can come here more often and update us with the progress.
     
  25. thelebaron

    thelebaron

    Joined:
    Jun 2, 2013
    Posts:
    261
    Ill be honest, if segi had an offline baking option it would be pretty amazing too. I find segi gi much more pleasing than enlighten.
     
    buttmatrix and RB_lashman like this.
  26. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    1,399
    Yeah, exactly. Thing is SEGI needs to be worked on more to make it run better in terms of realtime performance, I am not saying it won't get there, but realtime lightmap baking feature is one of those thing that can be used right away and right now and it is practically useful. No other assets in the store has anything like this.
     
    mtornio and RB_lashman like this.
  27. buttmatrix

    buttmatrix

    Joined:
    Mar 23, 2015
    Posts:
    609
    This might be the first time on this thread someone has actually referred to the aesthetic value of SEGI as a static lightmap baker - honestly, I'm not sure why this isn't top priority. Right now, as a VR developer, SEGI is basically a screenshot tool for me, it simply is not performant enough for any realtime use case that I might encounter. What's more, SEGI seems to be at a standstill with respect to its development roadmap, without any indication that it will graduate to a full release and there is no ETA on voxel cascades.

    Even if the only thing I could use SEGI for would be to bake lightmaps, that would at least be a return on my investment.
     
    RB_lashman likes this.
  28. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,447
    OR @buttmatrix...

    ...single pass stereo rendering.

    SEGI is quite fast, just not when rendering both eyes. Sharing the data between the two cameras should be far easier to implement than a baker.

    It's killing me I probably won't be able to use SEGI in my VR work currently...
     
    TerraUnity, buttmatrix and RB_lashman like this.
  29. buttmatrix

    buttmatrix

    Joined:
    Mar 23, 2015
    Posts:
    609
    Yeah, that is a fair point. It seems many image effects still do not unanimously support single pass stereo, which is surprising given Unity's advocacy for VR. I don't think SEGI supports single pass yet (?) but I know it's been requested before.

    I recall there was an interesting archviz builder project that was posted on here a while back, and it looked great, but it was very modular and not really typical of most game scenarios.
     
    RB_lashman likes this.
  30. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,447
    No it doesn't, but it was on Cody's roadmap eons ago, so at the very least he's aware (or was made aware early on)
     
    RB_lashman likes this.
  31. Mauri

    Mauri

    Joined:
    Dec 9, 2010
    Posts:
    1,493
    He's working on new approaches, because he simply has no other choice right now. As sonicether said earlier, SEGI is currently "teetering at the edge of efficiency and visual robustness". That means "a step towards efficiency breaks visual robustness, and a step towards visual robustness breaks efficiency".

    I'd love to see an option to bake lightmaps, too, even though it would lead SEGI and its idea (being able to use true realtime GI inside Unity, which doesn't require baking at all!) ad adsurdum. But it's clearly not a high priority right now - for obvious reasons.

    (Also, regarding baking - it's on the Roadmap for ages now)
     
  32. buttmatrix

    buttmatrix

    Joined:
    Mar 23, 2015
    Posts:
    609
    I appreciate these opinions, and I don't disagree. Just to be clear, what I'm saying is that, if progress really is at an impasse with respect to performance and visual quality - and the experimental approaches that are under consideration are only going to contribute marginal improvements, even if successful - then now might be a good opportunity to *actually* start implementing some of these major features like single pass stereo and lightmap baking that have been on the roadmap for some time now.

    EDIT: SEGI one year anniversary is June 22, 2017
     
    Last edited: May 1, 2017
    RB_lashman likes this.
  33. Obsurveyor

    Obsurveyor

    Joined:
    Nov 22, 2012
    Posts:
    272
    I'm pretty sure it's been on his roadmap since he introduced SEGI.
     
    RB_lashman likes this.
  34. yohami

    yohami

    Joined:
    Apr 19, 2009
    Posts:
    124
    More progress on my game using SEGI - the character is a placeholder. @sonicether let's finish this thing!

     
    Malbers, SteveB, buttmatrix and 3 others like this.
  35. yohami

    yohami

    Joined:
    Apr 19, 2009
    Posts:
    124
    How about if SEGI does cascading - baking the cascades that are far, and updating the scene in your proximity once every X frames, could be configurable.

    Having a version that doesn't die on the distance would make this release ready, even if it just bakes the whole thing and it's not dynamic. Then keep iterating on future releases while you laugh your way to the bank.
     
    Last edited: May 2, 2017
    TooManySugar and RB_lashman like this.
  36. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,447
    As I said lol. (Reread what I wrote)
     
    RB_lashman likes this.
  37. amasinton

    amasinton

    Joined:
    Aug 12, 2006
    Posts:
    131
    Just ... testing ... to see if SEGI would run on OS X now that Unity 5.6 is doing editor support for Metal.

    Unsurprisingly, no - it doesn't work. No worries. Just testing, just for fun.

    When I imported the package into a clean project I got no compile errors at all (which is better than when I tried under 5.5).

    Except that the SEGITraceScene and SEGIVoxelizeScene shaders both report a warning: 'Program uses geometry shaders automatically setting target to 4.0'

    However, when I turn on the SEGI script on the camera (or when I assigned it the first time) Unity crashes immediately.

    The log reports: MTLTextureDescriptor has invalid pixelFormat (0).

    Google says this happened to someone when they were trying to compute a depth render texture with 4x MSAA. Does SEGI try to do anything like this? Just wondering if this error were (magically and impossibly) relatively trivial...
    ???

    SEGI on my Windows setup, of course, is a marvel that never ceases to delight me!
     
    RB_lashman likes this.
  38. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    1,399
    Try it with msaa turned off in your unity setting and see what happens?
     
    RB_lashman likes this.
  39. elbows

    elbows

    Joined:
    Nov 28, 2009
    Posts:
    2,231
    Metal in Unity does not support pragma target 4.0 or 5.0. It supports what they have called 4.5, and I'm afraid 4.5 does not currently support geometry shaders. Therefore I do not expect Metal compatibility to be as trivial as you had hoped.
     
    RB_lashman likes this.
  40. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Hey guys, I thought I'd let you know how everything is going.

    So, the new thing I've been working on is using Per-Pixel Linked Lists (PPLL) to voxelize the scene instead of directly rasterizing the scene into a 3D texture. This would involve storing fragments from three major directions (forward, left, and up) into a data structure where I can freely access all fragments that are stored in a given coordinate. While I was working on this, I decided to go ahead and try using this data directly instead of using voxels for cone tracing just as an experimentation. This approach could be described as rasterized pathtracing with global ray bundles. It yielded some interesting results, but overall cone tracing still has more strengths, so unfortunately that didn't result in anything beneficial.

    Using PPLLs to voxelize opens up a few more possibilities (like solid voxelization), but it's slower since the scene has to be rasterized three times instead of once. I'll keep this idea on the shelf for now because it's slower, but who knows, it may come in handy in the future.

    Oh well, it's just another thing I can be sure about whether SEGI would see significant improvements or not from it.

    For the next update with cascades, I just need to update the reflection cone tracing to the new data structure and then I'd say it's ready for initial testing. Once again, cascade behavior will not necessarily be the intended mode of use, since it's still rough, but I think I need some feedback on it before proceeding.
     
    ksam2, BruceJRBarratt, SteveB and 9 others like this.
  41. yohami

    yohami

    Joined:
    Apr 19, 2009
    Posts:
    124
    Thanks for the update @sonicether. Many of us don't need the realtime GI, and could use one that bakes the cascades on the distance, and refreshes the scene around you once per frame. Something like that would beat all the other lighting solutions in the store at the moment.
     
  42. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    4,190
    Also have an experimental folder for those who can actually offer insight with playing with new features (aka not me). After all even with the first beta people found quick way to optimized and tune the performance to something usable (that infamous trigger script that let you have only hick up when you move!). Who know what user came from with, even with broken features!
     
    RB_lashman likes this.
  43. amasinton

    amasinton

    Joined:
    Aug 12, 2006
    Posts:
    131
    Ah well. Thanks for the explanation, though. That's very helpful. Maybe some day Metal will do geometry shaders...
     
  44. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,447
    I appreciate the update Cody. Can you at least address single pass stereo rendering? Is it non-trivial or could it in fact be implemented easily? I ask for all the VR dev's here as, despite everything SEGI is quite useable for me in production, barring this one thing.
     
    madrobotMK and RB_lashman like this.
  45. amasinton

    amasinton

    Joined:
    Aug 12, 2006
    Posts:
    131
    This is still very WiP, but I just wanted to post this screenshot here.

    @sonicether SEGI is quickly replacing my conventional rendering pipeline. It's a real joy to use and suits my needs so very well already. It's a marvel. Thank you.

    (Also, HUGE thanks to @JohnRossitter for Helios and its gorgeous screen capture abilities.)

    NAFromNE2k.jpg
     
    yohami, OnlyVR, ftejada and 7 others like this.
  46. OnlyVR

    OnlyVR

    Joined:
    Oct 5, 2015
    Posts:
    50
    Cody, when you decide to optimize SEGI for VR could you check the latest NVIDIA "VRWorks" that already implemented the single pass for VR(and other very useful features) and available in "Unity 2017" ?
    https://developer.nvidia.com/nvidia-vrworks-support-unity-engine-now-available
     
    madrobotMK, RB_lashman and SteveB like this.
  47. BruceJRBarratt

    BruceJRBarratt

    Joined:
    Jun 20, 2015
    Posts:
    46
    Thanks for the update @sonicether. Sounds like you have some potentially useful stuff for later. What do you mean by solid voxelization? Would that mean that SEGI could do less work per frame outside of voxelization? Could we then get potentially get higher performance by lazy updating?
     
    arnoob and RB_lashman like this.
  48. BruceJRBarratt

    BruceJRBarratt

    Joined:
    Jun 20, 2015
    Posts:
    46
    Oh sorry rereading that you say rasterization is done 3 times so nevermind. I guess you could use the voxel data for other as yet unknown processes is what you're saying.
     
    RB_lashman likes this.
  49. edan_kuva

    edan_kuva

    Joined:
    May 11, 2017
    Posts:
    3
    Does anyone get it working with Standard Surface Shader? My application requires some GPU vertex transformation. It doesn't seem to be working with SEGI. Is it because of the voxelization? Any one knows how to get it working?

    CC @sonicether
     
    RB_lashman likes this.
  50. FPires

    FPires

    Joined:
    Jan 5, 2012
    Posts:
    151
    Not clear on this - is SEGI currently working with 5.6 in any platform, or not at all?
     
    RB_lashman likes this.