Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[RELEASED] SECTR STREAM: Seamless Scene Streaming

Discussion in 'Assets and Asset Store' started by MakeCodeNow, Feb 21, 2014.

  1. mansonet

    mansonet

    Joined:
    Jan 11, 2014
    Posts:
    8
    Hi
    Thanks for taking a look into this. Also i've found a workaround which seems to be working fine so far, you need to activate the hidden root gameobject in each sector scene file, looks like unity recognize static batching without any issues.

    In the other hand, i'm trying to minimize the lag when a sector is being loaded, and i've found that the whole "parenting" and "activating" chunks code, leads Unity to re-calculate colliders that are being marked as static.

    Screen Shot 2014-08-08 at 5.07.28 PM.png

    Am i doing something wrong?

    Thanks!
     
  2. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Hmmm, tricky tricky. To make sure I understand your profile, you're seeing that large amount of time in Chunk::FixedUpdate() when you make the change to enable the root node in the scene in order to fix static batching, right? In other words, with stock SECTR you don't see that big physics rebuild time, but you also don't see static batching working correctly. Is that right?

    If so, it's a tricky intersection of Unity behaviors. The main reason for all of the activation and parenting is that it allows me to have more control over when chunks actually incur most of their main thread cost. In the original version, chunks were exported active and just came in whenever LoadLevelAdditiveAsync finished. Unfortunately, this meant that many chunks could activate in a single frame, which caused CPU spikes due to excessive first frame script execution, object construction, etc. The current behavior exists so that I can control the number of activations per frame, which helped in a lot of cases, but appears to interfere w/ static batching (an issue I wasn't aware of until now).
     
  3. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    You can also try to replace your SECTR_Chunk::parentChunk() with this:

    Code (csharp):
    1.  
    2. private void _ParentChunk()
    3. {
    4.     _FindChunkRoot();
    5.     if(chunkRoot)
    6.     {
    7.         //Parent thenewly loaded chunk to ourselves
    8.         Transform chunkTransform = chunkRoot.transform;
    9.         chunkTransform.parent = transform;
    10.         if(recenterChunk)
    11.         {
    12.             if(chunkTransform.localPosition != Vector3.zero)
    13.             {
    14.                 chunkTransform.localPosition = Vector3.zero;
    15.             }
    16.             if(chunkTransform.localRotation != Quaternion.identity)
    17.             {
    18.                 chunkTransform.localRotation = Quaternion.identity;
    19.             }
    20.             if(chunkTransform.localScale != Vector3.one)
    21.             {
    22.                 chunkTransform.localScale = Vector3.one;
    23.             }
    24.         }
    25.         loadState = LoadState.Parented;
    26.     }
    27. }
    28.  
     
  4. mansonet

    mansonet

    Joined:
    Jan 11, 2014
    Posts:
    8
    Hi, sorry for the lack of detail in the profile picture, the profile is after enabling the root of all sector chunks. But if i leave the root node as disabled, the profiling time for Chunk::FixedUpdate is the double in time. Static batching works consistently among all chunks only if i have all chunk root nodes activated.
    After some more tests, i think what is causing this spike is the call to _ParentChunk(), maybe i'm wrong about it but i think since most of the game objects inside of the sector are static they shouldn't move around the scene, and in the case you did move them, which i think is happening when you re-parent the sector, Unity needs to re-calculate some collider information.
     
  5. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    So I totally get you on needing the root to be active for the static batching to work. I'll take that as a given in Unity and see what I can do about balancing it with my other options. I may have to add an export time option or something.

    I definitely think that _ParentChunk is the cause of the FixedUpdate time as it both parents and moves the SECTR from the origin to the correct place* and that would certainly cause static colliders to rebuild. What I don't understand is why the time is doubled if you leave the root node disabled. I would expect it to go away when the root is disabled and appear when it's enabled. My reasoning is that I always _ActivateChunk() after _ParentChunk(). I would think that the Chunk is saved with an inactive root, then _ParentChunk would happen on an inactive object, which should not cause any colliders to rebuild. The activate would cause the colliders to get built, but presumably in a similar fashion to when the scene is loaded normally. I have no idea why an inactive root would cause the colliders to be rebuilt 2x... Are you sure that's the case?

    * This step isn't strictly necessary and wasn't part of the original SECTR. I originally just exported Sectors where they were, but one client wanted to be able to re-use the same Sector many times in a single scene. I may need to make this also be an export time option.
     
  6. mansonet

    mansonet

    Joined:
    Jan 11, 2014
    Posts:
    8
    Yes, i don't know why it's taking more time, i thought that i could be because the activation and re-parenting happens at the same time, but it doesn't look like it's in that way.
    In the old Profile screenshot "Mesh.Bake Scale Mesh..." takes around 27ms, but in this one takes 52ms. For this test i haven't done any changes to SECTR code and the chunks root nodes are saved as inactive.

    Screen Shot 2014-08-08 at 8.48.28 PM.png
     
  7. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    That is totally banannas. I'll dig in more tonight/tomorrow, but as a last request, can you try the code I pasted a few posts ago?

    BTW - if you want me to send you some preview code as I go about addressing this, email support@makecodenow.com and reference this thread.
     
  8. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Hmm, looking at your profiles again, I see one important difference. In the first profile, the time is directly in _FixedUpdate() which I assume is really from _ParentChunk()*. In the later profile, all of the time is coming from the call to Activate from within _FixedUpdate, which must mean it's happening from _ActivateChunk(). Why _ActivateChunk takes 2x as long as _ParentChunk() is a complete mystery, but definitely a problem I'll have to address.

    * I'll be curious to see if this time goes down with my updated code pasted above.
     
  9. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Ok, I think I have an update that addresses all of your issues and preserves all of the good parts of my previous behavior, in particular gating of scene loads. I'd love to test this with you directly; please email support@makecodenow.com if you're interested.
     
  10. Hamesh81

    Hamesh81

    Joined:
    Mar 9, 2012
    Posts:
    405
    Is there a way to set priority for chunks? For example you want to stream in 3 chunks at once but you want to load chunk 1 in first before chunk 2 and 3. Can this be done?
     
  11. Rico21745

    Rico21745

    Joined:
    Apr 25, 2012
    Posts:
    409
    Very interested to see the fix for masonet's problem since it sounds like a significant performance boost when streaming in chunks. I'm currently getting a lot of stuttering when loading in new chunks and I saw the same issues he spotted but I was assuming it was on my end. I do have a ton of meshes/colliders marked as static so I'm guessing we're experiencing the same issue.

    My thought was to try and change the SECTR activation code for the chunk game objects to activate only X per frame or something, so I could have the chunk being loaded in over longer time rather than all at once. Since my view distance is somewhat limited, I don't think players would notice this happening and even if they did, the stuttering is far more noticeable a problem.
     
  12. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Right now there is no priority mechanism. It's left up to Unity to order the loads. SECTR will guarantee that only one scene is materialized per frame, which cuts down on hitching. If you really needed to you could expand on this logic to implement some scheduling, but it's tricky enough that I probably won't add it to the base product due to maintenance overhead.
     
  13. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    We've been working together on this. I have one change that takes care of about 50% of the static collider issue. Now there's another issue in the way, where apparently setting the parent of an object forces a collider rebuild, even though the actual positions aren't changing at all. I have two ideas for how to address this issue and am debating pros and cons. If anyone wants to help test before the next release, just email support@makecodenow.com.
     
  14. julianr

    julianr

    Joined:
    Jun 5, 2014
    Posts:
    1,212
    Hi. Great product. My scenes are quite large with lots going on, so I've had to reduce the number of assets, not from a SECTR Complete point of view, but from a development point of view where my i5 machine lags when I try and copy and paste large assets around that I've created on another scene, before I can even use SECTR Stream. Looking forward to Unity 5 when its 64 bit on the editor, which will improve performance. My question is; will we be able to use SECTR Stream with Unity 5s multi scene editing? Not sure how its all going to work yet, whether you can save all scenes into one scene, then use SECTR Stream to cut them up?
     
  15. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    13,286
    Can i stream very large maps ?

    For example now i have very big maps with lots of things and my own activation/deactivation mechanism based on camera distance (not occlusion for now, i guess Unity does some occlusion for objects not in camera view)

    Is it easy to have SECTR stream the next stage as i come close to a load next level portal for example ? And how will SECTR know which level to start loading in the background ?

    Also how does it do the loading ? Is another thread used in the background ? Since pre-loading a big level seems to pause the game, does SECTR offer a totally seamless loading ?

    Also the items disabled could have some important scripts attached, is there a way to control what should not be culled in the scene for certain critical objects ?

    And finally how does SECTR differ from Unity's auto occlusion ?

    Thanks
     
  16. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Good question. I plan to keep SECTR compatible with 5 in general. Multi Scene editing is happening sometime after 5.0, and that could be a while yet. I'll be able to be more specific once that's in a beta that I can actually run against. Most likely it'll just be a cleaner editor experience for the existing chunk stream process.
     
    julianr likes this.
  17. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    SECTR 1.1.1 is now live in the UAS, just in time for the end of the August Madness sale. It includes some bug fixes, and optimizations to Stream hitching.
     
  18. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Wow. Lots of questions!

    There's no limit to the size of maps you can stream, other that memory and floating point precision. SECTR STREAM has two parts, Chunks, which manage the loading and unloading of parts of the scene, and Loaders, which contain the logic for when to load and unload Chunks. SECTR comes with a number of Loader components that can load based on basic distance, portal connectivity, airlock doors, etc. You can also hook in your own Loader logic.

    The loading logic sits on top of Unity's LoadLevelAdditive(Async for Pro users). It adds a lot of logic on top of that to do things like limit the number of scenes that are added per frame, handle streaming lightmaps from individual chunks, etc. The loading is as seamless as it can be, but it's not guaranteed to be seamless no matter what. Unity's loading does some main thread work, and anything SECTR or your scripts do in Start/Awake/Enable has to run in the main thread and can account for hitching. Of course I do whatever I can to keep the SECTR costs to a minimum, hopefully Unity will improve their end in 5.0 and the rest is up to you :)

    STREAM exportes Sectors and their Children. Anything you leave in the main scene will not be exported. There are support components to allow these global objects to be "Sector Aware", including a Hibernator component that will shut down it's object if it finds itself in a part of the world that's unloaded. Any references between exported Chunks are up to you to restore.

    Check out the VIS manual for a detailed comparison of VIS and Umbra in Unity.
     
  19. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    13,286
    Thanks for the clarifications :)
     
  20. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    SECTR 1.1.2 is now live in the asset store. This follows 1.1.1 with some major reductions in CPU time spent during chunk load in STREAM. SECTR time when loading chunks is now close to zero and the loading logic allows Unity to do as much asynchronously as possible. I think this is very close to as fast as one can get doing additive loads in Unity 4.x.
     
    julianr likes this.
  21. Meceka

    Meceka

    Joined:
    Dec 23, 2013
    Posts:
    423
    Is it faster in unity free too? Thanks.
     
  22. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Yes, but you'll still see hitching on larger loads because free loads everything synchronously.
     
  23. Meceka

    Meceka

    Joined:
    Dec 23, 2013
    Posts:
    423
    Thanks. A recommendation, reduce sample levels file size with reducing texture sizes of lightmaps and other stuff. This asset shouldn't take 230mb size.
     
  24. Woodlauncher

    Woodlauncher

    Joined:
    Nov 23, 2012
    Posts:
    173
    How does Stream deal with texture unloading? Read a post by the Frontiers dev and he couldn't find a solution for his open-world game, he ended up loading all the textures into memory.
     
  25. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    I just call UnloadUnusedResources after I destroy the chunk. Works great for unloading meshes, textures, etc.
     
  26. Meceka

    Meceka

    Joined:
    Dec 23, 2013
    Posts:
    423
    How can I set it to unload resources right after destroying a chunk? Which script and function does destroying and where should I add UnloadUnusedResources line?

    I thought sectr was doing this automatically. My game sometimes becomes laggy after driving around map. Probably it's filling up memory as it's not unloading textures.

    Thanks.
     
  27. Woodlauncher

    Woodlauncher

    Joined:
    Nov 23, 2012
    Posts:
    173
    I'm pretty sure that by "I just call UnloadUnusedResources" MakeCodeNow means it is indeed done automatically for you by the asset.
    Thanks.
     
  28. Meceka

    Meceka

    Joined:
    Dec 23, 2013
    Posts:
    423
    Thanks Woodlauncher, you are probably right ;)
     
  29. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Quite so :)
     
  30. reocwolf

    reocwolf

    Joined:
    Aug 1, 2013
    Posts:
    182
    I'm making an open world game that is divided in regions. For example: One of the scenes is an entire peninsula with 3 villages. There's several "transporters" placed strategically that ask the player if they want to travel to somewhere else. And that loads another region (maybe a city or something). So apart from that those regions are pretty much very open worldly (and big enough). Also I made my map using Wold Composer and Terrain Composer so it's multiple terrains stitched together and organized with x,y coordinates suffixes.

    So, how well do Vis and Stream work in an open world? Can I set it up for L.O.D functionality? Would you recommend this product for my open world game?

    Can it automatically create regions based on my tiled terrain or even slice them up even further?

    I just hate baking and this seems like a beautiful pack of tools. As long as I can make stuff appear in the distance without it looking too weird and control the L.O.D. as the player approaches we are good. This seems like a very very nice product you've created.

    I think I'm just worried that it wont work as I expect after I buy it.

    BTW I know Unity 5 will have level streaming and scene management features but I think that you'll keep up with whatever they bring and make this even better. So, that's why I'm still buying this if it does indeed work well for my game.
     
  31. Rico21745

    Rico21745

    Joined:
    Apr 25, 2012
    Posts:
    409
    I can vouch for this asset. It can handle large scenes and is great to work with. Awesome support too. Just do it, one of the best purchases I've made.
     
    reocwolf likes this.
  32. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Hi, Reco. You game sounds like exactly the kind of game SECTR was designed for.

    SECTR includes support for terrain composer and will sectorize using the TC information. It will also preserve TC data during STREAM import and export.

    SECTR VIS supports distance based LOD, which is simple enough. SECTR STREAM has a similar feature called a proxy, which is a mesh that is displayed when a Sector is unloaded and automatically hidden when the Sector is loaded. It's a great fit for things like a walled village where in the distance you just want to see the outer walls but then load the detailed version of the whole thing when you get close.

    The plan is to keep SECTR updated with 5.x. It's hard to commit to exactly how it will handle features that are pretty far away (like the scene management), but the goal is to keep things compatible.
     
  33. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    :D
     
  34. reocwolf

    reocwolf

    Joined:
    Aug 1, 2013
    Posts:
    182
    Thanks guys. Going to buy it right now. XD Sounds perfect.
     
  35. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Awesome. FYI that I'm fixing an issue in VIS culling on terrains. It can cause bad performance in some scenarios. Hope to have a new version pushed tonight.
     
    reocwolf and julianr like this.
  36. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    SECTR 1.1.3 is now live on the UAS. Includes bug fixes and some terrain window improvements.
     
    reocwolf likes this.
  37. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    SECTR 1.1.4 is now out on the UAS. Includes AUDIO improvements, lots of bug fixes, and early Unity 5 compatibility!
     
  38. loadexfa

    loadexfa

    Joined:
    Sep 2, 2008
    Posts:
    214
    I seem to be doing something wrong with creating my sectors. When I open the Stream window I can only export one of my sectors, the rest don't have the export option. Do you know what I could be doing wrong?
     
  39. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Only static Sectors can be exported.
     
  40. loadexfa

    loadexfa

    Joined:
    Sep 2, 2008
    Posts:
    214
    Ahh, makes sense. How do I specify a sector is static?
     
  41. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Just make the game object its on static.
     
  42. loadexfa

    loadexfa

    Joined:
    Sep 2, 2008
    Posts:
    214
    Doh! So simple. Thanks for your help, that fixed my issues. :)
     
  43. Sendatsu_Yoshimitsu

    Sendatsu_Yoshimitsu

    Joined:
    May 19, 2014
    Posts:
    691
    So first of all, this is an amazing product- I only purchased SECTR complete this evening and it's already solved several problems I've been smashing my head against for months without progress.

    Now this may be an obvious question, but one of the things I'd like to do is simulate NPC movement across sectors. To that add, is there any sort of pre-existing "pathfinding" functionality, by which I can get a list of the doors and sectors that connect one sector to another?
     
  44. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    New SECTR update out now. Includes improvements to terrain splitting and culling, plus some Unity 5 updates.
     
  45. delinx32

    delinx32

    Joined:
    Apr 20, 2012
    Posts:
    417
    Is it editor only, or can you set up sectors at runtime?
     
  46. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    You can set up Sectors and Portals and all that at runtime, however for streaming specifically the chunks must be exported into sub-scenes and that requires functions that are Editor only.
     
  47. eridani

    eridani

    Joined:
    Aug 30, 2012
    Posts:
    655
    Hello, some questions for you:

    Can this asset be used on Unity Free to create a huge open world driving game (think "Mad Max") on older Mobile devices (iPhone 4/S, etc)? How will it handle huge flat environments with line-of-sight to the horizon? Will there be hitching all the time as new sections of the map load while driving (are workarounds such as tunnels necessary between sections)? Can NavMesh based enemies chase the player across newly loaded sections?

    Thank you!!
     
  48. MakeCodeNow

    MakeCodeNow

    Joined:
    Feb 14, 2014
    Posts:
    1,246
    Hi! Good questions.

    SECTR works without Unity Pro, but you really need Pro to make a hitch free open world game (on any platform) because Unity binds all of the async APIs to Pro.

    In Unity 4, you can only have a single, immutable, global nav mesh, so you can't really load in new sections. Unity 5 looks like it'll fix that issue, but I haven't taken that part of the API through it's paces yet.
     
  49. eridani

    eridani

    Joined:
    Aug 30, 2012
    Posts:
    655
    Ok, thanks for the quick response! Does having a huge nav mesh kill performance on mobile?

    Also, performance-wise, is there a huge difference between completely unloading a section once you move through a tunnel portal (using SECTR STREAM) or just occluding that whole section (using SECTR VIS). Using SECTR VIS would eliminate any hitching in this case right? I'm thinking a large desert-like scene with not that many complex meshes, so memory usage for even the entire map shouldn't be all that huge.
     
  50. red2blue

    red2blue

    Joined:
    Feb 26, 2013
    Posts:
    200

    Hi, sorry for hijacking here, but I am also interessted in open world with Unity Free. Is this a problem regarding the nav mesh, so it could be fixed with another pathfinding solution (I use A*), or misunderstood I something? Sorry for the maybe dumb question.

    Thanks a lot!