[RELEASED] SECTR STREAM: Seamless Scene Streaming

What you're trying to do should "just work". It's designed to produce a standard Unity mesh so that you can do exactly the kinds of things you're trying to do. A few things to check out...

The proxy mesh verts are in local space. If you built them from an object with a large scale, it might be that they are very tiny when applied to a unit scale GO. Simply scaling the GO up may resolve that.

Regarding it being pink, you need to essentially copy the Chunk's ProxyMaterials array into the MeshRenderer's Materials array. Have you tried that?

In general, though, glad to hear you're getting good results from the feature. What context are you using it in?

 

I have some questions, Ive moved from using just viz to using stream and am trying to understand if the issues I'm seeing are my setup or expected.
I'm on Unity 4.6 Pro
-Is it normal to export a chunk and it takes 20+ minutes, even changing one mesh or option?
-everything is slow now, saving my game, even exiting Unity takes 4 minutes, My full exports are also MUCH longer.
-Ive read earlier posts about what I need to look out for for streaming hitches, but this is beyond hitching.!i hang up for seconds sometimes.
- how much does fhe exact shape of the blue bounds matter? Some have some overlap, others have a good amount of overlap. Should we strive for no overlap?
- Does async loading happen automatically with Pro or do I need to set it up?
Thanks!

 

What you are describing is definitely not normal, though I'm not sure why it's happening. Answers to you individual questions:

* Exporting is really just saving the current scene and then deleting everything that's not part of the chunk in question. It's telling that exporting and saving the scene are all slow. Are you scene files large? I wonder if there are a ton of hidden objects from another asset that don't play well with SECTR.

* A full export is just a matter of exporting each chunk, so if individual chunks take forever then it makes sense that all chunks would be a multiple of that. As another idea, do you have any assets that perform custom work on scene save or build?

* I wouldn't worry about streaming hitches until the previous issue is resolved. Something has gone very wrong in the scene files and we should figure out what that is.

* The amount of overlap doesn't matter much. All that really matters is how many sectors are getting activated at a time.

* Async is automatic if you have pro. However, in the Unity disables quite a bit of streaming logic when playing in editor, and 4.x is rather worse than 5.x.

Lastly, I know you've been using SECTR for quite a while. What version are you using?

 

SECTR 1.1.5a is now live on UAS. It includes some bug fixes, major reduction in garbage generated by Member update, and an optimization to how STREAM reclaims memory.

 

I see huge spikes in the editor when loading new sectors, but figured that was just in the editor. But in a Mac Standalone build (running on a high-end 2013 MBP), I do get a significant spike -- the loading process uses 48ms for a moment (see screenshot). Is that normal? I worry it's going to be much worse on mobile platforms. Is that based on the size/complexity of the sector?

 

The spikes are due to work that happens on the main thread. They will probably be worse on mobile, but not necessarily because of architectural differences. Here are the primary causes:
* Resource finalziation. This is work Unity needs to one the main thread after the async work is done. It's usually the #1 cause of spikes (as is the case in your snapshot), and is roughly proportional to the number of assets being loaded. If you have a fairly recent SECTR, I do some work to spread out the time as much as I can by only creating one chunk per frame, but there's only so much I can do. The major improvements in this area involve going from 4.x to 5.x, because Unity dramatically improved the quality of their async loading code in 5.x.
* Script startup logic. Script logic in Awake/Start/Enable is called on the main thread for every object in the chunk, all at once. If you have scripts that do a lot of work in start/enable, then you want to minimize that where possible.
* SECTR chunk init. This is really a special case of the above, and I've worked hard to ensure that it's very close to zero. You can see that it's very close to zero in your profile.

 

thanks for the help! Answers in Bold

What you are describing is definitely not normal, though I'm not sure why it's happening. Answers to you individual questions:

* Exporting is really just saving the current scene and then deleting everything that's not part of the chunk in question. It's telling that exporting and saving the scene are all slow. Are you scene files large? I wonder if there are a ton of hidden objects from another asset that don't play well with SECTR.
the levels are from 10mb to 65mb, they are unoptimized. I should probably so a solid optimization pass before. is 65mb a large size for SECTR?

* A full export is just a matter of exporting each chunk, so if individual chunks take forever then it makes sense that all chunks would be a multiple of that. As another idea, do you have any assets that perform custom work on scene save or build? No, Nothing custom. I had all my levels in one large 'Level1' file, split with numerous portals. Now I have a master level1 file with a bunch of chunks that I import to work on then export when done. Is this right?
I assume I still always work in my master file...

* I wouldn't worry about streaming hitches until the previous issue is resolved. Something has gone very wrong in the scene files and we should figure out what that is. I agree! I am learning SECTR streaming, so was looking for a baseline.

* The amount of overlap doesn't matter much. All that really matters is how many sectors are getting activated at a time. I only ever need 3 at once. Current and one behand and front.

* Async is automatic if you have pro. However, in the Unity disables quite a bit of streaming logic when playing in editor, and 4.x is rather worse than 5.x.

Lastly, I know you've been using SECTR for quite a while. What version are you using?
I am on the latest as of last week, updating to the current one now.

 

SECTR doesn't especially care how big your files are. It'll work with whatever you throw at it, but of course it's limited by the performance of basic Unity functions like scene saving. Because Unity .scene files are just descriptions of what goes where, I worry that having single chunks as large as 65 MB indicates some kind of bloat in the files. If you can, open up one of the larger ones in the editor and in a text file. Check to see if there are nodes that don't make sense. There may be many hidden nodes that you can only see in a text editor. I suspect something along those lines is the root problem.

Working out of the master scene is ideal, and yeah, just import and export as needed. You can work in the sub-files, you just have to be a bit careful to keep everything parented properly.

Prev/Current/Next room is an ideal approach for streaming. It should work well once we have the other issue ironed out.

Gtk that you're on latest. 1.1.5a will help give a bit more performance but probably won't fix whatever underlying problem is bloating your scenes.

 

Thanks! I am seeing something else, when I import an area, my entire editor slows down, 3 sec lag when clicking, everything is sluggish. Does it do updating or anything during editor use? This isn't during play, just regular editor work. I'm on the latest SECTR.

 

After import nothing extra happens. It's as if you'd never exported it. I think the sluggishness is another sign that these scenes somehow have tons of invisible nodes. Probably time to take a look at the scene in a text editor and see if you can find the things that shouldn't be there.

 

Yeah thats my next move... Thanks!

 

@MakeCodeNow : A quick question for you, I have a level made of 9 unity terrains, put together with TerrainComposer.
Everything is in a single scene file, how would Sectr stream this ?
Just tryng to understand how would this work ?

Thanks

 

First, use the included Terrain tool to create sectors for each terrain. That tool is terrain composer aware, so just check the Connected option and follow the prompts. Then, add a Region Loader component to your player. Lastly, use the Stream window to export for streaming. Then run!

PS - you can get a lot more elaborate than this, but the above is the basics.

 

looking at the chunk level file size, its huge! That makes sense why its going nuts... you say open it in a text editor, which file? the level files seem encrypted

 

It's the .unity file for the chunk scene. You my have to change your Editor Preferences->Asset Serialization Mode to Force Text to be able to open it in a text editor.

 

Thanks for your help! Working great now

 

Interested in SECTR. I've read about using SECTR Stream for large open worlds.

Are there any examples/videos available for this specifically?

Thanks

Edit: Oh, and how's Unity 5 Compat?

 

The best examples are from real games like Firewatch and Mute, both of whom use SECTR for pretty elaborate large worlds. I haven't had time to build any dedicated tutorial videos around it, but it's pretty simple to setup.

The only real issue is if your open world is so large that you'll need recentering tech to avoid physics precision problems. Unity's additive level code and shifting make for really bad performance so much so that it's not worth supporting in SECTR.

Unity compat is just fine. I have a couple of open bugs for 5.0, one is todo w/ VIS performance (which I cannot work around) and the other is todo w/ STREAM chunks not loading (which I can work around).

 

Firewatch looks great. From some scenes in that video, it looks like what I'm looking for in terms of scale.

How is it handles? Obviously with large landscapes, unless using fog, it's a bit hard to have sections of terrain just turned off completely. Is it better to mix with LOD, say... have a low detail terrain LOD that deactivates at a certain distance, and the Stream section loads in at that same spot, or is all that handled in SECTR? At the moment I have 16 terrains at about 500 x 500, because I've been testing performance (which gets slaughtered with adding any trees right now lol).

It looks like SECTR is most likely what I'll be looking for, mixed in with InstantOC, could drastically improve my performance.

 

Firewatch uses many techniques, including fog and all lot of just really good art design to break up sightlines. You can also use a grid of terrains that are always loaded and just use very aggressive LOD or you can stream in distant terrains and use the chunk proxy feature to show low res geo before the real geo is loaded. The right approach depends a lot on your game.

If you want occlusion culling, just get SECTR COMPLETE. That will give you stream, VIS (for OC) and AUDIO

 

I already have IOC, which is why I mentioned it, but I'll likely grab SECTR Complete anyway.

Alright, I look forward to having a mess around with it when I get it, expect to see me around here asking stupid questions when I do haha.

Thanks MCN.

 

SECTR 1.1.5b now live on the UAS. Includes a critical bug fix for 1.1.5a.

 

Hello, I just realised that sectr_sector components cause a huge slowdown in unity editor scene while moving gameobjects with transform gizmo

Since sometime I had that performance problem in editor scene, I was disabling parent gameobjects that I wasn't modifying to save some performance. Today I realised that just disabling 12 sectr_sector scripts in my large scene improves editor scene performance while moving other gameobjects from 3-4fps to 60fps.

 

Sectors need to have accurate bounds, which means that in the editor in edit mode they recompute the bounds from their children during every update (which is once a frame when dragging objects). It's usually not as expensive as you're seeing, but if you have fairly complex sectors or an older computer, it could add up.

The current algorithm is somewhat wasteful as most of the time the children haven't changed. Unfortunately Unity doesn't give any good callbacks for when the children of an object change, so I have to do this current logic. As you saw, you can work around it by disabling the SECTR_Sector component of expensive Sectors.

Do you have 1.1.5b? I rolled out some performance improvements to Sector and Member updating in 1.1.5a and 1.1.5b that might also help you.

 

Hello, performance drop happens on all our computers including my pc that has i7-4790k overclocked.

Most of sectors have "override bounds" ticked but they still cause same performance drop. And sectors aren't that populated, about 7-10% of the scene is in sectors.

Yes I have 1.1.5b, when I realised performance problem is caused by sectr, I updated from an older version but nothing changed.

Disabling the bool works fine for me.

Well, there is a difference since some time and it may cause the performance impact. I have some objects that have parents in world 0,0,0 point, and they are out of bounds from sectr, but their child meshes are in sectr range. Right now I can see there are white bound boxed drawn for each of those objects when I select a sectr parent. I don't remember seeing them about a month ago, and I don't remember when they appeared. There were only blue ones that shows sectr's bounds.

An irrelevant recommendation, scaling sectors like scaling a box collider with dragging would be a great time saver.
And, I told you this one earlier, but demo scenes lightmaps are huge and everytime I update sectr it downloads them again, You should scale them down a little bit.

 

Those perf issues are very strange. Can you point the profiler at the editor, drag a bit and send me a snapshot during one of the spikes? I'd like to see where the timing is going.

No idea if the white boxes are related. White box means member component of its part of SECTR.

Not sure about SECTR scale tool. I see why you want it but edge cases are tricky.

I'll look at reducing lightmap size in future updates.

 

Screenshot of editor profiler:
https://www.dropbox.com/s/ujpnhykpoc7gn5y/Screenshot 2015-03-23 15.21.08.png?dl=0

Note that it only happens when moving any object in the scene (doesn't need to be connected to sectr, moving any gameobject, a new gameobject or a sectr doesn't matter)

Edit: In my latest reply I said that disabling sectr scripts removes performance impact. I didn't change anything but now disabling scripts doesn't work. I have to disable gameobjects one by one.

 

Thanks. Can you post another with deep profiling enabled? The right side shows a bunch of function calls that are not in SECTR at all.

 

I editted my first reply, and right side objects are all objects that are childs of a sectr (a disabled sectr right now)

 

https://www.dropbox.com/s/9a588yzbuzwnjw9/Screenshot 2015-03-23 15.28.37.png?dl=0

 

Sectr's and chunks enabled: https://www.dropbox.com/s/3d85rjqkuko5r7s/Screenshot 2015-03-23 15.31.18.png?dl=0

It doesn't only happen when moving a gameobject, but also it locks up a sec. when enabling/disabling any gameobject in the scene.

Edit: It happens even while I am modifying a unity layout. Profiler looks same.

 

I see the problem. You probably have a large number of objects in the scene total? I'll post a code patch later today.

 

When I select all objects that have sectr_member, scene looks like this, many white boxes. It didn't used to look this way before.
https://www.dropbox.com/s/6jdqqigznfh9lze/Screenshot 2015-03-23 15.48.10.png?dl=0

 

I just tested the scene with removing 90% of objects, and performance impact is reduced, but still editor is about 5-6fps and slow. profiler impact is about 180ms.

Does sectr auto create sectr_members? Will it mess things up if I delete all sectr_member components, or will it create them again?

When you fix the problem, can you please send me the updated package so that I won't have to wait for asset store approval?

Thanks.

 

It should be safe to remove any member components you don't want.

SECTR will not create members unless you tell it to. The only way to create a lot of members is using the fix overlapping children button.

I'll post the code fix here when it's ready. Should be pretty small.

 

PS - nice looking city!

 

Thanks

 

The fix for your bug is simple. Just delete the code below in SECTR_Member.cs. You can also remove the legacyBakeMode field entirely.

Code (csharp):

1.  
2. #if !UNITY_4
3.                 legacyBakeMode = false;
4.                 SerializedObject serialObj = new SerializedObject(FindObjectOfType<LightmapSettings>());
5.                 SerializedProperty workflowProp = serialObj.FindProperty("m_GIWorkflowMode");
6.                 if(workflowProp != null && workflowProp.intValue == 2) // 2 is legacy mode
7.                 {
8.                     legacyBakeMode = true;
9.                 }
10. #endif
11.  

 

Does Sectr support automatic terrain placement and camera reset to enable planet wide navigation as described here?
http://unity3d.com/learn/resources/talks/gis-terrain-unity

 

It does not because that technique, when combined with streaming in Unity, causes big performance issues. SECTR is compatible with that technique, but you'd have to write the "glue" yourself.

 

So if I would write the "glue" myself, what do I need to do in order to prevent those big performance issues?

 

Either avoid using static objects in your world chunks or get Unity to improve their code

 

That's interesting... I was under the impression that static objects which don't move are good for performance. Or is removing and placing static objects the problem?

To clarify, the terrain chunks don't actually move. They are just placed and removed successively. Or is that the same effect as moving a static object as far as Unity is concerned?

 

Yes, exactly. When you import an additive scene it gets loaded at that position. If you try to move it at all (which you have to do for the shifting world case), it'll force static collision rebuild.

 

Damn. They conveniently left that out in that talk.

 

I need a little help with a script I'm working on to take a large open world with lots of meshes and create the appropriate SECTR scripts on each object.

I've written a script that will split a mesh terrain (not a unity terrain, an actual mesh) a number of times (say 4x4) and in that script I add a SECTR sector to each chunk. I think that now I need to create a portal between each sector. Next, I need to create portals between each sector and set the position of the portal and the the front and back section of each portal, right? Then I think I need to set the coordinates for the portal between each sector but I'm not sure how to do that? Can you provide any suggestions?

 

I'd recommend you check out SECTR_TerrainWindow.cs. The work it does is very similar to what you're doing, except that it works on terrains. The portal creation logic should be quite similar, though. You can probably take it almost exactly. The basic idea is that you create a portal at the mid point between the sectors on each side, use a quad for the geo, and set the scale to be based on the dimensions of the sectors.

 

I recently upgraded my project to Unity 5 and I'm having some issues with SECTR STREAM. I'm using Terrain Composer and baking lightmaps. I have a 5 x 5 grid in Terrain Composer so there are a total of 25 terrains. When I'm done baking the lightmaps, Unity says I have 26 lightmaps. I assume there is one for each terrain but I'm not sure what the extra one is yet.

I use SECTR to create sectors for each of the terrains in Terrain Composer, then use STREAM to export them. I have a region loader on the player object so that the terrains load when the player moves over them. This all seems to work well except for the lightmaps. They work at first, but after loading several sectors, then moving back to the previous sectors, the lightmaps don't show up anymore (they worked at first). In the editor, I get the following red error message:

"Can't append 26 lightmaps, since that would exceed the 254 lightmaps limit. Appending only 20 lightmaps. Objects that use lightmaps past that limit won't get proper lightmaps."

There are only 26 lightmaps total from the baking, so I'm not sure how it is hitting a 254 limit, unless it is not unloading the loaded lightmaps after a sector is unloaded? It also seems that it loads all 26 lightmaps every time a sector is loaded, but the sectors should only have 1 lightmap each.

Is this a known issue with Unity 5? I'm using 5.0.1 and SECTR Complete 1.1.5b.

 

Hi, Chines. You're doing everything correctly. The problem is a bug in Unity 5. I'm working on a solution now for the next update of SECTR. If you'd like to get a preview of the fix, email support@makecodenow.com.

 

Thanks for the hint. The only thing I struggled with was needed to force an update on my newly created sectors before creating portals so the bounds would be updated properly.