[RELEASED] SECTR STREAM: Seamless Scene Streaming

I actually only tried it in editor. Will make a quick build.

But can you answer my question whether or not the async loading is still only for Unity Pro (as written in the docs)?

 

No this feature is included in the free version already. Generally there is not much left that is restricted in free (thanks unity!).

 

Yes, exactly what zkw said. SECTR is async (for 4.x pro or any version of Unity 5), but the editor has a bunch of extra overhead. Make sure to text in an actual standalone build.

 

Thanks. It works fine now.

Different question: is it possible to dynamically move a portal based on the player's position? I have a game with portals (like in the game Portal) where the player can look through walls (he basically creates a window in the wall). Therefore, I would like to move the SECTR portals around. In the editor, when I move them manually, it seems to work OK, but how can I determine the front/back sectors in code? Is this just too far fetched?

 

Hi, I'm torn between buying SECTR Stream or World Streamer: I need efficient streaming for an open world urban game with tons of streets and props -- so no interiors.
I know the two assets use radically different solutions. Which would work best in an urban game heavy with prefabs? Thanks.

 

(adding to my previous question)

I would like to use Viz occlusion culling in a first-person 3D game. We use a see-through shader that makes a hole/window in walls close to the player (discarding the fragments), so that the player can see what is on the other side of the wall. It's a bit like Portal, but where the portal/hole is cut through the whole wall.

This makes it impossible to pre-define the SECTR portals, since the player can in theory look through all the walls (if close enough to the object).

I am considering working with a dynamic SECTR portal that is placed in front of the player. I'm thinking about defining the SECTR rooms as usual and then parent a SECTR portal to the player. The problem is that the portal needs to know about its front and back sectors. Would it be possible for the portal to know which sector it is currently in (back sector) and then make a raycast in the forward direction of the player to find the other sector?

I've made a crude drawing of the scenario:



The player is in sector 0 and is going towards sector 1. Can I get access to these two and assign it to the portal? The portal would then be placed along the forward direction of the player (in a specified distance) and thus make the occlusion culling work as expected.

 

Hi. Good question. You can certainly create, destroy, move, or otherwise manipulate portals at runtime. You can simply assign the FrontSector and BackSector properties of the Portal as needed to update how they are connected. I can't really tell you how to determine which sectors to hook up to the portal as that's likely to be very game-specific.

 

I'm not super familiar with World Streamer, but at a very high level I'd say that they are very focused on outdoor spaces. They handle some things that SECTR STREAM doesn't, like auto-recentering for very very large worlds. On the other hand, SECTR STREAM is more general, can be used indoors or outdoors in in hybrid spaces. And of course if you get SECTR COMPLETE it also includes a realtime occlusion culling module and an audio module.

 

@MakeCodeNow thanks, I didn't expect the author himself to compare the two. I'll do some more research and then probably opt for Sectr.

 

I would say, they have different methods and goals (although World Streamer do all the things Sectr Stream does and more, but in exchange you have more modules in the Sectr if you need them).
Sectr is better when you're creating room by room experiences or you have only one layer and you're relying on triggers to load/unload the "rooms".
World Streamer is more robust, has solution to stream layers (terrain, huge objects, small objects, etc), it can stream based on position, but it handles triggers as well. It has the "ring stream" mode, when you have the current place, some near distant places and the far places and you have a long viewing distance. You can do it. In exchange it seriously mess up your naming convention and parent-children object relationship. It also has "reset position" feature if you're working on a huge world with possible physics bug (~ >10000 unity units size).

Choose the one which is more suitable for your needs. I own both of them.

 

Cool. Is it possible to access the current sector that the player is in, e.g., via SECTR_Member?

And do the sectors actually consist of colliders that I can access, e.g., via Collider.bounds.Intersect()? Or maybe via a raycast and specific layers?

 

I have parented a sector portal to the player. I use an empty object in front of the player with a collider to detect the sector in front of the player (using Collider.bounds.Intersects()). It seems like the front and back sectors are assigned correctly (looking in the Inspector), but for some reason the drawing of the objects doesn't seem to reflect the change (run-time).

If I manually reassign the front and back sector in the Inspector, it works.

Do I need to call some kind of "apply" method in order to update the portal?

I basically loop through a list of all my sectors and set the front and back sectors of my portal accordingly.

Code (CSharp):

1. public SECTR_Portal PlayerPortal;
2.     public Collider PlayerCollider;
3.     public Collider NextRoomCollider;
4.  
5.     public List<SECTR_Sector> AllSectors;
6.  
7.     public SECTR_Sector CurrentSector;
8.     public SECTR_Sector NextSector;
9.  
10.     private SECTR_Sector _currentSectorTemp;
11.     private SECTR_Sector _nextSectorTemp;
12.  
13.     private void Start()
14.     {
15.         var allSectors = FindObjectsOfType<SECTR_Sector>();
16.  
17.         AllSectors = new List<SECTR_Sector>(allSectors.Length);
18.         for (int i = 0; i < allSectors.Length; i++)
19.             AllSectors.Add(allSectors[i]);
20.     }
21.  
22.     private void Update()
23.     {
24.         _currentSectorTemp = null;
25.         _nextSectorTemp = null;
26.  
27.         for (int i = 0; i < AllSectors.Count; i++)
28.         {
29.             if (_currentSectorTemp == null)
30.             {
31.                 _currentSectorTemp = FindCurrentSector(PlayerCollider, AllSectors[i]);
32.                 CurrentSector = _currentSectorTemp;
33.             }
34.  
35.             if (_nextSectorTemp == null)
36.             {
37.                 _nextSectorTemp = FindCurrentSector(NextRoomCollider, AllSectors[i]);
38.                 NextSector = _nextSectorTemp;
39.             }
40.  
41.             if (_currentSectorTemp != null && _nextSectorTemp != null)
42.                 break;
43.         }
44.  
45.         if (CurrentSector != null && NextSector != null)
46.         {
47.                 PlayerPortal.FrontSector = NextSector;
48.                 PlayerPortal.BackSector = CurrentSector;
49.         }
50.     }
51.  
52.     private SECTR_Sector FindCurrentSector(Collider myCollider, SECTR_Sector sector)
53.     {
54.         if (myCollider.bounds.Intersects(sector.TotalBounds))
55.             return sector;
56.         else
57.             return null;
58.  
59.     }

It seems like no matter how I assign the FrontSector and BackSector via a script, it doesn't update properly. But I assign them via the Inspector, it works ...

EDIT:
It seems like the problem happens if I repeatedly assign a sector to a portal. If I assign it null and then give it the proper sector, it looks like it works.

 

So, a few thoughts...

FrontSector and BackSector properties will, by design, ignore calls where the new Sector is the same as the old Sector. I'm not sure why that's causing you a problem here, but that's how the code works. Maybe you are assigning the same Sector to the Front and Back slots? The code doesn't protect against that, but it's not a good idea.

If you need to get all of the Sectors, you can call SECTR_Sector.All, which is much faster than FindObjectsOfType. Similarly, you can call SECTR_Sector.GetContaining() to get the sectors that overlap a bounding volume.

If you use the SECTR_Member component, you can call the Sectors property, which will give you the list of all Sectors that Member is in.

-Nathan

 

Thanks a bunch. It was really helpful!

Instead of setting the portal's sectors to null before assigning it the new sector, I just save the previously-assigned sector and check if it is identical to the new one. Seems to work

Code (CSharp):

1. public class SECTR_Player : MonoBehaviour
2. {
3.     public SECTR_Portal PlayerPortal;
4.     public Collider PlayerCollider;
5.     public Collider NextRoomCollider;
6.     public SECTR_Member SectrMember;
7.  
8.     public List<SECTR_Sector> CurrentSectors;
9.     public List<SECTR_Sector> OverlappingFront;
10.     private List<SECTR_Sector> _tempNextSectors = new List<SECTR_Sector>();
11.     private SECTR_Sector _prevBackSector;
12.     private SECTR_Sector _prevFrontSector;
13.  
14.     private void Update()
15.     {
16.         CurrentSectors = SectrMember.Sectors;
17.         SECTR_Sector.GetContaining(ref _tempNextSectors, NextRoomCollider.bounds);
18.         AssignSectors(CurrentSectors[0], _tempNextSectors[0]);
19.     }
20.  
21.    
22.     private void AssignSectors(SECTR_Sector newBackSector, SECTR_Sector newFrontSector)
23.     {
24.         //PlayerPortal.BackSector = null;
25.         //PlayerPortal.FrontSector= null;
26.  
27.         if (newBackSector == newFrontSector)
28.             newFrontSector = null;
29.  
30.         if (newBackSector != _prevBackSector)
31.         {
32.             PlayerPortal.BackSector = newBackSector;
33.             _prevBackSector = newBackSector;
34.         }
35.  
36.         if (newFrontSector != _prevFrontSector)
37.         {
38.             PlayerPortal.FrontSector = newFrontSector;
39.             _prevFrontSector = newFrontSector;
40.         }
41.     }
42. }

 

Awesome!

 

Sorry for keeping asking questions ...

Is it possible to use both Stream and Vis at the same time? If so, how?

When I try using the neighbor loader and culling camera at the same time (having exported a bunch of sectors), I get this error:

Code (CSharp):

1. NullReferenceException
2. SECTR_CullingCamera._HideAllMembers () (at Assets/Plugins/SECTR/Code/Vis/Scripts/SECTR_CullingCamera.cs:1746)
3. SECTR_CullingCamera.OnPreCull () (at Assets/Plugins/SECTR/Code/Vis/Scripts/SECTR_CullingCamera.cs:559)

Second question: is it possible to move the sectors around in the main scene after having exported? E.g., can I create a template for our master scene (exporting the sectors to create the chunks scenes) and then later move them around (not in run-time). We are multiple people working on the scenes, so it would be easy if I could just create some kind of template and then let people create stuff in each of their individual chunks scenes.

 

It is definitely possible to use STREAM and VIS together, though VIS may or may not be a big benefit if you are streaming efficiently.

Those null errors are fixed in SECTR 1.3.2 which came out a week or so ago.

You can move Sectors around, but here's the workflow I recommend:
* Check out the main scene and the sector to be moved.
* Import the sector into the main scene.
* Move sector to new position.
* Export sector.
* Check in main scene and newly positioned sector.

The above is pretty easy and safe, and similar to what the Campo Santo team did on Firewatch.

 

Ok, thank you.

It seems like you need to put everything into the master scene at least one time. That means that the master scene becomes somewhat of a bottleneck, no?

Is it possible to go to any individual scene and export it as a sector chunk and then import that chunk to the master scene? Or maybe make some "dummy" sectors in the master scene that can later be replaced with any other scene I chose.

It looks like the stream window (import/export) only shows sector scenes that have already been placed in the master scene.

 

The main scene doesn't actually need to be a bottleneck. Everyone can open it in a read-only way and you only need to check it out when you are adding, moving, or removing Sectors. You can also have people work in the exported Unity's scenes, provided they are careful about keeping everything organized under the right transform.

 

I am having problems with adding global objects in a chunk scene. What am I doing wrong? In the manual, it says New global objects can be created in the Chunk scenes. However, they will not function correctly until imported into the main scene and re-exported.

Here is a short screen recording:

When I re-export it from the main scene, the change is gone?

 

I'm sorry for the confusion. That's a documentation bug. It used to work before Unity's multi-scene editor APIs but doesn't work anymore. Now global objects have to be created in the main scene. I'll fix the documentation for the next release.

 

Ok. I hope you will eventually find a different solution, since it can become cumbersome to manage objects that are specific to room/sector (e.g., position), but should work as a global object and not be loaded/unloaded.

EDIT:
To clarify what I mean: it's practical to have everything grouped into a room/sector, but sometimes there are objects that should be global (e.g., not load/unload), even though it belongs to a room. Would it be possible to add some kind of SECTR_Member component that simply flags an object to not load/unload, even if it is part of a sector?

My situation:
I have a lever that the player can pull. The lever is in room B. It has references to objects in room A and C (dragged into public fields in the Inspector). How do I make sure it doesn't lose these references? Do I really need to make the lever global AND the objects in A and C global?

 

I'm getting lots of errors (one per game object) when trying to Export a scene made from ProBuilder objects:
Can't remove pb_Entity (Script) because pb_Object (Script) depends on it
UnityEngine.ObjectestroyImmediate(Object)
SECTR_StreamExport:ExportToChunk(SECTR_Sector) (at Assets/SECTR/Code/Stream/Editor/SECTR_StreamExport.cs:400)
SECTR_StreamExport:ExportSceneChunks() (at Assets/SECTR/Code/Stream/Editor/SECTR_StreamExport.cs:678)
SECTR_StreamExport:ExportSceneChunksUI() (at Assets/SECTR/Code/Stream/Editor/SECTR_StreamExport.cs:621)
SECTR_StreamWindow:OnGUI() (at Assets/SECTR/Code/Stream/Editor/SECTR_StreamWindow.cs:138)
UnityEditor.DockArea:OnGUI()

Is anybody else using ProBuilder with SECTR? Seems a real shame that I would have to convert everything to a normal mesh in order to use it.

- edit: Found that even trying it has broken things more than I realised. I tried removing the Chunks component on the failed exports, then turning into meshes (removing all ProBuilder scripts) but the STREAM window still thinks that these game objects have been exported. All I get is the option to Import.
- I've deleted the folder with the chunks (scenes) in it. Just can't get back to my oringial, even the back-up is affected. Any ideas on how I can roll back the failed Export?

 

So after reverting to a pre sector export copy of the project, and making sure that I deleted all ProBuilder scripts from the scene, the Export worked ok.

What a shame that this is necessary... I realise that ProBuilder isn't your asset, but it's used by a lot of people. I really expected SECTR to be compatible with it...

It kind of makes it unusable for me... Another example... I then built my own doors, forgot to remove the ProBuilder scripts before clicking Run, and now I'm getting a new error:

HasKey is not allowed to be called during serialization, call it from OnEnable instead. Called from ScriptableObject 'pg_Editor'.
See "Script Serialization" page in the Unity Manual for further details.
UnityEditor.EditorPrefs:HasKey(String)
ProGrids.pg_Editor:LoadPreferences() (at Assets/ProCore/ProGrids/Editor/pg_Editor.cs:190)
ProGrids.pg_Editor:OnAfterDeserialize() (at Assets/ProCore/ProGrids/Editor/pg_Editor.cs:426)

Again this has completely broken the scene, even after I deleted my new doors, and stripped out all ProBuilder scripts, I still get the error.

I'll contact the ProBuilder dev to flag up the issue at his end, but I expect he'll just say that SECTR isn't his asset... Maybe you two could get in touch and try to resolve this? I feel really disappointed that what was potentially so useful is going to be useless to me

EDIT: after further testing I realise that this is a ProGrids issue
- even after stripping out the ProBuilder scripts, running the scene throws up the same 'HasKey is not allowed' error.
(possibly something I've never come across before because I've never had to convert my objects to meshes and strip out the ProBuilder scripts?)

 

Incompatibilities between plugins can occur. In terms of how you put your project together, maybe you can create your levels in one project (using ProBuilder) and export those levels to your final project, which uses SECTR ...?

I think it's fairly common to use one project to create each of the resources to make something - like particle effects or levels - and then import the finished products into the project you will ultimately build your game from.

 

@hopeful Love your handle (name)! says a lot about your attitude, which I like and is reflected in your comment

It really is a shame though... There's a handful of assets where I would do as you suggest but ProBuilder is so much about rapid, in editor prototyping that it seems a crying shame to not to be able to use it. Other streaming assets I own have no problem with it all, so I'm certainly 'hopeful' that it's a problem that can be solved...

 

Thank you! I'm not sure where the problem is cropping up with Pro Grids, but it seems like an odd collision to have. @MakeCodeNow will probably be able to help you sort it out.

 

Sorry, another query... I've been learning by building my own version of the demos. So after making a simple NeighborLoader type map (which includes doors), I went ahead and converted it to using the LoadingDoor component.

It kind of worked but was running at a very slow framerate, with hundreds of warnings accumulating in the console (one per door, per frame, as far as I can make out). They're all of this kind (the hash is different for each door instance):
Parameter 'Hash 673404623' does not exist.
UnityEngine.Animator:SetBool(Int32, Boolean)
SECTR_Door:Start() (at Assets/SECTR/Code/Core/Scripts/SECTR_Door.cs:108)

So I worked out (by chance/comparing the actual demo) that the problem was that each of my SECTR_LoadingDoor scripts included a CanOpenParam value - I'd left it defined as 'CanOpen', as per the NeighborLoader demo.

Can anyone explain why the warnings appear, and how the CanOpenParam is used (i.e. why is it used in one demo but not the other)?

 

Thanks for taking the time to write up these issues. In general, I can't guarantee compatibility with other assets in the store except for the ones I officially support (Playmaker and Terrain Composer). There are too many assets, even if you just limit it to the popular ones, for me support, especially since they get updated, Unity versions change things, etc.

That said, I did look into this issue and can give some ideas.

The problem you ran into w/ ProBuilder appears to be due to the fact that you have ProBuilder components on the same GameObject as the Sector component. When the Sector is exported for streaming, I have to strip off components on that GameObject, and that seems to cause a problem when those are pro-builder components. Probably you can fix this if you make the Sector a parent GameObject with only the Sector component on it.

Regarding the ProGrids issue, I'm much less sure what's going on there. It seems like they are doing something kind of non-standard that Unity does't like when the chunk scene is saved out.

 

The idea is that when you are streaming, you don't want the door to open until the sector on the other side has loaded. In the mean time, you want to play a looping animation and then transition to the open animation when the other side is ready. This looping animation is optional and not relevant at all if you aren't streaming.

The more basic issue is that with Unity animation there's a contract between the game object/logic and the animation. In this case, you create an asset without the param and then told SECTR to look for that param, hence the warning spam.

 

Spot on! That now works great, and I can confirm that you can Import chunks, edit in ProBuilder and reExport all without any problems (or console errors).

Thanks for your help.

 

Please do, because these details can be really confusing and in fact leads to a nagging doubt about the validity of the documentation as a whole. Among many other little things : in Creating Proxy meshes chapter you wrote :
"As a convenience, the Chunk component also includes a tool that
can create Proxy Meshes for you automatically. Simply expand the Create Proxy
Mesh foldout, select the child meshes that you wish to include, and press the Make
Proxy Mesh button."
Cool...but...where is the expandable "Create Proxy Mesh foldout" and " the Make Proxy Mesh button."
Cheers!

 

The original doc bug has been fixed.

Regarding proxy tool, it's in the Chunk component but it only appears when there are meshes that are children of that Chunk. If the Chunk has no children, that tool foldout auto-hides itself.

 

We are having some problems with using SECTR Stream and Neighbor Loaders.

We are using SECTR as both a workflow tool and a optimization/streaming tool. Basically, we are working in the individual chunk scenes, e.g., one level designer is working in room A, while another is working in room B, C, etc.

However, we sometimes want objects to reference each other across different chunks.

For example:
We have three rooms: A, B, C. In room A there is a lever that the player can pull. This lever should somehow activate something in room C. In this example it could be a light that should be turned on or maybe some audio effect that should start playing. The problem is that the C chunk isn't loaded yet. We don't want to load all of chunk C in order to activate this light.



Is it somehow possible to access an object in a specific chunk even if it isn't loaded? Maybe with some kind of global middle man that can give us references to the objects we need?

We do not want to make said objects global (by placing them in the master scene that loads all of the chunks), since we want each scene to contain its own objects. It is much more practical to have each object be contained in its corresponding scene. If we were to make the interactable objects global (by placing them in the master scene), it would be difficult to maintain when somebody makes changes in the level design.

Asked in another way: Is it possible to make a local object in a chunk global during run-time?

 

Would it make sense to make the lever and light a separate chunk that loads with both A and C?

 

Good question. The short answer is that this is doable but it's something that winds up being very game specific, so you'll need to implement it yourself. In general you need to create some kind of id/reference type attribute and have a central system that hooks into the SECTR load events and then fixes up the references at least time. Another variation is to have the new reference type simply resolve itself when accessed.

 

We could do that, but it kind of defeats the workflow purpose of having each chunk being self-contained. Say we have chunk A, B and C (as shown in the drawing) and then a chunk called InteractableChunk that has the lever and light. If we decide to make level design changes in e.g. A or C, we need to make sure that the changes also happen in the InteractableChunk, like, if we move the lever or something. To me, this sounds like a logistical nightmare.

After having thought about it some more, I don't think we can manage to do something like that. We might have to just make the unique objects global, even though I would prefer a different solution.

 

I have a new question:

My level designer is working in the master scene with all of the chunks. However, he found that the boundary boxes of the chunks always starts in origin (0,0,0), even though the actual bounds is supposed to be smaller.

Why is this? Since all of our chunks start in the origin, it defeats the purpose of using a neighbor loader, since the player is inside most of the chunks from the beginning.

Of course, we could use the "override bounds" manually, but it seems redundant.

On the picture below, we have two sector chunks. One is loaded (the one with purple outline), but both of them start in the origin.

 

Hi. That's very strange. SECTR should freeze the bounds of the sector when the chunk is exported and those bounds should match the Geo in it. A few questions:

1) is it possible someone is moving sectors in the master scene after they were exported?

2) is it possible that you have some objects like lights or particles that extend well beyond the meshes?

3) are there any scripts that run in the master scene that could be modifying sectors?

 

So I split up my scene into chunks very similar to the Stream demo, but I'm finding that the game keeps hanging when moving from chunk to chunk (loading and unloading chunks). Really not sure why because these are just rooms and corridors, no scripts.

So I gave up for a few months and have now decided to try splitting my level into bigger chunks and to use Loading Door loaders. Bigger chunks worked fine while still using the neighbor loader but as soon as I switched to the Loading Door loader, for some reason now my entire level loads at once - after the initial Start Loader loads the first chunk, all of the other chunks then get loaded (before the player has moved anywhere)

Are there any obvious things I might be doing wrong?

 

Hard to say but I would put a breakpoint into the add reference function in chunk to see who is requesting the rest of the scene to load.

On hitches, make sure to test in standalone build. Hitches in editor are common and due to unity disabling many streaming opts in editor.

 

Thanks for that. Ok worked it out. As I thought I was going something stupid..
For anyone with similar issues: because the loading door component's layer filter defaults to all layers ticked, I guess everything in the scene was triggering every door. I unticked everything and just selected my player layer, that worked.

Re. hitches. Sadly I know about the editor not properly streaming and was testing on the device. Must be something specific to my scene assets.

Btw. Did you ever get my last email, or resolve the issue of errors on loading new scenes (as opposed to sector chunks). Not being able to completely switch scenes is still a bit of a deal breaker for me, but I'm hoping it can be solved.

Thanks, as always

 

Hi!

I've noticed that it doesn't seem possible to have a sector with only invisible geometry in it (e.g. invisible collision meshes). Reading the docs, it seems like sector membership depends on Renderer components, so I imagine this is a fairly fundamental restriction.

Right now I've gotten around this by putting some cubes that won't be seen at the corners of the area I want to stream in and out, but is there a better approach?

Thanks!

 

Hey, Jerome. I did get your email but have been busy with other obligations. I'll try to flush out the issue with full scene loading and send you an update soon.

 

That's correct. You could use the bounds override to force a certain size, but right now it's intentionally only based on the visual bounds.

 

Is there any easy way to "sectorize" an existing Unity scene?

As far as I understand, the current workflow is like this:
You have a master scene and select parts of this scene to export it into its own chunk. These chunks are now saved as individual Unity scenes and can be modified as such.

Is it possible to:
Have X amount of traditional Unity scenes and make them into chunks that are then loaded into the master scene?

The reason is that our team just wants to work on many different scenes, but we do not know beforehand how many we will create. We don't want to have to export a chunk in the master scene every time we want to make a new scene (middle-man pipeline).

 

That should be possible but it's unsupported. You'd need to write the logic yourself that creates a master scene from the chunk scenes.

 

I have read the tutorials twice, watched the videos and read this forum, but I am still not quite sure when I should be using this, how to use it correctly, or what it's purpose is.

Edit: Ok, to be more specific, I am thinking about my items which go into my inventory. If I pick up a spear in scene 1, and it is in my inventory, and I move to scene 2 and drop this spear, what does this spear need to have. Does it need to be a global object, should it have a hibernator, and should it be portal determined?

 

SECTR STEAM is really intended to handle the big chunks of world data that take up a lot of memory. If you need to remember where every dropped spear is, then you want to write a custom system to save and load them, independent of SECTR.

 

Thanks, I have a save/load system for the inventory items and it works fine. I was more asking about that checkbox in Sectr (the one in the image), and what it should be used for and when. I can't seem to find much information on it in the manual or the videos. I haven't used it at all, and I was just wondering if I should be using it.