[RELEASED] SECTR COMPLETE: Spaces, and the Connections Between Them

Glad you like it!

Unfortunately I can't thread the export because Unity API isn't thread safe. There are ways I could make it faster by launching a bunch of background command line unity processes, but it would add a lot of complexity, require a lot of memory, and have an uncertain speed benefit, so while it's possible, I don't plan on doing it anytime soon. Usually, though, it's less of a problem over time because you get every scene exported and then it's just incremental exports from there.

 

SECTR 1.1.1 is now live in the UAS, just in time for the end of the August Madness sale. It includes some bug fixes, and optimizations to Stream hitching.

 

Maybe it is to early to ask but will sectr complete will be updated in unity 5 and also backwards for 4.6 later?

Also i read this thread about level loading/streaming:
http://forum.unity3d.com/threads/unity-5-level-streaming.235037/

and hope that sectr will also take benefit from this new techniques like better multithreading und some other things they talk about.

 

The plan is for SECTR to work on everything from 4.0 onward. The streaming improvements you mention in 5 should "just work" as long as Unity keep the loadLevelAdditiveAsync API reasonably similar.

 

According to what I've read on that thread from unity staff, it should just work better in the background and not induce hic ups/freezes while adding new scenes.

By the way is there a way to avoid this on mobile at this time ? Streaming is great because without it we probable couldn't load a scene of that size but the hic ups are bad enough to be a dead end for a feature in our current project.

 

That's my read, too.

As for hitching on mobile in 4.x, it depends what the cause is. 1.1.1 includes some improvements in collision and I have some big SECTR side fixes in 1.1.2 that are bring tested now. Hopefully that'll fix your problem, but if its bottlenecked on things like texture decompression and fixup then that's not something SECTR can fix. Email supports makecodenow.com if you want an early preview of 1.1.2.

 

I have some sectors with fractured buildings inside so there might just be too many meshes to load even though I made the chunks pretty big to limit the amount. It just seems out of range for mobile, at least on low end devices.

I'll make some more experiments and if needed send the mail to try the new version, in case there's a chance it solves things. Thanks.

 

SECTR 1.1.2 is now live in the asset store. This follows 1.1.1 with some major reductions in CPU time spent during chunk load in STREAM. SECTR time when loading chunks is now close to zero and the loading logic allows Unity to do as much asynchronously as possible. I think this is very close to as fast as one can get doing additive loads in Unity 4.x.

 

Hi,

I purchased SECTR COMPLETE off of the Asset Store a while ago and Ive managed to set up all my sectors on my terrain and the rest of my scene, as well as all the protals and their connections to their sectors. Ive also set up some Occluders as well just to get some basic testing going.

However, I seem to be running into a few problems:

1. The game hangs when taking a few steps forward and moving the camera, Ive noticed that this is connected to the Terrain Sectors though as whenever I turn them off and only work with the other Sectors that I set up (for my caves and buildings) no hangs occur and I can run through the scene smoothly in editor and game mode. Im wondering if this might not perhaps be connected to the fact that my Terrain Sectors' portals dont seem to have a mesh (as all of their Mesh HULL slots are empty).

2. Cull In Editor doesnt want to work properly, I need to have the Player selected for it to display in the Scene Window, the biggest problem though is that the only things that are being culled in the Scene window are lights, none of the meshes are are being turned off in the Scene Window, I can only see lights switching on and off in the Scene view depending on where the frustum is pointing and the distance between the player and the light ofcourse. The Meshes cull fine in the Game view which I can see easily with and Occulder or two, but I dont see those same results in the Scene Window which makes testing very difficult. I must also mention I am using UFPS and a 2 camera set-up as the First Camera is for the Scene and is used as the Main Camera and the second is used for the First Person Character's hands and weapons so that they cannot clip with anything in the scene. I have followed the instructions in the SECTR VIS manual about uncommenting LAYER_CULL_RENDERERS from the SECTR_Culling Camera.cs script and I have set up an Invisible layer and connected it to the SECTR_Culling Camera's Invisible Layer atribute, just like the maunal stipulates (although, Im not sure what to do with this layer though as I understand it is also intended for the Terrain to make the terrain able to cull and display but Im not sure if I should use this same layer for all the other meshes and Im also not sure which camera should receive that layer).

3. Terrain wont display when incorporating the workaround method mentioned in the SECTRE VIS manual. I created a new layer and set both the Terrain on that said layer as well as the SECTR_Culling Camera's Invisible Layer attribute to that same layer. Sadly though the entire terrain just becomes transparent and it wont display. I have tried attaching a Sector component to the Terrain, which didnt help, and I also tried attaching a member component to no avail.

Please any help on the above issues would be immensely appreciated as I cant make heads or tails of any of it. Just to give a little bit more info on my setup, I am running Unity 4.5.3 Pro, Windows 7 Ultimate 64 bit and I am running the latest version of SECTR COMPLETE (1.1.2)

Please just let me know if you need any other infom so that we can get these problems sorted out as quick as possible.

Thanks in advance for any and all help and also thank you for creating such amazing plugins, all of the SECTR modules are really amzing and so extensive, I cant wait to get more hands on as I dive deeper into SECTR COMPLETE.

 

Ok, a few things. First, comment LAYER_CULL_RENDERERS back out. Those instructions are for multi-camera games where each camera needs to cull and have very different views of the world. In your case, the weapon doesn't do any culling so you don't need this step. That will fix several of these issues, including meshes not culling in the scene view. I'll update the docs to be more clear on this point.

Second, when using the Invisible Layer technique, don't change the terrain of the layer. Just create the invisible layer and set it in the camera and you're done. You won't see these cull in other views, but they will be culling (you can verify in stats). The docs don't say to change the terrain layer, so I think you were just getting a bit ambitious

I'm not sure why you see the stalling in #1. Make sure the terrain sectors are static and then let's see if it doesn't go away with the above changes. If it persists, it'd be really helpful to see a Profiler capture so that I know where the time is being spent.

 

Thanks so much for the fast reply.

Also thanks for the advice and the corrections, apologies I didnt realise that the 2nd camera techinque only applies to 2 simultaneously culling cameras, I will be sure to comment LAYER_CULL_RENDERERS back out.

Also apologies on the Terrain confusion I dont know why I thought I needed to switch the Terrain to the Invisible Layer, thanks for the correction.

As for #1, I have my Terrain Sectors set to Static already. But it is still weird that none of the Terrain Sector Portals have Mesh prefabs connected to them (unless ofcourse, it needs to be this way and the meshes for the portals are created procedurally during playmode, which I checked as well and it didnt seem like they even hade meshes connected to them during playmode). The portals are physically there in the hierarchy and their Back and Front slots are connected to their corresponding Terrain Sectors, the Portals also display the back and front direction gizmos (yellow and purple lines) but the actual green shape and frame gizmo of the portal itself is missing. This is just a bit odd as I could swear previous version of SECTR created actual full portals (with green shape gizmos and mesh connected) for the Terrain Sectors and Portals.

Again thanks for the quick reply and the help I will try out the changes that you suggested in a few minutes and update you on my progress, Im just quickly waiting for my project backup to finish copying.

 

Terrain portals have never had geo. They tend to be so large and they are right next to each other that it's not worth doing all of the frustum manipulation.

 

Okay, just like you said, it seems commenting out LAYERS_CULL_RENDERERS again seemed to have fixed most of the problems, also the Terrain doesnt turn invisible anymore which is awesome, but how can I confirm that the terrain is indeed being culled during the right times as I cant see the terrain being turned off in the scene view ever, is there something perhaps in stats or in the profiler that could indicate to me that the terrain has been culled?

Now the only real problem I have left is the Terrain Sectors.

Luckily, neither the editor nor the build crashes anymore nor do they hang, however whenever I get close enough to the Terrain to make the Terrain Sectors start activating the game slows down to a crippling slow speed (like 1 frame every 2-3 seconds).

Additionally to make it more bizarre I created new Terrain Sectors and Portals, just leaving the old ones off, just to test if its maybe something inside a specific Terrain Sector maybe just causing a conflict, but thats the weird thing it still does the crippling slow even if the Terrain Sectors are empty.

Im wondering though if I cant maybe just use normal Sectors and just leave the Terrain Sectors out as they definetely seem to be casuing the problem.

This is quite baffling, but I am pretty sure we will be able to figure it out. Thanks again for all the help so far and let me know if I should send you any other info regarding the Terrain Sector problems Im having.

 

You have to look at batch counts to confirm that the terrain culling is working. I'm not aware of a better way. The layer trick is all a workaround for a Unity terrain rendering bug in 4.x, so there's not much I can do about it.

Regarding the terrain sectors, I'm not sure what's going on. Static sectors do basically zero work on update, and the other things you've tried don't really help me understand what's happening. What I really need is for you to use the profiler during these super slow frames, then select a slow frame, and post a screenshot so that I can see where the time is going. You can post that here or email to support@makecodenow.com.

 

Hi, Apologies for the late reply Ive been going at SECTR like crazy for the last 2 days trying to figure everything out.

Awesome, thanks for the advice will check out the batch counts to make sure the terrain culls.

As for the terrain sectors, Ive struggled with a bunch of different methods in trying to find the route of the problem. The best I can get it at the moment is by either just using Terrain Sectors with normal Pass Through meshless Portals or Terrain Portals with Meshes attached and their Flag set to none (the weird thing is the second method actually runs a bit better and more strable).

But as soon as I Turn on my other Sectors form my buildings and caves I get a memeory overload crash in the editor casued either by VertexData.cpp, Texture2D.cpp or dynamic_array.h, another weird thing is that everything runs fine when running it in the build even if the Terrain and non Terrain sectors are switched on, and when just running the Terrain Sectors or just running the Non-Terrain Sectors in the editor the editor doesnt crash (although the Terrain Sectors are running extremely slow and jittery when compared to the other non-Terrain Sectors).

I will email you the error reports that I got as well as a screenshot of the profiler to maybe help showcase the root of the lagging on the Terrain Sectors.

One final thing, I also just want to check how Occluders should be set up, as my Occluders dont seem to be blocking the Culling cameras view from Portals. I just wanted to check should they be parented to a Sector or be on the same hierarchial level? Also Ive noticedthey also tend to move with the Culling Camera's movements so I assume it is best left on Dynamic and not Static.

Thanks again for all the help so far, hopefully we can get this probelm with the Terrain Sectors sorted out. Sending through the pics in the next few minutes via email.

 

Maybe there could be a demo scene showing off terrain and occluders?

 

There totally should be. I'm working on an outdoor demo, and that will have occluders in it.

 

Ok, thanks for emailing me your profiler shots and logs. Here's the scoop:

1) There is almost certainly a bug with VIS culling on grids of pass-through/geometry-less portals. I'll dig into this and hopefully have a fix tonight or tomorrow. In the mean time you can put quads on all of those portals and clear the pass through flag. That should do the trick and be a reasonable indicator of real culling performance. If the culling is too slow, check out the optimization tips in the VIS manual.

2) The two memory crashes that you sent me are memory related. In both cases, Unity is trying to make a single, fairly large (4-5 MB) allocation and failing. This could be because you're in the editor and Unity is nearing it's 2 GB max or it could be because Unity's heap is too fragmented. In either case, there's not much I can do except suggest workarounds. If you're nearing max Unity memory, then you want to cleanup your project to reduce memory. If it's fragmentation, then you want to that no Sector has large resources like tons of static collision, huge textures, super high poly meshes that get static batched, etc.

3) Occluders don't care if they are static or dynamic or if they are the child of a Sector or not. Usually you want them to auto-orient around an axis, but it depends on the shape of the object you are modelling. A wall should not auto-orient, a cylinder shaped thing should. In any event, occluders have to fully block the entire AABB of the portal or mesh or terrain in order for that object to be culled by them. For very large meshes and chunks of terrain, you may need to be very close to the occluder or have it be very large for it to really fully block a large object.

 

Thanks so much for the fast reply.

1) Good to hear that the profiler screen shot and info could be of use in order to help pinpont a bug, hopefully it wont be too difficult to fix. Thanks also for the heads up on the Flag settings, Mesh and the Optimisation tips, Ive gotten the Terrain Sectors to run pretty smoothly in editor (when just running the Terrain sectors and leaving the rest off though).

2) Thanks for the heads up on the Unity memory limits and fragmentation problems. My project is about 16GB but Ive deleted a few unnecessary assets and have gotten it to 14.7GB, although I am still getting the crashes, which leads me to believe that it is most likey a fragmentation problem. I have quite a few dead body models which are quite high poly, have pretty high res terxtures and are also using their normal mesh colliders so I am thinking that these dead body models might be the culprits. Also I wanted to check would combining meshes pre-runtime (in-editor) help to alleviate possible fragmentation problems as I have a LOT of rock models in my scene, my vegetation is already combined, so I know it cant be the vegetation, but I have thousands of rocks in my scene that arent combined so I am also wondering if they cant be the culprits.

3) Thanks for the heads up on Occluders, like hopeful mentioned it would be nice to get a demo scene perhaps to see a proper demonstration on what the best Occluder practices are.

I will keep you updated and let you know what I can figure out with the memory overload crash.

Thanks again for all the help so far.

 

Okay, Ive finally gotten all my sectors to run without any crashes in editor. It was actually due to a few unnecessary Sectors on my building meshes as soon as I removed their sectors and set them to children of their respective Terrain Sectors instead (as well as putting members on them) I could actually run my entire scene in editor.

I am geussing that sorting out those static high poly meshes will icrease performance even more though. Also I just wanted to confirm, but I am guessing that particle systems also need members on them as I see they arent being culled by the Sectors.

Finally will SECTR Stream help alleviate any of these performance problems? Apologies if the question is a bit vague.

Will keep you up to date if I run into any other problems or make any progress on better performance.

Thanks again for all the help.

 

Apologies for the multiple messages but Ive noticed that the view frustum of the Culling Camera doesnt appear to be accurate, it seems to pick up stuff behind the view frustum as well and it doesnt seem to adhere to the actual drawing distance of the camera. Is this perhaps a feature or setting that just needs to be tweaked or is it a bug?

 

That's just a debugging tool. You can make it longer or shorted using the debug frustum length property.

 

Monolithic assets tend to be the worst case for fragmentation because they are the hardest to fit "into the cracks" once memory is fragmented. Generally I wouldn't recommend modifying assets to address fragmentation until you had done a lot of investigation, and usually then it's splitting, not combining.

STREAM is like the intimate form of culling. It saves memory and CPU and GPU because the unloaded objects don't exist at all. You just need to hide the loading behind a door or fog or distance or whatever.

For now particles that are the children of a sector will need a member to cull, but I don't like that requirement and will fix it in the next update.

 

Thanks for the updates.

Where exactly do I change the debug frustum length property? I cant seem to find it on the Culler Camera or anything.

Thanks for the heads up on the fragmentation I will investigate my poly, texture and physics intensive assets first before comibning things. What do you mean by it is usually splitting intead of combining though?

Okay, awesome so STREAM should sort out some of the hitching form me when moving from sector to sector as it aids VIS in culling, although I want to get the scene to reun as good as possible with VIS first as I still need to finalise my light baking and Nav meshes etc before I export the chunks for STREAM and set up any loaders.

Thanks also on the confrimation of memebrs on particles, I noticed that it was the only way I could get bounds over them, but just wanted to confirm. Glad to hear that you are trying to simplify this process though.

Another thing that I have noticed is that my Terrain Portals completely ignore Occluders, Ive set up a few Occluders but they only seem to block the portals of my non terrain sectors, I can see straght through all of my terrain sectors though from the one point of the map to the opposite end, which is baffling me a bit.

Finally, Im also struggling in getting the Portal/Sector graph to run through properly, I have a few portals connecting the Terrain sectors to cave sectors but when ever I look at the portal connecting the cave entrance and the terrain sector the culling camera freezes its rendering, the game doesnt freeze as I can still hear the character walk around and the transformation values of the culling camera can still be seen changing as the camera looks a round but the rendering window just freezez completely when the frustum comes into view of thise particular portals.

Apologies for all the problems, I have quite a complicated scene, with terrain, underwater and above water cave systems, as well as buildings sitting on the terrain with interiors. So I expected that I would run into a few problems in trying to get the workflow going, I just hope that I can figure it out soon as I still have quite a bit to do in my project besides just optimisation.

Thanks again for all the help so far, I will keep you up to date.

 

Such questions!

On fragmentation, imagine a skyscraper full of offices. Companies come and go, some are big and some are small. A new company joins and wants everyone to sit together. The bigger the company, the harder it is to do that, right? Even if you have enough empty space total, its hard to find a big empty chunk. However, if you can split the company in two, it's easier to fit. The same is true of fragmentation. Big assets are like big companies, and the computer can't split it without your help.

There should not be any hitching (sudden frame rate drops) in VIS as it's pretty constant costs per frame. I suspect it's something else but you should use the profile to investigate. Send me a screenshot if it's something in SECTR.

Not sure why the Occluders aren't blocking terrain n portals. How do you know they aren't blocking them? Are the terrain portals huge? If do are you sure they are completely occluded?

The terrain to cave portal thing could be a bug but I'm not sure. Make sure no portals are children on sectors. Beyond that, I think you should setup some simpler, test scenes. It's very hard to debug a huge scene and do middleware integration all at once. Much easier if you can simplify, get it working, then add complexity. That way you have a functional baseline and can easily tell when an addition breaks.

 

Apologies, I realize I do have quite a large and complicated scene, so yes I should have started on a smaller area and work my way from there.

Thank you so much for the analogy on fragmentation it makes a lot more sense now.

Thanks also for the clarification on the hitching in VIS.

As for the occluders I think you might be right the terrain portals are extremely big, so I will try making them smaller and the occluders bigger.

I also just want to check again, where I should change the debug frustum length property?

 

Oh yes I forgot ask, is there perhaps a more in depth profiler asset available, even if it is a paid asset that you know of. Its just that the profiler doesnt always give detailed enough info and it is difficulot to pinpoint sometimes what exactly is causing the performance problems.

 

Happy to help.

Debug frustum is in CullingCamera inspector. I can't check right now but you may need to expand something. I'll follow up this eve.

 

I doubt there is a profiler more detailed than the Unity one.

The frustum distance attribute is called Gizmo Distance on the Culling Camera.

 

I bought your sectr complete asset, and as usual, the first time I go to use it my scene makes things more difficult than they should be
I have a level consisting of a big old house, with multiple rooms on multiple levels - one of the lower areas is only accessible via a trapdoor, though it is 'exited" via a normal door opening - can I lay a portal down between sectors, ( horizontally) in the floor opening) or should I just include the lower area in a sector with the nearest upper area that uses a conventional portal position?
Thanks.

 

Portals can be placed in any orientation. Just make sure to get the front and back sectors right w/ regard to the general direction. Also, they don't have to be placed right at the edge of Sectors, they can be wherever they need to be.

 

Great, thanks for your quick response, but your reply begs the question: if I don't put the portal at the edge of a sector, doesn't that screw up the vis, and what would be the point of having a portal in the middle of a sector somewhere? (My apologies for the noob query, but I didn't come across this in the docs or videos).

 

It does not screw up VIS. If it did, I wouldn't recommend it.

Portals should be placed in things like windows and doorways which are usually near the edge, but are often not on the exact edge, of Sectors, especially for rooms that are irregularly shaped. The culling technique in VIS is very robust and designed to work in a wide range of spaces.

 

Ah, ok, got it, and thanks again for your quick response.

 

SECTR 1.1.3 is now live on the UAS. Includes VIS optimizations and some bug fixes.

 

Hello guys! So I've been working with most of Sectr Complete except audio (yet) and it's working better than I expected. It is really easy to use. So I recommend it.

Anyways. So I was watching this video on the making of AC Unity and around minute 5:07 they start showing a tool they made for their engine which can generate complex city setups in just seconds and easily editable too. Here:

So I was thinking that someone should make this for the Unity Engine. I think it would bring something new and necessary to the store. I know I would pay a lot for this thing. Make a code now! what do you say? Unless I missed something and someone already did this. In which case please point me towards it!

 

There's Buildr by Jasper Stocker.

 

I was looking at that. But it seems like it's more for creating a model than actually placing it. Buildr is useful if you don't want to model your building in and exterior software. And still if you want to get better result you have to export it to add details.

What I was talking about was more like a model layout placer. Where you have to have your own models but the tools would allow you to list different models under different themes. Then make the layout and the tool places them inside the drawn layout but also according to the theme you selected.

Like I'm an artist (not much of a programmer) and I have lots of models ready. But when the time comes that I have to place all these models. It gets really nuts. For example: I has a village that have five sectors controlled by five different clans. The buildings reflect the wealth and general ideals of these clans. So with this, it would be really easy for me to make a list for each different clan zone and each different district. I could have Clan 1 and then under that group another list that describes the wealth of that section. Clan 1 > poor / Clan 1 > rich etc. And then just drag each model into their respective list. Then all I have to do is draw the layout > select the theme (clan 1 > rich) and generate. And I don't have to go nuts.

Of course it doesn't have to restrict to buildings only.

BTW sorry guys. I know this has nothing to do with Sectr but I wanted to give you this idea.

 

I get it, that's something different if you have your own models. But to be fair, I was talking about this with Buildr:

 

Hello again! I just updated Sectr Complete and noticed that I can no longer look through more then one portal at a time (in a sequence). The room after the one I'm looking into is no longer being rendered. I looked through both vis and core manuals if something changed but couldn't find the answer. I wont be surprised if I missed it though.

 

Hi! There were some VIS opts in the last update that could be related. Of course it is possible to look through many portals as you can see in the demi scenes.

My first recommendation is that you double check your data. Make sure portal geo is good and that front and back sectors are hooked up correctly. I know of folks using latest VIS in very complex scenes, so I tend to suspect a data issue (most likely front and back are reversed)

If you've double checked your data and the problem persists, post some images here or email a sample problem package to support - makecodenow.com.

 

SECTR 1.1.4 is now out on the UAS. Includes AUDIO improvements, lots of bug fixes, and early Unity 5 compatibility!

 

I am having this error when trying to build on Windows Phone 8.1;

Assets\SECTR\Code\Vis\Scripts\SECTR_CullingCamera.cs(279,15): error CS0246: The type or namespace name 'Thread' could not be found (are you missing a using directive or an assembly reference?)

 

That bug is fixed in latest SECTR or you can just disable the threaded culling define at the top of culling camera.

 

I wanted to let you know that there was a bug in 1.1.4. I have a fix in test now, will post it as 1.1.4b ASAP. If you want it earlier, email support@makecodenow.com.

 

Just posted 1.1.4b. Now live with the last known VIS bug fixed.

 

I have purchased unity pro, testing on windows phone (lumia 520) but it's making the same hitch while loading(feels completely same), and it seems like it's loading chunks in 1 frame. Locks and loads in about 0.7 second. Is there anything I should do?
I've tested SECTR_Modules.HasPro (); and it returns true.

 

There are a few things possibly going on:

1) Make sure you have latest SECTR. There have been some good improvements lately.

2) It could be limited by script main thread time. Now that you have pro, use the profiler to see where the time is going. If the time is going into script components you can try to optimize them.

3) Unity async methods may just be bad on Win Phone. I know the threading primitives have been disabled on that platform and maybe that affects a sync loading too. You'd have to follow up with Unity to know for sure.

 

Just loaded SECTR into a new project, newest version of unity, and I'm getting this error:
Assets/3rd Party/Utilities/SECTR/Code/Core/Editor/SECTR_Menu.cs(82,32): error CS0029: Cannot implicitly convert type `UnityEngine.Camera' to `Camera'
Any thoughts? Thanks.

Edit: I'm actually running into this problem in a number of places in the 4.5.whateverisnewest...

 

Do you have other scripts or packages imported? It sounds like there is another script in your project that defines a class called Camera which is confusing the Unity compiler.