[RELEASED] World Streamer ! Forget about your game memory usage and create big world!

This what collider streaming could do but it load objects into memory, not make them vissible or not. Ummm I have similar script for my vr game but I don't decide to put it on asset store yet.

 

Still do you plan in future to solve this in different manner? Or there is some technical limitation to make it working for parented game objects? Or what about pointing world streamer directly to parent gameobject, practically defining layer with this.

As this approach with big count of flat objects in scene without parent gameobject is somehow against base guidelines. And it clutters your hierarchy.

Thanks
Peter

 

Hello! Is it possible to try this with already splitted scenes? We have a lot of them and I'd love to give it a try with World Streamer without merging them alltogether just to split them afterwards with your tool.

Cheers
Martin

 

Another question popping up: How big are the terrains in your car demo in units and which settings are you using? Detail, Height etc. ?

 

I am working on a game with a very large world. My world is 32x32 terrain pieces with each terrain piece being approximately 1500 until on each side.
Is there any limit to the number of terrain pieces that I could load using world streamer? Could it handle having a total of 1024 terrain pieces being in the world?
Does your loading and recentering system play nicely with different pathfinding systems such as A*?
Would a lot of coding be needed to integrate pathfinding into world streamer?

 

Sorry for bothering on a somwhat regular base. I felt the urgent need to buy it and give it a try. For testing purpose I used a complete fresh new project in Unity 5.3.5 added the GameFloat Fix Scene to the build settings and alle scenes in the SplitScenes folder. Opened the GameFloat, hit play and it works nicely. Unfortunately the build doesn't do more than showing me the loading screen and thats it. After deactivating the loading screen the result was ethan on one terrain tile but nothing else happened. Quick help is appreciated. I am in a hurry and need to find out wether it works with our project or not.

 

Just purchased your assets,

I am having problem to align the position of the Player and the terrain, it seems that the Player's position is not on top of the loaded terrain -> when I click Play the player drops off to infinity, How do you position the Players position relative to the loaded terrain so that it is "spawned" on top of the terrain.

Thank you for your assistance

 

First of all back from holidays a 2 weeks ago.. I made reply to everyone and after unity has backup forum... I lost every changes....so I have to answer again to everyone.... SORRY for late replay if somone didn't get any reply for his problem. Unity made big mess with forum.

We dont want to brake prefabs, lods and construction that you made in hierarchy. There is also problem with undo option when we will use catalogs, it could be hard to pack everything as it was before.

Terrains are 1000x1000x6000. Heightmap resolution 513, Detail resolution 256, detail per patch 16 (only unbuged unity value don't asky me why unity broke terrain in this place), control resolution 128 (it should be bigger), base texture resolution 128 (it should be bigger)

There is no limits I worked with several thousand of pieces. Sure it will handle that. Create few navmeshes and load them when player will be close enough. You should bake navmesh with whole models at the one place. You could only load small part of the world or whole world by local area updater. Than you bake navmesh for this area and put it on new layer. So this should be pretty easy. I dont know how about many navmesh connections but it should be fine. Even standard unity navmesh have such function. You could also use it but you coudn't use it with floating point fix system. We are not able to move unity navmesh until they will save it as objects (soon).

Add TAG into your project. That was the first thing that we mentioned in manual, it is very important. Without it, unity couldn't find our streamers and reference to them.

Could you give some screens? Do you use floating point fix system? Does player position match terrain position? Um simply we don't change any position until we dont use floating point system. Hmmm check if you use Y axis for streaming. If you are not creating game with space like "space games" you don't need it.

 

Info update about progress, we it lost few days ago because of forum backup:

upcoming update "1.7"

- Bug fixes with looped world which start from "-" positions. (done)
- New gizmos for sectors. You will be able to visualizehow far you stream every layer, compare them etc. (done)
- System speed up (done)
- Changes in UI to more intuitive (done)
-Collider streaming. You will be able to stream sectors that only show up when player hit collider. Usefull for building interiors! Collider could be looped, it will be also compatible with floating point fix system. With this system you could avoid loading big group of invisible models. (done)
-Ring streaming. You are able to load objects in specific range and than replace them when you are close or far enought. It could be usefull as LODs for far view terrains. At close distances you will have terrain at far you could load low quality mesh. You also could use it to replacing objects/models in different distances.(done)
- New tutorials and manual doc. It will be more based on typical asked questions.
- More world examples
- Streamer will get prefab that is easy to setup

 

Is the update already submitted? If you provide access to the updated package, I can prepare a good tutorial on youtube.

 

Not yet. We will change UI, We also working on new huge test scene and prefabs so please wait until we will upload it

 

anybody test this on console ? i bought sectr and its crap , really interested in this one but i fear it end like sectr any chance for trial ??

 

Is it true that before I Split Scene, I must put all objects as a root gameobject, so they do not have a parent at 0,0,0?
Will this work?


How do you assign terrain, models, gameobjects to a layer?

I'm having much trouble and confusion with current documentation & tutorials

 

Yes you have to unparent them.
If you want to set them into layer:
- put prefix Terrain for terrains so it will find all objects that have Terrain in name.
- cube for all objects that have cube in name or left empty prefix to catch into this layer all objects that left at the scene.
- you could also separate models by size in different layers to load small models in close distance, big in far distance. This will save a lot of memory cpu gpu vram usage look:
Before split:
https://gyazo.com/078c7bc4a7938cd2978bbfae5836ff6e
After split:
https://gyazo.com/22a449cb9b9ca3931945922d4d26eaa4
- new tutorial scene and tutorials are comming

The idea of our streaming is from console. Console had to load very fast and only things that are needed because of low memory capacity. You could adjust layers and sector size again in few seconds so you should be happy If it will not work for you, simply conntact me. If I will not able to help you, you always could take refund.

 

@NatureManufacture:
I have been playing with World Streamer and I like it so much!!!
But the current documentation/video tutorials are a bit confusing ....

Please, deliver the new tutorials and documentations... will be sooo appreciated.

 

We working at this We had few additional things to do because of community request

 

Thanks.

Could you please give us a rough estimate of the completion date for the new documentation?

 

2-3 weeks new version will be online. We finished whole systems. We need to test it a bit, add few gizmos, sort out ui and rewrite docs. UI changed a bit and new docs must be compatible. We also made new tutorial scenes. Tutorial scenes will show how to replace expensive unity terrain by cheap low poly mesh in far distance. without visual changes. This is huge step forward. Now you are able to replace objects by objects if your close/far enough. You could have very far view without performance drop. There will be also example with building. Streamer will stream only visible floors, so you could put very expensive and detailed interior and it will not affect your cpu/gpu/ram/vram.

 

@NatureManufacture
Could you please put/write/etc "Tooltips" on your config UI?
Thanks.

 

We will try Ah give them a hand and they will eat whole arm Np. We will try to make this tool as clear as possible to everyone. As we saw some devs have no problems other have few. From our side as a tool dev is hard to assess what is incomprehensibly. If something is very confusing please send me pm and point things that you have problem with. We will check this at our docs and update it. We will do our best! We added a lot of gizmos to help visualize streaming settings. We rebuilded UI to make it more clear. Anyway every additional things takes time so tooltips will took 1-2 days more probably;/, but yes they will be helpfull you will not have to check in doc settings so often.

 

I'm an asset store publisher as well as running a popular youtube channel for Unity/VR tutorials. https://www.youtube.com/nurfacegames/

I often hovered my mouse over items in the inspector to wait for a tooltip, and there were none. So tooltips would be great (and I don't think tooltip request is the "whole arm" xD)

As a native English speaker, I will tell you your documentation/videos are very difficult to understand and follow. It took me days to figure out WorldStreamer when it could have been easily learned in a proper tutorial video. I think the comments on the youtube are telling:
"Is there a video tutorial with voice and explanations, Im new to unity and I have this plugin."
"I can only hear music, how do you use this???"
"Sir, I just recently bought this asset. I tried reading the manual and its like greek to me. The tutorial above with just subtitles doesn't help either."
"I did not understand nothing from that tut,"

As previously mentioned, I am going to make a tutorial for this package on my channel (as it's badly needed). Again, you're welcome to use my tutorial for your official tutorial, as I'll much something much easier to understand. Either way, I'll post a tutorial here to help your users out!

 

We will add tooltips hehe that was just joke about this arm
Yes that why we going to make tutorials when new version will be ready. We changed the UI so we have to wait until everything will be finished. If you also would love to make tutorial feel free, just wait for new version I also could add it as official. We don't have native english peoples so we would use you in tutorial. We had few peoples that want to make tutorial but we said to everyone - wait until we finish that version.

 

Great, thanks NatureManufacture! We really like your assets and can make some really cool games with the tools

 

We decided to create VERY detailed doc. There will be huge sections about each object that you could stream.


WORLD STREAMER MANUAL.

0. About World Streamer.
1. Idea of streaming - very important.
2. World System project requirements.
3. Quick Start - Test Scene.
4. World Streamer details and settings.
......4.1 Scene splitter.
......4.2 Streamer objects.
......4.3 Teleport or Respawn.
......4.4 Loading Screen UI for game start/teleport/respawn.
......4.5 Floating Point Fix system..
......4.6 How to loop your whole world or specific object/models.
......4.7 Ring streaming idea.
5. Terrain Streaming..
......5.1Terrain preparation (unity terrain, low poly mesh, custom solutions)13
......5.2 Scene split and sector setup.
......5.3 Streamer setup at your game scene.
......5.4 Ring streaming and how to replace detailed terrain object by low poly mesh at far distance.
......5.5 Huge amount of details at Unity Terrain and performance solutions.
......5.6 Workflow solutions for terrain in pieces.
......5.7 Create looped terrain.
6. Model/Objects Streaming..
......6.1 Model preparation.
......6.2 Scene split and sector setup.
......6.3 Streamer setup at your game scene.
7. Streaming by colliders.
......7.1 Object preparation.
......7.2 Scene split and generation.
......7.3 Collider streamer setup at your game scene.
8. Multiple layer streaming - why you need this.
9. Local Area updater.
......9.1 Workplace setup.
......9.2 Loading your sectors in one or multiple layers.
......9.3 Update your sectors.
10. AI navmesh solutions.
......10.1 Custom navmesh solutions.
......10.2 Unity navmesh.
11. Version changes.
12. TIPS, additional info, performance problems and solutions.
13. FAQ..
14. Our other assets that might be helpful

If you want additional information about something please send me a sign.

 

Thank so much NatureManufacture for taking care of the documentation needs of your customers.

 

Update: I've added few additional sections into tutorial. At the moment it have 33 pages without images.... and I just started point 6. This will be book about solutions, with small tutorials.
I will say probably this will be a "bible" of optimization and asset management.
Probably I will need help at the end to check if everthing will be clear for users.

 

Count on me.

 

I'll definitely check out the manual with the update

 

Today I will have first version without images and with weak one section (collider streaming). Our coder just goes for unexpected holidays (last minute), so probably we will get 1 week delay. Anyway I will provide new doc before we release this version. Most parts of the streamer didn't changed. We add some prefabs and new UI, regardless of this dosc probably will help you alot.

 

Manual link:
New manual for World streamer 1.7 version. It is underd evelopment (no photos with UI)

You don't have prefabs that are mentioned in manual but... to be honest, you could create them from test scene that you have in 1.5 World Streamer Version. Object names are mostly the same. Why I attach this manual now? This is huge doc/library which will help you even if doc it's not ready. It contains ideas, solutions, tutorials and previes of upcoming and actual Asset Store packs, which will cure your problems. If something is not clear for you, give me a sign. This is alpha document. All language mistakes I will remove during next week. I will actualize this doc probably everyday. Feedback from your side will be cool. Read it carefully!

Ah, tommorow 30.07.2016 our Multiple Terrain Brush will be on 24h sale. So if someone wondering about buying it, tommorow will be nice chance.

Multiple Terrain Brush new video:

 

The new manual is much better, I actually understand how it works now! The new features in 1.7 look really good so I can't wait to get it once released.

A few things I am still slightly unclear about are:

- the way you setup a player's position. I want to be able to use PlayerPrefs to enable the player to save their position (provided they are stationery with their feet on the ground), and then enable them to load from the last saved position

- the way the loading screen is setup to avoid seeing the world load as the game starts. Does this also prevent the player from falling through the ground before the ground has loaded?

- can you use the Local Area Updater to load Collider scenes?

Thank you for the new manual!

 

Hi!
- You could do this by creating teleports/spawners or by moving teleports/spawners objects
-nope every game have different architecture. You could do this on few ways
a) simply turn on platform under player foots and turn off it when loading will end
b) for multiplayer games you could sent player to safe place, replace in streamers "player" to object that is player spawn position. When loading will end, move player to spawn position and replace "player" by real player
c) edit: we always load area under player feet first, so in many cases you don't have to change anything.
-nope yet, but you simply could move scene into hierarchy. We will add this in 1.8version
Thanks for feedback.

Edit 1. Ah we rebuilded whole UI, there are less place where you have to drag and drop something so.. streamer will be easier to use few times. Also we added new feature for collider streaming.
- You could stream something for example when your player hit the collider.
- You could stream something when a bullet or other object hit the collider

Edit 2. System is ready now I must finish docs and example scenes. Everything will be available next week

Update "1.7" Upload date: 09.08.2016. Please wait few days, unity must accept it. If someone need it earlier please e-mail us with invoice number.

- Bug fixes with looped world which start from "-" positions. (done)
- Bug fixes with looped non-square world (done)
- Bug fixes with local area updater (non default scene directory) (done)
- Bug fixes with scene splitter (non default scene directory) (done)
- New gizmos for sectors. You will be able to visualize how far you stream every layer, compare them etc. (done)
- System speed up (done)
- Changes in UI to more intuitive, we removed some windows and we make system more automatic (done)
-Collider streaming. You will be able to stream sectors that only show up when player hit collider. Useful for building interiors! Collider could be looped, it will be also compatible with floating point fix system. With this system you could avoid loading big group of invisible models. (done)
-Ring streaming. You are able to load objects in specific range and then replace them when you are close or far enought. It could be usefull as LODs for far view terrains. At close distances you will have terrain at far you could load low quality mesh. You also could use it to replacing objects/models in different distances.(done)
- New tutorials and manual doc. It will be more based on typical asked questions. (done)
- Streamer objects will get prefabs that are easy to setup (done)
- Tooltips (done)

 

Can you please elaborate on these two points?

I would like to create an endless world and the only way to do this is with procedural generation.
How do we create blank terrains at runtime in the right position in the world (and in the streaming asset), and how do we set which streaming layer they belong to, etc.

Does the asset handle all of this for us or must we do it manually?

I imagine that with these features you could create a minecraft remake easily but I simply want to create an endless 2d platform game with a room-based structure. Could I design the rooms beforehand and arrange them procedurally at runtime then stream them in and out with this asset?

 

Hi!

"in few clicks"
- When you have your terrain in parts, you only have to create layer (virtual grid) and click split. Virtual grid element size must be the same as terrain part width, length
- Then you have to click generate button - system will create scenes from virtual grid elements and prefab "SC" scene collection that holds info about scenes.
- You drag and drop streamer prefab and fill it with "SC" prefab. Done, basic functionality is working. If you want to loop the simply check checkbox "looped".

Endless = looped or extremely big
Looped - Your world will behave like a planet. You will always back to the same place.
Extremely big - There is no world size limit, we even handle engine problems with big position numbers and the accuracy of calculations.

Here is manual for 1.7 upcoming version, please check it. This version should be available today

Manual should help you a to understand our streaming ideas

"Could I stream procedurally generated terrain?"
Manual : Page 37 - real time generated terrains.
We think this idea is correct, but we didn't tested it, thats why didn't write much about this kind of streaming.
There is also big adventage of this solution: You could back to the same place. When you fill prefabs by data, you could stream the same area again, without generating it again.

Nope, blank terrains are must be prepared before streaming, they will be split into virtual grid and system will generate scenes from them. Thats why they will have proper layer etc. You only fill streamed objects by your data. You have to fill terrain part prefabs by data by your-self/your script.

Hmmm I will write few words more about this in our manual if you want. Manual should be actualized this week. We will also release 3-5 videos about world streamer in upcoming 2-3 weeks.

Could you tell something more about room-based structure? Procedurally generated world is always problem, from a logical point of view, if you generate it in run-time, you should also unload your assets at runtime. Your system should do both things. I see few ways to connect it with world streamer but there is always work from your side, you have to "fill" streaming data. We could load/unload this data.

 

Endless = infinite.

As you explore the world keeps procedurally generating so you can never reach the end.
Thus it is impossible to generate an infinite terrain before runtime and slice into parts.

So if you want to have 16 tiles/parts loaded at once you need to make 16 blank tiles before runtime then at runtime you modify the 16 tiles somehow to represent the different parts of the map?
Or are the blank tiles only required for the newly created parts and the existing parts can be loaded from disk?
Or are they not required at all if you generate the parts at runtime before placing them?

With a room-based structure where rooms (or parts of rooms) are already constructed by hand before runtime and saved as scenes/tiles can the World Streamer load and unload each room as the player moves around (as long as it has a guide of where to place each room eg. a map or array which can be generated procedurally), and thus rooms can be reused in multiple locations procedurally which perhaps eliminates the need to continuously modify blank tiles.

I guess the World Streamer is already loading and unloading the individual tiles or scenes from some type of map or array so it would just be a case of accessing this and/or creating it at runtime (and possibly saving each part as it generates if persistence of changes is required). Of course the map/array would need the ability to expand in any direction (or in 4 directions in my case).

 

Hi!
New 1.7 version is already available!

Here is a changelog:

Update "1.7"
- Bug fixes with looped world which start from "-" positions.
- Bug fixes with looped non-square world
- Bug fixes with local area updater (non default scene directory)
- Bug fixes with scene splitter (non default scene directory)
- New gizmos for sectors. You will be able to visualize how far you stream every layer, compare them etc.
- System speed up
- Changes in UI to more intuitive, we removed some windows and we make system more automatic
- Collider streaming. You will be able to stream sectors that only show up when player hit collider. Useful for building interiors! Collider could be looped, it will be also compatible with floating point fix system. With this system you could avoid loading big group of invisible models.
-Ring streaming. You are able to load objects in specific range and then replace them when you are close or far enought. It could be usefull as LODs for far view terrains. At close distances you will have terrain at far you could load low quality mesh. You also could use it to replacing objects/models in different distances.
- New tutorials and manual doc. It will be more based on typical asked questions.
- Streamer objects will get prefabs that are easy to setup
- Tooltips

Upcoming changes "1.7.5", we are going to upload this probably at the end of next week :
- minor bug fixing, they do not affect system workflow (done)
- video tutorial series (simple and more advanced)
- manual laguage changes (50% done)
- more info about real time generated data and world streamer workflow.

Yes if you look at this from this way that is true. Before you hit play you have to prepare as many blank tiles for streaming as you need. If you use floating point fix and looping you could use small trick to create unlimited world.

Here is the trick idea, you don't have to modify anything and you system will work as you need:
1) If you want hold in memory 3x3 blank tiles and you will prepare (create) 8x8 blank tiles. Those tiles will be only used for looping and floating point fix system.
2) You have to check looping at streamer and add "world_mover object" at the scene and connect it with streamer. At world mover you could set distance after which system will restart whole world position, for example after 4 terrain tiles.
3) After you hit play, system will load tiles so you have to fill it by your terrain data. Even if system will load tiles that have been used before because of looping, you will also overwrite them by new data.
4) Terrain tiles pool will never end, because in this case we use looping to recycle tiles. You also have floating point fix so all problem with phisics or big numbers just don't exist.
5) Blank tiles in this solution will be only used as folder where you put your data, they will never end, because they are looped. There is probably a disadvantage of this solution. You probably could't back to the same place, because we don't store previously generated data. Ofc i don't know how your generating system works, maybe it will handle this.

I hope this will help. If something is not clear feel free to ask, we could expand this idea. I didn't asnwer your few questions but probably this "trick idea" probably will give you a clear view.

System will work fine with your room construction. It's even easier than with terrains. You could stream in X, Y, Z axis. 4 directions? XYZ and? time?. You could use our collider streaming or normal world streamer workflow.

Yes your 16 parts represent different parts of map, ofc you could load the same tile few times so "different parts" is not so sure.

I love such discussions, they open mind and shows new possibilities of use for our World Streamer.

 

would love a step by step tutorial on how to use this with ufps multiplayer

 

We will add few tutorials soon 1-2 weeks probably. Btw check multiplayer section in manual.

 

Will World Streamer work with Unet or other multiplayer/MMO systems? Can it be made to work with client/server configuration?

Oops, noticed the post above..lol..and found it in the manual. Thanks.

 

i cant figure it out, i tried but it seems to spawn my guy a whole bunch of times while loading the scenes so when it eventually shows it its all abit whacked.

 

Could you say something more about that? Do you use floating point fix ? Do you use looping? You spawn a player, our loading screen shows and when it ends what happend?

Btw this what you do with your player until loading screen has dissapear is your choice. You could move it to safe place or block at starting place until streamer will end loading. You also could simply left your player in starting place add some y distance over the ground/models and system will handle this, world streamer is loading tile under player legs first

Btw please update your World Streamer Assets, There are small changes in manual - >looping section and we also fix advanced demo scene.

 

oops sorry i should have come back to this earlier but i needed a break from it lol.
So installed ufps and the ufps multiplayer add-on, then world streamer. i used the simple floating point fix and added a spawn to it. adding the prefab of the mphero to the managers, removing the thirdperson guy and setting the build scenes. So as i play and login i see the hero is obviously spawning when its loading the scenes but not just once but several times for some reason. when it finally shows the scene there is alot of the hero players (lots of arms and people with no heads lol) and they move when i do. But also its not streaming the other parts of the map, i guess it is something to do with the way the ufps multiplayer prefab is cloned (mphero(clone)) so i picking the original is pointless it seems. As for it making loads of my player i have no idea. I'm sure its more of a problem with the way the multiplayer is set up in ufps than this system but im not really very good at coding so it boggles my mind. If anyone else has managed to get it working i'm all ears and eyes.

 

Hi!
Basically World Streamer have nothing to network solutions, it only have additional info for rpc when you use floatingp oint system.
Here is some questions which should help.
- It only happens when you add worlld streamer?
- Did you try without floating point fix system?
- Did you use difference in position between player and server (it happends when you use floating point fix)? World Mover hold it?

Yes streamer have attached objects which system fallows. You could change it dynamicly.

Call me by skype we could take a look on it together.

Upcoming changes "1.7.2" uploaded 29.08.2016
- Faster gc solution, it works 10x faster now
- Hide few debug values
- Own terrain culling system based on camera view. With it you probably will save 90-10% of terrain cpu/gpu usage.
- Minor manual changes
- More info about real time generated data and world streamer workflow.
- Minor bug fixing, they do not affect system workflow

Upcoming changes "1.7.5", anticipated upload 1.09.2016:

- video tutorial series (simple and more advanced)
- manual laguage changes (50% done)

 

Hi!
Here is our first tutorial!

 

Nice, cant wait for next one on large scale streaming.

 

Few min ago unity accept our 1.7.2 version!

Update "1.7.2"
- Faster gc solution, it works 10x faster now
- Hide few debug values
- Own terrain culling system based on camera view. With it you probably will save 90-10% of terrain cpu/gpu usage.
- Minor manual changes
- More info about real time generated data and world streamer workflow.
- Minor bug fixing, they do not affect system workflow

 

@NatureManufacture will world streamer give us close to the same performance working in the editor?

 

Just curious, how you found a way how to enable terrain without apparant spike?

 

Should be fine, we use Bolt . UConstruct has Bolt , Unet etc working with it and integrated World Streamer...I believe Gloria Victis MMO was made with World Streamer. So shouldn't be an issue.

 

In editor world streamer is 10000% slower because during loading it must do operations which are done in game build process like:
- batching
- collider batching
- collider baking
- asset baking
- also access to memory in unity editor is veeery slow
At build all this things will disappear and game will be very smooth. In manual you will find many tips/tricks about editor workflow..

I fighting with this about few years and to be honest it's changing from version to version. I';m not completely satisfied because we left material update in cpu (about 0,01-0,1 ms) per terrain.

Gloria Victis use bugged uLink. Stay away from uLink system... their support and bug fixing time is about 1 year... to fix few lines stupid bug. They will probably change it because uLink even do not support unity 5.4.... They also use very old World Streamer version, so they don't have 80% of updates. It's hard to actualize each system in such big project.