[RELEASED] World Streamer ! Forget about your game memory usage and create big world!

hi,
I have some questions before i buy the product,

1- From what size is recomended to use the Floating point error fix system? My terrain is 3x3(9 km2).

2- I am using terrain composer and chunks are 512x512.Is TC2 compatible with WS?Any guide?

3- Navmesh and dynamic objects.I will have animals in my scene that patrol and wander by the terrain.I have seen in videos that SECTR has an hibernate script (seems to hibernate children,behaviours,colliders,rigidbody,every one optional with a checkbox).When the chunk is deactivated the object hibernates and when is activated stops hibernating and continues whatevet it was doing before.This is what i have understood from their docs and videos, may be i am wrong.

-Is posible to do something similar with World Streamer?
-In case is possible, what will matters if i have a flee behaviour in an animal and runs away opposite the player and reach a chunk that is deactivated?Will fall down or the navmesh is intact and at the moment it is in the deactivated chunk should hibernate?

Thank you and excuse i am not very good in scripting

 

1. In my opinion if world is bigger then -10 - 10k or -20k to 20k
2.Yeah it will works. So 1 terrain is 3km x 3km terrain heightmap resolution is 512x512. Hmmm How far you would like to have terrain at the screen? How many terrains would you like to hold at memory at the same time?
3. Hibernation is bad idea... You still have this objects at memory and you rise object amount at scene which will slow down Garbage collector and it will also slow down cpu (it must check all object must) Solution with hibernation is good for small area small amount of models but then you don't need streaming. If you will call GC (unloading assets from memory) you will get spike and it will depends from number of objects at the scene so hibernation will kill your CPU. Basically their idea is wrong in this area in my opinion. You should spawn Ai only when you need it in close distance. You dont need navmesh you could use grids or ai without navmesh - Apex solution. It work fast even at mobiles.Btw I give you a tip. If you want to use navmesh you could let him stay (not stream) or stream in other layer (larger). In far distance you could replace your AI by pointers that moves across the world. You will have to build your own script which instantiate models and animators inside pointers only when you are close enought. This script should be very easy it simply spawn aobject inside pointier which represent AI in far distance. This is best solution and it will remove 99% of cpu usage from AI. It "ms" will just dissapear.

Navmesh is pretty light object you could let him stay in the scene to be honest. Ofc you could load/unload it in layers which will hold them in very long distance.

 

Thanks for your quick answer,

Well, mi terrain has 36 chunks of 512x512.Many trees as forests have,animals and objects.I was thinking to have active the players chunk plus another chunk around the player, so total 9 chunks every time.I haven´t tested deeply already if it is a low number and if i will have visual problems when chunks are higher than others for the distance.I mean, things appearing suddenly.The total square is a bit more of 2 square km at the same time.

- Does WS streams the hole chunk of terrain or can be configurated to stream for example every 300 sqm?

- The replacement by pointers is a good idea.
I´ve been thinking to spawn AI randomly within the next chunk of terrain active from the player and unspawn it the same way, may be with an script that measures the distance from player to be spawned, lets say in my case 500 - 600 meters from player and with patrol points so it nevers go far from the chunk that is active in case of flee.I have read you have some trigger script to spawn objects.What do you think?

Thank you

 

Yes this idea about chunk is correct you also could have more "far view" by LQ mesh in background. It's easy to do, I will make tutorial about this today. World streamer only stream terrain chunk one by one. So everything depends from chunk size.
Trigger to spawn? Hmmm we have collider streaming system. While you are in collider system holds data for example building interior but this could be anything even AI. Ofc you could also stream AI like other objects

 

Upcomming Update "1.8" Probably Upload 31.10.2016
- World Streamer Tutorial 5. (done)
- Fix for player position gizmo if you are not using world mover. (done)
- Save/load setup for Local Area Updater. (done)
- Additional support for spawned players. (in progress)

World Streamer Tutorial 5 :

 

Is this compatible with TerrainComposer 2? I know it says about TerrainCoposer but number 2? lol thanks

 

Sure

 

I was actually going over a video of TerrainComposer 2 and it does come with its own terrain streaming though? so what would you suggest?

 

You know that you can only stream procedural/made by graphs objects and terrain ? You couldn't stream manual made things. At terrain composer 2 site you also have info about that if you want to stream manual placed/modified things you should use World Streamer We are in co-operation and close relations with Nathaniel for almost 2 years.


We also happy to say that we uploaded new world streamer version today

Update "1.8" Uploaded 31.10.2016 waiting for accept
- World Streamer Tutorial 5.
- Fix for player position gizmo if you are not using world mover.
- Save/load setup for Local Area Updater.
- Additional support for spawned players.

 

Oooooo you learn things everyday, thank you for the info - and thanks too for the recent release

 

I still didn't exactly get how could it be possible to bake lightmaps with the "Local Updater".

I mean, say I have a huge city and I use local updater to bake lightmap of a part of the city, then after that I'd use the "Local Updater" to bake light map of another part of the city. Now, wouldn't all the baked data of the first part of the city get deleted when I used the "Local Updater" for the other part? (I won't bake lightmaps for terrains and I use Unity 5.3.6)

@NatureManufacture Please answer. Thanks in advance.

 

I was wondering if MapMagic will work with this? I'm considering buying both on payday if it will.

 

Thought the same, even together with VoxelLand.
Can't think about an easier solution with those three to get a good running open world without the need to open a modeller for everything.

 

Well I think voxeland is getting incorporated into MapMagic, so it does seem like the ideal tool to use with World Streamer to me as well.

 

You might have to bake your terrains to mesh but mesh terrains for World Streamer is unknown/untested at this time. Voxeland has it's own chunk loading system and as an editor or real time mode the way it is wont work. Also you can only have one Voxeland per scene but you can make it unlimited if you want. I own both assets and have asked the same question. The new Voxeland and MapMagic +integration might be different..it's coming out in a month or two.

@NatureManufacture about adding the new spawner support this isn't the same as what I was requesting before with a Ring Streamer script to follow an object and replace any mesh nearby with in a set distance?

Also would I have to delete World Streamer folders for each update? Thanks.

 

Uh so many questions. Ok one by one.

Yeap and I love that. Everytime somone need help from us and show us how he/she want to use world streamer we are surpriced

Hm.....When you bake light in another part and you load last part it will be overwrited because you bake it again. I undestand you need last parts because it contains light which will affect neighbour parts?

Hard to say we didn't test it. If you can save world in tiles then you can load it by our system

You could load voxel terrain in parts or save them as mesh and also load/unload them. We use meshes in ring streamening to replace heavy unity terrain by something simple in far distance.

Spawn script is to support players/objects which will be spawned at the scene, they not exist at the start. This also could be used to many things For example streamer wait for bomb which will be spawned and connect into it

No just download new version and done

 

No, I actually won't re-bake a part. What I ask is something like this:
Imagine two objects which are in the work scene. the first one is in the the south of the scene and the other one is in the north. Imagine I want to bake light maps for them. So I use "local updater" to load the object in Local Update scene and then I bake lightmap and save it and close it. Then I redo all of these processes again only for the second object. the question is does baking the lightmap for the 2nd object delete the lightmap of the 1st object?. I hope it's clear.

 

Lightmap is per scene so it will not delete/replace lightmaps at scenes which you didn't load.

 

So I was wondering about using this with emerald AI on a dedicated server to put mobs on a large world. Can I use this to make a Low Rez Proxy of all the terrains for the server? The client would use your plugin but the server would need to run with all the land for collisions for wandering Monsters / NPC's etc..

 

You mean:
- Player use streaming for world
- Server is in batch mode (without or even with streaming) it only holds collisions, navmesh and AI?
Yes, World Streamer support such construction. You don't have to stream assets on server side if it's in batchmode. Even whole MMO world takes 300-700 mb of ram. ( I know because I build such constructions and games) If you use batchmode whole rendering is off so your cpu and optional gpu is used only for calculations (AI and collisions). Client/Player will heave streamed world with details and server will only have "World Shape" You could even cluster server and handle floating point fix system because we add outside values which help in client-server communication when you use floating point fix system.

 

Excellent!! On Friday/Payday you have a new customer. If you have any docs or tutorials about setting this up properly I would love to get the links for them.

 

On first page of this thread you will find:
- huge manual
- 5 video tutorials
- in manual you always have contact to me if you will find any problems
Feel free to ask any questions

 

What is the recommended way of handling persistent physics objects with this tool? If the terrain gets removed they will just fall forever : p
Since they can be picked up and moved, they cant be tied to a layer either, so maybe the only solution is to create some proximity script that turns them of when they get a certain distance from the player.

 

I've been asking about almost this exact question and the response seems to be that the new Player/Object Spawner script may be the solution.. because for my project I need an object replaced that is distant from the Player and is not effected by the regular World Streamer process/LOD that works from only the player's distance/view.

 

Phisics objects should be unloaded earlier then terrain You could freeze phiscis by yourself if objects is far enough this is even cool because you save phisics usage (turn on cinematic in rigidbody). or... We could make such script 10 -20 min and it's done. I will try to catch coder tommorow.

 

Yeah, i can do it myself, thats easy : D
Was just asking to see if you had anything.
My idea is to just add the script to the object when the player drops it, and then when the player gets far enough, it just disables the physics. Or, maybe its more efficient to have some kind of global object manager that has a list of all physics objects. Have to think about it : D

 

I've had this for quite a while, but just finally started testing. I'm using the Advanced_Game for testing and I'm seeing an error that says Player is not connected to Position Gizmo. I see mention of some fix coming for Position Gizmo so maybe this is the issue? Is there something I can do to make the error go away? The scene seems to run fine in the editor, but when I run on the Wii U, all I see is a blank scene with the default skybox. I don't even see the player. And no other errors are reported except the player gizmo one.

 

a) Just turn off this object. Yes patch which fix that is waiting for unity accept.You always could ask at e-mail for fresh version.
b) Scene with streamer objects should be first one at build settings or simply just before small streamed scenes.

 

I'm so new I have no idea what object you might be referring to.

I'm pretty sure this is the case, but I'll double check when I get home tonight.

 

Could not resist buying this now!
Setup was really easy and quick, thanx for a great asset.
I do of course have a problem though : D
All was working good wit hthe streaming until ai added the culling scripts, now the streaming does not seem to work at all and the camera culling dont do anything either.
Rendering Distance option, is that for culling terrain in front of the camera behind that distance?

 

Ohh, and your asset generates a crapload of warnings when making a build, would be happy if you could fix that so i dont have to sort through it everytime in search for some important warnings : )

 

Check in game camera view. In editor/scene camera could not be refreshing. We have to fight with this a bit. We use Low level api to do this and this api works only with game camera. I don't know why ask Unity hehe... We will do our best to solve that. Streaming should work but terrains will not show in scene view without game view mode on.
Yes rendering distance is from camera to object center.
This warnings are because we support 5.1.4 to 5.5+ . Unity messsed alot in loading functions in this versions,. We could not write streamer for each unity version but... probably soon we will upload specific version for 5.3+ only, without any warnings

 

I did check in game camera, and everything is messed up now : p Only 2 tiles are loading, and no streaming at all. Streaming worked perfectly until i added the culling script to the terrains.

 

But culling system have nothing to streaming. Streaming streams it anyway culling just hide objects behind camera so do not connect this 2 things. Check if all scenes are added to build settings (at streamers) and pass few screens from hierarchy and streamers.

 

I do not connect anything : ) Just saying that the culling scripts are affecting the end result in a bad way : )
I just tried to remove the culling scripts and now the streaming is working perfectly again.

 

I found the Position_Gizmo object in the scene and turned it off. What happens is that now I see the same results when running in the editor that I got on the Wii U. I just see an empty scene with a skybox. And I checked and the first scene in the build is the Tutorial World Streamer - Advanced_Game, which has the streamer objects. Just to be clear though, the first thing I see is the World Streamer loading screen. Once that completes loading then I see the empty scene.

 

Very strange.
a) Do you have all tags on streamers, managers etc which are included in manual? In editor this could somtimes work but at build everything will crash without this tags.
b) You have all scenes at build settings? (no mesages at streamers)
c) Does loading screen behave normaly? or it just skip loading?
d) This is almost 99% sure this is the reson - maybe player is under the world and you should hold his position until streamer will load data? This could happend, even if we in first step load data under the player. At pc it load so fast so this have no effect. 1st you could try to move up player a bit by 100 units or block it. Devs have different way to spawn/load player. Some of them turn on cinematic at collider, other move player into safe place until world will load etc. It depends from game AI and much more things that we could expect. This is the reson why we didn't put our finger there but maybe we should. Simple script/system which will hold player until loading will end it's about 5 min of work.

Hmmm Hmm Hmmm but you said they were not streamed so they were at hierarchy and was inivisble or they simply didn't load. Our culling system only hide object nothing more. I would love to take a look on this. Btw you use this culling scripts on unity terrains only?

 

Ok, case closed, lol.
I totally forgot that i had another camera in the scene, so it was picking the wrong one : D
If you use Camera.main, you will pick the first one tagged MainCamera. but if you instead use Camera.current, it will use the camera thats actually used for rendering. Might be worth changing that.

 

Hmm, setting it as Camera.current messed up another scene instead, what a bother : D
Just have to make sure i disable any unused cameras then : )

 

I'm using your scene. So I assume it has everything it needs.

Yes.

Totally normal and goes to 100% before empty scene.

I was thinking this too and will probably try this. But why the empty scene when position gizmo is turned off on PC dev station?

 

@NatureManufacture I forgot to ask you if the new spawner script is also a solution for mesh terrains bypassing the need to use the terrain neighbor script

 

I guess we can add whatever stuff to the terrain layer even if its not terrain?

 

Yeah but in my opinion it's not good deal. Unity terrain is heavy to load and in most cases is an object which you see in far view. The idea is to load only necessary objects so small stones in close distance etc. This will eliminate culling from cpu and lod refreshing. Ofc you could try why not.

I don't know what do you mean. Neighbour script just blend edges. Spawner in 1.8 is just a bool with few functions to search/wait for player/object appear

If you go under the terrain until loading screen will gone you are far away under the world so scene seams to be clear.

Maybe we will fight with that in next release 1.9
.
In monday we will give a list of 1.9 features which is based on your needs.
.

 

You're missing the point. The scene works fine on the PC every single time except when I turn off the Position_Gizmo object. How does turning that off cause this scenario?

 

Ok, im not planing on loading loads of objects with it, just have some cave parts that would be good to load at the same time, so i dont get big holes in the terrain : )

 

Caves etc you could load by collider streamer. They will be unloaded until player will be close to the cave enter This will work better then occlusion culling becasue they are gone from cpu. So you could load models that hide holes but interiors for example only byc collider streamer. When player will be outside above cave - interior will be unloaded

Hmmm this is very strange! This is only gizmo with position which is connected to player and world mover if you use floating point fix? You don't have any error? You could even remove this object. Btw version with fixed gizmo should be available in 2-4 days. It was uploaded 1 week ago. Anyway this shoudn't affect streaming. Maybe something else you setup. Could you pass few screens or met by skype?

 

Yeah, but i need to load the initial part of the cave with the terrain : )
This system is awesome btw : D

 

Thanks don't forget to rate it on assetstore Yeap you have right with this cave,

 

You had said over a week ago or so that you don't know if mesh based terrains would be compatible with World Streamer and said something about you needing to know what references each other. I had explained that I was using Voxeland terrains which did have some kind of referencing system and it's own LOD.

I'd like to use some mesh based terrains but if there is no problem with that then great!

 

Any news about this? Did you handle that?

upcomming Update "1.9"
- System that holds or moves player to safe place during loading time.
- System which manage phisics when player is far from objects
- Terrain culling camera setup changes