[RELEASED] World Streamer ! Forget about your game memory usage and create big world!

OCC is saved at scene like lightmaps AO, Navmesh. Yes it could be streamed.

 

Collider streaming will allow you to stream manualy but... it would be heavy. Streaming should be part by part step by step.
Anyway you could hack:
1st Island - 1 st group of streamers
2nd island - next group of streamers
All islands are at the same position and:
When player leave island you move him to 0 0 0 or other point and:
- turn off 1st group of streamers
- turn on 2nd group of streamers
etc...
Um just a tricky idea What do you think?

 

@

Thanks, I'll have to test the steaming out for smaller static content, you made a good point about the nav mesh, thanks for reminding me that it needs a ground level not to fall through at close range lol maybe a meshes collisions could be loaded only when the camera isn't looking? then it would make sense/be safe to stream meshes at close range.

Also another question.. I've seen previous posts about people requesting the ability to undo adding content to World Streamer is this now possible? I think if you don't already have it maybe the current working scene could be made to be saved in a folder with the state of the scene before being processed and name of the action and when pressing undo the scene could be loaded inactive within the current scene so it can be unloaded. There are many things that may make us not think to do save scene manually.. many people will assume that Undo exists and just works.

 

I just got a message from Unity that this has been fixed in the new beta. I was also told it was fixed in version 2017.3, lol, so maybe this time it is fixed. (This error stopped once I stopped using the foliage culling, so maybe this fix fixes that as well, not sure exactly what they fixed.)

 

Is there a recommended way to handle the fact that it takes several seconds for World Streamer to load the initial terrain, but while that's going on, the scene is already running and my characters need something to stand on. Not to mention other things I've set up to happen at the beginning of the scene, assuming the scene will be fully loaded and ready to go at Awake time. Do I need to add extra code to freeze everything in my game mid-air until this has loaded? Or is there some way for the system to load some things at awake, so they're ready before the Update loop starts?

 

Navmesh could be dynamic or not. There are something in manual about navmesh. Anyway there are few cool solutions which really works good for streaming. I must think about this, anyway we have few ideas for new streamer version. We will take this into account.

Yes, move player to save place.
Game is starts - streamer fallow pointer where player will spawn, atm player is in save place/platform
Game is running - you move player to correct place nad replace streamer fallow from pointer to player object.

Also system starts to loading from area near player so it shodunt happend.

 

I'm sorry, I don't quite understand. Are you saying I should place the player character on a special platform at the startup, and then spend many frames (and Update calls) waiting for the terrain to load? And then when it's done, World Streamer will move my player object to the real starting position?

The problem is that I've assumed so far that I can start all the gameplay mechanisms at Awake/Start. And I have more going on in the world than just the player. I spawn groups of non-player characters that start performing their routines etc. and require something under their feet as they do so (doesn't have to be the best quality). So it sounds like I'd have to freeze all of that, skipping all Update calls, freezing all physics etc. while I wait for the plugin to load. This is possible but pretty awkward. It would me great if there was an option to load the low-quality terrains for all areas, and the good quality terrains for the player start position, in Awake.

Also, please clarify: What shouldn't happen (because it starts loading from the area near the player)?

 

And how will you get this low quality terrain? (you could get by 3rd party tools) and what if user dont use terrains? This tool is universal not for one game solutions. We load point clouds, terrains,models, lightmaps,navmesh, online objects etc.. We not creating solution for direct game or app type so what will happend and when is on user side. Ofc we give solutions and tips how to handle few things so:

This means that terrain under player should be loaded at first file. Does terrains are your main streamers? Maybe object should be moved up a bit even 4-8 units until unity engine will start loading anything (loading is async thats why you have problem, its not 1 frame but many to keep it smooth). Object during this frames moves down and it's under colision when 1st tile of the world is loaded.

Yeah freeze is good for single player (world streamer throw info when loading screen is finished), for mmo and multiplayer games safe place and object representation is better.

I hope I help a bit.

 

Thanks for the responses so far!

I don't claim to know all the technicalities of this. All I'm saying is that it would be nice to be able to have something loaded at Awake (I don't mind some lack of smoothness), so I can design my scene in the usual Unity fashion. The freezing solution will work for me but it's not ideal. I'm lucky that I update most of my objects in a top-down fashion. If I was using Update, LateUpdate, etc. calls normally, it seems to me I'd have to add "has World Streamer loaded" checks to all of those functions. Or design my whole initialization logic with World Streamer in mind, for example by spawning everything by code instead of just putting it in the scene. But maybe I'm misunderstanding something?

 

Yes, when world streamer off the loading screen he send info to ui that loading is finished.

 

I'm sorry, I don't understand how that helps me avoid the issues I mentioned. If I read your answer correctly, you're saying that when the World Streamer has finished loading, it informs the UI about that. Yes, I know I can check when the loading is finished, for example by checking the Streamer.LoadingProgress on all Streamers. That's exactly what I believe I'll have to do in every Update, LateUpdate, etc. Or I'd have to spawn everything after that's happened. So I'm not sure what I misunderstood.

 

You could use this message to unfreeze objects

Nah dont do it every update, try to do it async and from time to time. or... why you just dont send this message to all objects which need it?
There is a rule if you have huge amount of objects... not objects should check if something is done. Process should send info that is done to all objects. Ofc it will or maybe even not generate a small or bigger spike but you will not check anything everyframe. We used this in mmo games where server have to handle tons of data when player log in.

 

I'd still need to add something to all Update, LateUpdate etc. calls, such as:

Code (csharp):

1.  
2. if (!worldStreamerEventReceived){
3.     return;
4. }
5.  

Plus code to catch that event and change worldStreamerEventReceived.

Otherwise my objects would be doing their normal things when the world is not ready for them.

Edit: By the way, our project uses one Streamer for a low-quality version of the entire terrain, so that's how we do the "low quality" bit.

But I already did the freezing thing, so it's probably OK for us now.

 

yeap, that is true. Anyway maybe not all need it

 

Hi @NatureManufacture ,

Do you plan to make some World Streamer 2 plugin, more streamlined, with many tasks automatic and using something different than having to follow naming conventions. Or some paid update making it lot better.
I don't ask a "push button to auto stream" but something heavy simplified.

Or it is to much work and you have other plugins to work on instead or you think it's good as it is ?

 

Even at the moment you could use folders instead of names.
What do you mean by with many tasks?
Yes it will be simplier, we have vision how to do it.
Nah update will be for free like always from us probably .Maybe at unity 2018 we will add paid update 2-3$ in few assets.We will see.

 

I bought the asset, but before using it I want to ask a question that I had to ask at the time prior to the purchase.

Does it work in multiplayer mode with Photon?

I explain.

I have a car racing game. Will there be problems when they cross, when they collide, to be detected in the minimap, etc., etc.?

Thank you very much.

 

Game will have server or peer to peer connection?

 

Hello again, @NatureManufacture,

I've successfully set up several ring streaming projects with WS.

However, in one project the major_streamer grid is OFFSET from the loaded terrains from the Scene Collection, as shown in the image below. For simplicity in my description, I've turned off the minor_streamer in this case, but it is also offset:


This is a problem because the Player object can cross into empty space before queuing a scene tile to load, since the streamer grid and the actual terrains are offset as shown. The Player object is in the center of the major_streamer grid, which I take to be the normal and intended setup.

Here is one of my working scenes, where the major_streamer grid aligns with the loaded scenes from the Scene Collection:


I'm using the same loading range, loading range min and deloading range in both examples, this is, both in the working and non-working scenes those values are: (X:1, Y:0, Z:1).

I've read this complete forum and have been through the documentation several times, but see no mention of this.

Any ideas?

 

Do you use floating point fix?

 

Offset could be because for example grid is from 0 -500 500-1000 and terrain start for example in 100 to 600 and 600 to 1100 etc. So grids dont match the terrains or terrains are gets offset.

 

Thanks for writing back, @NatureManufacture,

To answer your question -- no, everything is in true position, I'm not using WorldMover for the floating point fix in this scene, though I have in other scenes.

What do you mean by 'terrain start'? What values do I need to pay attention to in order to get grids to match terrains?

If you mean that the input terrain positions must be divisible by the virtual grid dimensions, what is the origin of the virtual grid? That is, where should I translate the terrain tiles before applying the virtual grid -- should the bottom left terrain tile be (0, 0, 0), or should the center point of the middle tile be (0, 0, 0), or something else?

Thanks again!

 

Ok so if youbhave terrain in size 512 ( just example) they should be in such positions:
0 512 1024 1536 2047 etc
If they will be moved a bit there will be offset. Virtual grid size must be same size as terrain object. We check object positions to put data in correct place.

 

hi.
A simple question. I have many islands, but with no terrain between them. only a ocean later. Can I still stream the scene with World Stremer, so that the islands still have the same distance between each other or must there be terrain?

 

Sure terrain dont have to be continuated.

 

thx for the fast response. Can you make a videotutorial about the floating point fix? From 0 to the goal would be nice. I know the manual, but seeing it would be much better in my opinion.

 

Thanks for writing, @NatureManufacture,

My terrain tiles prior to splitting into a virtual grid are evenly divisible as you show.

That is, my layer [X, Z] sizes are both 2664 and the X positions include [-2664, 0, 10656 (2664 * 4) ,18648 (2664 * 7)], et cetera. So the values are multiples of the size, as you suggest, and yet I am seeing the offset shown in my images above.

I asked about the terrain 'origin' WS expects since that will determine if there are any negative X and Z positions. What origin do you expect for a tile group?

Thanks again for any advice, this is something I'd really like to resolve.

 

For terrain 0 0 you have orgin 0 0. Unity terrain have position in the corner.we took this into account. Terrains are childs of the other game objects?

 

It may be helpful for other users to note that if you hide the streamer objects (click active off), WS will not clear virtual grid for these hidden items.

That is, if meshes or terrains are hidden, they will not revert when virtual grid is cleared, causing an unintended mix of states. This is no doubt because GameObjects need to be visible (active) in the Editor; if not, they are inaccessible to scripts. There are workarounds to traverse hidden objects, but since this is not a bug you need only be aware of this: one more thing to check. ;-)

 

The terrains and coordinated meshes are at root level (not children).

Thanks for your answer on origins in VS; I'll start with a clean scene and regenerate to see if the offset is persistent.

 

Another separate question, should I add ‘Terrain Culling System’ script to all prefabs just prior to splitting the virtual grid? Or after?

Is terrain culling distance based, or does it cull meshes & terrains out of the camera view?

Thanks again!

 

Here are steps to reproduce a strange result: the WS generate command (clicking 'g') in this circumstance moves terrains and meshes out of alignment with the virtual grid -- leading to the offset problems I wrote about. After tracking through the steps I’ve shown that it happens reliably, requiring me to reset the positions each time they move after cycling the virtual grid.

Here are my steps:

• We start with a set of terrain tiles and low res meshes created from these terrain tiles; their positions are coordinated and all positions are multiples of the [X, Z] WS virtual grid size.
• Next we hide the meshes to work on terrain streaming first
• We call the virtual grid split on the terrains, import into the player scene, iterate
• Each terrain revision we clear the scene split, amend the terrains, and split again
• Once we have the terrains in the desired spots, we unhide the meshes and split both terrains and meshes, then generate the output.
• At this point: BOTH TERRAINS AND MESHES MOVE!

The positions are now NOT multiples of the [X, Z] WS virtual grid size, e.g.:

Position before: ‘Terrain_LOD0_41’: X = -7992, Z = 10656
Position after: ‘Terrain_LOD0_41’: X = -12230, Z = 10600

 

@NatureManufacture

Apologies in advanced if this has already been asked, is it possible to add a quick check so if World Streamer is running in "Headless Mode" i.e. SystemInfo.graphicsDeviceType == GraphicsDeviceType.Null.

Or at least point me in a direction where i can add the check my self, the end goal is for a dedicated UNET server to fully load the entire scene, while players (clients) simply load scenes are requested for loaders.

Hope im making sense?
-J

 

It use unity deep functions its camera and distance base.

Server will load scene?

 

Has anyone successfully integrated uNature and World Streamer? Everything streams fine, but the foliage doesn't show in the Game scene.

 

Hello there,
---- Editing to remove a bunch that I answered for myself after a while of using this.
------------------------------------------------------
My end goals is to have my game be multiplayer, I will probably just start out with simple cooperative play of probably 3-4 players, but they may not always be in the same area at the same time. I suppose ideally peer to peer would be best for that, but if I had to I could probably setup some sort of server structure, whatever ends up being needed and recommended by you guys. Are there any special considerations that need to be made when setting up how you are going to be streaming your world when it comes to multiplayer? How would multiplayer work with the world mover and the floating point reset? Bought uMMO2 which I had planned on using for multiplayer.

I appreciate your time if you made it this far and I thank you!

----------Edit - I just thought of more. Is terrain loading distance able to be adjusted at runtime? Example is, if the player is on the ground, or below height X, only visually load a certain amount of terrain, if the player is above X height, it means they are on a flying mount, so load more visual distance (preferably lower quality, as it is not really super important to have high detail when flying, only when on the ground), etc.

 

Good afternoon. Your asset is compatible with RTP. How correctly to adjust, that rtp correctly worked with yours. what steps need to be taken? (is there a demo setting?)

 

Have you checked that there is controller in the streaming scene? Do you have multiple cameras by chance? If you have a very large build or have a lot to stream in the area that your controller is spawned, it might just be a very long load.

 

----- This is fixed

Thanks for the reply. I only currently have my terrain, a character, and a mount in my game for testing purposes. I have the WS loading screen, spawner, etc all setup. I built it as a development build and the profiler was attached. When I started the game it showed the unity splash screen but then just sat there, the profiler didnt show it was loading anything, it was only using like 0.03ms cpu and 0.0gpu. The world streamer loading screen never showed up and nothing seemed to be loading. I let it sit for about 10 minutes and got nothing. My pc is an i7, gtx 1080, 32gb ram, and 2 512 SSD's in raid 0, so I don't have any issues with loading typically. It fires right up in editor and the loading screen and everything works fine, puts me in game, I can go around, I get no errors or anything, no errors when building, or when loading from a build, it just... doesn't do anything.

Though, what do you mean by controller? I have my terrain streamer in there, everything is hooked up right to the loading screen and the spawner, they all work right in editor as I mentioned. Is there something else I am missing?

 

Maybe if you check out the logs, it can tell you more? https://docs.unity3d.com/Manual/LogFiles.html

 

Ah, good call! It looks like Microsplat might be to blame for this.

Shader 'MicroSplat/WC_TerrainPart_99': dependency 'AddPassShader' shader 'Hidden/MicroSplat/AddPass' not found

Strange that it compiled just fine.

 

RTP, has nothing really to do with World Streamer, its a shader, for terrain RTP.. so it will work like any other terrain shader that will work with World Streamer. So no need for a set up...set RTP as you normally would for Multi tile or depending on your set up meshes, or w/e I been using World Streamer for a few years, and Unity for many years, RTP for awhile as well, don't use often anymore... but I use others like it.. So set it up like you would. with out WS. Just be sure to set up the terrains properly as per docs, and videos.

 

Update on my issue, I looked up the microsplat error and the maker had said it would in no way keep the game from working, it would just make the other shader slower. That is pretty much the only thing in my log. : /


Direct3D:
Version: Direct3D 11.0 [level 11.1]
Renderer: NVIDIA GeForce GTX 1080 (ID=0x1be0)
Vendor: NVIDIA
VRAM: 8086 MB
Driver: 23.21.13.8871
Begin MonoManager ReloadAssembly
- Completed reload, in 0.089 seconds
<RI> Initializing input.

<RI> Input initialized.

<RI> Initialized touch support.

Shader 'MicroSplat/MicroSplat': dependency 'AddPassShader' shader 'Hidden/MicroSplat/AddPass' not found
Setting up 4 worker threads for Enlighten.
Thread -> id: 2454 -> priority: 1
Thread -> id: 1f00 -> priority: 1
Thread -> id: 49ac -> priority: 1
Thread -> id: 27a4 -> priority: 1
UnloadTime: 1.558275 ms
PlayerConnection accepted from [192.168.0.16] handle:0xad8
Setting up 4 worker threads for Enlighten.
Thread -> id: 5a60 -> priority: 1
Thread -> id: 5878 -> priority: 1
Thread -> id: 2224 -> priority: 1
Thread -> id: 2b34 -> priority: 1
Waiting for finish
##utp:{"type":"MemoryLeaks","allocatedMemory":25182743,"memoryLabels":[{"Default":664},{"NewDelete":6291213},{"Manager":792},{"FMOD":640},{"Serialization":11328},{"BaseObject":560},{"Terrain":176},{"String":2762},{"DynamicArray":64},{"HashMap":18874368},{"PoolAlloc":-176},{"CloudService":120},{"SceneManager":232}]}


That is all thats in there. (minus a little bit more above it about connection and what not.)

 

If you want someone to take a look, or just test your built version, send me a PM. Otherwise I will leave this to the authors because I have no idea at this point.

 

I appreciate it, got that part worked out. : ) I think I am ok for the time being.

 

Hey there again. I have things working pretty well for the most part, minus this below. I am using Vegetation Studio and it seems the only way I can keep the particular vegetation system assigned to its correct terrain is to have it be a child of the terrain and load when the terrain does, but it hits like a truck when it does load. Is there some sort of way to break this up, but still allow the veg system to load within the scene of its assigned terrain after the terrain loads so that it can find its preassigned terrain and they both are not popping at once and slapping me in the face?

If anyone has any ideas, please do let me know!





Also, is GC supposed to be dropping me down that far as well?

 

Has no one heard from the dev at all lately? : (
Is root motion the only way in which you can use world mover?

Also, how do I do the World Mover if I am mounted? When mounted I become a child object of my mount. It ends up making me go who knows even where and separates me from the mount but then my character and controller still think I am actually mounted.

Is there an event that is fired or anyway to know when scenes are done loading as you are moving around the world so that another script can execute something once a terrain is done loading?

 

Good afternoon. I very longly can not solve a problem .GC.Mark.Dependenciesfunction. what to do ?? each time a large burst when the pieces are loaded. I have already divided into 5 layers of objects, but the spikes continue to show and the game is constantly suspended because of them. all the checkboxes also put the player's settings. What should I do ??

 

Unfortunately, I can't answer that, but I am having similar issues with huge spikes. I have tried making larger terrain (2k x 2k) medium, small (512x512), lowering all texture maps to 512 or even 256, no matter what I keep getting getting them while loading, which sometimes take between 150-400ms 3-4 times in a row when loading maps, and then also on the GC cycle. Is there a way to smooth things out more? I literally only have terrain, my player, vegetation studio, and Enviro in my scene. I also have an i7, gtx 1080, and 32gb ram, and raid 0 ssd's so I have no idea why in the world the spikes are so big when loading a new area. Its like they are not "streaming", it's more like they are a startling firework someone threw at you.

 

Does anyone have a solution for this? I have uNature and CTS on my terrain and I'm getting a similar error. As soon as a terrain is despawned the game freezes.