[RELEASED] World Streamer ! Forget about your game memory usage and create big world!

Each page of the document is finished by contacts:

https://gyazo.com/e905dbc981d64fd91d7e74b530cd53da

 

That's fine, I was just curious really. I'll post if I have any trouble with it.

 

Thank you. My manual did not have that for some reason. So, I will test and then contact you

Here is what my manual looks like:

 

This manual is old like hell - 1year? Please update your world streamer to 1.9

 

Hello,

I have a question for anyone who uses RTP with this World Streamer asset. How do you use the _RTP_LODmanager gameobject? Do you just have one _RTP_LODmanager in your first main scene and then delete the ones in all of your other scenes?

I am receiving large lag spikes whenever a terrain tile is loaded, and I was wondering if the _RTP_LODmanager being in the scene could be the problem?

But when I build the game, RTP automatically adds this _RTP_LODmanager to my scenes (automatically). So I'm wondering how to deal with this.

Thanks for any help.

 

Can this system work with a 10x10 km open world where I can render mountains in the distance, but NOT load game objects like trees, animals, houses etc. that the player can not see?

I don't want to lose the terrain visibility far away.

 

Yes, you could replace HQ terrain by low poly mesh in far distance like here:

Old bugged unity version but it works good

 

I've just tested GC at huge scene with huge amount of trees, grass etc. 8750 units of loaded data (36 HQ terrains 625x625, 64 Low poly mesh terrains 625x625) and :
- editor 3-7 ms
- build 0,5 ms
So GC work very good at unity 5.5.2 a bit better then in 5.4.1 (about 1-2ms)

 

Hi,

A doubt in manual of 1.9.1 page 6 has:
"All models and objects in "Work" scene should be unpacked from catalogues in hierarchy""
But in page 54 has:
"Streamer works with catalogs in hierarchy."

What this mean exactly, I still need unpacked objects?

 

We add catalogs support in last update. Sorry for confusion.

All objects that you stream shoudn't have any parents, but from 1.9.1 they could be for example in 1 parent like models etc.. Check our work demo scenes with catalogues and without, unsplit and split with setup which you will find there and you probably will know everything - you will see how system searching objects to streaming etc...

 

ya sadly Gizmo's is a editor only, but you could use a Cube or line renderer, I know its NOT as good, but we tend to use that, plus we use a plug in which I know won't do you any good if people don't own it..

So I suggest using a cube and using the Line render better than nothing, that is how we used to do it in the old days to. Also could make your own visually image of a gizmo.

 

I could create own shader with outline only but hmmm anyway it will need cube.

 

(I made the same comment on the first Tutorial Video but then felt like here would be a more appropriate place)
I'm curious about a certain thing:

Assume you have a GameObject X which, before splitting, referenced another object Y, via the Editor.
Assume you have Scenes A and B, created by splitting your world.
After the split, what happens to this reference if X and Y happen to be in A and B, respectively? (That is, in different scenes)

I would assume all references are lost, because Unity can't handle this kind of thing. At the same time though, since we have access to those references at the time of the splitting, I feel something could be done to preserve them. Something like redirecting them to dummy objects somewhere on the Streamer itself, which would only load once the Scene which the object lives in is loaded, and then the Streamer could substitute the references back to their desired state.

Anyway, just curiosity for now. Doesn't seem impossible, if not far fetched. After all, if you're using the technique of separating bigger and smaller objects in different scenes it seems relatively easy to fall in this case. More so than for objects far apart, I feel.

 

Reference will be broken. You could always put together this objects or.. attach script to the objects which will find reference or wait for reference after loading process. At mmo game streamer streamed whole network castles, interactive flags etc and we simply fill scripts with references after end of the loading process

 

Hi,

To Bake light, what is the workflow? I'm trying:

1- Open gameplay scene with direct light, sky, and lightmapping setup.
2- Append a Work scene
3- Bake light
4- Split Scenes

Is this the correct workflow?

 

Other question.

I created 2 groups: Rocks and Trees.

Added the "Object_Parent" script in both. Prefix small for both.

This is the question: can be small for both or need be small_rock_ and small_Trees_ ?

Because when I cleaned the splits, WorldStreamer mixed the objects, put some rocks in trees, and some trees in rock.

 

Nope because light is saved at scene file so you should:
- load whole split scenes by local area updater or you could make operation at only part of split scenes.
- bake light
- save all scenes

My mistake that I didn't put this in manual.

Im sure it works in other way like: object parent with prefix "small" with trees, stones inside.
Yeah it mix because system during unsplit don't know what is small tree, or small rock until you will separate them by tree and stone prefix. Hehe users always find another way which we didn't expect

 

@NatureManufacture works! Thanks for your help.

For Occlusion and navMesh (small world) the workflow is the same that baked light?

 

Occlusion yes, navmesh yes and no. If you have small world you could bake navmesh and put it into gameplayscene manualy. If you want navmesh in parts then you could bake it for each scene like light.

 

Any news on Navmesh able to work with the floating point system ?

 

At 5.6 navmesh is part of the objects so if we move object we also should be able to move navmesh We have to check this
Anyway custom solution solve this and you don't need 5.6

 

I doubt its possible in post 5.6. In 5.6 navmeshes are a component though, so should "just work" there since it can be attached to the gameObject itself?
Edit: To slow!

 

I have to play with this before I will trust unity devs and new feature

 

Very good approach to life i would say : )

 

This is how new features usually works in life

 

Hello,

Is anyone else having large garbage collection issues when terrain tiles or assets are unloaded from memory? I am still getting a large performance hit from "GC.Markdependencies" specifically.

I'm still not sure what's causing it exactly I am using 2k x 2k terrain tiles. I am using RTP from the asset store for my terrain shaders etc.... I have about 10 prefabs in my Resources folder. I am using 1024x1024 textures and normal maps on my terrain tiles.

When Resources.UnloadUnusedAssets() is called, big performance hits ensue. Wondering if anyone might know the cause?

 

This is kinda how we been doing it for over a year now, we make sure our roads connection end at each tile. Basically.

 

I would disable RTP make sure its not related to that first. making it easier to pin point which asset is causing it, unless you know for sure its World Streamer. I personally have not seen large GC in World Streamer, I do not use RTP, as I develop for mobile these days and console with Unity,.. I use Mega Splats and had no issues with GC. But that don't mean much you could have an issue some where, so again I would start disabling assets, or run a scene with just WORLD STREAMER.. in it to see if you can get it to have issues by it self. This is what I do alot with assets.. Then i will run the two assets in a scene and only them two to see if they are not playing well together.

 

I use terrain composer 2 for making my tiles..works flawless even TC1 did and was using with World Streamer so glad you found a tool, but this has been around for years... But this free tool looks cool size you can set the size of the tile? I know in TC you can change the amount of tiles in each direction. but this feature sounds good to set the size of it..

 

I have a question. Since I've read articles that says marking objects "Static" could help performance and on the other hand none of the streaming objects should be static in order to work with "World mover" feature. Then what should I do to for the static objects? Thanks in advance.

 

Static = batching , it cut number of objects and...calls on cpu. Basically streaming do the same but this is visible at big scenes where in far view small models just gone. You could batch models yourself by custom batcher system from assetstore as we mentioned in our manual. Basically batched objects will behave and become one object so you cannot move it's components. You could batch each scene independently but this without some kind of automatic script could be anoying for many scenes and updates. Big companies have automatic scripts to batch each prepared scene and merge objects. If you tell me more about your problem I can advice you a bit more

 

But I couldn't find the asset name in the manual would you recommend me a good asset for it? Thanks for the reply,though.

 

Hi, NatureManufacture!
I'm one of that guys who wrote that review about your asset yesterday. There is no convenient possibility to have some discussion there so I'll put more info here.

1) Imagine situation:
- your are experinced programmer with 10+ years experience,
- working on a some project with 2000+ CCU online,
- your artists not use some special tools to work with terrain - Unity editor give enough functionality for them
and some day your chief say: "It's time to try much bigger scenes, we bought few assets some time ago - use it and show prototype".
- One month ago you update asset to latest version and start to work with it.

This is

tutorial video I've worked with, where you are talking about only one tool for terrain splitting and wrote "We will make own terrain splitting system in close future to save your time". This video published 09/20/2016. And this system is still not implemented. Well, I've try to use proposed free tool - it's not feet my requirements. Review explained why.

Finally I spend one day to create own - it works perfect. It can create 16 500meters chunks in one second. Without seams. With some name convention. Main idea was taken in Sectr split script and little bit optimized.

Main reason why I wrote about splitting system in review - you give no alternative in your video tutorial, and all other splitting tools cost at least as World Streamer. My chief think he is buying all-inclusive solution .

2) Then I say "Well, let's start to use World Streamer!".
I understood from the beginning - it's bad idea to use some kind of prefixes to group objects. And my artists say "we need some better solution". Ok, let's make some changes in the code.
It's no reason to ask something on the forum in this case. All World Streamer customers use prefixes and your will not write something special system for us only.
At that moment I hoped only - asset written good and well constructed system will be easy to refactoring for our purposes.
And, as I sad in review, I was upset - asset is written bad.

I was not absolutely correct when sad "all data fields are public". Yes, you use private fields somewhere, but when we start to talk about some shared between classes data - you use public everywhere!
I recommend you to read http://www.clicktocontinue.com/books/EffectvCShrp.pdf book, at least "Item 1: Use Properties Instead of Accessible Data Members" chapter. Protecting data is a one of the main OOP idioms!

I gave many other examples why we can't use asset with such kind of code quality. And it was not complete list.
I can give some clues how to make your code better if you wish just because I like your tutorial quality very much and I'm sure - you ready to make you asset better. But we have no time to wait and have enough experience to make our own asset.

So, after discussion with other developers, we wrote that emotional review.
Best regards.

 

I suggested few in manual as I remember but probably it's time to refresh data from assetstore. Yeah we need to check is there new asset with this function available then we could suggest something.

I will start from this sentence:

If you have any problems, ideas, conflicts, solutions, experiance, task or you need help this is place where you could ask about everything, you even have my personal skype at each page of the manual, also with mail... Many devs use it to share their questions, ideas, or they need custom scripts which in 99% cases apear in next world streamer version but customer gets them much much earlier (ASAP). So please first ask here and if you would not get any help or you will be ignored then write such .... "review" at assetstore.

About 1) I'm not sure what is this "-" for, not only you and your team are experianced. Many our forum users are from huge AAA companies and products. From my designer, system designer, level designer, optimization dev, etc etc (this is stupid) side with "bigger" experiance with open world and MMO with open world done at Unity (I'm not the coder just system designer, manual creator, our coder will probably answer him self about this situation).. I will told you that without special tools you will not going to make open world game at Unity engine, This impossible without special way of thinking about assets and scene architecture (our manual teach this thats why it have almost 60 pages). This is unity engine not unreal which most things with open world has done. All huge projects have even special teams to manage streaming and assets for streaming. We made as automatic system as it's possible and as simple and easy in use as it's possible from our point of view. We adjusted our product to be as open and universal as it can be. Anyway your artist will have to change way of thinking if they want to create open world with unity without spending years on development. Standard unity tools for terrain are slow and waste of time. Custom solutions which fills Unity holes in performance, workflow at big world are necessary and nothing even your own huge amount of scripts will not change this. You will drown in creating script after script, system after system and update them because you will have to change unity version because of bug or improvment.. Assetstore is full of solutions for problems which devs meet during development. Thats why in our manual we put as much info about possibilities and solutions as we can. This is not possible to create every aspect our own and support them from version to version, specialy when unity change alot in past and future version, even whole blocks of functions were changed. We put references to 3rd party solutions which will care about problems and support them from version to version. Ofc Unity is changing and this solutions also anyway... We also do not watch, change for our needs other products at asset store - we are not alowed and don't want to..

We never could give all inclusive solutions when unity is changing all the time and we have to support old, new , future versions all the time. This asset is updated as often as it can be also I talking about manual etc. We cannot create xx movies after each month because new possibility come, old are outdated etc.... From perspective of time I probably should change most tutorials again and again after 5.6 and again in unity2017 (Navmesh changes for example).... Thats why most important thing is manual "book" from our dropbox (assetstore link) which we update as often as we can. Anyway thats is cool that you solve terrain spliting problem, but you waste 1 day and you probably could meet some bugs or not. World composer, terrain composer which are great tools have this already and with huge amount of tools for terrain workflow, nvm just my opinion. I know I know own is always better and personalized

About 2) Yes peoples ask us about new solution in this area and we made it specialy for them. If you will write us a mail and simply ask for custom or new feature we will make it for free like we always do for our customers. Thats why we made many aspect of world streamer even physcis manager... This solution with catalogs was made specialy for 1 customer which ask about this (in very short time)... About code, probably programmer will say more then me but nobody have problems with this before as we said. Ofc we are not alpha and omega I will not going to arque with you.

In such things there shoudn't be place for emotions this is uncomfortable for both sides and create long discussions without a goal. It's better to take refund or rate asset down.

I hope I wasn't rude or anything, I tried to show also my view about this confusing "review".
We wish you all best with your personal streamer, probably in some cases it's better to have own tool, personalized and which fit only your goals. You will avoid many problems which had to be handled by universal system.

Regards,
Bartlomiej Galas

 

Hi, I like to comment if I could here.. I been a AAA developer for half my life and now an indie developer with doing some contract work still for some AAA . I use Unity for my own companies stuff for mobile and other platforms in the past. We mainly use Unreal, Cry no assets needed for those engines..... I been using Unity since before Unity 3. I can say first hand experience, I charge about $120-$180 an hour. for my work..I bought World Streamer for around 40 bucks at the time, I believe..... My team and myself, saved many hours...Our world is for mobile, and tested on PC, about 15 KM x2 for one world and smaller from there for others .Console release will be larger more than likely...

We own many assets as they save time.... Being someone who is a programmer, and UI designer by trade, professionally, and I program just about anything, we did make our own solution even as a team, it took months to do..With Unity always changing, we spend to much time updating, and such which is a pain..Time is money, so as a company we need to save money as well, but we want quality products.. World Streamer is quality , I have worked with many custom solutions with AAA in house and Unreal and many others...I find it odd that you think his code is bad due naming conventions....This happens with any coder as we are all programmers and most dislike the nexted naming conventions, I dislike just about anyones... I hate people who use single letter variables.......Guess what there is a nice little tool in VS renaming, takes 2 seconds...This is what I do, you also can refactor..I do this for 99% of assets used..because an asset developer its impossible for them to PLEASE us all, you should know this as a game developer that made a game before. That you can't please everyone.

I can say from first hand experience I'm very critical of assets , and I can say Bart has always made changes to make World Streamer better.... and what I find a little bit unfair saying he has the worst code ever....I have developed over a 100 games, and worked with thousands....and he is far from the worst...I believe he is a better coder than most..Working with Unity for large worlds is no easy task at all..... Mobile its even worse, due to larger worlds and we so far have had hardly any issues that Bart did not fix, or we didn't add or selves with ease, we never asked Bart to add some stuff as we wanted to do our selves ..

But regardless you have a right to give your opinion, but I think its a bit unfair... and not true.. as far as being the worst .. I think his tool is one of the better ones out there, Best for Unity..Hands down.. and saved our company tons of time and money...We are using for two games right now across platforms.. now.. which will release soon..

I wish you the best with making your own solution...Keep in mind if you think this is the worst code, I suggest you TO NOT use any more assets at all... because I have seen bad code in some assets on the store... even some known assets...You will even see by my comments to those developers that I even like that I told them there code is bad....


Regardless I wish you all the luck, and time spent.

PS: Far as splitting terrains, I use TC2 used to use TC1, works great to make terrains and for multi tile... even stitches them up..Great tool GAIA will have soon, and Map Magic and even World Creator.....All top notch tools, I used them all..... The " FREE ONE" well its free. enough said...It's not Barts tool, I think he was being nice by letting people know its there if they couldn't afford something else...

If you made it in minutes, I would of made it anyways... that way you have it.. I make anything myself, if I can do in under a week or so.. But if its a tool that will take months and extend the engine, I always consider buying code assets.. as I never buy art ones typically.

Best of luck!!

 

And your are good in your profession! Only one thing you may improve on your side - update slide in video where you wrote "We will make own terrain splitting system in close future to save your time". You can add there one more sentence about other tools for terrain splitting. And all questions about splitting will be closed.

Main claims were to the quality of the code.
recon0303, what do you think about constructions:
1)
if (!someCase)
someCase = true;

2) about comments like
/// <summary>
/// SomeFunctionName
/// </summary>
void SomeFunctionName()

3) about algorithm where
a) collect some data on each frame
b) check is data != null to show button
c) if button were pressed - call some function where you are collect same data again instead of sending already prepared data?

4) count layers in such manner:
int layerNum = 0;
foreach (var layer in sceneLayers)
{
layerNum++
}
do you know at least about foreach performance issue in unity?

These examples are only top of the iceberg.

We've used assets like MeshBaker, Final-Ik, etc - no one have such problems.

 

Mesh baker is cool but UI is pretty unclear.https://www.assetstore.unity3d.com/en/#!/content/5017

 

I've got no one comment with arguments why my claims to your code quality is "a bit unfair... and not true.."

Can your confirm that case

already happend?

 

I see you are "strange" man or even hater...lol... I said that our coder will answer you so please wait pilitely. There is something like weekend and free time, all people have it, also life and other customers, users.
From first view "3" is related to old world streamer version so....

 

Hi, I'm Pawel programmer/developer who wrote code for world streamer, as Bartek said he is designer behind it. I have 8 years of experience in programming. Mostly working with c# in unity but also doing .net web projects.

Most of the code for world streamer i try to write on easy level for most people to understand. Thats why many times try to use simplest sollutions, but as optimized as possible, rather than "advanced" futures of .Net like properties.

Could i made more encapsulation with properties and privates - Yes, but that would made it less readable for beginner programmers who also want to use the code... Second thing is that this project is fast evolving and changing for better. We try to work with our clients and supporters to have the best possible streaming sollution which will satisfy everyones needs. This means that sometimes we make quick fix (even during skype call) and not always have time to refactor it for better "looking" code right away...

As for your points:
1) could you give what sollution in this case would you use?
2) this is just basic comment generated by the ide, i write my own comments when i see there is something more complicated in code
3) if this point is about streamer editor that was a bug and was changed for better sollution, still i would do it bottom up different way but it will come with our big 2.0 update which we plan for end of this year.
4) thank you for turning my attention for this "bad code" i already changed it for next update

As for loops for/foreach it depends on situation where you use it, in most cases for is fast and sufficient but sometimes even foreach can be helpful.

To be truthful the point you submited can be changed in 15 minutes to 1h so i don't see any big issues with them?
What was main problem you had with changing the code for what you needed?
What other sollution rather then prefixes would you suggest we could also use in next version of world streamer?

Best Regards

 

I'm pretty sure I've latest version and in the StreamerEditor class you are still using AddScenesToBuild() in inefficient way.

As far as you are not coder - it will be better for you to do not try to answer on technical questions .

Look, you wrote novels to explain why your asses is so cool and perfect and my review is unfair. But say nothing concrete about code quality. And trust me, every experienced uninterested programmer will confirm each my example.

 

Thanks for your answer! It was much close to the subject of my posts.

1) could you give what sollution in this case would you use?
Here you are:
if (cancel) {
cancel = false;
} else
listSizeCollections = currentCollections.Count;
Even more, you can remove whole this snippet because it's much better (safer) to create private property for retrieving currentCollections.Count value or even use it directly

2) this is just basic comment generated by the ide
And it is ridiculous without helpful information

3) if this point is about streamer editor
yes, StreamerEditor.AddScenesToBuild()

4) thank you for turning my attention for this "bad code"
Am I right that
int layerNum = sceneLayers.Count;
much more clear and effecient?


When I say "the worst code we’ve ever seen" it was subjective. Subjective things depends on experience of subject.
Sorry, but in my 15 years career I really saw so many strange examples in commercial project first time.

What other sollution rather then prefixes would you suggest we could also use in next version of world streamer?
You could create one root object per each layer and place all related objects to them before splitting without renaming. It will decrease complexity of your code and will be much convenient to use. My artist try this workflow in my prototype and like it.

 

Hi Pawel, it's good you're here! First of all, thank you for this asset, it's incredibly useful.

But, I also have some unsolicited advice and comments.
Especially about the comment you made above. No. I think you're missing the point of your own asset. World Streamer (in my mind) should be the ultimate world streaming asset. And being that, it should be optimized to the end. I think you shouldn't care about "beginners", who try to read your code. In the 99,999999*% of the time the computer will read your code and act accordingly. Make it super-fast and super-low-resource-needy. Of course, you always can put comments in the code to guide yourself and us what does what and why it's doing that way.
I think while you're developing a tool like WS the speed and lightweight should be the first and second priority. Everything else (including code readability) should be just third or so.

BTW, I do not understand the logic behind the "change names to certain prefixes in order to use it with our product". Is it possible to think about it for a little bit more in the future? Even change it if you can? I'm planning to take a look at it on one of the upcoming weekends, but I really don't do your work.

--

Don't get me wrong, I'm not upset, and I do not think WS would be a bad or damaged asset. On the contrary. I'm trying to give some ideas, because I care about it.

 

Oh, man, thanks! It's hard to say better then you do!
It's exactly what I'm thinking about but I'm not native speaker and it's really not easy to explain my thoughts when I wrote all these posts.

There are many places in this asset where used inefficient algorithms and I spend up to one week trying to optimize it and only then decide to write own tool. It is first time I've such kind of negative emotions .

 

@NatureManufacture May I ask a question? From what number the "floating point error" would usually occurs? knowing the number could help choose the world mover or not. Thanks in advance.

 

Root folder is ok but this is way which force you to do manual work after each update, local updates and during changes - you have to move objects, find them etc. This is not automatic and generates mistakes. As MMO developer we worked like that few weeks and we give up... That was very bad idea. With proper names and our actual solution you simply rename prefab and click unsplit and split buttons again - and object is in proper layer. If your scene is not a mess with tons of broken prefabs everything will be very fast and comfortable. With broken prefabs ah.. you have to rename objects - this is not cool even with good rename assets. Good side of root folder is that you have more controll without creating/rename objects/prefabs that should be at many layers at the same time. As an option it's probably good way. Btw at the moment we use root folders (actual version) but prefix must be still with objects. We didn't want to use tags layers because they should be used in other operations and stay free for developers. Anyway as an option we will include this in new versions.

Depends from world scale, object scale and few other things. Basically if you are far from orgin shadows start flicker and looking bad, physical operations loses quality. Much depends from the game. I hear that after 20k unity units everything starts to lose quality. We always advice to not do restart position too often Probably 10k is better. Wrong shadows we notice even after 15 k units, but that was old unity version.

Good luck with your tool.

 

NatureManufacture, thanks for this comment!
In this case it was really good analyze of problem. It is show that I was right - designer on this project is professional.
Could you prepare an example scene where prefabs problem will be good visible?
Maybe I'll find good solution for this case.

 

Hey LurkingNinjaDev,
you misunderstood me, making world-streamer "super-fast and super-low-resource-needy" is top priority and we always try to make it this way. The code for beginners was about using public field vs properties and things similar to that which doesn't affect optimization.

about btw i think Bartek responded to it fully:

 

Hehehe scene mess is life, sometimes you are too lazy to find or fix something or you dont know about that something has broken (reimport, unity version change also could broke prefabs). Sometimes online library or crash could broke something. In first few weeks everything is ok but after changes and with 2-4 milions of objects and xxxx prefabs...you will get small mess. Ofc if your designers pedanticntic you could avoid some problems. Its hard to give world scene which looks like after few months of development by 20 devs but you could be sure it is a mess . Im not sure even how to prepare it without huge source of work.

Anyway big world could be done with as automatic workflow as its possible. Less manual work less mistakes more time for POI and design. Monkey work generates mistakes and it burn time alot.

 

about 1) in which script is it?
about 4) yes but this variable was used for setting layer.layerNumber = layerNum; not counting collection size, i changed entire function from foreach to for