Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

[RELEASED] World Streamer ! Forget about your game memory usage and create big world!

Discussion in 'Assets and Asset Store' started by NatureManufacture, May 13, 2015.

  1. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    In unity roadmap there is playmode editing but...:D
     
  2. jnbbender

    jnbbender

    Joined:
    May 25, 2017
    Posts:
    367
    Looking into this Asset I just have one question. Can Scene Streamer stream in scenes from HTTP. My scenes are very large (smallest is 1.2GB) and I'd hate to have to deliver them with the game. Will I be able to pull in a new scene from the hosted environment as the game progresses?
     
  3. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,121
    In short -- No. That is a LOT of data to come up/down the pipeline. Streaming game worlds is nothing like streaming video, if that is what you might be thinking. It's an easy hypothesis though, as with enough standardization, it very well could be.
    But really, the fact is, pretty much all streaming games deliver the bulk of that content locally, either directly, or later through updates to the game via content patches (especially in the case of any MMORPG game). This has the added benefit of being able to fix bugs and other such things you couldn't otherwise do by simply grabbing content on the fly from the cloud, which is why most developers do this. In short, "Streaming" is simply a way to save memory at runtime so your game doesn't crash and burn while loading that 15GB+ world in GTAV, for example.


    Honestly, there is no magic. World Streamer does the most simple streaming possible -- it loads files from your hard disk via asynchronous loading, using standard Unity calls. The only "special" thing it currently does is that it handles the floating origin situation for you (with a lot of setup on your part first, of course). Beyond that, World Streamer is pretty basic all around.

    This is exactly why people have been pushing for the super-secret 2.0 update. People who bought it (like me) are seeing just how limited and underwhelming it is, especially in the face of the new capabilities of Entities and scenes shown in the MegaCity demo. There is a free origin-shifting script already out there if you really want that, and as long as you manage the separation of your scenes yourself (which you should be doing anyway), automatically splitting scenes is really all that's left that World Streamer does for you (again, with a lot of setup on your part, and, again, you should also be doing this yourself already). And then even after all of that, in the end, the crippling draw-distance problem of tiles that "pop-in' when they're loaded must also be handled by you somehow (i.e. with fog, or with the purchase of the Curved World addon, all in an attempt to hide this critical failing of your streaming solution). As far as I've heard, this fix is not in 2.0 either. My question then, if this is true, becomes a very important one -- What good is a huge epic world when all you can see is fog?
     
  4. jnbbender

    jnbbender

    Joined:
    May 25, 2017
    Posts:
    367
    Thanks for all the great info. I was hoping I could do more.
     
    awesomedata likes this.
  5. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    No its not possible to stream from http without own code and rebuild
     
  6. taymdevr

    taymdevr

    Joined:
    Sep 18, 2013
    Posts:
    4
    Hi, I have doubts about two possible separate projects:

    1. Works for procedurally generated environments, for example a procedural dungeon whit hand-made rooms (prefabs)?

    2. Any problem/consideration using with a multiplayer project?

    Thanks!
     
  7. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    1) Not out of the box and scene parts must be in scene files
    2) no but server shoudnt have streaming
     
  8. SickaGamer

    SickaGamer

    Joined:
    Jan 15, 2018
    Posts:
    1,004
    Do we have an ETA on 2.0? Jan 2020, Summer 2020 ect...
     
  9. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    We ware blocked by 2019.3 and other changes at other packs but... we just almost finish. I think jan/feb is resonable time. Basicly system is done but we add more features to give much much better performance.
     
    Bartolomeus755 likes this.
  10. SickaGamer

    SickaGamer

    Joined:
    Jan 15, 2018
    Posts:
    1,004
    Gotcha! Multi-threaded? Right now I want to compare you to PW Storm/SECTR.
     
  11. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    Even now streamer is multithreaded like unity load data.
     
  12. SickaGamer

    SickaGamer

    Joined:
    Jan 15, 2018
    Posts:
    1,004
    Imposters? GPU instancing of Prefabs ect...
     
  13. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    You have srp batcher in hd and lw rp. I think even standard have instancing on gpu - you got checkbox at materials
    We have better solution then imposter.

    We will not build custom hard to work solutions. Unity functions are enough. Every platform must work with it.
     
    SickaGamer likes this.
  14. SickaGamer

    SickaGamer

    Joined:
    Jan 15, 2018
    Posts:
    1,004
    ok! Thanks for answering!
     
  15. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,121
    That's interesting to hear! :)

    "Better" in what way? -- At the moment, draw distance is what's really killing my projects with World Streamer. Imposters are a great way to draw something highly detailed far away. I can't really imagine something "better" than that for far away objects and scenes.
     
    SickaGamer likes this.
  16. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    ;) You will see, but this take time:)
     
    awesomedata and SickaGamer like this.
  17. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,121
    In the sage words of Austin Powers... -- "Yeah, Baby!! Hahaha!"
     
    SickaGamer likes this.
  18. nyobu

    nyobu

    Joined:
    Mar 9, 2017
    Posts:
    9
    In this Cyber Week, I will buy World Streamer.So if you release version2.0, is it free or discounted?
     
    SickaGamer likes this.
  19. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    For sure discount and grace days aswell.
     
  20. recon0303

    recon0303

    Joined:
    Apr 20, 2014
    Posts:
    1,634
    Update 2.0 We start working on world streamer 2.0 please put info what you would like to see in new version:)

    I seen post about asking for features. from tools like Atavism.. and other networking libraries. I suggest having some basic tutorials .. Since Unity don't have double Precision Positioning support ...

    I suggest to add better ways to handle larger maps. and possible some tutorials for users. Multi Player related ..... possible Unity, Atavism, Photon ones... some of the more common ways. This could help users a lot.

    . I get this prolly once a week, someone asking for the passed few years. handing this in Unreal is a lot easier. in my opinion. but I know you have the experience to prolly make this easier for Unity users. even for some of the more experienced ones they have struggled.

    I also get Easy Roads, support of some kind, making it easier to work with.

    and I get some VS pro support. Users having issues setting World Streamer up with these two tools.


    I would like to see Layer support. for World Streamer 2. it would be great if you had some sort of way to handle distant terrains other than mesh. Sure mesh work but its kinda out dated way of handling larger maps. more and more games use Shader like Horizon on asset does. ( sadly not supported .


    I basically was posting, on what I get the most DM's from on Discord, and the most messages I got over the years of using World Streamer. Most of it is related to the Multi player issue.. dealing with physics and such, its a common topic over the years here as well. I suggest having an easier way for users advance and not to understand, and possible some tuts.. the docs always been pretty good, but for that topic, it has lacked. I understand its not a easy solution. We had fixed this in our own game, but it was a task in Unity... Unreal it was a piece of cake.

    Anyways those are some of my recommendations for World Streamer 2. I may think of more. :)



    Edit I like to add, I know and worked with many solutions World Streamer is great and I know impossible to share every single solution, of way of handing these problems...but I suggest having some based on some more common set ups... This will atleast give some of the users a better understanding on how to handle this. I would as I was saying, work with Atavism, Photon and Unity .. are the top three I would use. anyways I hope this helps, with World Streamer 2.

    Can't wait to see what you have .
     
    Last edited: Dec 24, 2019
  21. Shadex

    Shadex

    Joined:
    Dec 18, 2013
    Posts:
    88
    So what's going to be the big difference in world streamer 2.0? I'm was going to Integrate it in to a project monday. Is it mostly multiplayer upgrades? Or is there any big advancements, like better FPS in 2.0?
     
  22. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    181
    I'd say have an option to load streamers one after the other (wait for a streamer to be loaded before loading the next one)
     
  23. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    FPS always epends on setup. I will make a list soon. Basicly more options to manage huge landscapes with better quality result:) and ofc more effective ui and streaming. Make note that unity change alot in this area in past few months.
     
    awesomedata and Lars-Steenhoff like this.
  24. Shadex

    Shadex

    Joined:
    Dec 18, 2013
    Posts:
    88
    If i use 1.95, cut the terrains and set everything up in the project, will upgrading to 2.0 be an easy process that i can do in a day, or am i going to be redoing everything anyways?
     
  25. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    Rasther easy, anyway it's still under development. We have delay because of other assets and winter endless sickness;p
     
    MostHated likes this.
  26. Shadex

    Shadex

    Joined:
    Dec 18, 2013
    Posts:
    88
    I Hear you on the winter sickness. Hope you all are feeling better. Thanks for the answer. No reason to wait then. if i can just upgrade when 2.0 comes out.
     
  27. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,121
    Just curious -- anything new with World Streamer 2.0 yet?

    Also, I was under the impression that it will support 64-bit positioning data -- For example, I have 64-bit transforms on a world that I've made outside Unity and exported into .fbx files. I would now really like to import it into Unity to have World Streamer stream it for me. This is possible, right?
     
  28. SickaGamer

    SickaGamer

    Joined:
    Jan 15, 2018
    Posts:
    1,004
    They are sitting on it
     
    awesomedata likes this.
  29. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    We still fight with 1 cool feature :)
     
  30. f1chris

    f1chris

    Joined:
    Sep 21, 2013
    Posts:
    326
    Hope you'll win the fight, even if it's to be a TKO on 12th round ;-)
     
  31. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,121
    Is that feature 64-bit positioning for large worlds as described above?
     
  32. razzraziel

    razzraziel

    Joined:
    Sep 13, 2018
    Posts:
    168
    It took so long, maybe releasing a beta with discounted price would be better? What are the plans?
     
    awesomedata likes this.
  33. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    because 2019.3 release destroyed our tech, and URP force us to rebuild shaders etc... alot new things that happend in meantime delay our work ;/
    Ask @hollegar :D
     
    malkere and awesomedata like this.
  34. RPowers

    RPowers

    Joined:
    May 14, 2015
    Posts:
    114
    Hey, does anyone have any experience with World Streamer and VR games? I'm wondering if I should bother trying implementing World Streamer into my VR game, or if streaming is just way too difficult to implement without any lag or showing of the VR compositor.

    Also I should note my game doesn't use very low poly models. It's fairly detailed and each enemy can have 10k-30k vertices. I've only seen streaming done in VR with low poly games before, so I'm doubting if my game is even suitable for streaming.
     
    Last edited: Mar 9, 2020
  35. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,121
    To be fair, streaming is great for any kind of game with a high memory overhead and is meant to solve this issue to some extent, but your original question complicates things a bit...

    I think you're definitely going to have problems with lag or visuals at some point with an "automated" streaming system like World Streamer -- This is assuming you're working with an "organic" world layouts though (which is usually the case when considering a streaming solution). On top of this, World Streamer, in its current state, really isn't setup for "not difficult" in any way whatsoever -- especially for organic (usually hex-grid-based) worlds. World Streamer isn't really setup for this, not to mention it could take months of testing and heavy reading to really understand the nuances and limitations of the system compared to a more suitable system. There are many unexpected scenarios that will arise in streaming worlds that World Streamer simply doesn't handle or have very good documentation for. And to be fair, your usage is very edge-case, so it probably won't ever fully address your specific situation.

    Also, the difficulty factor probably isn't the most likely reason you don't see more streaming VR games though -- It is probably due to the fact that streaming is a bit like pushing in a push-pin on a cork-board with a sledgehammer. In general, most VR games handle physical camera/player movement very carefully and deliberately due to potential motion-sickness, and realtime streaming is often not even necessary in the first place. For example, in many cases, a simple fade in/out to teleport handles most movement, so depending on your motion system, a streaming solution isn't always necessary. For example, load in/out the next area manually when a user enters a group of trigger volumes at a certain point in the level (before the user actually sees that part of the level). Another option is to divert that loading factor into the duration of the fade in/out (in the case of more bite-sized areas).

    While the trigger areas can be more "work" (in certain high-visibility scenarios), in the case of an FPS-styled VR game, this "work" would be better-handled manually anyway. Check out the GDC on youtube for Firewatch. Although that game is not VR, it seems to be the closest analogue to what you seem to be going for. It might offer some valuable insights on your approach.
     
    Last edited: Mar 10, 2020
  36. SickaGamer

    SickaGamer

    Joined:
    Jan 15, 2018
    Posts:
    1,004
    Still waiting for WS2
     
    razzraziel, f1chris and awesomedata like this.
  37. RPowers

    RPowers

    Joined:
    May 14, 2015
    Posts:
    114
    Wow I REALLY appreciate this. THE GDC talk on youtube for Firewatch was extremely helpful and their case is very similar to mine. They went with Sectr so I'll look at that as well. I might be able to pull off the same thing with World Streamer, but I have more research to do before I decide on anything. Thanks!
     
    awesomedata likes this.
  38. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,121
    No problem! -- I've done my research on this topic, so I'm glad I could offer some insight! :)

    World Streamer has it's pros/cons, but "portals" aren't that straightforward in it as it currently stands (and the lack of tuts or docs really hurts the product). World Streamer is more automated in its streaming design, and it is this reason I actually chose World Streamer over Sectr, but in your specific case, I think World Streamer (as it is now) might be too much. Not sure what World Streamer 2 is bringing on board (since the author hasn't brought up the list of features he promised us), so I can't say whether the 2.0 version will be any easier to use for your purposes. From my experiences so far with the author though, World Streamer is kind of seeming like a back-burner project. Not sure about Sectr, but Sectr is not as high-tech as World Streamer is from what I've seen. You would probably be just as happy using trigger volumes and writing the async load/unload operations yourself. Even in Firewatch, they had to do their own custom stuff in Sectr, so I wouldn't write it off as an option if you want to save a bit of cash.

    Regarding World Streamer though -- I'm starting to wonder about it. I feel like 2.0 (at least what has been mentioned so far) really is what 1.0 should have been by now. The current workflow is incompatible with worlds too large to load at once in the scene (which I didn't actually know until long after I bought it), so I'm really hesitant about paying any "upgrade" price for 2.0, since the author has clearly not been very attentive to World Streamer 1 in the past. Maybe things will change, but I'm personally just _very_ close to jumping ship from World Streamer altogether. What is there is workable (in some cases -- but not mine), but it is very clearly not up to the streaming task of huge, geometry-heavy, worlds (as it currently stands) since most computers simply cannot load all that heavy geometry into the editor at once and run a very heavy scene-splitting operation on it at the same time. I keep waiting though. Maybe 2.0 will actually blow my mind.

    Either way -- Good luck on your game! -- Glad I could help! :)
     
    razzraziel likes this.
  39. Irondust

    Irondust

    Joined:
    Jun 30, 2012
    Posts:
    7
    Hello just wanted to ask will I need to call Resources.UnloadUnusedAssets Manually or will WorldStreamer do this for me?
     
  40. SickaGamer

    SickaGamer

    Joined:
    Jan 15, 2018
    Posts:
    1,004
    World Streamer 2 is coming out soon Ithink :)
     
    Barritico likes this.
  41. razzraziel

    razzraziel

    Joined:
    Sep 13, 2018
    Posts:
    168
    "soon" is pretty flexible thing. any new info?
     
    awesomedata likes this.
  42. SickaGamer

    SickaGamer

    Joined:
    Jan 15, 2018
    Posts:
    1,004
    I know. No clue to be honest
     
  43. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    We do our best to push it;) We had few delays because of world situation and winter sickness.
     
  44. jobobbel

    jobobbel

    Joined:
    Mar 22, 2018
    Posts:
    10
    Hello, @NatureManufacture! I'm trying to use the terrain culling system for low qualy mesh terrain... is that possible? If yes, how? I did not find any hints on that...
    Thanks for your work and effort!
     
  45. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    Nope and it have no sense. Culling is for objects which generate cpu overhead like terrain.
     
  46. Jacubson

    Jacubson

    Joined:
    Jun 11, 2018
    Posts:
    5
    Hi, iam trying to stream terrains with Vegetation Studio applied. I cant find any guide how to set up it properly - could you give me aby tips? the problem is that i have 100km race track so I have to use World Mover feature (Main terrain is divided into 512x512 meters). For now I have in my main scene all neccesary streamer assets and vegetation studio assets. Streaming with foliage works well until i reach terrain size set in vegetation studio.
     
  47. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,121
    Maybe the elusive new version will pop its head in and claim support for Vegetation Studio.
     
  48. NatureManufacture

    NatureManufacture

    Joined:
    May 20, 2014
    Posts:
    1,807
    It's rather VS question, to support our floating point fix system.
     
  49. Seth_Knight

    Seth_Knight

    Joined:
    Sep 1, 2015
    Posts:
    12
    Hi there, I just bought this asset yesterday as I'm finally going to take the plunge into streaming my game's world and I knew I'd need this asset from the get-go. I've already go the basic scene streaming stuff working which is nice, but I have a question for you guys that will be important to the saving/loading of characters and items throughout the game world.

    Before, I was able to save all entities to xml files and regenerate the scene at run-time. This works well. The issue now is that since I'll be unloading/loading these entities as their scenes get unloaded, I'm making some changes to it. To keep it consistent with the loading system you guys have, I'll be saving data for every grid tile that will then be loaded when WorldStreamer loads that gridtile.

    My question is, how can I check which scene/virtual grid element an object should be in? Are there any utility functions I can use to check?
     
  50. razzraziel

    razzraziel

    Joined:
    Sep 13, 2018
    Posts:
    168
     
unityunity