Search Unity

  1. New Unity Live Help updates. Check them out here!

    Dismiss Notice

How a team of five created a next-gen MMO with Unity and SpatialOS

Discussion in 'General Discussion' started by ggambetta, Feb 18, 2016.

  1. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    Here's how a team of five at Bossa Studios created Worlds Adrift, a sandbox MMO set in a world of a scale, complexity and level of detail never attempted before, where thousands of players can interact simultaneously, with no more effort than it takes to build a single-player game.

    http://improbable.io/2016/02/18/how-a-team-of-five-created-a-next-gen-mmo-with-unity3d-and-spatialos

    This post also goes into some detail, for the first time, about the Entity-Component-Worker model, and how it enables small teams to build games of such a scale and complexity in a simple and efficient way.

    SpatialOS is available to Unity developers right now -- we can’t wait to see what new kind of games become possible when scale and complexity are not an issue. 10,000 players in a single massive FPS battle? Incredibly realistic worlds on your mobile phone? The kind of immersive experiences VR deserves? Up to you!
     
  2. Tomnnn

    Tomnnn

    Joined:
    May 23, 2013
    Posts:
    4,137
    SpatialOS is a bad name. OS typically stands for operating system. And those aren't very spatial since they are virtual. Who wants a spatial operating system? Sounds like one of the early computers that occupied a whole room!
     
    Ghosthowl, HemiMG, Acissathar and 3 others like this.
  3. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    Why next-gen ? On physics or server management ?
    Because i don't call cartoon graphics next-gen.

    I have some doubts about indies able to get 150 servers.

    Impressive work so far.
     
    Martin_H likes this.
  4. Kiwasi

    Kiwasi

    Joined:
    Dec 5, 2013
    Posts:
    16,658
    I always did ask for someone to come back once they had finished an MMO. Congrats!
     
    CDMcGwire, Martin_H and GarBenjamin like this.
  5. Neoptolemus

    Neoptolemus

    Joined:
    Jul 5, 2014
    Posts:
    52
    What does their choice of art style have to do with the capabilities of their technology and game design?


    What they're talking about is cluster computing. Each "server" is actually a node in the cluster and can actually be relatively weak hardware (could even be someone's old laptop for example). Such clusters scale horizontally, so when you need more computing power you just hook up a bunch of new nodes rather than buying expensive new hardware to replace what you've got (vertical scaling).

    So when they say 150 servers, they don't mean the big, powerful servers that normally power MMOs.
     
    Freakyuno, Ryiah and GarBenjamin like this.
  6. GarBenjamin

    GarBenjamin

    Joined:
    Dec 26, 2013
    Posts:
    7,442
    This sounds like an awesome game with the focus placed exactly on the areas I always think about right down to the core of interaction and life cycles of the inhabitants. I'll have to check it out. I am actually a bit surprised they did it in Unity because it is not the typical game people seem to focus on around here.
     
  7. Martin_H

    Martin_H

    Joined:
    Jul 11, 2015
    Posts:
    3,975
    Sounds impressive!


    But why would you ever say something like this?


    Seeing what you've done I'm quite sure that most randomly assembled 5 person mmo teams would fail miserably trying to achieve the same.
     
    Kiwasi likes this.
  8. Kiwasi

    Kiwasi

    Joined:
    Dec 5, 2013
    Posts:
    16,658
    You have to remember that these forums are not an accurate cross section of the people using Unity.

    This is a sales pitch for the engine they used. Sales pitches lie.
     
  9. GarBenjamin

    GarBenjamin

    Joined:
    Dec 26, 2013
    Posts:
    7,442
    @Martin_H .... True and also most randomly assembled 5-member teams will fail to complete anything at all... even Flappy Bird.

    3 to 5 enthusiastic and skilled developers focusing on such a project could certainly complete it. The issue is around here generally the only folks who want to make a "big" game are the people with no experience and skills. Outside of here there are people with experience and skills building such things. Which I guess basically is the same thing @BoredMormon stated. ;) Yeah @BoredMormon I agree.
     
    Martin_H likes this.
  10. Socrates

    Socrates

    Joined:
    Mar 29, 2011
    Posts:
    715
    @ggambetta It's an interesting sales pitch, especially considering I've been watching Worlds Adrift for a while with interest in playing once it's released.

    But how about more information on the technology package you're selling? I looked at your web site but did not see any information on your revenue model. One-time purchase? Revenue share? Monthly fees? That could certainly make a big difference in people's interest in your product.
     
  11. 00christian00

    00christian00

    Joined:
    Jul 22, 2012
    Posts:
    821
    Congrats for the game, but I hope that it is just a proof of concept for fun because :
    1 server per each 10 players seem like the perfect recipe for a commercial suicide, even if you had a on the fly scaling solution.
    That's why it's always advised against doing mmo, because there is a difference between doing it and doing it in a way that is financially sustainable and even expert studios often fail even when backed by investors with deep pockets.
     
    Socrates, Ryiah and Martin_H like this.
  12. Metron

    Metron

    Joined:
    Aug 24, 2009
    Posts:
    1,014
    Looks interesting. I'll give it a shot in a couple of weeks. I'm wondering though what the business model looks like. There isn't any information available about pricing structures or license models.
     
  13. zombiegorilla

    zombiegorilla

    Moderator

    Joined:
    May 8, 2012
    Posts:
    8,041
    I signed up up out of genuine curiosity. The demos look interesting as are the claims. The pitch and marketing is polished and nothing set off my bs detector. (working in Silicon Valley, my bs detector has gotten pretty battle tested ;) )

    My two concerns are 1) the tech details are mostly non-existent (understandable in a marketing site). And 2) as mentioned above, revenue model. It does claim that it is open source, which makes the revenue model even more confusing.

    Hopefully getting to test it in action will address those concerns. If it is as it claims, it could be pretty amazing.
     
  14. ironbellystudios

    ironbellystudios

    Joined:
    Jul 21, 2015
    Posts:
    402
    @ggambetta Wow Gabriel! You're ALIVE! I haven't seen you around (granted I wasn't really looking) since Wild West Wendy!

    I know this forum is FULL of people who, for good reason, get uncomfortable with anyone who uses the term MMO. That said, for anyone else, I can vouch that Gabriel has been around forever and is in fact, the real deal. No clue if his engine is good, haha, but at the very least he isn't some noob with dreams that don't match his years of experience.
     
  15. sowatnow

    sowatnow

    Joined:
    Jun 12, 2014
    Posts:
    304
    Sounds interesting.

    I wonder if this system could be used to map out road networks of a real city, and to manage them more efficiently.
     
  16. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    12,477
    It might not be servers per player, it could be servers per area. Still... even that area statistic seems very marketing-ey. It quotes a huge area, but as the video clearly shows the vast majority of that area is either empty or populated only by clouds, and achieving it required "625 Unity engines collaborating". The lack of technical info means we can't really evaluate what those markety numbers might actually mean for developers.
     
    Martin_H, zombiegorilla and McMayhem like this.
  17. zoran404

    zoran404

    Joined:
    Jan 11, 2015
    Posts:
    519
    Ok now the way you get that mass of players together in a single world is by splitting the world into many regions running on many servers and players/entities on different regions of the world (on different servers) can't interact.
    So that FTS battle in your sails pitch ain't gonna happen, unless you count 1000 fps rooms a single battle.

    I'm pretty sure the way they set it up is each of their islands is running on a different server and empty space might also be a separate server.
    And the reason their islands are so far apart and hidden by clouds is (besides the nice atmosphere) because for you to see other islands would require a hell of a lot of bandwidth and cpu time, which costs a lot.

    The difference from most MMOs is that it's just that transition between servers appears smooth, they don't use portals.
    But still none of the ideas they mention is new, they are just trying to sell their product with few big unspecific words.

    That said I really like the aesthetics and, as a fan of an old 2d game where the main way of movement is using hooks, I'm looking forward to playing this!
    Gameplay remains me of Attack On Titan :D

    Neither of those are even remotely related to anything.
     
  18. zombiegorilla

    zombiegorilla

    Moderator

    Joined:
    May 8, 2012
    Posts:
    8,041
    Indeed. Though now that you point that out (sparse smaller areas nearby enough to interact if needed, but obscured enough to not require full 1:1 constant sync), that is kind of an elegant solution. Whether that is the actual case or not. I am a fan of creative solutions in design to compensate for technical challenges.

    But yea, reading through the marketing, it raised a lot of questions. Distributed servers sounds plausible, but there is still some syncing that needs to happen, and in my mind, I see that as a challenge. Is there replication? Is there still a central authoritative server/node? What happens in a lost/dropped server? Is the resource and traffic cost more risky or unstable than a dedicated server farm with tons of ram?

    It sounds like they have it worked out, a lot of pie in sky ideas rely on "imagine if... " nonsense. I didn't really get that from site. Plus @ironbellystudios saying he is a straight shooter backs that up. Looking forward to seeing where this goes.
     
    angrypenguin and Martin_H like this.
  19. Metron

    Metron

    Joined:
    Aug 24, 2009
    Posts:
    1,014
    From the announced features it surely sounds like an implementation similar to Planetside 2's network structure. Even though there are several hundred people playing in the same area, this area is sub-split and only the closest players sync very often. (the farther away, the less sync)
     
  20. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    @Tomnnn It is a distributed operating system, in the sense that it can run an "application" (your game) in a cluster of machines, where the processes are what we call "workers" (AI engines, Unity engines, etc). There are more details about the interaction between SpatialOS and Unity in this other blog post.
     
  21. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    @zenGarden You don't have to get any servers! You develop your game locally and then use our web interface to deploy it to The Cloud™. We take care of the unpleasant infrastructure details :)
     
    elias_t and tatoforever like this.
  22. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    @GarBenjamin That's exactly what we hope! SpatialOS lets you use Unity in ways that weren't really possible before. We've made unmodified Unity physics engines co-simulate, for example. In this Worlds Adrift test we used 625 physics engines to simulate an area of 160km x 160km, way beyond what you can normally do with floating point coordinates. We hope people invent new genres of games with this tech!
     
    elias_t and tatoforever like this.
  23. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    @Martin_H Making an MMO of this scale is anything but trivial! The fact that 5 developers pulled it off is impressive, even if they weren't randomly picked :)
     
    Martin_H likes this.
  24. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    @00christian00 Yes, it was an internal test, and the goal was to push the limits of the scale the tech could reach, not necessarily optimise for cost at that time. And in this kind of simulated world, everything (down to each tree and rock) has interesting behaviour, so don't discount the millions of individually simulated things that were being simulated at the same time.

    But yours a very valid concern, and hopefully Worlds Adrift will serve as proof that this is economically viable once it is released.
     
  25. neginfinity

    neginfinity

    Joined:
    Jan 27, 2013
    Posts:
    7,115
    @ggambetta: I remember that unity had some restrictions on remotely connecting to the engine, or having multiple versions running at the same time.

    It might be worth checking it out. Not entirely sure if it applies in this case.

    See (Ctrl+F) "Embedded System Restriction" and "Streaming and Cloud Gaming restriction" in unity eula.
    https://unity3d.com/legal/eula
     
  26. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    @Metron @Socrates We'll be publishing something about the pricing model soon.
     
  27. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    @zombiegorilla Worlds Adrift will be open source at launch, not SpatialOS itself. But yes, we're just coming out of stealth, this blog post is the first time ever we show snippets of code. In the near future we're going to do in-depth technical posts, including fully open source demos. Stay tuned!
     
  28. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    @ironbellystudios Hahaha, I'm alive! I spent four years hiding at Google, but now I'm back with exciting game related stuff :D Thank you so much for your kind words!
     
  29. Tomnnn

    Tomnnn

    Joined:
    May 23, 2013
    Posts:
    4,137
    Node and other technologies have been doing this for years. Cloud computing has also been a thing for quite a while. Why call a well known concept an operating system? They don't even call it that at my work where worker servers are doing this.

    That said, good for gaming. Distributed systems sounds like a good idea for mmos which so many newcomers wish to make. It sounds a little much for an indie though. I'm with @zenGarden on this. What makes an indie isn't just a small team, it's a small budget! I'd be surprised if you guys could maintain a model that cost less than aws.
     
    Ryiah likes this.
  30. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    @sowatnow That's a very insightful observation. Different groups are using SpatialOS for that kind of application, see this and this for example.
     
  31. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    @zoran404 "I'm pretty sure the way they set it up is each of their islands is running on a different server and empty space might also be a separate server." Not really, we actually support co-simulation (see here). And we're explicitly saying this 160km x 160km world is a single continuous space. No boundaries, no loading screens, no instances, everyone can interact with everyone else.

    Surprisingly, VR experiences and mobile games are related :) If you build not a game but a simulated game world, both the VR headset and a mobile phone are just two different ways to interact with that world that is there.
     
  32. ggambetta

    ggambetta

    Joined:
    Sep 8, 2015
    Posts:
    25
    @zombiegorilla Thanks for giving us the benefit of the doubt :D Yes, sync happens within nodes in the cloud deployment, and between workers, both internal (physics engines being authoritative for subsets of entities) and externals (Unity clients that are sent state updates, and in turn send player inputs upstream).

    But all of these gory details (along with fault tolerance, persistence, etc) are taken care of! You don't have to write a single line of networking code (this is literally true, not marketing BS). This blog post goes into a bit of detail of how you write a game using this new Entity-Component-Worker model, including some snippets of code, but also we'll be releasing a new demo, video and blog post with full source code very soon.
     
    tatoforever likes this.
  33. BFGames

    BFGames

    Joined:
    Oct 2, 2012
    Posts:
    1,543
    Looks really interesting, and i like Bossa's work.

    But it does seem like financial in regard to server structure it is not really for indies? Wouldn't call Bossa studio indie at all :)
     
  34. Tomnnn

    Tomnnn

    Joined:
    May 23, 2013
    Posts:
    4,137
    Then again, mmos have the opportunity to be a source of perpetual income. Maybe they can bet everything on trying this software out for a while and see if they can make a profit... it'd be nice if certain producers finally went under and were unable to continue making games :D *nod towards the 100 unit z clones on steam greenlight*
     
  35. BFGames

    BFGames

    Joined:
    Oct 2, 2012
    Posts:
    1,543
    Well they sure can bet everything! One thing though is the server cost for small indies another is to even find the players to fill it. It is easy for Bossa to get a lot of press because, well they are Bossa....
     
    Tomnnn likes this.
  36. Tomnnn

    Tomnnn

    Joined:
    May 23, 2013
    Posts:
    4,137
    And an audience willing to buy something. Not too difficult these days, but it's hard to compete with some of the really high quality free games.
     
  37. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    I would say don't make an mmo or make one that is really unique on gameplay.
     
  38. darkhog

    darkhog

    Joined:
    Dec 4, 2012
    Posts:
    2,219
    Interesting. I wonder what are the cost for the developer who want to deploy SpatialOS service? As in, hosting costs.
     
  39. GoGoGadget

    GoGoGadget

    Joined:
    Sep 23, 2013
    Posts:
    690
    Look, I'm all for cool new tech that's built specifically for Unity, but when it comes out with a number which at best is a wildly incorrect estimate, and at worst is just a straight up magic number (lie) made up for marketing purposes, it's just disappointing.

    Just because you can get 10,000 cubes to sync their transform (barely, even moving that many GO's doesn't work well) in Unity, that doesn't mean you can go ahead and claim to have "10,000 player FPS" support.
     
    Last edited: Feb 20, 2016
  40. BFGames

    BFGames

    Joined:
    Oct 2, 2012
    Posts:
    1,543
    Yeah good luck having the Client handle 10000 players in a small area, just handling animations on that amount in Unity will kill any client FPS. I know you can have worlds with a high amount of players, but how will they meet and actually fight in larger numbers. Never seen that handled well.

    At the end of the day the clients normal computer have to handle whats on the screen.

    I know some are testing streaming the actual game, but that will never work (well for now) for games that require fast response times.
     
  41. Tomnnn

    Tomnnn

    Joined:
    May 23, 2013
    Posts:
    4,137
    How about real life for the location? An mmo for mobile devices where players near you is based on adhoc connections between the physical devices and map seeds are given by your general area / coordinates.
     
  42. Neoptolemus

    Neoptolemus

    Joined:
    Jul 5, 2014
    Posts:
    52
    @ggambetta

    Does your model operate like Hadoop, in so far that each node is abstracted to one or more computing "slots" based on capability, with some sort of master node delegating compute tasks based on available slots?

    Really interesting idea if so. I'd pondered the possibility myself but didn't get around to actually trying to draw something up.
     
  43. Carve_Online

    Carve_Online

    Joined:
    Jun 28, 2014
    Posts:
    550
    Looks really great. I expect to see 500 kickstarters going up in the next week.
     
    GarBenjamin likes this.
  44. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    It would be hard to play any time , any weather and any conditions.
     
  45. Tomnnn

    Tomnnn

    Joined:
    May 23, 2013
    Posts:
    4,137
    Real life locations just give you the seed. From that point you could move around in game with minimal movement IRL.

    That's just a gimmick to help sell the main idea - a game where you play with people nearby over LAN or a direct connection. Anything to make LAN more popular, to free us from the chains of DRM
     
  46. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    15,317
    Great. We get to deal with our annoying neighbors in game as well as out of it. Can we at least have a flamethrower? :p
     
  47. zombiegorilla

    zombiegorilla

    Moderator

    Joined:
    May 8, 2012
    Posts:
    8,041
    Which part?
     
  48. GarBenjamin

    GarBenjamin

    Joined:
    Dec 26, 2013
    Posts:
    7,442
    Well they can probably do it. Having 10,000 people in the same area at the same time is not the same as having 10,000 people on the screen at the same time. I mean really how could you even tell? Render out the closest 256 and call it good. The screen would be absolutely packed already. Not the same as a birds eye view of an army game. Now if this is a birds eye view type of game then yeah that is going to be a real challenge I'd think.
     
    angrypenguin likes this.
  49. Carve_Online

    Carve_Online

    Joined:
    Jun 28, 2014
    Posts:
    550
    birds are very low poly, it could be done.
     
    GarBenjamin likes this.
  50. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,456
    What kind of pricing model? There is no mention of price :)

    10,000 players would require a large amount of server resources too...
     
    Tomnnn and Teila like this.
unityunity