anyone using Node.js as server?

I'm currently trying out Photon Cloud for Unity and I'm pretty impressed with it so far. But It got me thinking about multiplayer networking in general, and I thought I would throw this out there. disclaimer: I don't have network game programming experience. Never have made a MMO, nor do I want to. But I am generally interested in the concepts especially around cloud computing.

But what about making a server with NodeJS http://nodejs.org/

• Node is great at handling massive # of connections (limited only by the # of filehandles or sockets on the box)
• Node excels in non-blocking async mashup situations
• Use Node's HTTP streaming aka keepalive!
• No firewall or NAT punchthrough issues because HTTP just works
• "But HTTP is slow" ... yes not if the TCP socket is already open and streaming in keepalive.
• "But TCP too is slow, real games use UDP". Maybe just dogma with origin that pre-dates http servers that support streaming?
• The simplification of using HTTP streams and Node's programming model should make it easy to deploy in a cloud environment with distributed shared memory.

But obviously have not done any benchmarking It would be a fun open source project for Unity. After I finish my current project I will probably do some prototyping.

 

Node is not built to handle application that are as latency sensitive as real time simulations.

HTTP is a bloated protocol, and it's text-based. It will not work. Maybe you can make some facebook game or something, but building a multiplayer game in the sense most people here think about games, will not work ontop of HTTP.

Also, why would "TCP being slow and UDP being fast" be dogma that is somehow related to before HTTP servers supported stream? Mixing up the concepts a bit?

Also TCP is not slow, not slower then UDP (or anything else that runs ontop of IP). The problem with TCP is with how it deals with dropped packets.

Also, rant:

What is with everyone trying shoe-horn Node.js into every frekin' type of software in the world. Node.js is nothing "revolutionary". You can get as many connections on a normal old win32 socket using IOCP. Node.js just happned to dumb down async programming enough for the masses to understand it I suppose.

 

Thanks fholm, good feedback. I would only add that "real-time" is mentioned twice on NodeJs homepage

[edit: of course they probably mean "real time" in the sense of watching a twitter feed ]

 

That's not the real-time I'm talking about...

Basically there is _NOTHING_ magical about Node.js, for example your quote:

The above statement is true for ANY other socket implementation that exists today, on ANY platform pretty much. It's just that most people do not know how to write properly async:ed networking code.

Node.js does nothing magical at all.

 

Yeah I edited my reply to say "they probably mean "real time" in the sense of watching a twitter feed"

 

Hmm I honestly wonder why microsoft, ebay, linkedin, yahoo are all using it. Surely they have people on staff who know to do some nice socket programming?
Anyways I appreciate your feedback.

 

But it's Javascript ON THE SERVER! Can't you see how awesome that is?

Maybe you're biased, like this guy: http://teddziuba.com/2011/10/node-js-is-cancer.html

 

Ha ha so true. I'm going to bookmark that one. I write in C# for all my Unity work. For my part-time (non-gaming related) day job as software engineer, I unfortunately have to sling some Javascript code. Client side only. My boss was asking about Node and what I though about it.

 

Of course they do, node.js just happened to become popular. JavaScript was basically the first language a lot of people learned, it was inherently async due to the browser, node.js leveraged this to build a very nice server-side environment that a lot of people (old javascript programmers) were very familiar with. Boom, Node.js becomes big.

Node.js is the new ruby on rails, except it uses a language most people can stand (or at least gotten used to by now).

Other companies jumped on the hype (not saying it's only hype, Node.js is damn good, no doubt about that). Do you really think that two people could write something that all these companies could not? Of course not. There has been many many attempts at things similar to node js inside microsoft and yahoo before. The technologies just never got a good foot hold in the community, were forgotten and then deprecated and dumped.

Node.js has not even done any "new" socket programming, it's basically a patchwerk of libraries (epoll, v8, etc.) that have gotten exposed to a tidy JS API that looks and works like the browser async model.

It's really brilliant, I agree. But it's nothing new, it just happened to be the one tech to become "famous".

 

JavaScript on the server existed long before Node.js.

I'm not trying to say Node.js "is bad" or anything. It's just nothing new. It's a bunch of libraries put together that exposes an API that i similar to what people were used to (web browser javascript engine-style).

 

Just a couple of observations for you.

This is true, but also kind of true about anything. The challenge for game servers is generally optimizing for latency. You might very well be able to support 1 million connections, but if the latency is 500ms, it will not be useful for lots of games.

While HTTP is "most likely to work", there is no silver bullet for firewalls. People that aren't allowed to create arbitrary TCP connections may very well have issues with HTTP-keepalive style connections. We have had HTTP tunneling as an Electroserver feature for years, and while it generally works pretty well, there are still lots of situations where it just won't work: Caching firewalls, stateful or inspecting firewalls, etc. The traffic basically isn't exactly the same as "normal" HTTP traffic, so there can be issues.

HTTP still has additional overhead vs a binary protocol over TCP/UDP. There is always at least a bit of increased latency and the message size is increased, even if you are cleverly packing it as a binary protocol.

This is an oversimplification of the problem. The reason this type of scalability is 'relatively' easy for things like websites is because they rely on large parts of their architecture being stateless. This is typically not the case for many multiplayer games, since they generally require users to interact in "real time" where there is also an emphasis on low latency interaction. This basically means it is very difficult to employ typical horizontal web scaling strategies for multiplayer games.

All that said, there is no reason you could not use Node.js, or any other general purpose "web/http" tech for multiplayer games. It is just a matter of choosing the right tool for the job and knowing the various strengths and weaknesses of the platform.

 

@tigeba, thx good info there.

@polytropoi wow that guy is hardcore:
http://teddziuba.com/2011/10/straight-talk-on-event-loops.html

 

While we usually agree on IRC, I can't help but feel that this is not correct

Trying to build an FPS game on-top of some standard of-the-shelf HTTP server like node/apache/IIS would surely not work out well.

 

Fholm,

I may have just been unclear and I believe we do in fact agree. I was merely suggesting this was a valid alternative provided you work within the limitations of the tech and choose the right game. Obviously I feel there is a place for specialized game servers since I make them

 

More evidence that Unity is a victim of it's own success.

 

OK Ms. l33t AND snarky... whatever! I've been a Unity user since 2008 and Torque user before that.

 

Lots of people are using it because it's useful for lots of stuff. The notion that it could be used as a game server for some things seems pretty reasonable to me - i.e. sending/receiving json with a nosql backend. There are other types of games than MMOs, you know. The biggest pain it seems to me is like the Ted guy said, you end up having to run a real webserver in addition to Node.js to do some things, so it's not as simple as it first appears. But I don't care too much what the "Unix neckbeards" think about it.

My experience in Unity has made me appreciate single-threaded "event-based" programming, and how it differs from "traditional" i/o, where every new request is a new thread. When I started looking at Node.js, it reminded me of Unity - no guaranteed order of execution, lots of coroutines and waiting for callbacks - like big Update loop.

 

That's a really good point! And I *love* coroutines in Unity

 

Web stuff.

No one in the right mind would ever use a text based protocol like JSON for anything other then turn based games or those facebook "web games" (or w/e they are called)

Who mentioned an MMO?

 

Nobody's pissing in your garden, man. The web games you majestically disregard have their place, and it's pretty profitable one. If/when I need a socket server, I'll look at your stuff, since you've obviously got The Answer.

 

Did I even mention my networking library? No.

 

mmkay. and yet, I was able to read your bias between the lines. It's OK, most people are biased in one way or another. Here's mine: I've never built anything (yet) that might require a socket server, I've built tons of stuff (yes, games!) that just send horrible, awful, nasty text strings to and from web/database servers, and while I'm not always "in my right mind", it strikes me that Node.js might hold some promise for these applications. My general sense is that few non-MMO games really need a socket server, but almost every game needs some kind of data persistence. I'll admit, though, that I'm resistant to taking the step into binary encoding and socket service - it's a little scary. Maybe if you tell me a nice story, I'll feel better about it.

 

All OSes are asynchronous wait loops, even real-time (vxWorks to name one) have device interrupts they must wait on.

Socket programming is layered over the OS...

HTTP is layered over those...

and javascript is embedded in that...

X-Windows and all window servers are just events loops....games are event loops. It's just interrupt service layer abstractions.

It's not a big deal if you're not transferring meshes or things across the connection and only have a few connections to use node.js

 

Well thanks for the replies all. An interesting discussion, aside from 1 snarky.

A lot of hype about Node have been shattered for me. Key points are that async socket programming a.k.a. evented I/O is nothing new it's been around for a while. Also the obvious but easy to forget fact that HTTP is pretty bloated and text based.

I'm just happy to be developing in C# and really have no desire, I realized, to write more Javascript. At my day job I have to write enough Bash, Ruby and Perl, that I really savor the statically typed and compiled goodness of C# in Unity.

Also realized that now that I'm using Photon Cloud, well I probably won't be looking back. Cheers

 

@ mindlube, sorry for the snarkiness, if you were referring to me, and it's your thread so you can close it if you want to, but I haven't yet seen anything I'd consider both informed and objective regarding your question, and I'm honestly interested in it too. The notion that "it's been done before" and so it can't be useful is silly, everything's been done before, that doesn't address the relative merits of Node.js here and now. The bias against javascript is just that - a bias. Everybody's got a preferred language, and dismissing Node.js because it doesn't use the language you prefer is not what I'd call an objective evaluation of it's merits.

Has anybody out there actually USED Node.js with Unity? I have done some simple experiments, and as I say it seems promising as an alternative to traditional i/o for those cases that don't need a full-blown socket server, but want to decrease latency and increase concurrency at a low cost in both money and complexity. Which applies to almost everything EXCEPT an MMO, doesn't it? Why use a socket server with binary encoding unless it was necessary? And it usually isn't, unless, as some here seem to do, you regard everything except MMOs as beneath your concern. The relevant comparison is not to Photon or fholm's library, but to apache/php/mysql, or IIS/.NET/SQLServer.

 

Polytropoi: I would say the reverse is true, a small and fast binary protocol is important for all games except the simplest ones (turn based for example).

 

@polytropi, no not you, I was referring to msleeper

I wasn't bashing Node.js and I found your comments informative, and mostly agree with you. Although I haven't tried, I think Node would work great for turn-based multiplayer games in Unity, or anything that needs to communicate with your game for metadata, like I don't know - a lobby or leaderboard or saving screenshots or ... lots of things.

What it comes down to is just prototyping it and seeing if it's fast enough for your use case and more important: if you like how it works

re: biases, true and I've got a love-hate relationship with Javascript and dynamic languages in general. It's actually a very cool language in some respects. An interesting lecture series here http://yuiblog.com/crockford/

Cheers

 

@fholm - thanks for your reply, but can you explain why? Really, I'd likes to know. I'll grant that a big multiplayer game needs it, and I can see the advantages in communication efficiency, but keeping to strings has it's own advantages: you can read them! Of all the things I've needed to optimize, parsing strings has never been the bottleneck. Doesn't encoding/decoding the binary stream have it's own overhead? I'm sure the information density of the binary compression is much greater than sending strings, but where is the point this becomes the overriding concern? I'm sure I'll understand these things better and will probably agree with you when I've built a realtime multiplayer game. I'm just saying there's a whole universe of stuff that's not that, built on top of traditional internet protocols and string handling. One of the strengths of Unity, IMO, is it's transparency to all that - you can build great Unity experiences using cheap, familiar internet technology.

 

It's hard to separate the hype about Node.js from the old-school dogma about what "can work" for network multiplayer. [edit: which I guess is the reason I started this thread ]

It could work for real-time multiplayer games. Someone just needs to prototype it Here is an interesting case study and backed up by having a solid 4.5 - 5 star rating in the itunes app store:

(http://mashable.com/2011/03/10/node-js/)

 

There is a big difference between delivering "near instant audio", which easily could be 100-200ms delay - as they don't specify, also note that node here is simple a pass-thru fan-out distributor of the pre-packaged audio file.

HTTP is pretty ok for delivering HTTP, as they overhead of the HTTP envelope versus the size of an audio file is basically nothing. It's different when you send tightly packed 1-4 byte commands on the wire. Also note that they are using TCP (which do work for some games, but far from all).

 

fholm, of course keep in mind the http keepalive/ streaming eliminates a lot of the http envelope overhead.

I don't know what audio format Voxer uses under the hood, although it's interesting to note it has unlimited participants in the chat and persistent rooms and scrubbing of the audio clips. It sounds a very cool app I am definitely going to try it out later.

However, I guess it all gets back to what is your personal definition of real-time. I agree with fholm that games needing "twitch-quality" realtime won't fly unless you are using UDP with byte sized commands. But I would wager that lots of other semi-realtime games could probably work very nicely with Nodejs.

 

I have to say I just bought some books for Node for some browser games/apps I've been working on and I believe it to be on the "upswing" just as Unity has been and still is.

First off, this is exactly what I need for my mobile and browser games and general web apps http://vimeo.com/24149718

I bought some books on javascript patterns, object oriented js, and node.js and it changed my view of javascript (in unity I prefer c#), especially as a web developer familiar with javascript. This looks amazing:

and this

so simple to work with....

Also in combination with http://socket.io/ As a web developer thats a pretty good arsenal.

It also follows Unity's mantra of "write it once and it works almost everywhere", as well as unity's co-routines being similar to event-driven callbacks http://developer.yahoo.com/blogs/yd...ps-and-writing-great-code-for-node-js-part-1/ not to mention Unity being "component based" is akin to javascript's "module based".

Even microsoft likes it: http://blogs.msdn.com/b/windowsazur...in-windows-azure-to-the-cloud-and-beyond.aspx wonder what the odds of node# are lol

edit:: http://developer.att.com/developer/forward.jsp?passedItemId=9600004

 

FYI Node.js does not have to use the HTTP protocol at all, you can use bare sockets and it works wonderfully

 

yupp but then you need to implement the whole messaging protocol again on top of it which makes its use quite a bit lower and that the scripting side of node.js gets broken appart to incompatibility every few months definitely does not help it as a production environment usable server if you want to stay up to date with its development (which you want)

 

Sorry to open a fresh wound here but this is exactly true. Yes, people have been using it for http (Web) servers as of late but it can also be used as a TCP/Socket server. You definitely don't have to use the HTTP protocol for something like this.

 

Now that more than a year has passed...what are people's thoughts on using Node.js (socket server) as a way to do networked gaming?

 

Just saw this thread on google while searching for Unity Server made with Node.js, have you guys check Node.js Pomelo Server? And here is a sample MMORPG created with Pomelo -> LordOfPomelo.

Your thoughts guys?

 

Pomelo server speed is quite decent. There is a Unity3D wrapper and they made some efforts to make a proof-of-concept MMO. They also have a lively peer support.


Obviously, some people don't like it because it threatens their livelihood. - like making bare-bone MMO solutions which costs tens of thousands to license and give no support for years on end, selling things on the asset store but never bother to update it.


Please do not listen to the fear-based people who discourage you from looking or experimenting with NodeJS or Pomelo Server or Photon or SmartFox.


You might even find these irrational people who say negative things to be fear-mongers.


In this forum, I've seen all the negative things said about how extremely complex Photon Cloud is, how difficult it is to code in SmartFox but I have yet to see the same people make viable solutions out of the things they sell.


Please, open your eyes and stop being so judgemental and negative about NodeJS.

 

I think Pomelo looks like an interesting architecture, although the demo leaves a lot to be desired in terms of responsiveness and just play-ability... I donnow. Curious what their Unity 3d wrapper is like- do you have a URL for that specifically? Thanks to CyrixCool for posting about it, interesting news.

Um nobody said anything negative for the last >1 year until this thread was revived

As the OP of the thread I'll just say I'm still sticking with Photon (Server) and I'm very happy coding in C# on the client server. Photon also has a very nice threading model using Fibers. Good tech support community, fast performance, low memory requirements, and licensing is a lot more affordable than SmartFox and Electrotank.

 

Please see:

https://github.com/NetEase/pomelo-unitychat

https://github.com/NetEase/pomelo-unityclient

 

Hi eskimojoe,

What server software are you using for Lands of Ammox MMO?


Thanks

 

Custom-developed. We licensed the server-side full sources from vendors and went ahead from there. The latest is the devs here optimized to nanosecond speeds.

 

Hi eskimojoe, may I know from whom your team license the server? And is there an evaluation version?

Thanks ^_^

 

Dunno. We usually do not tell anything at all.


If you want, you can invest something like Euros 500,000 non-refundable to know the answer.


We do not go by forum support with vendors. Usually we use private tickets.

 

Thanks for the Pomelo links, very interesting! I've been using node as rest+socket backend for a 3D Soundcloud kind of app I'm working on, check it out: http://elnoise.com.

Node serves the static webcontent, and speaks the same json to the webpages, unity webplayer, and android/ios. I was using uniweb + ws for socket comm, but I'll have to try some Pomelo.

@eskimojoe, somebody will be decompiling your stuff very soon

 

We know about that issue and took steps for that

 

Hey Guys well for that purpose I myself would use nginx. And according to my knowledge Node is not perfect to that.Thanks!!

 

I'm biased, because the node.js guys pissed me off right from the start claiming they were doing something new, when a lot of us had been writing event based servers for years before they came on the scene. And their whole idea of code reuse on client and server just hasn't panned out.

If you like tech that solves already solved problems, or are afraid to learn a language that is much better suited for the task, go ahead and use node.js. If you want something that kick's node.js's butt and is the right tool for the job, go look at Akka or Netty.

As for us unix neckbeards, we can teach you a thing or two. Who do you think is creating all of the latest highly concurrent distributed stuff out there? It's not game developers, that much I know

 

That's a fair criticism of Node.

While those may be great tools, have fun trying to integrate a Java client into Unity-as both of those are Java based servers.You would have to write your own client in C# and figure out how to have it into Unity's single threaded event loop. Not fun. [edit: and this is exactly one of the reasons too why Node may not be a good choice for Unity devs]

I take issue with this, being both a game developer and a unix neckbeard. Gaming has driven the development of a lot of highly concurrent distributed solutions. Look at couchbase for example, one of their main use cases and large class of customers is actually games.

For what it's worth, my game's network architecture (in development)

Unity c# photon clients <-> internet <-> Photon servers (win2012server) <-> VLAN <-> Couchbase cluster (Ubuntu linux)

 

As far as C# client networking, ya it's a pain but you have to deal with that regardless of what's on the server. FYI I spent some time on this myself and found a third party concurrent dictionary that works with mono 2.6. I use that to pass data from the network layer to the main thread. Works well even under load. Google Spicy Pixel, they have the concurrent dictionary implementation.

Social gaming in particular has driven some recent development, not traditional AAA game houses though from what I have seen.

Couchbase is the best thing going now for games IMO. Last game I worked on our studio wanted to use couchbase,but central tech made us use mongodb. I don't know if we had the largest cluster, but it was pretty big,and it failed horribly. When I left I think they were switching to Couchbase.

I don't mean to beat up on AAA game companies too much. It's just that if I see one more monolithic game server written in C++ with a thread per connection/user model, my head is going to explode

Chris