NAT Punchthrough, Connection Testing, and Best Practices.

Hi, I've been banging my head against the unity networking code, and I was hoping that one of the developers might be able to sound off on this. Please bear with me if it rambles.

Currently we're running connection testing, and for just about everyone behind a router it's reporting that NAT punch through is working. However, when it comes time to start a game, it doesn't work.

It appears that the connection tester reports PrivateIPHasNATPunchThrough even if the client has a symmetric NAT scheme. If it's known that unity cannot navigate certain NAT schemes, why doesn't the tester identify those situations? If it does, and I'm doing something wrong, how should I be testing the connection to see if the machine I'm on can host?

The master server reports the port that it received communications from the registering server, correct? So if I start a server at X, and the router talks to the master server on port Y, then when client C gets the server list, the HostData structure will contain port Y, correct?

If the above is true, then a player with a symmetric router can still host if he forwards port X to his machine, but then the client has to know to connect on port X directly, and not the port sent by the master server.

If the client can detect that its punch through won't work, then it could at least notify the master server, and the client can either connect to a known port, or the master server can send the correct port via the comments field.

Because our game is designed to be both peer-to-peer, and a facebook app, it has to automagically work pretty much all the time. We can certainly add proxy message routing support to our game using the provided API, but at the end of the day without robust connection testing, we have no way of knowing which way the client should connect to the server.

This is an area of unity where the documentation and support seem below average, and I would love to talk to someone about this in greater detail. At the moment it's hard to tell if I'm doing something wrong, if there are bugs or missing functionality in the engine, if it's just bad documentation, or if I'm simply asking too much.

Thanks for your time, I really hope someone can shed some light on this.

 

It's bad documentation... so that's a

+1 from me for better information on all this.

 

So here's what I've been able to learn through experimentation, but I need some help to fill in the holes.

1) The connection tester does not identify symmetric nats, because it's not checking to see if you can reach the server port. There's a case statement for message ID 253 which seems to do the loopback test. But my guess is that the unity engine never sends that ID, and there's no way to do so.

2) On a symmetric nat, the master server and the connection facilitator end up with different port numbers (which are also different that the internal server port). When you attempt to connect to the port provided by the master server, the connection fails because the facilitator doesn't know about anyone on that port. I once overrode the client connect function to connect to the port that the host registered with the facilitator on, and it attempted to do punch through but failed.

Something like this probably happens.

Server:
Creates a server on port X
Registers with the MasterServer, who maps port A to X.
Registers with the NatFacilitator, who maps port B to X.

Client:
Gets the MasterServer list.
Is told to connect via port A.
NatFacilitator says "Port A isn't connected to X" and fails immediately.

(By looking at the logs, I instead told the client to connect on port B... in this case)
Client connects to nat facilitator with port S.
Client tries to connect via port B.
Client talks to port B from S.
Server talks to port S from N (where N is a totally new port created by the router).
Connection times out because punch through fails.

Given the information I can get at the moment, it seems like the following would work best for dealing with symmetric nat (In my applications case)

Setup:
Server connects and is tested, determines he can either can't host, can host, or can host with nat.
If he's behind a symmetric router, connection tester will report that he can host with nat (but he actually can't).

Player creates a server. If he can't host, he never registers with the master server. He can play by himself, but won't create "Unable to connect" errors.

If he can host, he registers, identifying if he's using nat or not to the master server. The master server is modified to detect if the facilitator port and master server port are different. If they are then there are two options.

1) Do not add him to the database, since no one can connect to him.

2) Try to connect back on the internal server port. If you succeed, overwrite the port to be the internal port (He's got port forwarding setup), and overwrite useNat to false (We're connecting directly).

If you fail to connect back, last ditch option is to connect via a proxy. But from the documentation it's not clear if only the client or the server need useProxy set to true, or if they both have to be set to true (And set to true when the server is created).

Depending on how the proxy stuff works, it may be possible to setup the client to connect via a proxy without disturbing the server.

At any rate, it'd be really nice if the connection tester would report if nat is needed or not, and if the connection will actually work or not. But I get the impression that something needs to be changed on the Unity engine side for that to work. But it would be great if it could be make to work only by changing the connection tester. I'd be more than happy to do the work if this is the case.

Please, HiggyB, if you can get a network dev to look at this thread and contact me, it would be greatly appreciated.

 

Hi Factoid, any news on the subject? We are working on our second multiplayer game (our first one was Super Volei Brasil 2 - www.supervoleibrasil.com.br) and we're facing all the problems you described - again. This time we are thinking in using a Proxy Server to facilitate the connections in our peer-to-peer approach. But as you said, the documentation is poor on this subject. Do you have any suggestion?

 

I haven't heard anything yet. It may be possible to improve the connection tester by modifying the server side program, but I haven't had the time to look into it further.

Because the connection tester detects if you're running a server publicly, but doesn't seem to care if you're behind a NAT, it may not be really checking the loop back to the server.

To me, it seems like connection tester should report PrivateIPNoNAT if you're behind a router that won't punch through, and PublicIP if you're behind a router that has port forwarding enabled, that would at least let you know if you need to enable the proxy server before you start the host server.

 

And how about testing Public IP server? I mean, Unity asks me to start the server before testing if it is connectable. And after starting the serving running behind a "Public IP" I must test the machine again to see if it's really able to be reached by the clients. It's something very weird in my opinion. Among that, today I just couldn't connect to Unity's default Master Server (we have changed our ADSL router recently, maybe there is something related to this fact).

 

Unity's default master server is not meant for production use, and doesn't carry any guarantees of uptime. Sooner or later you'll probably need to run your own servers (Which can be helpful for testing anyway, being able to see the log output from them). They can be run on the same machine as your unity editor while testing, and moved to a dedicated box later.

You should only need to test once. NAT punch-through is only about checking if an outside connection can reach you. You have to be running a server to know if the connection was established, and if it fails due to a firewall or router, then it checks to see if it can succeed at punching through.

When it comes time to host, or connect, you only need useNat = true if the host is behind a firewall/router, or the client is connecting to a host which requires nat. The host will report it's nat requirement to the master server (for the client to retrieve), and the connection tester will establish if you should be using nat when hosting.

 

Have you sent a request to support@unity3d.com about this problem? They'll probably be able to get you moving.

 

This is why I always try to connect to the original server port after a failed connection attempt on the NAT-server reported port. But a unity-side fix would be great.

 

The big problem is that using a server solution other than that provided by the UT will require us redoing the entire network logic that will be a pain in the behind. We will have to rewrite all the connection logic to use our own facilitator/master server and we don't have the time to do that.

 

Dumb question from the peanut gallery (aka yours truly) but why would you have to rewrite anything, a single shared string of the IP and a shared integer of the Port is all you need, and you can have that on an EGO and have it persisted with 'do not destroy on load' on the object with its script.

Call me confused.
Never ever hard code values like these, your only using it during connect anyway, and server start routines.

What is so set in your code that you would have to start over?