Making localhost HTTP requests in Unity are slow? Help!

After exhaustive testing I have arrived at a threaded WebClient loop to try to poll a local server for data as fast as possible:

Code (csharp):

1. string urlAddress = "http://localhost:8080/data/json/";
2.  
3. void RunThreadAPI(object data)
4. {
5.     var tc = (ThreadController)data;
6.     while (tc.ShouldExecute)
7.     {
8.         using (WebClient client = new WebClient())
9.         {
10.             client.Proxy = null;
11.             string pagesource = client.DownloadString(urlAddress);
12.             Debug.Log("WWW Ok!: " + pagesource);
13.         }
14.     }
15. }

This results in less than 1 request per second.

I have also tried using the WWW class, which was slightly faster ~1.5 packets per second maybe..

What can I do to obtain at least 10 or even 15 request per second? I would like to reach > 50 request per second, the data is only a few hundred bytes and using Python it's very easy to achieve >50 requests per second. But Unity is crawling slow.

Do I have any options for faster response times??

[UPDATE]

I think the issue is in my server, not sure what is different between Python and Unity, but after reusing the connection in my server code responses are faster on my Unity client.

 

You're losing speed instantiating the WebClient object. You should probably manage this connection in a singleton and send packets over the already open connection.

"
When the lifetime of an IDisposable object is limited to a single method, you should declare and instantiate it in the using statement. The using statement calls the Dispose method on the object in the correct way, and (when you use it as shown earlier) it also causes the object itself to go out of scope as soon as Dispose is called. Within the using block, the object is read-only and cannot be modified or reassigned.

The using statement ensures that Dispose is called even if an exception occurs within the using block. You can achieve the same result by putting the object inside a try block and then calling Dispose in a finally block; in fact, this is how the using statement is translated by the compiler. The code example earlier expands to the following code at compile time (note the extra curly braces to create the limited scope for the object):"

"

 

I changed my thread to now use DownloadStringAsync.

It runs way better than before for a few seconds, then suddenly pauses, then continues running great, then pauses, then runs, then pauses... It's so frustrating while Python just works perfectly.

I have no idea what to do, I need consistent responses without pausing:

Code (csharp):

1.  
2. private ThreadController _threadController;
3. class ThreadController
4. {
5.     public bool ShouldExecute { get; set; }
6. }
7. private Thread tAPI;
8.  
9. void wc_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
10. {
11.     string text = e.Result;
12.  
13.     var mocapData = ParseLineData(text);
14.     RunOnMainThread.Enqueue(() =>
15.     {
16.         //Do stuff
17.     });
18. }
19.  
20. void RunThreadAPI(object data)
21. {
22.     var tc = (ThreadController)data;
23.     string urlAddress = "http://localhost:8080/data/json/";
24.  
25.     while (tc.ShouldExecute)
26.     {
27.         if (RunOnMainThread.IsEmpty)
28.         {
29.             MyWebClient client = new MyWebClient();
30.             client.DownloadStringCompleted += new DownloadStringCompletedEventHandler(wc_DownloadStringCompleted);
31.             client.DownloadStringAsync(new Uri(urlAddress));
32.         }
33.  
34.         Thread.Sleep(1);
35.     }
36. }
37.  

[EDIT]

I think I have made progress by adding a `activeRequests` variable to prevent multiple asyncHandlers from running. I no longer see any long pauses and I am getting ~100 requests per second:

Code (csharp):

1.  
2. private ThreadController _threadController;
3. class ThreadController
4. {
5.     public bool ShouldExecute { get; set; }
6. }
7. private Thread tAPI;
8.  
9.  
10. private int activeRequests = 0; //this prevents multiple async handlers from running concurrently
11.  
12.  
13. void wc_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
14. {
15.     string text = e.Result;
16.  
17.     var mocapData = ParseLineData(text);
18.     RunOnMainThread.Enqueue(() =>
19.     {
20.         //Do stuff
21.     });
22.  
23.     activeRequests--;
24. }
25.  
26. void RunThreadAPI(object data)
27. {
28.     var tc = (ThreadController)data;
29.     //MyWebClient client = new MyWebClient();
30.     //client.Proxy = null;
31.     string urlAddress = "http://localhost:8080/data/json/";
32.  
33.     while (tc.ShouldExecute)
34.     {
35.         if (activeRequests == 0)
36.         {
37.             activeRequests++;
38.             MyWebClient client = new MyWebClient();
39.             client.DownloadStringCompleted += new DownloadStringCompletedEventHandler(wc_DownloadStringCompleted);
40.             client.DownloadStringAsync(new Uri(urlAddress));
41.         }
42.  
43.         Thread.Sleep(1);
44.     }
45. }
46.  

It is working reliably for my testing. But I am afraid that if even a single wc_DownloadStringCompleted callback is missed then the thread will get stuck because activeRequests will remain above zero and no new request will be created.

I am sure it is not a proper solution, but definitely was an improvement.

 

I think I found the issue and it was very simple.

Never use "http://localhost" in the URL, because it was causing the connection to reset/retransmission after every 100th request (and was causing the pausing I was experiencing)

instead I changed the URL to: "http://127.0.0.1" and was no longer seeing connection resets!!!

So I was able to revise my thread loop to use a simple HttpClient pattern:

Code (csharp):

1. public class SyncAvatarMocap : MonoBehaviour
2. {
3.  
4.     private HttpClient client;
5.    public static readonly ConcurrentQueue<Action> RunOnMainThread = new ConcurrentQueue<Action>();
6.  
7.    
8.     void Start()
9.     {    
10.         WebRequest.DefaultWebProxy = null;
11.  
12.         _threadController = new ThreadController {ShouldExecute = true};
13.         tAPI = new Thread(RunThreadAPI);
14.         tAPI.Start(_threadController);
15.     }
16.  
17.     void LateUpdate()
18.     {
19.         if(!RunOnMainThread.IsEmpty)
20.         {
21.             Action action;
22.             while(RunOnMainThread.TryDequeue(out action))
23.             {
24.                 action.Invoke();
25.             }
26.         }
27.     }
28.  
29.     void OnDisable()
30.     {
31.         if (_threadController != null)
32.         {
33.             _threadController.ShouldExecute = false;
34.         }
35.         _threadController = null;
36.     }
37.  
38.  
39.     private ThreadController _threadController;
40.     class ThreadController
41.     {
42.         public bool ShouldExecute { get; set; }
43.     }
44.  
45.    private Thread tAPI;
46.     void RunThreadAPI(object data)
47.     {
48.         var tc = (ThreadController)data;
49.         string urlAddress = "http://127.0.0.1:8080/data/json/";
50.  
51.         using(var client = new HttpClient())
52.         {
53.             while (tc.ShouldExecute)
54.             {
55.                 if (RunOnMainThread.IsEmpty)
56.                 {
57.                    var text = client.GetStringAsync(new Uri(urlAddress)).Result;
58.                     var mocapData = ParseLineData(text);
59.  
60.                     RunOnMainThread.Enqueue(() =>
61.                     {
62.                        Debug.Log(text);
63.                         //DO STUFF WITH DATA
64.                     });
65.  
66.                     //Thread.Sleep(50);
67.                 } else {
68.                    Thread.Sleep(1);
69.                }
70.             }  
71.         }
72.     }  
73.  
74. }

I do not know why "localhost" causes reset issues while "127.0.0.1" does not. Anyone know why that might be?

 

I can't give you a reason, but I know this is very common and have ran into this myself in the past. I assume it has something to do with the machine resolving localhost, but never have found a true reason why.

I have in the past suspected it is something to do with machine name length, I once had a >15 length machine name operating on active directory and when I used localhost via a webservice it slowed to a crawl when i was debugging in visual studio. Changed the machine name and it fixed it, but it seems strange to me why it would affect it, so it could just be a coincidence.