Best HTTP Released

Hello,

How can see the content of a HTTP POST request I send to a server?
With response.DataAsText I can see the response content but for a request?
I need to know if the request body is well formed or not.
Thanks

 

@Kafar

I would suggest to use an intermediate proxy to see what really the plugin sending out to the server.
My personal preference is to use Charles, but any proxy would be equally good.
You can set up the plugin to use a proxy something like this:

Code (CSharp):

1. BestHTTP.HTTPManager.Proxy = new BestHTTP.HTTPProxy(new Uri("http://localhost:8888"), null, true);

 

Ok, thanks.
I have a doubt... Is this the correct sintax for a HTTP POST? Because monitoring with the proxy I can see the server return the method CONNECT only but no POST...

HTTPRequest request = new HTTPRequest(new System.Uri ("https://xxxxxxxx"), HTTPMethods.Post, OnSubscriptionRequestFinished);
request.AddField ("idxxx", idxxx);
request.AddField ("model_id", modelId);
request.Send();

 

@Kafar

CONNECT informs the proxy where it should connect. Ater this step the plugin will send the same request as it would otherwise.

 

And why I see the CONNECT only?

 

@Kafar

You are using HTTPS, the proxy can see only this part of your request. You can turn on SSL Proxying in Charles in the Proxy/SSL Proxying Settings... menu item, or you can use a non-https request.

 

Well, to be more precise, it obviously can see the whole request, but it's encrypted.

 

Sorry, not sure why it would depend on Unity when using BestHttp (not Unity's www) to make a web request.

 

Ok, thanks, I found my issue, I must do this request sending in JSON format. Can BestHTTP do this? Little sample?
Thanks

 

@gschemmel They had to enable/implement IPv6 support in certain places: "And over the past few months we have been working to bring support for IPv6-only networks to Unity’s .NET/IL2CPP libraries."

 

@Kafar

It's really easy. Basically you have to set the RawData property of your request:

Code (CSharp):

1.         Uri uri = new Uri("http://httpbin.org/post");
2.  
3.         HTTPRequest request = new HTTPRequest(uri, HTTPMethods.Post, (req, resp) =>
4.         {
5.             // All error handling omitted for clarity!
6.  
7.             Debug.Log("Response: " + resp.DataAsText);
8.         });
9.  
10.  
11.         request.AddHeader("Content-Type", "application/json");
12.  
13.         string data = "{'user':'username', 'pass' : 'password'}";
14.  
15.         request.RawData = Encoding.UTF8.GetBytes(data);
16.  
17.         request.Send();

 

I'm having an issue using BestHTTP to poll our server database. Intermittently we will get a status code of 0 in our request callback. The server Apache logs always show status code 200 and the server shouldn't be able to send status code 0 at all. Most of the time everything works fine. Leaving our application running over the weekend this only happened about 9 times in total. When it happens we get several logs in a row with status code 0 in a group of requests. Here is the relevant snippet of code doing the requests:

Code (CSharp):

1.  
2. private HTTPRequest RequestGameData;
3. private bool RequestingGameData;
4. private float GetDataStartTime;
5. public float GetDataTimeout = 5.0f;
6. public float GetDataFrequency = 0.75f;
7.  
8. private void Start()
9. {
10.     // Repeating request to get game related data
11.     RequestGameData = new HTTPRequest(new System.Uri(ServerRoot + "endpoint.php"), GetGameDataFinished);
12.     GetDataStartTime = Time.realtimeSinceStartup - GetDataFrequency;
13.     GetGameData();
14. }
15.  
16. private void Update()
17. {
18.     GetGameData();
19. }
20.    
21. public void GetGameData()
22. {
23.     float timeSinceLastRequest = Time.realtimeSinceStartup - GetDataStartTime;
24.     if( (!RequestingGameData && timeSinceLastRequest >= GetDataFrequency) || timeSinceLastRequest >= GetDataTimeout )
25.     {
26.         RequestingGameData = true;
27.         GetDataStartTime = Time.realtimeSinceStartup;
28.         RequestGameData.Send();
29.     }
30. }
31.  
32. public void GetGameDataFinished(HTTPRequest request, HTTPResponse response)
33. {
34.     RequestingGameData = false;
35.     if (request.State == HTTPRequestStates.Finished)
36.     {
37.         if (response.IsSuccess)
38.         {
39.             // Parse response
40.         }
41.         else
42.         {
43.             string errorMsg = "Get game data got error response from server. Status code: " + response.StatusCode.ToString();
44.             Debug.Log(errorMsg);
45.         }
46.     }
47.     else
48.     {
49.         string errorMsg = "Get game data did not finish properly. Request state: " + request.State.ToString();
50.         Debug.Log(errorMsg);
51.     }
52. }
53.  

We are definitely logging a status code of 0, which according to the comments on that enum shouldn't be possible. Am I doing something wrong? I get the feeling this is maybe a race condition or something to do with threading that only happens in a random intermittent edge case and so it's not usually a problem unless you're constantly polling a URL. If I can get around these 0 status codes by waiting until there is a valid status or something all of my connectivity issues will be solved. I was thinking of creating a new request object each time and just reading the properties on that request in an update until I feel it's time to send a new request. Does that seem like a valid route?

 

@bingomoon

That workaround would surely work.

I'm not sure what can cause a status code of zero.
One problem could be if there are data left in the network stream and when the plugin reuses the very same connection it will read out 'garbage'. For this as a prerequisite the length of data must be greater than what the server specified in the content-length header. It's very unlikely, but you can try to disable connection pooling to test:

Code (CSharp):

1. HTTPManager.KeepAliveDefaultValue = false;

You can also turn on more detailed logging:

Code (CSharp):

1. HTTPManager.Logger.Level = Loglevels.All;

Maybe we can see something in it.

Also, I will send a download link to a new package that contains a lot of bugfixes and improvements.

 

Hello, I am having trouble with rejected payloads on a POST request. I am using HTTPRequest as follows:

HTTPRequest request = new HTTPRequest(new Uri("https://hooks.slack.com/services/[redacted]"), HTTPMethods.Post, OnRequestFinished);
request.SetHeader("Content-Type", "application/json");
request.AddField ("text", "testing");
request.Send ();
yield return StartCoroutine (request);

In response.DataAsText I get the reply invalid_payload, though I can get this to work with curl in the shell.

...the server website here says that "by declaring the content type, no further encoding of the POST body is needed — just provide valid JSON in UTF-8." They give the curl example:
curl -X POST -H 'Content-type: application/json' --data '{"text":"This is a line of text.\nAnd this is another one."}' https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX

They provide a different URL-encoded example:
curl -X POST --data-urlencode 'payload={"text":"This is a line of text.\nAnd this is another one."}' https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX

I'm not sure where I'm going wrong. Are you able to help? Thanks in advance.

 

Sorry, one more question for a separate problem. I am trying to send a 4Mb string to my server (this need never be retrieved by the client). Your UploadStream example is helpful for understanding how to aggregate and prepare this for transport. I wondered whether you might also have a suggestion for the most appropriate protocol with which to transmit this stream: is this also a job for an HTTP Post request with JSON?

I have a PHPmyAdmin server with an empty MySQL database ready to receive these upload requests, but have not identified a suitable script or software to connect with and store input from clients. Do you happen to know of any off-the-shelf software solutions for the server side to receive POST requests?

 

@popmem

In your and with the first curl example you have to use the RawData property of the request:

Code (CSharp):

1. string json = "{'text':'This is a line of text.\nAnd this is another one.'}";
2.  
3. HTTPRequest request = new HTTPRequest(new Uri("https://hooks.slack.com/services/[redacted]"), HTTPMethods.Post, OnRequestFinished);
4. request.SetHeader("Content-Type", "application/json");
5.  
6. request.RawData = Encoding.UTF8.GetBytes(json);
7.  
8. request.Send();
9. yield return StartCoroutine(request);

This will not use any further encoding.

On the other side, the second example has no "Content-Type" header, and the content should be sent in the "text" field:

Code (CSharp):

1. string json = "{\"text\":\"This is a line of text.\nAnd this is another one.\"}";
2.  
3. HTTPRequest request = new HTTPRequest(new Uri("https://hooks.slack.com/services/[redacted]"), HTTPMethods.Post, OnRequestFinished);
4. request.AddField("payload", json);
5.  
6. // It must sent as UrlEncoded
7. request.FormUsage = BestHTTP.Forms.HTTPFormUsage.UrlEncoded;
8. request.Send();
9. yield return StartCoroutine(request);

 

@popmem

The UploadStream example helps when you don't have all the data and you have to wait for additional one. If you already have a 4MB string, you can send it all at once.
I would recommend to send it through the RawData property as it will send the raw bytes, while sending it as a form field (AddField) would do additional encoding that would eat up too much resources (cpu and memory garbage).

A very basic php script should read the sent data through php://input.

 

In the first example, I had to use the quote-escaped version of the json string from your second example, but it then worked perfectly. Thank you very much! I have been stuck with this problem for quite some time. Thank you also for the uploading advice. I will give it a try.

 

Hi and thanks for a great product!

I'm implementing IAP and I noticed that during IAP (with Google's Play Store) Unity goes to background of sorts and when the IAP process is complete and Unity is again in focus, my BestHTTP socket disconnects.

What's the best practice here?

 

@MoranShemi

When your app goes to the background after some time the operating system closes the tcp connections associated to that app due to inactivity.
There are several ways to handle this case, but it might be the best to disconnect manually when the app goes to background and reopen the connection on focus.

 

Hi

I'm getting errors each time a new event is received. It seems to be related to calling Socket.Off, as it runs fine without this.

Below is an example to reproduce the issue.

Code (CSharp):

1. using UnityEngine;
2. using System;
3. using System.Collections;
4. using BestHTTP.SocketIO;
5.  
6. public class Test : MonoBehaviour {
7.  
8.     private SocketManager _socketManager;
9.  
10.     void Start ()
11.     {
12.         _socketManager = new SocketManager(new Uri("http://chat.socket.io/socket.io/"));
13.         _socketManager.Socket.On(SocketIOEventTypes.Connect, OnConnect);
14.     }
15.  
16.     void OnConnect(Socket socket, Packet packet, params object[] args)
17.     {
18.         Debug.Log("OnConnect");
19.         _socketManager.Socket.On("new message", OnNewMessage);
20.     }
21.  
22.     void OnNewMessage(Socket socket, Packet packet, params object[] args)
23.     {
24.         Debug.Log("OnNewMessage");
25.         _socketManager.Socket.Off("new message", OnNewMessage);
26.     }
27. }
28.  

I get this error each time a event is received (after the first one):
Ex [WebSocketTransport]: OnMessage - Message: 1: Object reference not set to an instance of an object at BestHTTP.SocketIO.Events.EventDescriptor.Call (BestHTTP.SocketIO.Socket socket, BestHTTP.SocketIO.Packet packet, System.Object[] args)

The same issue occurs using Socket.Once too.

Any ideas?
Thanks!

 

@db82

Thanks for the detailed info and repro code. I was able to reproduce it and sent a link in private to an updated package.

 

We seem to have possibly encountered a defect in BestHTTP when using mixes of Async and Sync type GET responses. Most of the time everything is fine but we have seen where about 1% of the time the wrong response will come back.

e.g. an Asyc to site1/page1 response is sent to Sync query site1/page2

We tend to send out multiple GET requests and can overlap and after many checks and retests over the past two weeks we have confirmed that sometimes the http response comes from the wrong query.

We would like to avoid converting them all to single query at a time but we cannot have the "wrong" reply. Has anyone else encountered this?

 

Hi, I'm using Best HTTP for a year long. This is a great plugin. Thank you!

Recently I found some strange behavior related caching.
According to the document, BestHTTP doesn't access to the server if the server returns a response with "Expire" header with the future date.
The issue is that even if my API server returns with future date expire header, BestHTTP still accessing to the server. Sometimes it accesses and sometimes not.

I dug into the BestHTTP source code and found that the problem has occurred at cache library key loading/saving part.
The URI, which cache mechanism doesn't work, is like "http://example.com/?artist=Earth Wind & Fire".

At line 641 in HTTPCacheService.cs, it saves cache key (URI object) as string using ToString() method. And line 597, it loads string and rebuilds URI object. BUT original URI object and rebuilt URI is not equal if original URI contains %26 (or character needed to be URL encoded).

As a result, HTTPCacheService couldn't find cached response from its library and accessed to the server.

Probably, it's a bug and should be fixed. How do you think?

 

@Saqoosha

Thanks you for the detailed report. Sent a download link to an updated package in private that should fix this issue.

 

@malek256

Never heard about an issue like this and can't even imagine how it would be possible. Can you share some more details even some code?
What's an Async and Sync request in your implementation/interpretation? Waiting a request to complete with a yield request (, or yield return StartCoroutine(request) on older Unity ) is basically the same as passing a callback to the HTTPRequest constructor.

 

This is for an online-dependant game, so I can't restore the session easily after disconnection. I ended up solving/avoiding the issue by increasing idle timeout. Thanks!

 

I get the following error when I use bestHTTP v1.9.11 along with the latest Json.NET(v2.0.0).
Could you inform me on what is happening and perhaps a solution to fix it?

 

@playnext_jp

Thank you for letting me know. Sent a link to an updated package in a private message.

 

This plugin seems like it will fit right into my project but I need to know a couple of things that aren't detailed on the documentation. I intend to use Websockets with a specific framework and I need to do use url queries and a custom handshake which basically changes an id on the url query. Does this plugin let me do this? or is it extensible enough that will let me implement this?

 

@Ownstoppable

Url queries are supported OOB, other modifications can be done too as the Pro version of the plugin contains all the source code. And I can help too to add missing features or point to the right direction.

 

@BestHTTP
Sounds good, I'll proceed with the purchase. If I need some help should I just post in this thread?

 

@BestHTTP

Will you support Socket.IO 1.4 compression? Can get alot of savings on an online game with it.

 

@Ownstoppable
You can post here in the forum, in a private message or you can send me a mail to besthttp@gmail.com. Whatever is best for you.

 

@AwDogsGo2Heaven

Well, the harder part of the support is already done: WebSocket Per-Message Deflate Compression extension added in version 1.9.9 (2016.02.16).
So it will negotiate the compression extension details with the server, and it will decompress server-sent compressed messages, and will automatically compress sent messages that longer than a predefined value.

With polling decompressing server sent messages also done transparently, and supported since years.

What is missing is to be able to control it from the Socket.IO api level and be able to set it per message. What i don't know is how compression is done on client side for polling, if it's even implemented.

 

Thanks, so it should work provided your ok with everything being compressed if I understand you correctly, there is no 'fine tuned' control over it yet.

 

@BestHTTP
Hi I'm about to decide to buy your awesome product. I intend to use it to SignalR servers and I need to not use hubs API and use the persistent connection with byte array messages. Is it possible to do this with bestHTTP ?

 

@AwDogsGo2Heaven Yes. The client should decompress the server sent messages correctly, and the client will compress most of the messages.

 

@Ashkan_gc

Yes, it's possible. The SignalR Connection class has a Send function to send an object to the server.
The ConnectionAPI sample also uses this function to send non-hub messages to the server.

 

I usually run my requests through Fiddler by setting the proxy property on a request.
For some reason this broke with the last update. I can see the SSL tunnel being established in Fiddler and then nothing happens. Eventually the request callback is invoked with a null response.

 

@fantastisch_
If you have the Pro version of the plugin you can revert to the previous behavior quickly by editing the \Assets\Best HTTP (Pro)\BestHTTP\HTTPResponse.cs. You have to uncomment lines 647 through 649 and 664 through 681.
Let me know if you need any help.

 

Dear @BestHTTP !

I have an issue that since I upgraded my project to Unity 5.3.5p4 (also tested with p5), an iPad running iOS 8.4 does not handle cookies well. It seems to not keep server-sent cookies and use them in subsequent requests, so they all get 403-ed.

It doesn't happen on other devices running iOS 8.4 (tested on iPhone 5), and it doesn't happen on Unity 5.1.1.
I also tested with older BestHTTP version (1.8.2) on Unity 5.3.5p4 but no change.

Although it's not necessarily a BHTTP issue, I thought I'd ask for known issues or ideas.

Thanks!

 

@MoranShemi

I will try to reproduce it tomorrow.
Can I ask you to increase the loglevel and send the log to me in a private message or in a mail?
You can set the log level like this:

Code (CSharp):

1. BestHTTP.HTTPManager.Logger.Level = BestHTTP.Logger.Loglevels.All;

 

@BestHTTP Sent to besthttp@gmail.com

Thanks!

 

@BestHTTP We began using HTTPS with our WebPlayer today, and now on our Response methods with (HTTPRequest request, HTTPResponse response) as parameters the response is coming in null. Do you have any idea why this would be?

 

@Ben BearFish Is it really a WebPlayer build and not a WebGL?
Anyway I would recommend to handle the request's states in the callback, like in this sample request:

Code (CSharp):

1. Uri uri = new Uri("https://httpbin.org/get");
2. HTTPRequest request = new HTTPRequest(uri, (req, resp) =>
3. {
4.     switch (req.State)
5.     {
6.         // The request finished without any problem.
7.         case HTTPRequestStates.Finished:
8.             if (resp.IsSuccess)
9.             {
10.                 Debug.Log("Request Finished Successfully! Response: " + resp.DataAsText);
11.  
12.                 // TODO: Request finished. Process server sent data.
13.             }
14.             else // Internal server error?
15.                 Debug.LogWarning(string.Format("Request Finished Successfully, but the server sent an error. Status Code: {0}-{1} Message: {2}",
16.                                                 resp.StatusCode,
17.                                                 resp.Message,
18.                                                 resp.DataAsText));
19.             break;
20.  
21.         // The request finished with an unexpected error. The request's Exception property may contain more info about the error.
22.         case HTTPRequestStates.Error:
23.             Debug.LogWarning("Request Finished with Error! " + (req.Exception != null ? (req.Exception.Message + "\n" + req.Exception.StackTrace) : "No Exception"));
24.             break;
25.  
26.         // The request aborted, initiated by the user.
27.         case HTTPRequestStates.Aborted:
28.             Debug.LogWarning("Request Aborted!");
29.             break;
30.  
31.         // Connecting to the server is timed out.
32.         case HTTPRequestStates.ConnectionTimedOut:
33.             Debug.LogError("Connection Timed Out!");
34.             break;
35.  
36.         // The request didn't finished in the given time.
37.         case HTTPRequestStates.TimedOut:
38.             Debug.LogError("Processing the request Timed Out!");
39.             break;
40.     }
41. });
42.  
43. request.Send();

If you are building a WebGL build, you can also check the browser's console, it may contain additional details.

 

Hi,

I am having a problem with the library. I'm using the Pro version.

Recently, it started returning null responses randomly. One request finishes fine, the next might or might not time out. I've talked with the colleague responsible for the server, he is 100% sure it's not coming from him.

So what actually happens with BestHTTP is:
- I send a request.
- The request either times out (BestHTTP logs ConnectionTimedOut) or finishes perfectly fine.
- The response callback starts. If the request timed out, the response object is null.

I don't think I've changed the default options, but I'll post them anyways:

Code (CSharp):

1. MaxConnectionPerServer = 4
2. KeepAliveDefaultValue = true;
3. MaxPathLength = 255;
4. MaxConnectionIdleTime = TimeSpan.FromSeconds(30);
5.  
6. CookieJarSize = 10 * 1024 * 1024;
7. EnablePrivateBrowsing = false;
8. ConnectTimeout = TimeSpan.FromSeconds(20);
9. RequestTimeout = TimeSpan.FromSeconds(60);
10.  
11. DefaultCertificateVerifyer = null;
12. UseAlternateSSLDefaultValue = true;

I hope you're able to help me, as apparently the server is fine.

 

@bigticket21

Well, the connection part is made by the Socket implementation of Unity's mono. If you are using the same plugin version that worked before, than I'm out of any idea.
But, you can try out these:
1.) Decrease MaxConnectionIdleTime by halving it, or setting the KeepAliveDefaultValue to false.
2.) Double ConnectTimeout
3.) Double RequestTimeout

 

Thanks, I'll try all of these to see for any differences.

As for the plugin version, I updated it last week, but the problem started before that (I'm not sure on the exact date). I hoped the update could change something.

I think the problem may have started around the time when the library stopped using Good Ol' Sockets (I think that's what it was called?).

 

@bigticket21 Good Ol' Sockets only needed when it's used with free version of Unity 4.x, as Socket support was in Unity Pro only.