UnityWebRequest.Post sends strings with escaped characters

I have an API listening outside of Unity. I'm trying to send players score with UnityWebRequest.Post to the API. I am able to send score with Postman but not with Unity. After 2 hours of debugging I finally found what is happening with Wireshark.

What is actually happening here with UnityWebRequest.Post and why and how to fix this problem? Upper packet is sent from Unity, lower is from Postman.

Code (CSharp):

1. public static IEnumerator SendScore(Score score)
2.             {
3.                 string bodyData = JObject.FromObject(score).ToString();
4.                 Debug.Log(bodyData); // Check the second image
5.  
6.                 using (UnityWebRequest request =
7.                     UnityWebRequest.Post(GameSettings.ApiPostScoreUrl, bodyData))
8.                 {
9.                     request.SetRequestHeader("Content-Type", "application/json");
10.                     yield return request.SendWebRequest();
11.                
12.                     if (request.isHttpError || request.isNetworkError)
13.                     {
14.                         Debug.LogError("Error on sending the score: " + request.error);
15.                     }
16.                     else
17.                     {
18.                         Debug.Log("Sent score, possible answer: " + request.downloadHandler.text);
19.                     }
20.                 }
21.             }

Code (CSharp):

1. namespace Api.Models
2. {
3.     public class Score {
4.  
5.         public int Points;
6.         public string Name;
7.     }
8. }

 

Anyone? I mean the POST gets sent but my API is unable to map it into a model due to weird characters.

 

The Unity docs state that:

The Content-Type header will be set to application/x-www-form-urlencoded by default.

Note: Many server backend languages do not properly handle POST requests with Content-Type headers set to encoding others than application/x-www-form-urlencoded or multipart/form-data.

If you're sure that your API is able to handle unencoded data then you could change the POST headers in Unity to send it unencoded using multipart/form-data as the content-type (here's an example of posting form data: https://docs.unity3d.com/Manual/UnityWebRequest-SendingForm.html )




Perhaps this bug is also relevant? https://fogbugz.unity3d.com/default.asp?826626_htlchp13nh8th2to

 

Hello, thank you for your response.

I wrote my API to use Content-Type application/json. I can verify with Postman that this works (API is also able to parse data into a model correctly, checked it with a breakpoint). I can also verify that with x-www-form-urlencoded it does not work.

In Unity side I am setting Content-Type to be at the line 9:

Code (CSharp):

1. request.SetRequestHeader("Content-Type", "application/json");

Is there any ways to get around the encoding issue (or escaped characters - I mean those %22 %2a things)?

 

Normally it doesn't matter if it's URL encoded - the API on the server end usually decodes it automatically.

For example, if I send data like that to my PHP code it usually decodes it (or you can decode it when it comes in using the PHP command urldecode).

What language is your API written in? It probably has a function to decode the URL encoded string - might be worth trying to add that to your API.

 

i'm having exactly the same bug report. Which unity version are you using? 2017.2 seems to work, .3 is broken

 

I wrote my web API using ASP.NET Core, I have no experience on it and wanted to give it a go. I will read more about URL encoding and ASP.NET Core documentation tomorrow to see if I will find anything. Thank you!

I am using the 2017.2. Can't remember the first version I used which had the same problem. I had to upgrade to Patch 2017.2.0p4 for fixing another bug (broken WebGL build).

 

if you get the exact version number that broke it, you can submit a bug report with a test case and they'll fix it asap, they want to release .3 this week and breaking WWW calls is a critical bug

 

that's exactly what im doing already and the server responds with a NULL error. Code works perfectly fine on 2017.2

 

Hello,

how I can pass byte[] to UnityWebRequest.Post()? There is no such constructor as well as uploadHandler.data doesn't have setter.


EDIT: I get 404 from API with WWWForm... Seems to be incredibly hard with Unity to just send what you want without any escaping or other modifications. Also it seems that WebClient doesn't work with WebGL build...

Code (CSharp):

1.  public static IEnumerator SendScore(Score score)
2.             {
3.                 WWWForm form = new WWWForm();
4.                 form.AddField("Name", score.Name);
5.                 form.AddField("Points", score.Points);
6.  
7.                 using (UnityWebRequest request = UnityWebRequest.Post(GameSettings.ApiPostScoreUrl, form))
8.                 {
9.                     request.SetRequestHeader("Content-Type", "application/json");
10.                     yield return request.SendWebRequest();
11.                    
12.                     if (request.isHttpError || request.isNetworkError)
13.                     {
14.                         Debug.LogError("Error on sending the score: " + request.error);
15.                     }
16.                     else
17.                     {
18.                         Debug.Log("Sent score, possible answer: " + request.downloadHandler.text);
19.                     }        
20.                 }
21.             }

 

Code (CSharp):

1. public static IEnumerator SendScore(Score score)
2.             {
3.                 byte[] data = Encoding.UTF8.GetBytes(JObject.FromObject(score).ToString());
4.  
5.                 using (UnityWebRequest request = new UnityWebRequest(GameSettings.ApiPostScoreUrl))
6.                 {
7.                     request.SetRequestHeader("Content-Type", "application/json");
8.                     request.method = "POST";
9.                     request.uploadHandler = new UploadHandlerRaw(data);
10.                     request.downloadHandler = new DownloadHandlerBuffer();
11.  
12.                     yield return request.SendWebRequest();
13.  
14.                     if (request.isNetworkError || request.isHttpError)
15.                     {
16.                         Debug.LogError("Unable to send the POST: " + request.error);
17.                     }
18.                     else
19.                     {
20.                         Debug.Log("Sent the score to the API");
21.                     }
22.  
23.                 }
24.             }

I get 404 from the API. It seems request.method = "POST" doesn't work, I get following in the API console:

Any other ways to set the method?

EDIT: It actually works on local (running Unity in the editor as well as having a local version of API and unity connecting to localhost). Problem is with the WebGL build or with my server.. I have Linux, running API on http and having a reverse proxy with Apache2 which listens https. Needs more investigating..
EDIT: Removing https didn't do anything..
EDIT: Ah, found the reason.. On another thread:

EDIT: Ok, now everything works.. I had API on another host and subdomain. I moved the API now into the same host and subdomain as the game because of deadline, will maybe later research more on this topic. Still using proxy and it works fine. Will try https soon. If someone needs apache conf for such setup then here it is:

 

Hello there!
I'm afraid that I have the exact same problem with 2017.30f3.

I'm trying to do a simple login, here's my code :

Code (CSharp):

1. IEnumerator GetClientAccesToken()
2.     {
3.         if(errorCounter >= MAX_ERROR_COUNT)
4.         {
5.             StopCoroutine("GetClientAccesToken");
6.             errorCounter = 0;
7.             yield break;
8.         }
9.         else
10.         {
11.             var form = new WWWForm();
12.  
13.             form.AddField("grant_type","password");
14.             form.AddField("client_id","4");
15.             form.AddField("client_secret","kfbhpxTn0HA9Y4Nsf2ewa4JHCeMmaLo7oTNhvjS");
16.             form.AddField("username","desarrollador@guiara.com");
17.             form.AddField("password","gUIaRA1");
18.  
19.             var headers = form.headers;
20.             headers["Accept"] = "application/json";
21.  
22.             var rawData = form.data;
23.  
24.             var www = new WWW(_URI_TokenUsuario, rawData, headers);
25. //            var www = new WWW(_URI_TokenUsuario, form);
26.  
27.             yield return www;
28.  
29.             if(!string.IsNullOrEmpty(www.error)) ///Error en la descarga
30.             {
31.                 errorCounter ++;
32.                 if(Debug.isDebugBuild)
33.                 {
34.                     Debug.LogFormat("*** Error at <{0}> with URI <{1}> LOG <{2}>", "GetClientAccesToken", _URI_TokenUsuario, www.text);
35.                 }
36.                 StartCoroutine("GetClientAccesToken");
37.                 yield break;
38.             }
39.             else
40.             {
41.                 try
42.                 {
43.                     if(Debug.isDebugBuild)
44.                     {
45.                         Debug.LogFormat("*** Download succesfull at <{0}> with URI <{1}> LOG <{2}>", "GetClientAccesToken", _URI_TokenUsuario, www.text);
46.                     }
47.                        
48.                     CreateUserToken(www.text);
49.  
50.                     yield break;
51.                 }
52.                 catch (Exception e)
53.                 {
54.                     Debug.LogFormat("Exception at <{0}>  <{1}>", "GetClientAccesToken", e);
55.                 }
56.             }
57.         }
58.     }

In Unity 2017.2.0f3 my code works out of the box even with this two aproaches:

var www = new WWW(_URI_TokenUsuario, rawData, headers);
var www = new WWW(_URI_TokenUsuario, form);

But in 2017.30f3 it just fails. I've the same result with uwr in 2017.3. Ther's a solution or a "hack" to get arround this?

Thanks in advance.

 

It's a serious bug. It is fixed on 2018.1 but not on 2017.3p1 yet. It's been reported several times, I don't know why they don't consider it critical and why it wasn't fixed before release because it was reported before the cut date.

I'm on the same situation. I can't update to .3 because then none of my web calls work but i need all the fixes included on .3 so... basically stuck without able to release for weeks

 

Could we get UT to answer this. How are we to set byte[] as a parameter to a method that expects a string? Can you not just create another Post method that doesn't URL encode the request body?

Code (CSharp):

1.        //
2.         // Summary:
3.         //     Creates a UnityWebRequest configured to send form data to a server via HTTP POST.
4.         //
5.         // Parameters:
6.         //   uri:
7.         //     The target URI to which form data will be transmitted.
8.         //
9.         //   postData:
10.         //     Form body data. Will be URLEncoded prior to transmission.
11.         //
12.         // Returns:
13.         //     A UnityWebRequest configured to send form data to uri via POST.
14.         public static UnityWebRequest Post(string uri, string postData);

 

Thank You for the reply.

I can get the UnityWebRequest to work in the editor but not in Android.

Get requests over https work flawlessly, so I just set up a proxy server with a simple state machine. I make requests to the proxy via GET, then depending on the "message" type in the GET URL (POST, PUT, or DELETE) I parse the body from the rest of the data in the GET url and send it to the main server which responds to the proxy which then sends the data back to the app. It's a pain in the butt, but It works. In the mean time, I am going to keep working on the game it's self. This is keeping me in a rut and I am not getting anything accomplished. Once I have the game in the Play Store, I will circle back and get the web requests done.

 

I use Unity 2020.1 and Android 10 and Android 11

 

TlsException: Handshake failed - error code: UNITYTLS_INTERNAL_ERROR, verify result: UNITYTLS_X509VERIFY_FLAG_NOT_TRUSTED

Same as this Stack Overflow Answer where the answer very explicitly details the process to chain the intermediate certificates and I still couldn't figure out how to do this. https://stackoverflow.com/questions...e-failed-unitytls-x509verify-flag-not-trusted

The problem is, aside from the fact that I am not well informed when it comes to TLS/SSL issues, that I secured the back end with LetsEncrypt which doesn't allow access to the root certificate. To be honest I am not sure it is even necessary for the app to be able to access the root certificate.

From LetsEncrypt's Website at https://letsencrypt.org/certificates/

When I am at a better place in the development of the game, I will probably buy a certificate from one of the bigger certifying authorities.

 

Yes. I built a front end registration form and it works with the exact data I was sending. The last time I worked on it, however I added

app.use(bodyParser.urlencoded({ extended: true })); [ICODE]UnityWebRequest

urlencodes the data sent
[/ICODE] and I haven't tested the web form since. https://unadyne.com

I just checked, and I think I left the server off....

 

For some reason, GET requests using System.NET work on the Android builds.

 

Thank You. For your attention.

Thank You. My problem is not the urlencoding. I only mentioned it because I am not urlencoding the json from the web front end, so it may not work. My problem is I don't know how to chain my certificates to account for unity / Android not caching the intermediate certificates. I don't even know if it's possible with LetsEncrypt.

 

Try checking your server on:
https://whatsmychaincert.com/

It does a more detailed analysis of your provided certificate. The issue is that you usually should include all the intermediate chain certificates as well. This page can actually generate the proper chain certificate for you. Browsers might lookup the chain certificates themselfs, however not every TLS implementation may do this.

 

Thank You. I actually did that. https://whatsmychaincert.com/?unadyne.com I think that since LetsEncrypt keeps the root certificate offline, it will never work. I think I am going to just have to buy a certificate whose chain can be verified all the way to root. To be honest, I am not very knowledgeable on security subjects, but this issue has been a blessing in disguise by making me read about ssl/tls.

 

String jsonString = MiniJSON.Json.Serialize(finalParameter);
Byte[] formData = Encoding.UTF8.GetBytes(jsonString);

UnityWebRequest w = UnityWebRequest.Post(finalUrl,"");
w.uploadHandler = new UploadHandlerRaw(formData); //body
w.SetRequestHeader("Content-Type", "application/json;charset=utf-8");
w.timeout = 15;

I success

 

Thanks for sharing! This worked for me too!

If this call "public static Networking.UnityWebRequest Post(string uri, string postData);" is broken or only works in some weird cases, I hope Unity team can consider updating the documentation. Right now, even sample is not accurate: https://docs.unity3d.com/ScriptReference/Networking.UnityWebRequest.Post.html

 

This worked. But, why not just have a POST method which allows taking in a byte array and just handle all this for us?

 

Nice, glad to hear. I guess 'latest version' is tricky because of unity's insane naming conventions. We're on some 2021 LTS version, and will be for awhile. Glad to hear it's improving though!

 

What version is it implemented?

 

I'm not completely sure but just looking at the git history it wasn't in 2021.1.0a2 but is in the next update we got which was 2022.2.0a10. So it was introduced somewhere in between.