Mangled POST requests with UnityWebRequest on 2018.4.x [SOLVED?]

UPDATE
We have managed to fix this by manually patching the framework file after build.
See this post for more info.

Reference patching tool can be found at https://github.com/SWAG-MASHA/UnityWebRequestWebPatcher
---

We're experiencing a strange issue sending POST request in certain browsers on certain machines. The request body seems to be completely mangled (looks like random bytes of memory)



We haven't been able to find the culprit of the problem - it's relatively rare (it only happens on most browsers of 4 of the machines of our QA department and 1 browser of our developer). But even 3% reproduce-ability among our players means a lot of negative feedback, so we can't leave it as it is.

We tried updating to the latest 2018.4.19, but it doesn't solve the problem

I can't really find any info on these Unity forums or on the internet and can't file a bug because we don't have a 100% working way of reproducing it.

Does anyone have any info regarding this? Or anyone experiencing something similar?

 

I have the same on my 2 games and it seem that the UnityWebRequest POST function is affected.
Around 3/5 % of my players has been affected , so it has been really hard to find the problem.

I have moved everything to GET in order to solve the situation quickly but my code look horrible.

 

Affected players get this instead of "GetInfo:ffisjs87564fddd54qs658"
I'm using Unity 2019.2.11f1

 

We have managed to find the culprit of the problem.
Taking a look at the .wasm.framework.unityweb code and seeing the _JS_WebRequest_Send method, it tries to http.send the raw bytes of the body before converting them to a string:

Code (JavaScript):

1. http.send(HEAPU8.subarray(ptr, ptr + length));

It seems like most browsers on most systems can correctly detect raw bytes and convert them to a string automatically. But some can't (those 3% of cases).

What we did - we wrote a patching tool that takes compressed framework.unityweb file, decompresses it, changes the http.send code so it takes a pre-converted string (to a UTF-8 using a TextDecoder), and compresses it back to gzip again.
TextDecoder is a relatively unsupported tool, but it's more widely supported than wasm itself, so it shouldn't be a problem to use. There are also a lot of third-party js implementations that convert a sequence of UTF-8 bytes to a string.

It took some time to manage the cryptic nature of those compressed files, but POST requests work correctly now (including the internal ones like unity analytics)
Unity requires certain comments to be present in the gzip archive and we couldn't manage to find where exactly does Unity adds that comment to a gzip archive (7za instance which is embedded in Unity doesn't support adding comments).

It would be great to get some comment of the Unity Tech guys on this problem.

I will post a patching tool here a bit later.

 

Maybe you can patch the WebRequest.js file present in :
- Program Files\Unity\Hub\Editor\"YourUnityVersion"\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\lib instead of using your tool???

Also, posting your solution can be usefull. Thank you

 

I have just checked - files UnityNative.asm.js and UnityNative.js both contain this code, and it can be patched right on the Unity install.

It doesn't really suit our case though as we use a separate CI tool that automatically builds new versions. And patching it on a Unity instance instead of on a post-build process will increase the maintenance pressure on our dev-ops team. It's also less transparent on what's going on and why certain builds on certain machines work (which have Unity patched) and some don't. Explicit is better than implicit.

But again, maybe someone's running a one-dev-machine setup and rarely updates Unity, so engine patching can be a viable option

 

Maybe @Marco-Trivellato @jukka_j @alexsuvorov @Schubkraft can take a look and fix it on a future release.

@Kumo-Kairo Can you share how you fixed it (code).

Thank you

 

This is a reference implementation that we actually use in our project https://github.com/SWAG-MASHA/UnityWebRequestWebPatcher

 

@LeonhardP can you take a look.
Thank you.

 

Hi i'm suffering from this too , @roka could you please share some insight how you post with get as you mention at the top of this thread pls?

thanks a bunch for your help

 

@Tarrag I have updated the first post with our solution which we have essentially shipped with

 

Hi @Kumo-Kairo thanks a bunch, yes I tested decompressed and gzip and neither worked for me. I built and then applied the patch. Probably didn't work as you mention in your post cos i'm using 2019 (tested on 2019.3.0f5, 2019.3.0f3, 2019.3.9)

FYI I also raised this issue as a bug with Unity, I'll post when I hear back

 

Got here from another thread. Using C# to replace JS framework code is too complicated.
You can try this instead:

Under Assets/Plugins/WebGL create a file .jslib file (e.g. webrequest_patch.jslib) with this content:

Code (JavaScript):

1. var LibraryWebrequestSend = {
2.   JS_WebRequest_Send: function (request, ptr, length) {
3.     var td = new TextDecoder('utf-8');
4.     var sp = td.decode(HEAPU8.subarray(ptr, ptr + length));
5.     var http = wr.requestInstances[request];
6.     try {
7.         if (length > 0)
8.             http.send(sp);
9.         else
10.             http.send();
11.     } catch (e) {
12.         console.error(e.name + ': ' + e.message);
13.     }
14.   }
15. };
16. mergeInto(LibraryManager.library, LibraryWebrequestSend);

That should replace the existing JS function.

 

I made a development build and the jslib replace it perfectly on the wasm.framework.unityweb.
Well done. I do not have tested if it work but it's in the good way.