Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Cannot set Accept-Encoding header on UnityWebRequest

Discussion in 'Scripting' started by wwweh, Sep 27, 2016.

  1. wwweh

    wwweh

    Joined:
    Oct 15, 2012
    Posts:
    18
    Hi,

    I'm attempting to use gzip requests/responses with UnityWebRequest using:
    Code (CSharp):
    1. currentRequest.SetRequestHeader("Accept-Encoding", "gzip");
    which returns
    Code (CSharp):
    1. Message:Cannot set Request Header Accept-Encoding - name contains illegal characters or is not user-overridable Source:UnityEngine TargetSite: Void SetRequestHeader(System.String, System.String)
    Are there any work arounds for this problem?
     
  2. CloudKid

    CloudKid

    Joined:
    Dec 13, 2015
    Posts:
    207
    Your problem is that "Accept-Encoding" is a forbidden header name and cannot be set. I advice you to switch to another forum and ask about XMLHttpRequest to get and exact solution/workaround to this
     
    wwweh likes this.
  3. wwweh

    wwweh

    Joined:
    Oct 15, 2012
    Posts:
    18
    Interesting! I've never come across forbidden header names before. Thanks for the help
     
  4. Cripple

    Cripple

    Joined:
    Aug 16, 2012
    Posts:
    92
  5. Lars-Blaabjerg

    Lars-Blaabjerg

    Joined:
    Apr 5, 2011
    Posts:
    54
    The documentation for SetRequestHeader now states the following:

    "It is possible to set a custom value for the accept-encoding header but the resulting behavior is unreliable so it is strongly recommended to let it be automatically handled unless you can accept the risk of unexpected results."


    Does anybody now what is behind these ominous words? I tried it, and it works fine, though I do have to decompress the contents myself.
    What kind of unreliable, unexpected results could happen?
     
  6. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
  7. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    Bump, let me gzip the content :(
     
  8. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    You can, at your own risk. The messages are informational, so you know you might have issues.
     
  9. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    They are completely spamming my console. It's not informational it's distracting. Show that message once per play not every time uwr is used...

    These messages make everything worse as I may miss something really important.
     
    MadMojo likes this.
  10. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    Additionally that information is confusing.
    What does it mean? Is it set automatically to I don't know... Something? Is it doing any decompression/compression automatically? (no it doesn't checked headers with fiddler) So what's the point?

    Here's how informational these msgs are:


    Thanks Unity for treating me like a child, now I have to disable warnings forever.

    I consider switching to HttpClient. Even PlayFab support started to recommend Unity 2019 instead of 2020 bcs of it https://community.playfab.com/quest...arning-when-making-requests-on-unity-202.html.
     
    MadMojo likes this.
  11. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    It is either set to supported values or left out depending on platform. On platforms that do support compression, data will be automatically decompressed, but on on platforms without support for compression the data will be compressed.
    Finally, on WebGL, the value you set will be ignored.
    That's why the recommendation to leave it unset.
     
  12. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    Servers (and many cdn providers) won't return compressed content if there's no Accept-Encoding header. So what you said here is false. Obviously Windows supports compression but uwr doesn't do it.
    Request with header manually set:


    Automatic:
     
  13. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    Have you checked whether servers response is actually compressed and that the data you get out of UWR was properly decompressed? Compression wasn't supported on Windows in the past.
     
  14. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    Yes I guess it's not "supported" but few lines of code does it:
    Code (CSharp):
    1.  
    2.     string textResponse;
    3.     if ( string.Equals( request.GetResponseHeader( "Content-Encoding" ), "gzip", StringComparison.InvariantCultureIgnoreCase ) )
    4.     {
    5.         using ( var decompress = new GZipStream( new MemoryStream( request.downloadHandler.data ), CompressionMode.Decompress ) )
    6.         using ( var sr = new StreamReader( decompress ) )
    7.         {
    8.             textResponse = sr.ReadToEnd();
    9.         }
    10.     }
    11.     else
    12.         textResponse = request.downloadHandler.text;
    13.  
    How can this fail and give me unexpected results?
     
  15. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    On platforms that do support gzip the data you get from download handler will be decompressed already and your code will fail.
     
  16. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    Eh... Then Unity code is wrong as it should remove Content-Encoding header.

    But anyway...
    1. Then show that warning only on those platforms...
    2. My game runs on Windows only using many native calls anyway that are missing in il2cpp like Process.Start. Fun fact when using net Process.Start in editor everything is fine then it fails in the build.
    3. When it would happen i'd just disable that code from a platform
    4. Let us disable this warning
    5. There are better ways like
    Code (CSharp):
    1. byte[] bytes = www.downloadHandler.data;
    2. bool isGzip = bytes != null && bytes [0] == 31 && bytes [1] == 139;
    but that's not the point.
     
  17. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    Couldn't that message be moved to method summary? It's really annoying:

    (sometimes when I get completely unrelated bug)
     
  18. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    Did some tests to see how bad HttpClient is. Timings would be inconsistent but memory is something what can be tested:

    HttpClient:
    TotalAllocatedMemory 99,64 Megabyte
    TotalReservedMemory 273,09 Megabyte

    HttpClient with
    "Accept-Encoding", "gzip"
    :
    TotalAllocatedMemory 100,64 Megabyte
    TotalReservedMemory 274,09 Megabyte

    UnityWebRequest:
    TotalAllocatedMemory 116,39 Megabyte
    TotalReservedMemory 289,65 Megabyte

    UnityWebRequest with
    "Accept-Encoding", "gzip"
    :
    TotalAllocatedMemory 104,44 Megabyte
    TotalReservedMemory 278,09 Megabyte

    Amazing results, UnityWebRequest even with "Accept-Encoding" and spamming warnings to console/logs it allocates less memory than pure UnityWebRequest due to much smaller response size. Interesting to see how response size doesn't matter for HttpClient (yes I did test if response size was gzipped, was 800kb down to 200kb).

    Gzip tests include deserialization of the content, http client does:
    Code (CSharp):
    1. HttpClientHandler handler = new HttpClientHandler()
    2. {
    3.     AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
    4. };
    5. client = new HttpClient( handler );
    Uwr what I've sent before in this thread.

    #Edit
    One more test x100 requests:
    HttpClient
    TotalAllocatedMemory 100,70 Megabyte
    TotalReservedMemory 274,09 Megabyte

    UnityWebRequest
    TotalAllocatedMemory 185,12 Megabyte
    TotalReservedMemory 357,80 Megabyte

    OK I'm definitely switching to HttpClient.

    More in https://forum.unity.com/threads/unitywebrequest-allocates-way-more-memory-than-httpclient.964085/
     
    Last edited: Sep 4, 2020
  19. chad47

    chad47

    Joined:
    Dec 17, 2015
    Posts:
    4
    I am not familiar with this header accept-encoding topic but I am using Playfab and am getting this header accept-encoding warning that is clogging up my console. I hope Unity and Playfab can resolve this.
     
  20. alautiev

    alautiev

    Joined:
    Sep 14, 2020
    Posts:
    1
    Same problem here. Console full of playfab warning. Unity, can you simply add some way to just disable the warning? (config?)
     
  21. MadMojo

    MadMojo

    Joined:
    Oct 16, 2014
    Posts:
    19
    Same, it is quite irritating.
     
  22. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
     
  23. Lars-Blaabjerg

    Lars-Blaabjerg

    Joined:
    Apr 5, 2011
    Posts:
    54
    I agree. This warning is annoying and we should be able to silence it.
    My code handles the case where the content is already decompressed, and decompresses it when it isn't.
     
  24. UnityUser_21

    UnityUser_21

    Joined:
    Apr 18, 2019
    Posts:
    10
    A way to suppress this warning is needed. Since it gets spammed to console it makes hard to spot any other warnings and worst thing is that Debug Logs delivered with crash logs are often full of the same warning so Debug Logs that might help with the crash/exception investigation can not be seen.
     
    bearhugmo likes this.
  25. SamiKuhakoski

    SamiKuhakoski

    Joined:
    Aug 13, 2019
    Posts:
    2
    I tried

    Code (CSharp):
    1. Debug.unityLogger.logEnabled = false;
    2. webRequest.SetRequestHeader("Accept-Encoding", "gzip");
    3. Debug.unityLogger.logEnabled = true;
    But still spamming warnings.
    Seems like only way is to turn of all warnings :(
     
  26. CameronND

    CameronND

    Joined:
    Oct 2, 2018
    Posts:
    88
    Is it documented anywhere which platforms do and do not support it?
     
    dongch007 likes this.