Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Bug libCurl with Playfab bug in Unity 2021.1.3f1

Discussion in 'Editor & General Support' started by Morphus74, Apr 14, 2021.

  1. Morphus74

    Morphus74

    Joined:
    Jun 12, 2018
    Posts:
    174
    2021.1.3 release note said that libCurl was update in 2021.1.3, that totally broke integration with Playfab.

    Is there a way to "use" the old libCurl?
     
  2. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    148
    I'd like to know this, too.
     
  3. Morphus74

    Morphus74

    Joined:
    Jun 12, 2018
    Posts:
    174
  4. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    148
    You can also do:
    reqContainer.RequestHeaders["X-Accept-Encoding"] = "identity" in PlayFabUnityHttp.cs
     
  5. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    We are investigating the issue.
     
    SketchWork likes this.
  6. SketchWork

    SketchWork

    Joined:
    Jul 6, 2012
    Posts:
    244
    Brilliant. Please keep us posted as this is a real problem and I don't really want to turn off compression as the workaround suggests :(

    Cheers.
     
  7. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    148
    Apparently Unity Web Request does not support compression anymore. This is not limited to Playfab, but all SDKs that use Unity Web Request. I hope it's just an oversight and it will get enabled again soon.
     
  8. Morphus74

    Morphus74

    Joined:
    Jun 12, 2018
    Posts:
    174
    I've log a bug on that one, and Unity answer that they don't deal with issue related to 3rd party package... so let see what happen
     
  9. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    148
    Mine is still open and about UnityWebRequest. Let's see what happens.
     
  10. thmsn

    thmsn

    Joined:
    May 29, 2019
    Posts:
    6
    I also ran into this issue and downgraded to Unity 2021.1.2f1 as a "fix". It was with an ordinary UnityWebRequest.Get to a server that responds with a gzipped response. e.g. I am not using playfab.
     
  11. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    Are you setting Accept-Encoding header manually?
     
  12. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    148
    Unity says it's a third party problem and they won't do anything about it. Third party says it's a Unity problem and they can't do anything about it. Great.

    I'm pretty sure Unity Web Request not supporting compression anymore is not the fault of a third party.


    I'm not him, but I do set it manually. Works with 2021.1.2 (and every version down to 2018), doesn't work with 2021.1.3 and above (new libCurl).
     
  13. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    Actually, Unity never supported compression on Windows. Playfab had their own decompressor on top of UnityWebRequest that detected whether downloaded data is compressed or not and decompressed if necessary.
    Now, after we updated the version of libcurl that we use, it has a built-in safeguard that prevents you from requesting compression without built-in support.
     
  14. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    148
    Interesting. Thank you for the detailed response.

    "built-in safeguard that prevents you from requesting compression without built-in support"
    I have a hard time understanding that part. libCurl has a built-in safeguard, but is also supposed to provide built-in support (which it doesn't have)?
     
  15. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    That a thing we intend to fix. But, unfortunately, the built-in support is not enabled on Windows.
     
  16. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    148
    I'm using it on mobile (Android). From the looks of it compression is skipped on Windows anyway.
    Sounds like it really is a bug then.
     
    etopian likes this.
  17. etopian

    etopian

    Joined:
    Dec 11, 2018
    Posts:
    8
    +1 Would like to see this resolved.

    If this is an issue at the curl level, it may be appropriate to roll back the version until the built-in support can be enabled on Windows (assuming this was a routine library update vs being attached to new Unity features).
     
  18. SketchWork

    SketchWork

    Joined:
    Jul 6, 2012
    Posts:
    244
    +1 from me too. This is a real pain :(
     
  19. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    148
    Answer from Unity: "We came to the conclusion that this is still a 3rd party package issue.
    It's not considered a bug."
    They won't do anything about it.
     
  20. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    We intend to add gzip support for Windows.
    But the current issue is indeed third party relying on being able to set a particular HTTP header we explicitly advise against in our documentation (and were forbidding entirely in the past).
     
  21. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    148
    I'm using Android/iOS, not Windows.
    But seems like it's more about forcing the header. Well, let's see what third party SDKs do about it. Without compression it will get expensive.

    Edit: Yes, the documentation strongly recommends not doing what the third party did anyway.
     
    Last edited: May 18, 2021
  22. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    Wait, request gets reject on Android and iOS too? Those two platforms should support gzip, only on Windows the request should be rejected due to missing support for gzip, other too only give the warning about the header that is discouraged.
     
  23. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    148
    From what I can tell is that the editor (with platform Android) does produce the error with the new libCurl version.
    The editor (with platform Android) does not produce the error with the old libCurl version.
    Not sure if the editor is supposed to count as "Windows" or "Android" in this case. If it's supposed to count as "Android" regarding libCurl functionality, then it's a bug. If it should have "Windows" functionality, then not.
    Not dramatic in my case, since I'm not interested in "Windows".


    Anyway, when making an APK build the behaviour is different. I get no error, regardless of the Unity or libCurl version used.
    However it seems that compression is never used on the server. Even when using the old libCurl. The response bytes are wrong and no decompression happens (because it's not compressed). Compression does work in the editor (old libCurl only of course).
    This is not a problem for Unity (I think), but clearly an issue of the third party.


    Confusing :D
    I'll investigate a bit futher, to rule out any mistakes on my side.


    give the warning about the header that is discouraged
    Btw, there are no warnings at any time.
     
  24. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    By Windows I meant anything that runs on Windows: Windows version of Unity Editor, Windows Standalone, UWP.
    The active platform in Editor is irrelevant here.
    So what you describe makes sense, we intend to add gzip support for Windows, but for now it's not there.
     
  25. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    148
    Seems like the third party in question just needs to extend
    #if !UNITY_WSA && !UNITY_WP8 && !UNITY_WEBGL

    to
    #if !UNITY_WSA && !UNITY_WP8 && !UNITY_WEBGL && !UNITY_EDITOR

    and all is fine then.
     
  26. thmsn

    thmsn

    Joined:
    May 29, 2019
    Posts:
    6
    All I'm doing is the following
    Code (CSharp):
    1.  
    2. var www = UnityWebRequest.Get(url);
    3. yield return www.SendWebRequest();
    4.  
    and then I decompress the response afterwards with a GZipStream, it works as expected in 2021.1.2f1 and started to fail with the libCurl error in the newer versions. I'm using Windows 10.
     
    Last edited: May 20, 2021
  27. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    Yes, that's another side of this issue. In this case you technically have the server that is disregarding clients accepted encoding and sending gzip. You should make server not send Content-Encoding header set to gzip, you can still send the data as gzip and always decompress it. Note, that if you take this approach, you have to manually decompress everywhere, not just on Windows.
     
  28. thmsn

    thmsn

    Joined:
    May 29, 2019
    Posts:
    6
    I have no control over the server I'm talking with sadly, and it seems to always respond with gzip, I did try initially to set the Accept-Encoding header to gzip, but I would never get a response, just the libcurl error about identity.

    Are you saying it should work if I set Accept-Encoding to gzip, and continue to manually decompress it, or is unity/libcurl outright denying gzipped responses on windows?, even if I'm handling them manually?
     
    Last edited: May 21, 2021
  29. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    It's libcurl reject them from either side if support hasn't been enabled.
    I've got some successful results, so hopefully we will have an update soon-ish with gzip support on Windows.
     
  30. TCROC

    TCROC

    Joined:
    Aug 15, 2015
    Posts:
    150
    I am also having this issue on Unity 2020.3.9f1 LTS
     
  31. jan-rb

    jan-rb

    Joined:
    Oct 26, 2020
    Posts:
    9
    Same Problem here on Unity 2020.3.9
     
  32. TheSheyper

    TheSheyper

    Joined:
    Jul 3, 2013
    Posts:
    19
    Hi !
    Same pb for me, I'm using playfab and Unity 2021.1.5f1.
    I can turn off compression but that really not ideal, so I'm downgrading to 2021.1.2f1.
    @Aurimas-Cernius any ETA ? I mean is this planned for 2021 ?
    Thx a lot !
     
  33. thatnzguy

    thatnzguy

    Joined:
    Jun 8, 2015
    Posts:
    18
    On Unity 2020.3.9f1 I have the same libcurl failure with the mapbox plugin, when requesting vector tiles.
    After rolling back to 2020.3.8f1 the issue is gone.
     
  34. jan-rb

    jan-rb

    Joined:
    Oct 26, 2020
    Posts:
    9
    Same problem for us with Mapbox and Unity 2020.3.9
     
    JorgeLogan and Version10 like this.
  35. doseon

    doseon

    Joined:
    Mar 17, 2021
    Posts:
    36
    Same problem Unity 2020.3.9 or later.
     
  36. Version10

    Version10

    Joined:
    Oct 3, 2016
    Posts:
    1
    Same problem with Unity 2020.3.12... :(
     
  37. jan-rb

    jan-rb

    Joined:
    Oct 26, 2020
    Posts:
    9
    Any news on this?
    It's still not possible in Unity 2020.3.12 to get gzipped content from a Server on Windows.
    Please fix this. It's a real show stopper.
     
  38. RaventurnStefan

    RaventurnStefan

    Joined:
    Aug 1, 2012
    Posts:
    29
    I am also having this issue on Unity 2020.3.12f1 LTS :(
     
  39. doseon

    doseon

    Joined:
    Mar 17, 2021
    Posts:
    36
    I've same issue on Unity 2020.3.9f1 or higher versions.

    Today. In Unity2020.3.13f1 I modified the code as below just in case, but it worked.

    Code (CSharp):
    1. request.SetRequestHeader("accept-encoding", "gzip");
    to
    Code (CSharp):
    1. request.SetRequestHeader("content-encoding", "gzip");
    The reason is unknown. But my case has been solved, so I'm sharing it.
     
    JorgeLogan likes this.
  40. doseon

    doseon

    Joined:
    Mar 17, 2021
    Posts:
    36
    It was a mistake. There are some parts that don't work properly.
     
  41. SanctusSusanin

    SanctusSusanin

    Joined:
    Feb 9, 2019
    Posts:
    1
    Bump
     
  42. doseon

    doseon

    Joined:
    Mar 17, 2021
    Posts:
    36
    Setting Accept-Encoding gzip in the request header doesn't seem to be a problem.
    However, if you set Content-Encoding in the response header on the server, curl error 61 occurs. In my case, using a different word instead of "Content-Encoding:" worked fine.
    For example, curl error 61 did not occur when "blabla: gzip" was sent instead of "Content-Encoding: gzip".
     
  43. tmars

    tmars

    Joined:
    Jul 26, 2013
    Posts:
    35
    @Aurimas-Cernius FYI: the recent libcurl update in 2020 LTS has broken connection reuse on android as well - each new request opens a new connection instead of reusing the previous one. We too had to revert back to 2020.3.8.
     
    Last edited: Jul 2, 2021
  44. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    Are you using a custom certificate handler? After that update we do indeed use separate connection when certificate handler is set, but they should be reused when handler is absent.
     
  45. tmars

    tmars

    Joined:
    Jul 26, 2013
    Posts:
    35
    @Aurimas-Cernius Hello!
    Thanks for pointing this out, unfortunately removing the custom handler didn't help. Furthermore, switching back to plain HTTP didn't help either - each new request opens a new connection. Tested on 2020.3.13 LTS (while 2020.3.8 LTS is fine in both cases).
     
  46. jan-rb

    jan-rb

    Joined:
    Oct 26, 2020
    Posts:
    9
    @Aurimas-Cernius Any news on this?
    This bug is not resolved for more than 4 month on the latest stable LTS version of Unity.

    This is a show stopper for every game based on thirdparty web services where developers don't have control over the headers send by the service.
    Please take this bug serious.
    Thank you.
     
    Last edited: Jul 27, 2021
  47. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    The support for gzip on Windows is being rolled out ATM.
     
    RatherGood likes this.
  48. RatherGood

    RatherGood

    Joined:
    Oct 12, 2014
    Posts:
    21
    Does the 2021.2.0b4 beta version have the fix? Otherwise, I will downgrade to old 2020.2 version if I understand where this bug was introduced? My project using Mapbox is at a standstill at the moment. Thank you.
     
  49. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,511
    The 2021.2 is still in progress.
     
    RatherGood likes this.
  50. jan-rb

    jan-rb

    Joined:
    Oct 26, 2020
    Posts:
    9
    The latest version 2020.3.15 doesn't say anything in the change log about a gzip fix.
    Is this fixed in 2020.3.15 or is it postponed again?