Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Cookie not correctly set in UnityWebRequest in 2018.3

Discussion in 'Editor & General Support' started by rasca0027, Jan 25, 2019.

  1. rasca0027

    rasca0027

    Joined:
    Dec 14, 2016
    Posts:
    1
    It seems that Unity has changed the implementation of how UnityWebRequest handles cookies, and it's not documented anywhere.

    I have a RESTful API (written in Django), which has CSRF protection. And I have an Unity app that uses UnityWebRequest to POST things to the RESTful API.

    Because of the CSRF protection, I need to pass in CSRF token for each request; the token came from response headers.

    In a normal browser, browser handles the Set-Cookie header, so you don't have to manually set it; however, it is a know bug in UnityWebRequest that it does not set the Cookie header correctly, so I had to do some weird hack to set it manually, such as following (and this code works well in 2018.2):

    Code (CSharp):
    1.  
    2. IEnumerator Login()
    3.     {
    4.         string url = baseURL + "/accounts/login/";
    5.         WWWForm form = new WWWForm();
    6.         form.AddField("username", "username");
    7.         form.AddField("password", "helloworld");
    8.  
    9.         using (UnityWebRequest www = UnityWebRequest.Post(url, form))
    10.         {
    11.             www.SetRequestHeader("X-CSRFToken", csrftoken);
    12.             www.SetRequestHeader("Cookie", string.Format("csrftoken={0}", csrftoken));
    13.          
    14.             yield return www.SendWebRequest();
    15.  
    16.             if (www.isNetworkError || www.isHttpError)
    17.             {
    18.                 Debug.Log(www.error);
    19.             }
    20.             else
    21.             {
    22.                 string cookies = www.GetResponseHeader("Set-Cookie");
    23.                 sessionID = GetCookie(cookies, "sessionid");
    24.                 csrftoken = GetCookie(cookies, "csrftoken");      
    25.             }
    26.         }
    27.     }
    28.  
    Now I'm upgrading to 2018.3. I haven't changed any code on serverside, but now I'm getting all these 403 errors saying CSRF Token not matching.

    At first it seems like Unity finally "fixed the bug": to make it behave like a normal browser and handles cookies automatically. So I tried getting rid of
    Code (csharp):
    1.  
    2. www.SetRequestHeader("X-CSRFToken", csrftoken);
    3. www.SetRequestHeader("Cookie", string.Format("csrftoken={0}", csrftoken));
    4.  
    But then the serverside either does not get cookie in request headers at all, or the request header contains outdated token.

    Does anyone know how to fix this? Huge thanks in advance.
     
    Airmouse, sonnyb and irreverentQ like this.
  2. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,637
    sonnyb likes this.
  3. polytropoi

    polytropoi

    Joined:
    Aug 16, 2006
    Posts:
    681
    @Aurimas-Cernius is there a way to turn off the automatic cookie setting globally? I made my own cookie logic and now in 2018.3 I have to put the ClearCookieCache call in every method or my stuff doesn't work. Thanks!
     
  4. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,637
    No, we currently don't have API for that.
     
  5. sonnyb

    sonnyb

    Joined:
    Nov 5, 2013
    Posts:
    16
    I also ran into this issue, and I found that it affects both
    UnityWebRequest
    and the deprecated
    UnityEngine.WWW
    .

    I was able to work around the issue with
    UnityWebRequest.ClearCookieCache
    as suggested.

    (In my case, because my Unity game connects to an Azure web app, as of 2018.3 it was also automatically holding on to the auto-generated ARRAffinity cookie from the Set-Cookie response header, which was a breaking change for me.)
     
    Last edited: Mar 25, 2019
  6. ThisIsDangerous

    ThisIsDangerous

    Joined:
    Dec 3, 2018
    Posts:
    39
    UnityWebRequest.ClearCookieCache
    workaround doesn't work for me. It is clearly seen in Fiddler that after clearing cookies the next request goes out without them (this is correct) and there are some cookies set in the response. But the next request sometimes (not always) goes out not with them, but with cookies that were set before the
    UnityWebRequest.ClearCookieCache
    , so they are old.

    In other words, clearing cookie cache seemingly works but the following cookies don't always get set to what they supposed to be set, but to the old ones that were set before clearing the cookie cache.

    Worth noting that I am experiencing this behavior in the Unity Editor (2019.1.3f1).

    Please help.
     
  7. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,637
    Can you make a small project that showcases this issue and report a bug?
    Thanks.
     
  8. ThisIsDangerous

    ThisIsDangerous

    Joined:
    Dec 3, 2018
    Posts:
    39
    Unfortunately, I cannot. Serverside is under corporate VPN.

    I'll try to describe details I found meanwhile.
    1. Cookies get deleted and set in a single server response. Both delete and set instructions are present in headers. E.g.
      Set-Cookie: tralala=deleted;
      and
      Set-Cookie: tralala=123;
      afterwards in the same header.
    2. Probably worth noting that delete instruction is
      tralala=deleted;
      , not
      tralala="";
      .
    3. If I put my code in Awake that fires just after the application starts, I cannot seem to reproduce the issue.
    4. If I put my code in Start, then in some cases the issue does reproduce but in some other cases it does not.
    5. If I put my code in a coroutine Start after the
      yield return null;
      instruction, then it seems to reproduce in every case.
    By 'cases' I mean another two projects I have here.
    Regardless of the aforementioned points, the issue does not reproduce in every application launch. I have to continually launch-stop-launch until it happens.

    Probably this will lead you to some assumptions.
    If not, well, please tell and I shall do my best to try to write some simple server to be able to showcase the issue.

    Thank you.
     
  9. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,637
    Could show the code with your requests and cookie clearing?
     
  10. ThisIsDangerous

    ThisIsDangerous

    Joined:
    Dec 3, 2018
    Posts:
    39
    Certainly. I have attached a test project.
    The flow starts from MainManager. It instructs AuthorizationDataProvider to request authorization. Here the cookie cache clearing is called. And I can see in Fiddler that there are no cookies sent indeed. The response sets the cookies. After that MainDataProvider kicks in and requests main data. This is the request that often gets sent with wrong cookies.
     

    Attached Files:

  11. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,637
    Blind guess: UnityWebRequest is also used by Unity Analytics. Can you check in Fiddler the timing between your requests and analytics? The coockie sore is shared, so concurrency between different request can possibly cause issues.
     
  12. ThisIsDangerous

    ThisIsDangerous

    Joined:
    Dec 3, 2018
    Posts:
    39
    It seems you might very well be right. Please take a look at the timings (screenshot attached). You can see two sequences marked red where both times the issue persisted. Third sequence marked green was a success without the issue. To me the pattern is clear: the tunnel request to cdp.cloud.unity3d.com has won the race against the request to game.php. The screenshot cannot show more, but this pattern is evident every time with every successful and every error request.
     

    Attached Files:

  13. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,637
    Actually, I think this is a bug. IMO, clearing should leave incomplete requests out and prevent them from storing the cookies. Could you open one and post a case number here, I'll make sure it gets processed.
    I think a request to any website that sets cookies (even google.com does, I think), also link to this message in the bug report.
    Thanks.
     
    ThisIsDangerous likes this.
  14. ThisIsDangerous

    ThisIsDangerous

    Joined:
    Dec 3, 2018
    Posts:
    39
    Case 1158841. Thanks a lot for your assistance.
     
    Aurimas-Cernius likes this.
  15. ThisIsDangerous

    ThisIsDangerous

    Joined:
    Dec 3, 2018
    Posts:
    39
    Customer QA Team has replied back to me and told me that
    That's great news I say!
     
  16. sonnyb

    sonnyb

    Joined:
    Nov 5, 2013
    Posts:
    16
    @LanVision thanks for taking the time to find & report this bug. Could you provide a link to the issue in the issue tracker? I'm having trouble searching for it by case id (1158841).
     
    ThisIsDangerous likes this.
  17. ThisIsDangerous

    ThisIsDangerous

    Joined:
    Dec 3, 2018
    Posts:
    39
    Unfortunately I am not able to find it in the issue tracker either. I wonder whether it is present there at all.
     
  18. ThisIsDangerous

    ThisIsDangerous

    Joined:
    Dec 3, 2018
    Posts:
    39
    Actually the aforementioned version 2019.3.0a9 isn't released yet, not even listed anywhere right now. Probably once it is out we shall see the issue in the issue tracker.
     
  19. ThisIsDangerous

    ThisIsDangerous

    Joined:
    Dec 3, 2018
    Posts:
    39
  20. polytropoi

    polytropoi

    Joined:
    Aug 16, 2006
    Posts:
    681
    Please consider adding a simple way to disable this functionality globally. Fine to make it enabled by default. That would let those of us who don't need or want it to avoid these issues and hacky workarounds. I understand some of the reasons for this, and I would have appreciated it a few years ago, but black-boxing the web request machinery is a bit unsettling to me.
     
    Airmouse and marck_ozz like this.