Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

REST request to SSL server: Failed to receive data

Discussion in 'Scripting' started by Fenrirr, Mar 27, 2019.

  1. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    Hello!

    I am building a concept application that logs in my web server (build on ASP Core) with an SSL certificate.
    My code is working flawlessly on HTTP requests, but is returning a "Failed to receive data" on the "SendWebRequest" method of the ""UnityWebRequest" class, along with a "isNetworkError" being true.
    I would like to know if I am doing something wrong.

    Here is my current implementation of the REST calls:

    Code (csharp):
    1.  
    2.         UnityWebRequest request = new UnityWebRequest("https://URL:1111", "POST");
    3.         byte[] jsonToSend = new System.Text.UTF8Encoding().GetBytes("{ \"user\": \"administrator\" }");
    4.         request.uploadHandler = new UploadHandlerRaw(jsonToSend);
    5.         request.downloadHandler = new DownloadHandlerBuffer();
    6.         request.SetRequestHeader("Content-Type", "application/json");
    7.         request.certificateHandler = new CustomCertificateHandler();
    8.         request.useHttpContinue = false;
    9.  
    10.         yield return request.SendWebRequest();
    11.  
    12.      
    13.         //request.responseCode returns 0
    14.         var StatusCode = request.responseCode;
    15.  
    16.         Debug.Log("Return code: " + StatusCode);
    17.  
    18.         //request.isNetworkError returns true
    19.         if(request.isNetworkError || request.isHttpError) {
    20.  
    21.             //request.error returns "Failed to receive data"
    22.             Debug.LogError(request.error);
    23.         }
    24.  
    25.  
    And here is my implementation of the "CustomCertificateHandler" class:

    Code (csharp):
    1.  
    2.     // I removed the public key for obvious reasons
    3.     private static readonly string PUB_KEY = "PUBLIC_KEY";
    4.  
    5.     protected override bool ValidateCertificate(byte[] certificateData) {
    6.  
    7.         //After many failed attempts, I just return true to make sure the problem is not here
    8.         return true;
    9.  
    10.         X509Certificate2 certificate = new X509Certificate2(certificateData);
    11.         string pk = certificate.GetPublicKeyString();
    12.         if(pk.ToLower().Equals(PUB_KEY.ToLower())) {
    13.             return true;
    14.         }
    15.  
    16.         return false;
    17.     }
    18.  
    19.  
    I am really behind my schedule because of this problem and I will really appreciate any help!
    Thanks in advance! <3
     
  2. JeffDUnity3D

    JeffDUnity3D

    Unity Technologies

    Joined:
    May 2, 2017
    Posts:
    5,940
  3. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    Yeah, I bumped into this thread.
    The difference is that the error I receive is not "Unable to complete SSL connection".
    I receive no errors on SSL whatsoever. Only the "Failed to receive data" one. Are they related?
     
  4. JeffDUnity3D

    JeffDUnity3D

    Unity Technologies

    Joined:
    May 2, 2017
    Posts:
    5,940
    Not sure. One thing to try is to run Charles Proxy and watch the network requests. Be sure to install the root certificate and reboot after.
     
  5. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    That is weirdly dangerous.
    I just installed the Charles Proxy to run some tests and and magically the requests started working.
    On the past week none of them worked at all.
    Should I be concerned?

    EDIT:
    When running the Charles Proxy, the requests are received
    WITHOUT the software, I receive the same error as before
     
    Last edited: Mar 27, 2019
  6. JeffDUnity3D

    JeffDUnity3D

    Unity Technologies

    Joined:
    May 2, 2017
    Posts:
    5,940
    That is odd. Perhaps it has something to do with your local proxy or firewall settings, that Charles (perhaps) overrides?
     
  7. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    That should'nt be an issue. I do not use any proxy neither firewall on my testing computer.
    I will build a windows executable an try to run it on another PC
     
  8. JeffDUnity3D

    JeffDUnity3D

    Unity Technologies

    Joined:
    May 2, 2017
    Posts:
    5,940
    Agreed, and it seems like a possible bug to me. I might suggest posting your Charles findings in the other thread too
     
  9. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    I just tested on my company PC.
    The build without Charles running doesn't work, and with Charles running, it does work.

    I also did some other tests with. Aparently Charles roots all connections somehow (so it can read the packets), and, when you enable the SSL proxying on certain addresses (like mine), it uses an illegitimate certificate to re-sign the packages. (?)

    I don't know if this is directly related to the anothjer thread, but I will share it anyway
     
  10. eisenpony

    eisenpony

    Joined:
    May 8, 2015
    Posts:
    828
    I'm not sure if you can do this with Charles, but try observing the traffic without SSL decryption enabled. Does the secure tunnel handshake complete?

    You could try using Fiddler as an alternative to Charles, though I suspect the will be the same.

    These interception programs act as a proxy in order to observe your web traffic. That means there is an opportunity for the connection protocols to be different as compared to your real server.

    One theory: your game may be trying to open a TLS 1 or 1.1 channel. Your ASP Core service may only accept TLS 1.2. Charles, acting as a proxy, may be bypassing that problem by negotiating a TLS 1 connection with your client and a TLS 1.2 connection with your service and shuttling the messages across.

    Check your supported protocols in your game. What is the value of ServicePointManager.SecurityProtocol?
    Check your supported protocols on your server.
    What OS are you hosting your service on?
     
    JeffDUnity3D likes this.
  11. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    Hello @eisenpony!
    Yeah, trying Charles without observing SSL traffic indeed shows encrypted data, so, the communication is being made. And it also shows a HTTP 200 response, so my server is working properly.

    I have tried every value of the "SecurityProtocolType" with no success.
    My server is currenctly accepting TLS 1.0, 1.1 and 1.2 and running on Windows Server 2012 R2
     
  12. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    2,320
    So, with Charles proxy enabled everything works fine to you, but not when Unity calls the server directly?
    If that's the case, could you report a bug with simple project calling your server? That might help us a lot! Those intermittent issue with secure connections is quite a struggle due to difficulties in reproducing the issue, so we'd really appreciate anything that's more reliable in this area.
    Please, post a bug number here so I could put in on a fast track.
     
  13. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    Sure I can. Does the project necessarily needs my DNS or can I leave it empty?
    Also, what do you recommend for a workaroud? I am close to the project deadline and cannot afford to wait for a fix.
     
  14. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    2,320
    What do you mean by that?
    What we need is a project where the issue can be reliably reproduced. So far we can't and your particular issue is the first occurrence of such, so we need to have a look first before we can recommend a workaround.
     
  15. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    Nevermind! Haha
    I will post a soon as possible today!
     
  16. eisenpony

    eisenpony

    Joined:
    May 8, 2015
    Posts:
    828
    Fenrirr, the more accurate the repro is to your production system the better chance you have of seeing the error. The problem could very easily be entangled with your certificate or server technology stack. In the best case, you can reproduce the problem with a simple web service hosted on any platform and a basic unity implementation. However, I suspect the problem will go away as you simplify the repro.

    If the problem is related to your certificate, it might be impossible to give Unity a repro without either giving them your certificate or allowing them to use your server (what I think you called your dns). If you include your certificate, you might be giving away private information. If you offer your server, they might need your continued help to access logs etc. It is a trade off you will have to make if your certificate is involved in recreating the problem.

    If the problem is related with the server stack, you'll want to include as many details as possible regarding the version of OS and ASP.NET Core you implemented the service in.

    Of course it's possible the problem is only within Unity but I find it unlikely the Unity engineers would have had so much trouble reproducing internally if that was the case.

    As a workaround, I suggest you look into using System.Net.Http.HttpClient. I have found this the simplest way to make REST calls to a webservice.
     
  17. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    @Aurimas-Cernius
    I have just posted a bug with a project that fully reproduces my problem. It's a one-script thing. I think the bugt ticket is: 1141642_p7fv52ros56g7fc5

    @eisenpony
    I do understand the problem may be on something on my side, but since I do not own the server, it will far to impracticable for me to help :(
    Also, thanks for the tip on the "HttpClient". I'll give it a shot!
     
  18. andreasreich

    andreasreich

    Unity Technologies

    Joined:
    Sep 24, 2017
    Posts:
    49
    @Fenrirr
    Thanks for the sample project. I investigated it a bit and found out that this server immediately requests a TLS handshake renegotiation which we didn't properly support. That's curious behavior, but also an interesting case our tests didn't cover at all so far...
    I have a fix in the works and make sure that it will be backported to the 2018LTS version!
     
    wccrawford likes this.
  19. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    Thank you @andreasreich! Happy to know my sample project worked (or didnt work)!
    Is there any way for me to know when the patch is out? Or only checking the changelogs?
     
  20. andreasreich

    andreasreich

    Unity Technologies

    Joined:
    Sep 24, 2017
    Posts:
    49
    @Fenrirr I'll definitely have something in the changelog, will look after having the bugticket updated and will leave a note here.
     
  21. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    Thank you very much for all the help! Looking forward for this fix! haha
     
  22. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    Sorry for disturbing again. I am checking all the release logs for every version, but I don't know if I am searching right. What should I be looking for?
     
  23. doctorpangloss

    doctorpangloss

    Joined:
    Feb 20, 2013
    Posts:
    234
  24. Fenrirr

    Fenrirr

    Joined:
    Mar 5, 2015
    Posts:
    15
    Hello @doctorpangloss.
    Actually, they did found a problem. I'm just asking what should I search in the change log for.
     
  25. andreasreich

    andreasreich

    Unity Technologies

    Joined:
    Sep 24, 2017
    Posts:
    49
    There was some delay in the review process since people from the security team were out sick, but the fix is now in for 2019.3.0a1 and is now being backported to 2019.2, 2019.1 and 2018.3.
    Changelog will mention your bugticket (i.e. 1141642)
     
  26. StephenStrike-Interplay

    StephenStrike-Interplay

    Joined:
    Jul 22, 2017
    Posts:
    1
    Did this fix get released in a 2019.1 release? Trying to look for the bug ID in the changelogs but haven't found one with it yet? Getting a similar 200 with failed to recieve data on some requests that I am trying to work out the cause of.
     
  27. sathya

    sathya

    Joined:
    Jul 30, 2012
    Posts:
    235
    @andreasreich
    Using 2019.2.0f1.
    I get "Unable to complete SSL connection" error in the following code in few Android devices and works fine in a few

    Code (CSharp):
    1.  using (UnityWebRequest www = UnityWebRequest.Get($"https://itunes.apple.com/lookup?id={id}"))
    2.         {
    3.             yield return www.SendWebRequest();
    4.  
    5.             if (www.isNetworkError || www.isHttpError)
    6.             {
    7.                 Debug.Log(www.error);
    8.             }
    9.             else
    10.             {
    11.                 // Show results as text
    12.                 //Debug.Log(www.downloadHandler.text);
    13.             }
    14.         }
    15.     }
     
    Last edited: Aug 8, 2019
  28. bkachmar

    bkachmar

    Joined:
    Mar 15, 2013
    Posts:
    35
    In which version of Unity 2018.3 the bug is fixed?
     
  29. andreasreich

    andreasreich

    Unity Technologies

    Joined:
    Sep 24, 2017
    Posts:
    49
    @sathya Could be a network issue? I would be very surprised if cert validation for https://itunes.apple.com sometimes succeeds and sometimes not (also I think this gives a different error then)
    @bkachmar This particular one is fixed starting with 2018.4.5f1, 2019.2.0b8 and 2019.3.0a1