Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Curl error 51: Cert verify failed: UNITYTLS_X509VERIFY_FLAG_EXPIRED

Discussion in 'Editor & General Support' started by DarkDeivel, Oct 1, 2021.

  1. DarkDeivel

    DarkDeivel

    Joined:
    Aug 31, 2016
    Posts:
    127
    Hello everyone! Today, some Android users cannot play my online game. They write connection error messages. I have checked the logs and this error is - Curl error 51: Cert verify failed: UNITYTLS_X509VERIFY_FLAG_EXPIRED

    What to do now? Some of the players continue to play and do not know the problem, and some have now encountered this error.

    P.S. From a pc I do not have such a problem, from a build on android I get this problem and cannot connect to the server.

    I have UNITY 2019 4.28f1
     
  2. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    Strange. Does PC and android do requests to the same URL?
    Can you check the entire certificate chain in your browser?
     
  3. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    We have the same problem. Using Unity 2019.4.20f1, Addressables 1.18.13 and Unity Cloud Content Delivery.

    The problem started between Sep 30th and Oct 1st for us. It seems to be related to the Android version installed on the device. It seems Android devices with Android 7.0 trigger this issue, while devices running Android 9 and newer don't. I don't know about Android 8.

    It looks like the error is triggered when Addressables is initializing and trying to access Unity Cloud Content Delivery. Every Addressables request then results in the
    Curl error 51: Cert verify failed: UNITYTLS_X509VERIFY_FLAG_EXPIRED
    error.
    Code (CSharp):
    1. Curl error 51: Cert verify failed: UNITYTLS_X509VERIFY_FLAG_EXPIRED
    2.  
    3. (Filename: ./Modules/UnityWebRequest/Implementations/TransportCurl.cpp Line: 813)
    4.  
    5. Web request failed, retrying (2/3)...
    6. Unknown Error
    7. url : https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.client-api.unity3dusercontent.com/client_api/v1/buckets/yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/release_by_badge/latest/entry_by_path/content/?path=Android/r-audio-game-gui_assets_all_2ab2edc2ed97d26a0d042a1842245c1c.bundle
     
    DarkDeivel likes this.
  4. DarkDeivel

    DarkDeivel

    Joined:
    Aug 31, 2016
    Posts:
    127
    The URL is the same. Peter77 wrote correctly, I have the same problem with him.


    Hello! You are right, apparently that is why some of the android devices do not have an error, and some, like mine, have an error! I have android 7.0 and just this error occurs.
     
  5. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    This error indicates an expired certificate. Possibly newer Androids have updated certificate in their stores, so everything works, while older ones have expired certificate in there, so you get this error, which is actually perfectly valid.
     
  6. DarkDeivel

    DarkDeivel

    Joined:
    Aug 31, 2016
    Posts:
    127
    [QUOTE = "Aurimas-Cernius, post: 7538054, member: 365873"] This error indicates an expired certificate. Possibly newer Androids have updated certificate in their stores, so everything works, while older ones have expired certificate in there, so you get this error, which is actually perfectly valid. [/ QUOTE]
    So, what to do now?
     
  7. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    Peter77 likes this.
  8. amorimlukas

    amorimlukas

    Joined:
    Nov 12, 2019
    Posts:
    11
    We started receiving this error yesterday, users reported the connection error message earlier and yesterday our own devices started to show the message too. We have our app connect to our server, but in the Unity Editor and some newer versions of Android is working fine. In our devices the message was showed with Android versions 4.4.4, 7.0 and 6.0.1.
     
    DarkDeivel likes this.
  9. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    Can you perform a Charles Proxy capture? This will tell us which endpoint is returning the error. Even an encrypted capture would be helpful, but try to obtain an unencrypted capture https://support.unity.com/hc/en-us/articles/115002917683-Using-Charles-Proxy-with-Unity
     
  10. amorimlukas

    amorimlukas

    Joined:
    Nov 12, 2019
    Posts:
    11
    I've performed the Charles Proxy capture, it was the first time I've done it, so I'm not sure it's right or what to look at, I've generated a file of the session if you want to take a look.
     
    Last edited: Oct 1, 2021
  11. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    Thank you for the capture. Is this from an Android device? Did you see the error on your device during the capture? I'm not seeing any issues in the capture, although it is still encrypted. We might need an unencrypted capture, ensure to install the local root certificate as mentioned in the article and reboot. For Android, you'll need to add charlesproxy.unitypackage and create a new build. You can remove the package prior to release.
     
  12. amorimlukas

    amorimlukas

    Joined:
    Nov 12, 2019
    Posts:
    11
    Yes, the capture was from an Android Device, and the error appeared too. I've installed the Unity package and created a new build, but I don't see any differences on the capture.
    https://we.tl/t-ugUcY68MHM
    The first file is the same I've send you before, the others are with the new build. Both times the connection error appeared on the apk.
     
  13. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    Got it, thanks. The only error I see is with this one endpoint (myappswizard.com)

    CaptureError.png
     
  14. amorimlukas

    amorimlukas

    Joined:
    Nov 12, 2019
    Posts:
    11
    Got it. Any clue or ideas of what could be causing this error then? Since this endpoint doesn't seem to be the reason behind it, this app doesn't exist on other devices I have here with the same connection error appearing.
     
  15. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    Sorry I don't quite follow, I thought this was an error you were seeing coming from your app? Why do you think it isn't this endpoint, is your game calling it? How is the error appearing on other devices (what triggers it and how are you viewing the error). One note, you need to turn off local capture in Charles, we are only looking for network calls that you see when you launch your app on the device.
     
    Last edited: Oct 1, 2021
  16. amorimlukas

    amorimlukas

    Joined:
    Nov 12, 2019
    Posts:
    11
    The Curl error 51: Cert verify failed: UNITYTLS_X509VERIFY_FLAG_EXPIRE is appearing right after the game is launched. When tha game is open, there is the company splash logo, then an loading scene and then a menu where the user connects with his login. All the login process is validated through our site and online database, but before all that even starts the game fails to connect showing a message of connection error and when I debugged with adb the above error was displayed. We receive messages from different players reporting this issue before it appears on our tests devices. Then this morning I tested on a few devices and the Unity Editor, and the error only occured on the Android Devices with version below 8.0, all devices with 8.0 and above and the Unity Editor the game worked perfectly.
     

    Attached Files:

    DarkDeivel likes this.
  17. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    If you note in my message, that website is indeed returning Unknown Error, same as your screenshot. It looks like you are making a UnityWebRequest, are you calling that website or one of their services from your game? Their SSL certificate may be expired.
     
  18. amorimlukas

    amorimlukas

    Joined:
    Nov 12, 2019
    Posts:
    11
    No, we don't use that website, that is from an app that's instaled by default on the device, and the others devices don't have that app. I don't know why that's being called.

    The link below is a test using the same build on other device. It has the same error on the app screen.
     
    Last edited: Oct 4, 2021
  19. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    Please look for any endpoints that only your app is sending to. You should see them quickly appear in Charles as soon as you launch your app on the device. Are you using UnityWebRequest? How is the error appearing on the app screen? I thought it was in the logcat logs only. Are you writing errors to the UI? (which is a good debugging technique)
     
  20. amorimlukas

    amorimlukas

    Joined:
    Nov 12, 2019
    Posts:
    11
    Yes, I'm using UnityWebRequest, and if the request fails it shows some messages on the game UI depending the type of the error. The error showing on the UI is the connection error, that shows when the request fails because of network error. After seeing the UI error I've done the adb debug and got the topic error on logcat.
     
  21. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    You will want to check the SSL certificates for those endpoints that you are calling via UnityWebRequest
     
  22. DarkDeivel

    DarkDeivel

    Joined:
    Aug 31, 2016
    Posts:
    127
    So. If you use links http:// instead of https:// then requests go through.

    Tell! How can I determine the version of android via c#? Because I want to make a split, for those below 8.0 use http:// and those above https://

    But you need to somehow detect the android version.
     
    Last edited: Oct 2, 2021
  23. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    Because the problems are with establishing secure connection. Does your server use the certificate issued by CA I linked above?
    The most straight forward solution is to attach certificate handler to UnityWebRequest and manually verify the certificate. If devices store has an expired root certificate, there's not much you can do about it.
     
  24. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,589
    The problem vanished for me automagically after around a day it first occurred.
     
  25. DarekRusin

    DarekRusin

    Joined:
    Nov 15, 2013
    Posts:
    47
    @JeffDUnity3D We're having exactly the same issue. It looks like around 8% of players started having problems connecting to our server right after September 30. Random googling around issue led me to this thing that looks related:
    https://www.tomsguide.com/news/internet-disconnect-sept-30

    I can't find fault in the certificates on our backend server. It's managed via Let's Encrypt was issued on September 26 and is valid until December 25.

    Fortunately I was able to find an old Amazon Fire device and the game is indeed having problems there. The browser is working fine, but our game is not connecting to our backend. Checking with logcat I get this:

    E/Unity (14249): Curl error 51: Cert verify failed: UNITYTLS_X509VERIFY_FLAG_EXPIRED
    E/Unity (14249):
    E/Unity (14249): (Filename: Line: 813)

    This happens on each UnityWebRequest call.

    What can we do about this yet? Is this the problem:
    - with the device? This says it shouldn't be: https://www.tomsguide.com/news/android-cert-mess-averted
    - with our certificate? It seems valid and I can connect with it even on that Amazon Fire device when using it's web browser (Silk)
    - with Unity? It seems so, since it's the only thing having problems accessing the server

    How can we fix or work around this? Thanks for any help!
     
  26. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    I checked here, and unfortunately there are no additional suggestions https://forum.unity.com/threads/cur...x509verify_flag_expired.1176929/#post-7544086
     
  27. amorimlukas

    amorimlukas

    Joined:
    Nov 12, 2019
    Posts:
    11
    We changed our certificate to a new payed one and then the debug gives me the Cert verify failed: UNITYTLS_X509VERIFY_FLAG_CN_MISMATCH error when it tries do connect with our servers.
     
  28. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    What version of Android? Can you browse to your server via a browser on the device? (assuming you have an http/https GET endpoint)
     
  29. amorimlukas

    amorimlukas

    Joined:
    Nov 12, 2019
    Posts:
    11
    On all the Android devices we tested, with different versions doesn't work. And yes, when we go to the server through the device browser we can access it, but when is the game running it always ends up with the error.
    When we change the server to a different certificate is there anything that need to be configurated on Unity for it to work? Cause everything seems right on the new certificate, but with this new one the game doesn't work on the android devices.
     
  30. JeffDUnity3D

    JeffDUnity3D

    Joined:
    May 2, 2017
    Posts:
    14,446
    What Android versions did not work in your testing?
     
  31. amorimlukas

    amorimlukas

    Joined:
    Nov 12, 2019
    Posts:
    11
    Android 4.4, 7.0, 11, all devices with android when we use the new certificate we payed for the game doesn't connect.
     
  32. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    Possibly web browsers have their own certificate store (because they know that the store on device can be hopelessly out of date?).
    Have you tried installing server root certificate on device?

    The best option is to attach certificate handler to UWR and manually validate certificate.
     
  33. jnbt

    jnbt

    Joined:
    Jul 8, 2013
    Posts:
    11
    Hi everyone,

    we also expirenced this problem (" Curl error 51: Cert verify failed: UNITYTLS_X509VERIFY_FLAG_EXPIRED") on multiple projects we currently have. We were able to solve it in our scenario, and I believe that also others are effected by the same problem.

    Background

    The root cause of this error is that we used LetsEncrypt SSL certificates for our servers. Over the last year LetsEncrypt needed to switch to a new trust anchor (root certificate), because the formerly used DST Root CA X3 was going to expire (on end of September 2021, this will be important later). They switched to a quite new (their own) trust anchor (root certificate).
    Because most LetsEncrypt certificates are renewed every 90 days, this was performed automatically for us.
    More information about this can be found here: https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/

    The new root certificate is their own ISRG Root X1, but this certificate is not available on older devices (especially Android devices which don't get any OS updates anymore). This is a known issue for all new root certificates.
    But the smart people behind LetsEncrypt found a specific solution for Android devices:
    They include a seperate trust chain in the generate certificate, which includes a cross-signing of the ISRG Root X1 with DST Root CA X3. They knew that the DST Root CA X3 would expire on end of September 2021, but Android is a specific situation, because when Android was developed, it was decided to not check the expiry of trust anchors (root certificates). You can find more information about this trick here: https://letsencrypt.org/2020/12/21/extending-android-compatibility.html

    Because of this trick most websites, browsers and apps were not affected by this change, otherwise a lot of providers would have problems currently serving data to users using old Android devices (all Android versions < 7.1.1).

    For our games in Unity this trick also worked the whole year until end of September 2021.

    Problem

    We're pretty sure that Unity on Android doesn't use the Android's native functions to validate the SSL certificates when performing HTTPS calls via UnityWebRequest. Or it performs some additional / different checks. Also the errors "UNITYTLS_X509VERIFY_*" indicate that this might be the case.

    While other apps still work against servers using the LetsEncrypt certificate (we did a test with a native Android app), all apps build with Unity (we only tested Unity 2020.x) don't work.

    Technically I think the behaivor is correct, because it is up to the SSL's implementation to decide if to check for a trust anchors.

    Solution

    As far as we see there isn't a working solution with using LetsEncrypt-based certificates in combination with UnityWebRequest on older Android devices (Android OS < 7.1.1). (Maybe it works with newer Unity versions, we're always very conservative with upgrading Unity because of multiple bad expierences we had in the past. I'm looking forward for feedback).

    For us working was/is: We created a new non-LetsEncrypt certificate, which in our case is based on the USERTrust RSA Certification Authority. It is important to use a CA which has it's root certificate already installed on older Android devices.

    (In our scenario we also needed to re-use the existing private/public key pair, because we perform SSL certificate pinning, and just exchanging the certificate would have created different problems. But I'm pretty sure this is a very specific problem)
     
    crekri, amjaliks, LZbigfish and 2 others like this.
  34. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    All Unity versions pull root certificates from on device certificate store and validate against them.
     
  35. jnbt

    jnbt

    Joined:
    Jul 8, 2013
    Posts:
    11
    And this makes totally sense (The real problem is that the underlying OS doesn't get updates anymore!).
    Our doubt is that while the device certificate store is used the actual validation is done with a custom (or integrated) solution which doesn't have the specific Android behavior to ignore the expiry of root certificates ("Android has intentionally chosen not to use the notAfter field of trust anchors."). Therefor the trick LetsEncrypt applied doesn't work for our Unity-based games. (Also other projects have this problem, e.g. specific versions of OpenSSL don't accept the "new" LetsEncrypt certificates)
     
  36. DarekRusin

    DarekRusin

    Joined:
    Nov 15, 2013
    Posts:
    47
    @jnbt thanks for the excellent summary! It fits with our findings. We have this problem happening on older Amazon Fire device and web browser there works fine on our site, but when accessing it from within Unity (same HTTPS GET request) it fails with "SSL CA certificate error".

    What's more, I can connect to https://www.google.com from within our Unity game but can't do that to our server. So it would fit the theory that Let's Encrypt https certificates are the problem here.

    For now we've simply given up and for Android versions < 8.0 we fall back to unencrypted API calls. Hopefully this will be enough for now.

    But I've also found this article https://www.octopuce.fr/letsencrypt-certificate-expiration-consequences/ which says this thing about cURL, which I think Unity uses for connections:

    I'm not sure what to make of it though.
     
  37. amorimlukas

    amorimlukas

    Joined:
    Nov 12, 2019
    Posts:
    11
    Ok, after a lot of tests and changes we finally made it work. In the end our new payed certificate was working normally, but who made the certificate for us didn't send us one of the files of the certificate that was needed for the devices to be able to access the server. Now all the android devices are working as they should.
    Thanks to everyone on the Unity Team for the help while trying to solve this.
     
  38. jerome-lacoste

    jerome-lacoste

    Joined:
    Jan 7, 2012
    Posts:
    206
    Thanks @jnbt for the good write up. We had the same issue and also ended up using a new paid certificate.

    I understand that Unity's job to be cross platform might be made easier by sticking to their own SSL implementation, but in some way it creates unexpected experiences for users. "It works in the other [non-unity] apps ". One may argue it goes against the principle of least surprise. In some way Unity has become its own platform in that regard, and maybe should be proactive to ensure it supports environmental changes.

    As for using LTS, we're doing the same. We don't even upgrade unless forced, as we've found that there's too much instability in minor releases. And now we are super careful with features introduced in the current product cycle as we hit a disastrous bug last year.
     
    DarekRusin likes this.
  39. DarekRusin

    DarekRusin

    Joined:
    Nov 15, 2013
    Posts:
    47
    One other thing that worked for us is routing the traffic through CloudFlare and using their free SSL certificate. It's a relatively quick and easy option, if you are ok with passing everything through CloudFlare.

    And I agree about Unity's implementation. We've had exactly the same complaints from our users: "Everything works on my device except for this one [Unity] game". Not the first issue and not the worst though, so I'm kind of used to it ;)

    I'm happy to have it solved finally, and it looks like there are several solid workarounds.
     
  40. EdwinLyons

    EdwinLyons

    Joined:
    Oct 4, 2012
    Posts:
    84
    amjaliks likes this.
  41. amjaliks

    amjaliks

    Joined:
    Jul 11, 2015
    Posts:
    159
    Yes, you can create a workaround with custom subclass of
    CertificateHandler
    .

    The simplest implementation can accept any certificate. And here is my attempt to have some logic to check, if certificate is signed by Let's Encrypt.

    Code (CSharp):
    1.     private class LetsEncryptCertificateHandler : CertificateHandler {
    2.  
    3.         private readonly string host;
    4.  
    5.         public LetsEncryptCertificateHandler(string host) {
    6.             this.host = host;
    7.         }
    8.  
    9.         protected override bool ValidateCertificate(byte[] certificateData) {
    10.             var certificate = new X509Certificate2(certificateData);
    11.          
    12.             if (host != certificate.GetNameInfo(X509NameType.SimpleName, false)) return false;
    13.             if (certificate.NotBefore > DateTime.Now) return false;
    14.             if (certificate.NotAfter < DateTime.Now) return false;
    15.          
    16.             var chain = new X509Chain();
    17.          
    18.             var assets = Resources.LoadAll<TextAsset>("Let's Encrypt/");
    19.             foreach (var asset in assets)chain.ChainPolicy.ExtraStore.Add(new X509Certificate2(asset.bytes));
    20.          
    21.             chain.Build(certificate);
    22.             var valid = chain.ChainElements.Count == 3;
    23.          
    24.             chain.Dispose();
    25.  
    26.             return valid;
    27.         }
    28.     }
    29.  
    Resource folder has LE trust certificates in it.
     
  42. unity_NfjpRNVq3epQ_g

    unity_NfjpRNVq3epQ_g

    Joined:
    Jul 27, 2020
    Posts:
    1
    I've read a lot about this topic these days, because some of my players are affected aswell. Even when I overwrite the ValidateCertificate function it does not work.
    Error message:
    AndroidPlayer(ADB@127.0.0.1:34999) Curl error 7: Failed to connect to servername.net port xxxx: Connection refused

    It works on all other devices >Android 7.1

    Even when using
    Code (CSharp):
    1.  
    2. var cert = new ForceAcceptAll();
    3. UnityWebRequest www = UnityWebRequest.Get(fullurl);
    4. Debug.Log("Setting certificateHandler");
    5. www.certificateHandler = cert;
    6. yield return www.SendWebRequest();
    7. cert?.Dispose();
    8.  
    Code (CSharp):
    1.  
    2. public class ForceAcceptAll : CertificateHandler
    3. {
    4.     protected override bool ValidateCertificate(byte[] certificateData)
    5.     {
    6.         Debug.Log("VALIDATION CERTIFICATE!! "+certificateData);
    7.         return true;
    8.     }
    9. }
    It's the same result. Also did not work when using C# Lib HttpClient instead of UnityWebRequest.
     
  43. Danny9421

    Danny9421

    Joined:
    Mar 21, 2022
    Posts:
    36
    Have you found a solution so far ? I have the issues with android 6-7 devices.
     
  44. Barry100

    Barry100

    Joined:
    Nov 12, 2014
    Posts:
    200
    hi - who did you get your paid certificate from? I have this issue as well.