Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

Unable to refresh package manager list since early November (China specific?)

Discussion in '2018.3 Beta' started by bitinn, Nov 17, 2018.

  1. bitinn

    bitinn

    Joined:
    Aug 20, 2016
    Posts:
    552
    Screen Shot 2018-11-17 at 15.06.20.png

    It takes over 10 minutes to refresh just the In Project packages, the All Packages list was not able to load in even when I waited for half an hour, it's stuck at "Loading packages" (bottom left corner). This was not the case for the past a few weeks.

    I have 3 questions:

    - Does package manager respect system proxy settings like Unity core / Asset Store?

    - I am on 2018.3.0b10 with Package Manager UI (2.0.3), are there any updates that remedy this problem?

    - I am based in China so it could also be a China problem, but this wasn't the case last week, what's the domain being used for package manager repo?

    Thx.
     
  2. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    144
    Hi @bitinn!

    Here are the answers to the above questions. Please let us know if this doesn't solve your problem.

    1) The package manager should indeed use the same proxy settings, but to be sure, you can configure the proxy to use by setting the HTTP_PROXY and HTTPS_PROXY environment variables before launching Unity.

    The following domain names need to be accessible (whitelisted in the proxy/firewall rules, if required) through HTTPS for Unity Package Manager to work:
    There have been issues reported with using a self-signed certificate for the proxy; these certificates are considered insecure and will be rejected by the underlying HTTP layer used by the package manager.

    2) This issue should not be related to the specific version of Package Manager UI.

    3) I'm not aware of specific issues related to China.
     
    LeonhardP likes this.
  3. bitinn

    bitinn

    Joined:
    Aug 20, 2016
    Posts:
    552
    - My proxy is set via System Preference on macOS, does Unity apply this setting to package manager?

    - I usually launch Unity beta from Hub, please advice how can I launch it via Hub and set environment variable?

    - Am I right to believe dl.bintray.com is un-ping-able? I tried my local network and some remote servers, no ping were returned.

    - My local network ping (using China Telecom) to packages.unity.com are really bad, I can't speak for others but something to consider.

    Code (CSharp):
    1. $ ping packages.unity.com
    2.  
    3. PING bintray.com (119.81.184.206): 56 data bytes
    4. Request timeout for icmp_seq 0
    5. 64 bytes from 119.81.184.206: icmp_seq=1 ttl=44 time=161.055 ms
    6. Request timeout for icmp_seq 2
    7. Request timeout for icmp_seq 3
    8. Request timeout for icmp_seq 4
    9. Request timeout for icmp_seq 5
    10. Request timeout for icmp_seq 6
    11. 64 bytes from 119.81.184.206: icmp_seq=7 ttl=44 time=163.906 ms
    12. Request timeout for icmp_seq 8
    13. Request timeout for icmp_seq 9
    14. 64 bytes from 119.81.184.206: icmp_seq=10 ttl=44 time=160.975 ms
    15. Request timeout for icmp_seq 11
    16. Request timeout for icmp_seq 12
    17. 64 bytes from 119.81.184.206: icmp_seq=13 ttl=44 time=160.206 ms
    18. 64 bytes from 119.81.184.206: icmp_seq=14 ttl=44 time=162.467 ms
     
  4. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    144
    Hi @bitinn,

    Unfortunately, system proxy settings are not yet supported.

    Setting persistent environment variables on MacOS is not trivial (as it depends on the OS version), and that approach is not very good practice in the first place as you never know what other program/app would be affected by setting such a variable. Therefore the best advice I can provide for now would be to create a small shell script to launch the Hub with these environment variables. Afterwards, they will automatically be passed down to the Unity instances, and the Unity Package Manager processes launched in turn.

    To create that script, open a Terminal window and copy/paste the following snippet after replacing "proxy-url" with your own proxy info:

    Code (csharp):
    1. echo '#!/bin/bash
    2. export HTTP_PROXY=proxy-url
    3. export HTTPS_PROXY=proxy-url
    4. nohup "/Applications/Unity Hub.app/Contents/MacOS/Unity Hub" &>/dev/null &
    5. ' > launchHub.sh
    6. chmod +x launchHub.sh
    You can then move that file to a more convenient location (e.g. your Desktop) and double-click it to launch the Hub with the proper environment variables set. Alternatively, you can execute it from a terminal with the same effect.

    A few caveats:
    - If the Hub is already running, the script will just bring it into focus without relaunching it, so it will not update the proxy settings if you change them. You need to close the Hub first.
    - If you double-click the script, it will open a Terminal window and leave it open. This can be changed in the Terminal.app preferences.
    - Feel free to edit the script to remove either HTTP_PROXY or HTTPS_PROXY, or change the path to the Hub installation. Spaces in the path require you to leave the double-quotes around the path for the script to work.

    Regarding your last two questions about ping reachability and performance from China:
    - You're right that the Bintray servers do not respond to ping.
    - The main URL `packages.unity.com` is the endpoint to reach the metadata service; packages are downloaded from hosted CDNs which should, in practice, be much closer to your location, but I don't know the specifics of the routing rules or algorithms used. YMMV.
     
    bitinn likes this.
  5. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    144
    Please note that we're investigating easier methods of proxy configuration, including obtaining that information from the system settings. The proposed wrapper script solution is meant as a workaround until then.
     
  6. bitinn

    bitinn

    Joined:
    Aug 20, 2016
    Posts:
    552
    Thank you, I have created the launch script and observe that UnityPackageManager is finally going through my proxy (I have a client that can look at what requests are passing through this proxy), and with proxy enabled, I can see the list is now refreshed properly.

    Screen Shot 2018-11-23 at 18.10.04.png

    But if I set the request to go directly, I can see around 50% of requests to download.packages.unity.com, which is an CNAME alias of dl.bintray.com, failed due to timeout:

    Screen Shot 2018-11-23 at 18.08.00.png

    Since the domain download.packages.unity.com itself is accessible, I was able to test with my browser and confirm the same behavior (direct access failed, proxy access did see the package list output as HTML).

    So perhaps this is a regional network issue. I don't know, I have tested it with some local tools, here is the overall result from the 3 major ISP in China:

    Screen Shot 2018-11-23 at 18.39.02.png

    And China Telecom in particular:

    Screen Shot 2018-11-23 at 18.38.42.png

    Here is the result cache:

    https://www.boce.com/tool?type=get&taskId=e9d25d000e5fc62ce8691111ecd78477

    Hopefully you can work with local partners to check if this is indeed an issue or just isolated incidents. I suspect the problem might be more widespread.

    (My estimate: if the default timeout is set to 10s and the average response download time is ~8s for my region, then 50% request failure rate isn't that unimaginable.)
     
    LeonhardP and maximeb_unity like this.
  7. maximeb_unity

    maximeb_unity

    Unity Technologies

    Joined:
    Mar 20, 2018
    Posts:
    144
    I'm glad things are now working for you.

    And wow, that's amazing feedback! Thank you very much for that invaluable data, I'll pass it on to the relevant people.