Bug CI (Jenkins via Windows Node) Build stuck at uploading symbols

Hi,

We are trying to evaluate using Unity Cloud Diagnostics to help our development of a Quest 2 application, but our CI system which runs a batchmode build of Unity (Android, IL2CPP, 2021.3.4f1) seems to be unable to upload symbols.

We provide the command line the user and password for a valid unity license (as well as -serial) and it still seems to be getting stuck on this part. It specifically does not progress beyond this stage of the build process in the logs:

Code (csharp):

1.  
2. 17:27:18 Android PostProcess task "Calling IPostGenerateGradleAndroidProject callbacks" took 153.963 ms
3. 17:27:18 DisplayProgressbar: Performance Reporting Symbol Upload
4.  
5. 17:27:19 [Licensing::Module] Successfully cleared the access token
6. 17:27:19 [UnityConnectServicesConfig] config is NOT valid, switching to default
7.  
8. 17:27:21 [Licensing::Module] Successfully updated access token: "********"... (expires: 2022-06-28 18:27:38 GMT)
9. 17:27:21 User **** logged in successfully
10.  

Any help, pointers or anything would be great, getting to the point where I'm running out of options. (tried searching the forums to no luck, symbols are about 160MB at most, internet connection is fast)

 

Looking into this more, I've found the command line works when I manually do it, but when using Jenkins to do it, it doesn't get past the above log (went on for 12+ hours). Could be a permissions issue maybe? The firewalls on the build machines are off though so it shouldn't be blocking anything.

 

It's suddenly working now, when I did a manual command line (with a different account in our Unity org), which worked. Switched the account Jenkins was using, which worked and then went back to the original account which worked! Not sure what is happening, feels like a cache thing maybe? or an authentication thing. My account is an Owner on the org, but the jenkins account was just a Manager, maybe that was why?

Hoping someone with better knowledge of the internals of uploading symbols might be able to give answers

 

Problem has returned when deleting the library folder. Our CI system needs to be able to handle this scenario, so if anyone know anything that is cached in the Library folder to do with symbol uploads, that could be the issue?

 

Can anyone from unity help with this?

 

We aren't using Cloud Build, we're using our own build system locally in our office. Should I still make a ticket to the Cloud Build Support?

 

Our android builds stay more than 15 minutes in the 'Cloud diagnostics symbol upload' dialog.
Not sure if it has some relation to this, if its a bug, or if this is normal, but it's very annoying.