How to download AssetBundleManifest from Server with UnityWebRequest?

Hey,

so currently I am trying to make all my AssetBundles downloadable from a server.

Lets say, an AssetBundle 'assetBundle' contains 1 single Asset 'assetA'.
Unity3d will generate following files:
- assetBundle
- assetBundle.manifest

So when I try to download the AssetBundle with UnityWebRequest it works just fine, but once an AssetBundle is tried to be downloaded a second time, an error occurs "AssetBundle with that name is already loaded".
Currently, I only download the file 'assetBundle' from my WebServer.

So UnityWebRequest caches the downloaded AssetBundle. I read the docs and some posts and it was said, I could use the CRC inside the .manifest-file to see whether a Bundle was already loaded or not.

So I tried downloading the .manifest-file from my WebServer aswell, but it just does not work. The docs do not have an entry about downloading .manifest-files from a WebServer and I couldn't find anything useful with google aswell and I think thats pretty weird.

That's currently what I am trying to do:

Code (CSharp):

1.  UnityWebRequest manifestRequest =      UnityWebRequestAssetBundle.GetAssetBundle(urlManifest, 0);
2. yield return manifestRequest.SendWebRequest();
3. AssetBundle manifestBundle = DownloadHandlerAssetBundle.GetContent(manifestRequest);
4. AssetBundleManifest manifest = manifestBundle.LoadAsset<AssetBundleManifest>("AssetBundleManifest");

The code above fails with "Failed to decompress data for the AssetBundle 'http://someWebserver.com/getManifest/assetBundle.manifest'

I really appeciate any help.

 

So I figured out, that I shouldn't use the '.manifest' file but the AssetBundle-File itself. Sadly when trying to load the Manifest through the code above, it says its null..

 

So.. I changed my code to build the assets and used the StreamingAssets-Folder, before, I used to build the files outside of the unity-project-structure, there it only built 2 files. Bundle + Bundle.manifest.

Now that I built into the unity-project, it also generated a file "StreamingAssets" that was successfully loaded as a manifest.

But what should I do when I have 1000 different AssetBundles and I dont want only a single manifest-file with 1000 entries of AssetBundles?

 

AssetBundle with that name is already loaded means you're loading the same bundle (or a bundle with the same name) in the same app session twice. Once you've loaded it the first time, you should store a reference to it and use that instead of calling GetAssetbundle again.

That, or you can unload the bundle after you're done loading assets from it:

https://docs.unity3d.com/ScriptReference/AssetBundle.Unload.html

So there are three types of files when using Unity's built in assetbundle build system.

- The assetbundles themselve, stored as the name you gave them without extension.
- A manifest file stored as <bundlename>.manifest
- And an extra assetbundle containing exactly 1 asset of the type AssetbundleManifest, stored in the same output folder and named after the folder its stored in

https://docs.unity3d.com/ScriptReference/AssetBundleManifest.html

The files ending with .manifest are used by the editor for incremental builds, and serve no purpose at runtime. You should not upload them to your server or download / load them in your app.

The assetbundle manifest is an important one, as it allows you to query all available bundles and gather dependencies. You don't necesarily need this file though, if it doesn't make sense for your usecase to store all bundles in 1 file, then hook into the build system and store the info you need in a way that makes more sense to you.

Also, instead of using the legacy API's as I think you're doing now, you could look into building bundles via the Scriptable Build Pipeline, which is the way forward.

https://docs.unity3d.com/Packages/com.unity.scriptablebuildpipeline@1.15/manual/GettingStarted.html

 

You can try it... It is working for me.
Make sure to check you have permission to download the asset you want from the storage you are using (Your server / Firebase storage / Google storage / AWS S3 / Drive)

Code (CSharp):

1. //My asset url
2. string url = "blablabla.com/myassets/gun1"
3.  
4. public void Start
5. {
6.       StartCoroutine(DownloadAsset());
7. }
8.  
9.  
10. IEnumerator DownloadAsset()
11.     {
12.         using (UnityWebRequest uwr = UnityWebRequestAssetBundle.GetAssetBundle(url))
13.         {
14.             yield return uwr.SendWebRequest();
15.             if (uwr.isNetworkError || uwr.isHttpError)
16.             {
17.                 Debug.Log(uwr.error);
18.             }
19.             else
20.             {
21.                 // Get downloaded asset bundle
22.                 AssetBundle bundle = DownloadHandlerAssetBundle.GetContent(uwr);
23.                 var prefab = bundle.LoadAsset(partName);
24.                 Instantiate(prefab, new Vector3(0, 0, 0), Quaternion.identity) as GameObject;
25.             }
26.         }
27.     }
28.  
29.  

 

Hello @nilsdr! Thanks a lot for the info! This is very useful!
In my case, I have to load each asset on demand. When the user clicks on a product I have to download the asset bundle and cache it. So I have to use the manifest to get the Hash and CRC for the bundle. Do you know how to build a manifest for a single bundle?
If I build 5 bundles I want to create a manifest for each bundle and upload both the bundle and the manifest to the server. When the user clicks on the product I firstly will send an HTTP HEAD to check the Hash and CRC thanks to that info I will be able to check if that bundle is cached. Then if it is not cached I will use UnityWebRequest to get the bundle and load it. Anyway, I guess if I get the manifest info the UWR built-in function should do the job... I just need to know how to get that hash and CRC values for each bundle...

Thanks in advance!!
Chris

 

Assuming your bundles are generated in one go in the same command, the resulting assetbundlemanifest will contain all hashes for all bundles. You wouldn't need to have multiple manifests.

 

Hello, @nilsdr thanks a lot for replying!

Currently, I have a main unity project that will hold the logic of my app and a secondary unity project just for building AssetBundles. The main unity project is communicating with a CMS(Content Management System) to download information and resources about a specific product (3D Model).

On the CMS side, you can manage the products. Like name, description, image etc



And in the same product, I can upload the bundle for IOS, Android, and Windows...


At runtime, the app will connect to the CMS, and when the user clicks on the category I will make an API call to get the product list (a basic JSON) with all this information and the path of the bundles pointing to S3.

Everything works fine but at the moment I'm not caching anything... Each time I download the bundle (3 - 5 MB) and it's not that good...

This is exactly the stuff I don't understand. I know when I create a build the StandaloneWindows64 is the manifest. But I can't have all the info in one single file because the products will grow...


So my problems at the moment are:
- How do I cache the downloaded bundle? I need the Hash and CRC
- When I update the product on CMS the Hash and CRC will change and the app will automatically download a new bundle this is ok but how do I delete the old cache? Will the UWR replace the old cache?

I hope now that I explained the full architecture is more clear! What are your thoughts on it?
I also tried with Addressable Package but it's not compatible with the current system so I have to handle Bundles manually and it's really confusing!

Best regards!
Chriss

 

Thats very clear, if you can change the entity/model in your CMS your best approach might be to skip the assestbundlemanifest altogether and store the hashes for each given bundle there.

After building the assetbundles for each given platform, the resulting object (the assetbundlemanifest) will contain the hash for each bundle. Add a field for each platform your CMS supports and store the hash in there.

Then, from a client side perspective, pass that hash as a parameter to the uwr when downloading the assetbundle. This will skip the download altogether if a bundle with the given hash was previously downloaded.

Does that make sense?

 

Yes, It makes sense! That is what I wanted to do in the first place I just needed to know if I was doing the right thing!

So to recap:
1. Build asset bundle
2. Foreach platform GET the manifest (Android, IOS, etc...)
3. Extract the Hash of the bundle and the CRCs. I could use these functions:

Code (CSharp):

1.  
2. // GET HASH
3. Hash128 hash = manifest.GetAssetBundleHash("bundleName");
4. // GET CRC
5. BuildPipeline.GetCRCForAssetBundle(targetPath, out crc)

4. Upload everything to the Entity on the server

The client will load the bundle for the first time:
1. Call API to get the product entity (this will not be cached because its very light)
2. I will have access to CRC and Hash to check the cache with

Code (CSharp):

1. Networking.UnityWebRequest GetAssetBundle(string uri, Hash128 hash, uint crc)

From here everything is crisp and clear!
Now I have to figure out how to patch the bundle If someone updated it on the CMS:

From the DOC: https://docs.unity3d.com/Manual/AssetBundles-Patching.html
The more difficult problem to solve in the patching system is detecting which AssetBundles to replace. A patching system requires two lists of information:

• A list of the currently downloaded AssetBundles, and their versioning information
• A list of the AssetBundles on the server, and their versioning information

​

Anyway thanks a lot for the feedback! I hope this thread will be helpful . If somebody wants to use asset bundles...

 

That's the right approach. From a client side perspective you wouldn't care if the bundle was previously cached, UWR will handle that logic.

https://docs.unity3d.com/ScriptReference/Caching.ClearOtherCachedVersions.html

That api will remove older bundles, you can execute it after a UWR finishes for a given bundle and hash.

A mature pipeline could query your CMS api for the latest known bundle version in a post assetbundle-build step, and then let you know your local version is newer (or even automate the upload aswel)

 

That's great! Thanks a lot for the feedback!

 

I just tested a really rough implementation of the workflow above.

Code (CSharp):

1. public static async Task<AssetBundle> GetAssetBundle(string uri)
2.         {
3.  
4.             const string manifestPath = "http://localhost/wewear-app/StandaloneWindows64/StandaloneWindows64";
5.             using var manifestRequestRequest = UnityWebRequestAssetBundle.GetAssetBundle(manifestPath);
6.             var operationManifest = manifestRequestRequest.SendWebRequest();
7.            
8.             while (!operationManifest.isDone)
9.                 await Task.Yield();
10.            
11.             var assetBundleManifest = DownloadHandlerAssetBundle.GetContent(manifestRequestRequest);
12.             var manifest = assetBundleManifest.LoadAsset<AssetBundleManifest>("AssetBundleManifest");
13.            
14.             var hash = manifest.GetAssetBundleHash("cube.assetbundle");
15.  
16.  
17.             const int crc = 465257593; // copied manually from cube.manifest
18.            
19.             assetBundleManifest.Unload(true);
20.            
21.             Debug.Log("HASH: " + hash);
22.             Debug.Log("CRC: " + crc);
23.  
24.             using var unityWebRequest = UnityWebRequestAssetBundle.GetAssetBundle(uri, hash, crc);
25.             var operation = unityWebRequest.SendWebRequest();
26.            
27.             while (!operation.isDone)
28.                 await Task.Yield();
29.            
30.             return DownloadHandlerAssetBundle.GetContent(unityWebRequest);
31.         }

It works perfectly the BUT uwr is just creating a new cache if the Hash changes. So I need a way to delete the old cache and replace it with the new bundle.


Any clue on how to do that?
Do I have to keep track of the old Hash?
Unity patching: https://docs.unity3d.com/Manual/AssetBundles-Patching.html

 

Posted about that above, this is the function you need:

https://docs.unity3d.com/ScriptReference/Caching.ClearOtherCachedVersions.html