Is there an alternative to AssetBundles?

Hi folks.

I've got a little problem here that I'd like to ask you if somebody knows a solution to it.

The thing is that I need to have assets to be downloadable from the internet and to load them during runtime. The first and obvious thought would be to use AssetBundles. But AssetBundles have a downside that makes them pretty much useless to me: They NEED to be loaded COMPLETELY into memory to load an asset out of them. You then would either have them stay in memory in case you need another asset later on or you unload the AssetBundle from memory, but then have to load it (and therefore also uncompress it time-consumingly) again if you require another asset at a later point.

Usually, with today's machines, having it persist in memory is not that much of a deal, if you run out of memory the OS will cache it to disk without you having to do anything. But that's not the case on mobile devices where memory management is crucial.

The usual answer on this issue is always: Split your assets into multiple smaller AssetBundles in a smart way so that you will only have to load what you need. And if you need other assets on a later point in your application, load a different AssetBundle.
Yes, that might help. But also: Being able to handle a number of assets at once with only one file not kind of the textbook definition of, well, a "bundle"? And despite this being more of a philosophical argument, having to handle multiple AssetBundles is not always the ideal way to go. Downloading them, for example, is MUCH easier when only having to handle a single file instead of multiple bundles. Another case would be if you need all assets at one single point anyway, like when you have to load a scene. As I understand it, right now you would have to uncompress and load the whole AssetBundle into memory. Running on a mobile device, that takes some time and a considerable chunk of your total available memory. Then you would have to load all required assets out of the bundle into memory, which is fine because you would have to do that anyway. And then, after you're done loading and instantiating them you can unload the AssetBundle again to free up the memory it took, luckily there's a method that allows this without loosing your instantiated assets. But this, also, takes a wee bit of time.
And for the time frame directly before you unload the bundle, don't you have anything in memory twice? The original binary data of the AssetBundle and then the instantiated version that will persist after unloading the Bundle? That would only allow scenes that aren't bigger then half the memory that is available on the device. That's a little bit too restricting for my taste.
Oh, and then comes the idea that you could use multiple small AssetBundles, load the first one, instantiate all the assets, then unload it and load the next, and so on ad nauseam. But please be honest: Who want's to write code to handle this? With all the keeping track of what asset sits in what AssetBundle? Ugh... There's GOT to be a better solution.

Then I took a second thought and it came to mind that the Resources folder kind of does EXACTLY what I need: As you know, everything you put in a folder named Resources in the Editor will be put into a file called resources.assets at the time you build your project. When running your project, Unity is able to pull all the assets you need out of that resources.assets file without having to load the complete .assets file into memory. It seems to just load what it needs without touching everything else in this file, which is simply perfect in terms of performance and memory consumption. I was able, with a bit of memory management concerning visible assets, to handle some considerably large scenes on mobile devices, without ever running out of memory.
However, theres a but: The big drawback is that all assets MUST be build with your main application. It doesn't look like there would be a way to do this separately, like with AssetBundles.

And I think that's a shame, really. Because the technology required for this seems already build right into Unity. It's right there! And I can't seem to take use of it for what I need.

So, with that bit of story being told: Is there something that I don't know about that does what I want? Am I missing something obvious here? Or maybe a method, a workaround?

 

Its not that bad. If you just save it somewhere and you can load it up before you load your game.

 

What do you mean by "save it somewhere"?

 

It depends on the platform. When I did it for android I'd save it on the dd card. For iPhone I save it in the application folder. You just need to Checl every time you open the app if there's anything you need to load from there.

 

Now I think I got it: You mean saving the AssetBundle.
The AssetBundle is saved on the device. Of course it is, having to DL it every time would be WAY annoying.

That was not what I was talking about. I meant it is a drag to load it from the device's persistent memory and uncompress it, because that takes a while. (And by a while I mean a second or two. I am a perfectionist. I hate to wait that long if I only need a Texture or the like, which usually takes far less than a tenth of a second. I'm not a fan of watching a loading wheel, even not for a second. The hardware allows more performance, only the software doesn't. And Software you can fix.)
And on top of that having to have the whole thing in RAM is not very useful either, if you only need one or very few assets out of it at that point. It's just a waste of time and RAM, especially since the technology to do it differently is built right into Unity and already and is used very successfully by the thousands with the Resources class and folder.

 

if you use WWW.LoadFromCacheOrDownload it does not need to be decompressed each time. The cache api will decompress it on the disk.

 

Playing with Audio asset bundles on iOS I didn't experience the same results as you did.

When loading an Assetbundle I had a small pause for the initial bundle to be processed, but not a massive memory overhead that 10+ music tracks would entail. And loading individual tracks from the referenced bundle seemed to keep my memory usage to about 2 tracks at a time.
There is some memory usage when loading an AssetBundle, but I didn't find it to be close to the bundles contents uncompressed, although to be honest I didn't do all that rigorous testing as to what specifically was loaded at what stages.

My solution for music tracks was to load the bundle at startup and then pull out the assets I need (calling Resources.UnloadUnusedAssets() occasionally to make sure the no longer used assets were unloaded)

 

Thanks for the answers. I'll take a closer look.

 

Unfortunately, WWW.LoadFromCacheOrDownload only saves me the decompression time. However it STILL loads the whole freaking bundle into memory, just to get a handle on one single file inside the bundle. I would really like to talk to the guy who thought of and wrote the AssetBundle system to ask him what the hell he was thinking. Like I said: Take a look at the resources.assets behavior, which does exactly what I need: Bundle an arbitrary number of files to one bigger file to make it easy to download and handle, and then, when accessing it, read only the parts which I actually need, and by that I mean read them directly from disk and not from RAM. The Resources system that is build right into Unity does exactly what I need and I just can not use it outside of the initial build. And I just don't see the point in that! It's right there in front of my nose and I am not allowed to use it. This is so stupid!

In the end I ended up writing my own custom file format. Stuff like that happens quite a lot lately. Either I am becoming too knowledgable for Unity to be still useful to me (which I doubt) or Unity gets worse and worse.

 

I feel your pain, but unfortunately it is what it is and any access to an item in an assetbundle will load the entire bundle... what I ended up doing was create a bundle for each individual item or set of items that need to be loaded at the same time. You end up with a ton of bundles but it's either that or the highway.

 

Correct. This is what UT tells you to do anyway. But that defeats the purpose of having "bundles" if you still end up handling multiple files.

But Again: The technology is there. It's done. It's been tested by the hundreds of thousands or even millions during the past 5+ years and it works perfectly. It's called the Resources system.

 

Difference being that assetbundles inherently supports async loading, but I don't see why they couldn't just add an async resource load method for the pro version instead of inventing this whole separate AssetBundles thing... I was actually misled into buying the Pro version much earlier than I needed to because of this feature; imagine my surprise when I found out it loaded everything anyway.

http://forum.unity3d.com/threads/99980-Asset-Bundles-ram-usage-and-performance

 

it doesn't load everything but if you don't use WWW.LoadFromCacheOrDownload it has to load the whole bundle by definition cause its a 7z file which must be decompressed as 7z can not selectively decompress part of it, its all or nothing required to achieve the high compression ratio (3-5 times better than zip, the cache system stores it uncompressed on the disk just in case thats unknown, so its able to get away with FAR lower memory footprints and higher speeds! it works much more like resources actually)

 

I know, dreamora. You already wrote that. And judging by your track record around here I believe you know what you are talking about.

But the funny thing is that it actually DOES load the whole thing if I use WWW.LoadFromCacheOrDownload. I did half a dozen of test runs on an iPad and using Instruments to take a look at the iPad's system resources and Unity's RAM footprint jumped exactly the size of the cached, uncompressed AssetBundle, that it also takes sitting in the Documents folder on the iPad. That is, ONLY using WWW.LoadFromCacheOrDownload and without loading ANY asset out of the bundle yet. (You can check that bundle's size by downloading the xcappdata bundle from the device using XCode. And it was considerably larger than the initial AssetBundles I made for testing this, so it's pretty save to say that they were the uncompressed versions, just like you said.) No matter how large it was or how many files were in the bundle, it always loaded the whole thing. That, PLUS the assets you then actually load from out of that thing. So in a worst case, if you need all assets inside the bundle, you'll end up taking, for the time you load them until you unload the bundle again, twice the amount you'd actually need if Unity would read the assets out of the bundle from disk instead of RAM.

So either I did something completely wrong half a dozen of times, it works differently on iOS devices than it does on other platforms, it's a bug or WWW.LoadFromCacheOrDownload simply doesn't work the way you say it does.

 

Hello Whimsical,

I've also made some tests and this is what I've concluded so far:

When a bundle is not cached, it allocates a buffer to fit the whole size of the bundle plus a fixed 8Mb buffer used for LZMA decompression, after it's decompressed it releases the memory. After it's cached however, it does not add any memory, take a look at these graphs:


Loading from file://, the bottom graph is the first run, before the bundles are cached, the top graph is the second run


Loading from http:// quite similar to above, but taking longer times

To keep the bundles size small, instead of one huge file use the dependency management feature when building the bundles (PushAssetDependencies() and PopAssetDependencies()) and separate your assets into multiple bundles, and during load, load them in the same order you used to build. I've created a small bundle just to hold the name of all the bundles and their versions.

Also, make sure you dispose your www instance after loading or the memory will only be freed in the next garbage collection, I've put mine inside a using() block to be sure of that.

 

I was about to come on here and go on the exact same rant Whimsical went on until I read your post Capyvara.

Why isn't stuff like this more obviously documented somewhere??

Its kind of lame you can't export uncompressed asset bundles and just include them directly.

If only they added Resources.LoadAsync, everything would be so much simpler

 

I think they are planning to include uncompressed asset bundles in 3.5