Bug: 4GB limit to Textures in standalone build

I have the same problem. All game is pink if build is too big for Unity. Actually is not big, i just added a normal scene for standalone, i realized if i removed some parts it worked.

 

YOu can see it in the build log in C:\Users\nameofyourusername\AppData\LocalLow\Nameofproject\Nameofbuild

 

I get this error.

sharedassets0.assets'. Serialized files over 4.00 GB

What should I do? I didn't even created 10% of my game world!!!

 

I think your scene uses more than 4gb, split it in several scenes. Also for some reason i had also problems with a certain scene and when i moved all the gameobjects to other scene build didnt have any problem. In my case i had all pink and i solved it in that way

 

Yea , thanks I worked around this issue.. Streaming solves it pretty good.

 

How can I keep the links between objects if I create multiple scenes?

 

Unity Technologies gave a talk about this a while ago:

Spoiler: GUID Based reference workflow for System Shock 3 - Unite Copenhagen

Here are the files he was referring to in the talk:
https://github.com/Unity-Technologies/guid-based-reference

 

How do you do your streaming? id like to try if it doesnt involve too much work

 

ah yes, to be honest this solution doesnt work for me because its too manual to set up, if it can be automated it would be good, but there is no way i can mark up everything, I have too much in this scene. Ideally there would be a way to auto analyse a scene and generate bundles of graphics, sounds etc for that particular scene. I may write an asset for that sometime next year. For now im just happy with the 4 gig fix so I can soldier on with current resources system, but obviously this will become an issue one day when I need to release a small fix/patch. Thanks.

 

If you use steam to sell your game it wont be an issue..

 

Theres no real tutorial about it.. and streaming does the same.

 

why not? What if I fix 1 small bug, wouldnt they need to get the entire game again ?

 

No, it will only download changed files.

 

nice, i need to read up on how that works..

 

If you have a large Unity project you can tell the bug reporter to *not* include your Unity project in the submission. i.e. I have several gigabytes of assets and I don't want to even try to send that. The Unity team will still get your editor log, crash dump, and versioning in your bug report, which is all useful.

 

Thanks! For anyone else experiencing this issue, on older versions, I did the multiple scene thing and it worked pretty easily. Working with 16gb+ point cloud data files, had to split the assets into 5 or so files of 10 million per scene and then load them async/additive at runtime, grab the assets through a simple script, and combine them. Now the hard part is making it perform fast

 

I can't quite visualize the setup of the original solution, I've been reading the post a few times but it's not quite clicking in my head. Could you maybe provide some more details on the setup?

Update: I figured it out, it works amazingly well for a solution... prior to when 2020.1.0a19 is released.

 

Can anyone confirm this is fixed in 2020.1?

 

First come first, Thanks you for the work on this topic.
Second, is there a comparative between the loading speed of an asset bundle loaded from assets/bundle folder and the same weight of assets in direct reference way? I am working right now in a project which has Real location photos, text....and I am wondering if it would be a good practice to approach this from an Asset Bundle perspective.

Thanks.

 

I wouldn't say that's true always.

According to my tests, loading speed depends on platform and compression type. For example, I remember a specific console platform loads chunk-based LZ4 compressed bundles significantly faster than any other compression type.

It's worth to test your game on every target platform with every compression Unity supports. Track how much time each level requires to load. Then pick the compression that works best for you.

 

Hello! when will 2020 be out of beta?

 

In 2021 lol

 

Keep in mind LZMA compression will cause the bundle file to be entirely decompressed to memory in one go when it's loaded. This can easily double your memory usage if you keep your bundles around after loading assets from them (which is how Addressables work, if you are using that).

 

lol

 

I tried with assetbundles and once i can see the scene Unity freezes like 2 minutes and then show normal fps, which makes it not an option. Scene is already loaded as i can see it but takes fps to almost 1fps for some minutes, which doesnt happen loading a normal scene.
Tried also addressables but make Unity crash when load scene, and making multiple scenes would complicate much the development as the game is made. Dont know why scenes cant be bigger than just 4gb, that is very small scene

 

Same happened to me, after so many hours realized i couldnt make a scene bigger than 4gb, i understood why big companies dont develop with Unity

 

Asset bundles dont work, once you can see the scene game freezes for some minutes making AB useless

 

The asset bundle system has its flaws, but there are many developers that utilize asset bundles with great success, so I don't think it's entirely broken.

It could be something with your specific use-case that causes the asset bundle system to under-perform. If you debug/profile what's going on, you might be able to figure out what's causing the freeze and find a workaround

A common issue for freezes during scene load is scene activation, which I believe is split in at least two phases:

• Unity is doing something called "Asset Integration" which used to and probably still runs on the main-thread. To workaround this, you can split content across multiple smaller asset bundles, so Unity needs to integrate fewer assets per bundle.
• Unity calls Awake() on every Component that is in the loaded scene. If you do stuff inside Awake, it can sum up to take quite long. You could split your scene into more smaller ones. Mesh Colliders are also often found to be slow on scene activation.

This has been fixed/added in Unity 2020.1:
https://forum.unity.com/threads/bug...n-standalone-build.441116/page-5#post-5300130

However, putting a lot of assets in a single asset bundle isn't the preferred approach in many cases and can be one reason why the freeze occurs, due to the asset integration step.

I explained why splitting the content across multiple asset bundles has other advantages too here:
https://forum.unity.com/threads/bug...n-standalone-build.441116/page-5#post-4473202

 

Assetbundles and Addressables are a major pain in the butt: they are extremely complicated to design around, not only at the code level, but at the actual gameplay level.
I know, I have looked into it several times. It's a giant mess.

 

What are those pain points? Can you share specifics? I haven't looked at Addressables yet, but need to implement them in the coming weeks, so I'm interested to get some insights.

 

I have used successfuly assetbundles in other projects, for Android, with other Unity versions and not loading from hard drive the scene, in this case doesnt work, so its an Unity problem, not me
The assetbundle is the scene so i cannot split into several assetbundles unless i make several scenes in which case i wouldnt need assetbundles at all. As my game is i cannot split it in several scenes easily, i would have to move objects between scenes and change the code of my app.
Awake is not the problem otherwise the start of the scene would be sluggish too just loading the scene normally, which doesnt happen
I knew about 2020.1 but have some problems, its in alpha yet, also i tried 2019.3 and fps drop a lot for VR, VR is broken in 2019 and not sure in 2020

 

I looked into addressables with renewed hope when I knew they were releasing them. I hoped it would ease the pain of dealing with assetbundles, but actually it doesn't.
Addressables are designed to decouple the ACCESS to assetbundles, not to make them any easier to use: addressables is a layer of abstraction between you and the assetbundles to manage downloading them from the web plus loading and unloading them from memory, but there are several big flaws to this:
You can't use an addressable as a regular asset: you can't use an addressable like a pointer to a texture or to a gameobject. And that's what we would really need: proxy assets that act as pointers to unity objects that can be used in their place and would be loaded or unloaded automatically on demand BY THE ENGINE.

The current implementation of assetbundles is just a very low level way to import external assets into your project once it has been built, and that was the use cases for what they were designed: to load DLCs or texture packs into your game.
They were never meant to segment your actual project in different parts but to ADD new parts once the project is done. (hence the big emphasis on network download of assetbundles in the docs)

This is painfully obvious when you use them and Unity Tech is using them as an excuse to throw at anyone that speaks about the problem of managing big projects in Unity: "Oh, just use AssetBundles, they are meant for that."
No they aren't.
Asset management should work just like Async Scene loading and Texture Streaming does: automatically under the hood, were it belongs.
Using assetbundles or it's extension, addressables, is like using sparse textures to implement texture streaming yourself: it CAN technically be done, yet it's a massive pain in the but.

Let me be clear: I don't hate assetbundles or addressables. I think they are great for what they were designed to do: load external assets for your built application like texture packs, add-ons or DLCs.
But they are NOT a good way to manage your project internally.
It CAN be done and people actually use them to circumvent Unity's limitations, but yet again the fact you can use a fork to cut a steak if you REALLY have to doesn't make it any easier nor the best tool for the job.

 

I’m confused, are you saying people don’t download big apps up front and instead prefer asset bundle loading nightmares that never actually load and always interrupt the flow of the game to download some crap?

 

Actually, the end user is never the issue here: it's 2020 and we are all used to download GBs worth of updates for literally everything every time.
The problem is with store fronts like the AppleStore that limit the actual app size to 1GB and force you to deal with it.

 

AFAIK for iOS the limit is 4GB in the last couple of years (and I think it was 2 before that).

 

Yeah sorry, I was thinking about AppleTV ridiculous requirements for app sizes. (a tiny-weeny app size and then you have to download everything yourself)
I don't actually develop for iOS or TvOS though so it could have changed

 

That changed too btw (but we’re getting off topic)

 

I agree 100% with this. If Unity is the only engine you ever worked with it might not sound too bad, but if you have experience with other engines, Unity's run-time (and build-time) asset management is, to be blunt, terrible.

The biggest flaw is that Unity doesn't automatically know when an asset has become unreferenced, thus being unable to automatically unload them. The only way it has to detect unreferenced assets is by executing Resources.UnloadUnusedAssets(), which triggers a very expensive blocking sweep operation in order to do so. Fine-grained asset unloading is only possible when done manually. Even with addressables, you still need to manually release your AssetReferences, and the code complexity does add up on large projects.

I began studying Godot lately and I couldn't help but to smile when I saw this in their docs:

Then I remembered I have Unity projects to tend to and the smile faded away.

The second problem is that AssetBundles are so low-level you cannot use the same mechanism to load assets in edit-time as you do at run-time. You have to build a bundle before you can load it, and if you're targeting a different platform than the one you're running the Unity Editor on, it gets even more complicated. The workaround (which is also used by Addressables) is to "mimic" the access by using the editor-only AssetDatabase.

What is weird is that, before bundles and addressables, there was a system that worked exactly the same in both edit-time and run-time: the resources folder. The problem is that the way Unity packs and loads resources is suboptimal and doesn't scale well, however the Unity team never considered trying to improve the resource system and address its flaws. Sometimes I suspect the code behind the resources system is so arcane there's no one left at Unity who dares modify it.

 

Thanks for the reply!

 

I suspect that's what happened with surface shaders too, now that the new and shiny scriptable render pipelines don't support them anymore.

 

Guys wait a bit, make me understand. If I start building my 20gb project now, I will get an error that the size is to big?(I have 2k-4k textures for all of my stuff).Or is something specific to a platform?.

 

Read the proposed solution from ccsander here.

 

I was not looking for that solution, I am trying to understand what the problem is with that 4gb limit, is it per file, is it for assets added together? that type of question my friend.

 

Then read Karl_Jones answer here.

 

So you don't know either if you keep redirecting me to what I have read already.Every workaround found here is a bit to much for me, I used plugins and some tools to get my project up and running and in conclusion I cannot use those workarounds or atleast not without spending half of the year dealing with it.