Bug: 4GB limit to Textures in standalone build

Does this work for things like Watersystems, ufps, vegetation studio rendering solution?

 

Unable to open archive file: C:/Users/Unity/Documents/Asset Bundler Test/Assets/StreamingAssets/AssetBundles
UnityEngine.AssetBundle:LoadFromFile(String)
loader:Start() (at Assets/loader.cs:9)

I got that error with this code_ <<<

using UnityEngine;
using System.Collections;
using System.IO;

public class loader : MonoBehaviour
{
void Start()
{
var myLoadedAssetBundle = AssetBundle.LoadFromFile(Path.Combine(Application.streamingAssetsPath, "AssetBundles"));
if (myLoadedAssetBundle == null)
{
Debug.Log("Failed to load AssetBundle!");
return;
}

var prefab = myLoadedAssetBundle.LoadAsset<GameObject>("Plane");
Instantiate(prefab);

myLoadedAssetBundle.Unload(false);
}
}

 

Should the sceneloader script be on the 0 scene (splash scene)?

 

Any news about the resource file fix?

 

Aaah now I get it! This is clever! Much better than the assetbundle if you allready have big project do you know how to instead of prefabs lets say use it for textures?

Is it okay if I make a youtube tutorial about this?

 

This limitation is really bad news for me. I am working on game where I have a lot of different cloths and attachements for player (Almost 6 bg of different meshes + HD textures). Problem is that having that many assets means, that is impossible to have all of them referenced by some kind of config with standard unity references, because if I make references like this, than all of this assets will be stored in RAM memory on device. So I really need to use resources or something like this. Or is there any other way how to reference some kind of "database" of prefabs without loading all of them to memory? Or how to deal with this situation?

 

Any ETA on this this needs to be fixed asap without developers having to add work around, this issue is long known and should be fixed by now. For a true 64bit engine there should be no 32bit limitations plain and simple. A Unity dev should respond to this please. We are experiencing this issue with 2017.1.0

 

I'm fine with not being backported like a bug fix, but the feature has to improve the situation now, I mean look how old this thread is since 2016! We are in the era of 4K , I'm actually shocked that this limitation is something you guy's have not fixed or commented in your doc's or even stated as a warning message in older versions by older I mean at least Unity 5 and Unity 2017. We lost couple of day's taking everything apart until we found this post here, that's unacceptable for a subscription based Engine. Please let us know when this feature will be implemented ( In my opinion that's not a future feature but something a 64bit engine should off had from the start)

 

I agree that this needs to get fixed in Unity right away, and I also agree that it is ok for this to go into the latest version without the need to backport it to older builds. I am still having excellent results using the Multi Scene feature as a work around in a project that currently builds to just over 5GB in total size. I would love to have a real fix, though.

 

"A true 64bit engine should not have any 32bit limitations."

This should have been fixed at least two years ago.

 

Thank you Karl for keeping us in the loop, I really appreciate it!

 

@karl_jones We just ran into this 4Gb limitation and i cannot see that AssetBundles will help much.
We have a single scene with plenty large (FBX) models and a core "state-machine" that manipulates little parts of the big models. This state machine is essentially a list of GameObjects where each one has little scripts doing things with the model-parts - which means these little scripts have "public GameObject target" properties pointing at child objects of these models.

Problem is that we can not setup/configure this state-machine without having the many large models in the scene.

Is there any progress on the concept you describe in the quote above?

 

That is good news - "addressable assets"!
For now we managed to work around the 4Gb limit following a suggestion here in the thread that proved simple and genius - add a tiny Loader-scene as scene 0 in build sort-order, followed by any number of "resource-container" scenes containing just "naked assets" but no code, and finally the big scene, which automagically "re-uses" the resources already packed into the previous scenes.
The build came out at 8Gb and works fine.

 

I’m looking for this too. AssetBundles currently are a pain by the way that they are managed.

I did a similar solution for Android. I have lots of detailed buses/vehicles and more will come, with large textures and many different sounds. I was loading them from Resources folder (only one each time that the user plays) until I got the apk bigger than 100MB, and that could not be sent to Play Store anymore.

I switched Resources with Scenes and everything works well now. Just load tem as additive and find the root gameobject by name (generally the same name of the scene). Unity does a really good job while loading and unloading additive scenes in runtime.

So now they are packed inside obb instead of the apk. This is a tip for those who had problems with huge apk in Android, somehow similar to this problem on desktops as said on this thread.

I just wish a better build settings window, so we can add/see/group included scenes in an easier manner hehe At least it is working and that’s what matters.

 

If I hear "Asset Bundle" again, I am going to flip.

But seriously, I gave up on a project a few months ago, experiencing the same issue and thinking, maybe I did something wrong...like seriously wrong. Spent few days reimporting every asset and all.

And I see this.

Really didn't expect 4GB limitation to be the issue. Nice.

 

So if you are trying to make a open world seamless game that has say 10 gb content it can't be done?You would have to load scenes thus it would no longer be a open world seamless game?

 

@karl_jones
Are the content of all resources folders are packed into one file?

If yes, why not save each content of one resources folder into one separate file. Okay... the serialization size of one file can't exceed 4GB, but then it possible to hold the reference headers of each so called resources file into memory and collect the references from the headers to the corresponding file.

And for sure you can build up an additional header and iterate over by the job system to collect the references to the required data.

This have several advantages, you don't have to write a 64 bit serialization file system. It doesn't matter from what file you read, because on any platform is possible. Low cost for Unity, low work for you, and more happy developers by the way.

 

I have found a way around this.. Just put game logic in one scene. The terrain in another.. The rocks in a third.. Trees in a forth./ and so on... you split it all up.. Because every scene has it own resource file...

 

yeah but... It cant be down in a existing project.. In a new one yes its good but hard to use, bad tutorials and the docs dosent answer really questions that occur like how when and why..

 

Well im making it for the pc so asset bundles seems more for online content I don't mind a large disk size.Anyway can't i use both options resouce and asset bundles?

 

Not worked with scenes yet so you load all the scenes at the start you mean before the game starts proper?

 

In addition what karl said, perhaps you want to wait a little and then use the new Addressable Asset System, which seems to make asset loading / memory management a lot easier:
https://docs.google.com/document/d/1hPLNLdrF0qAvjEJTpKf-cuO_d4FCV0H2cqBeP1Zo6mA/edit

 

Hi Karl what does this mean?I'm still using 32 bit windows so my major concern is keeping memory use low.I read that asset bundles are confusing and difficult to work with as a entry level coder I struggle with overly complicated processes.

 

Karl how do i check the amount of MB/gigs that are been kept in the resources folders to date?I tried to look for a resources.asset file but there isn't one ?.Right now i am very much in the prototyping/experimenting stage so speed is very important to me I will look into asset bundles at a later date but im very happy at the moment with the way resources are working for me.

How does the amountof memory in the resources folder work i have for instance just prefabs in but the textures are outside the folder I imagine that the textures would be included in that 4 gigs cap u mentioned even if not in a resource folder?.

 

You add the scenes to the build list in the way you want them to load.. its the LoadScene.additive you use. theres tutorials about it...

 

sound much better than using asset bundels, it ridiculos much writing there.

 

Started looking into asset bundles now but I have a few concerns the major one at the moment is unlike with resources.load where the folder is always in the same place no matter where you install your project.In asset bundles because you have to write the script the points to the exact location of the asset bundle how do you ensure the asset bundles are always in the correct location for the script links to reach them.
This is in the case of storing the asset bundle on the local computer

. For instance i have my project on the c drive now if i wanted to sell the game the asset bundle would have to be loaded to the users computer on the exact path as it exists on mine or am i incorrect in this?

 

Don't hard-code the location from where to load.

I used these approaches so far...

1. The asset bundles are located in the "Assets/StreamingAssets" directory of the project. Thus they are part of the build. Use Application.streamingAssetsPath to get the file location.

2. The bundles are not part of the build, but downloaded at a later point, where we used different approaches for different games:
2a. Use WWW.LoadFromCacheOrDownload where just the URL is required, but no path.
2b. We implemented a "Launcher tool" that downloaded the required content before the actual game was started. The launcher saved bundles to "%APPDATA%/MyCompany/MyGame/" and the game used this path to load bundles.

 

Hey Peter wasn't aware that you could rename the folder that would help.But the files will be stored on the users local drive and not be streamed at all from any url.Would i still be able to use these methods or what would be a alternative to achieve a similar result?

 

From the documentation:

https://docs.unity3d.com/ScriptReference/Application-streamingAssetsPath.html

Means if you build a Windows Standalone player for example, "StreamingAssets" is located in the a sub-directory where the .exe is installed, which normally is the users local drive.

The name might be misleading to some degree, because it contains the word "streaming", but it has little to do with that. I recommend you read the documentation and google, as I believe this is what you're looking for.

 

KK cool thx Peter allays one of my biggest concerns

 

I moved to asset bundles, and there is STILL a 4 gig limit, werent they supposed to be free of limits!?!?!

 

@karl_jones seriously isnt this fixed in 2018 yet???

 

Argh more to learn, I hope someone makes a simple drag n drop tool for this.

 

Only if you're using 2018.2+. If you're on 2017 LTS ignore everything he said and make a lot of small asset bundles.

 

I tool a look at it.. still confusing.. And those damn dependencies... jeesus like if a house loads, just load it all!! dont need a million codes for that.. this is serious too much... Like the game coding itself isnt much..

 

If we have to use asset bundles to get around this - fine (I was also under the impression that it was a feature that could be implemented close to release to make patching and DLC easier). But the user should be told that they have breached the limit rather than some of their textures just messing up all of a sudden.

Now I have whole bunch of messed up textures. How do I fix them? Do I need to implement asset bundles across the project, and that will fix it? Or do I need to implement asset bundles and then re-import all of the textures again? (And hopefully not have to re-apply them and all their materials settings).

Edit: I think my problem was either with the files becoming garbled on my PC, or as part of using collaborate (maybe even on my freelancer's PC), because they were garbled in the editor itself, not in the builds. Oddly the bottom 30% of the texture was as normal, the rest was garbled. I replaced the textures with the originals (they were from an asset pack) and the problem was fixed.

 

You have 2 options...

You could do as I, split your project over multiple scenes and then load addtive... Like one main scene where stuff loads from other scenes. trees, houses, rocks, monsters, etc..

Or if you want to use asset bundles restarting your project is the easiest because asset bundles are a pain in the ass to start using later.. Are you familiar with them? I dont know if they have released the new asset bundles yet but its better then the previous verision, you had to write so much and load every dependencie... Im waiting for something like load.house .. Then it lods with everything in it and textures and the location from where its placed on the scene... And I would be happy if some one could actually make a real tutorial..

 

Is there any speed difference if you instantiate something from a asset bundles or the resources folder?

 

Performance wise, I would suggest using additive scene loading instead of asset bundles.

 

Far too broad of a generalization. There are very good uses for both bundles and additive scenes.

 

Fair enough. From what I have seen trying both additive scenes and asset bundles as work arounds for the 4GB limit, I personally found the additive scenes worked vastly better. The asset bundles feature really struggled when I tried to use it to bring in large amounts of assets. Everybody's mileage will vary, but I personally found the additive scene feature was the vastly better way for me to get around the silly 4GB limit in my project.

 

Sure, but smaller bundles for clothes, items, 2D art, and a bunch of other stuff doesn't really fit with scene additive even though collectively they could exceed the limit if all in one bundle.