Bug: 4GB limit to Textures in standalone build

Yea well.. 2d games are not so huge anyways.. Besides thats easy enough to use with bundles..

But Im making now a topdown survival game, and I understand that bundles is a great way to use dlc or game updates...

Any toughts on how to make dlc and updates without bundle?

 

I was referring to UI 2D art: icons for buttons and markers and inventory panels and such.

 

So you're saying it's fixed in 2018.2?

 

Thanks I read that and did some more investigating into the new Addressable Asset System but according to this post it requires editing the manifest file before it's available in the Package Manager. Also it's version 0.1.2 i.e. early beta. Instead I just did as recommended elsewhere and moved the bigger resources into separate AssetBundles and left a lot of stuff in the Resources to get it under 4GB.

 

I just ran into this problem too, I'm glad I found this thread since I've been struggling with this problem for a week and only got the error now.

 

I dont understand the asset bundle or how to use it ... easiest is to split the scene in more scenes..

 

This is completely unacceptable. It seems I just ran into this limit. Asset Management... all the kind of stuff we *don't* want to do, because all that stuff is exactly what Unity was made for. We want to make games, not manage assets.

Why is there no standard solution for this yet? I understand the file limit, but why are files not automatically split? Why do I need to read tons of manuals and study asset management features to get a working game in the first place?

We need a button that splits the files automatically, and we need it now. It's 2019, and apparently Unity still has not caught on with 64 bit.

I understand the underlying issue. But then at least come with a temporary workaround - completely automated - that splits everything automatically. That can't be too hard, right? Please?

 

Is this really still a thing? I'm asking, because about in January 2018 (about a year ago at the time of writing), it sounded like they're working on a fix:

What Unity version do you use?

 

Hi Peter, the latest version: 2018.2.18f1

 

And I confirm that this was indeed causing the issue. I am now splitting my scenes. I now got:

sharedassets1.assets.resS (50,113 kB)
sharedassets2.assets.resS (4,551,889 kB), which actually looks bigger than my previous file, but it seems the textures are displaying correctly. Unless I run into nasty surprises at the other end of the level which I can't see from here.

 

Guys.. Just use Sectr streaming, very easy to set up , it fixes everything

 

Actually I spoke with the developer of Sectr, and he says this is *not* the case and may introduce lots of other issues.

 

By the way, that is Peter right above me

Quote:

I just read up on that issue, ended up in the long forum thread where you just posted in the end. From how I understood the problem, you can run into that issue either by stuffing up the resources folder with more than 4 GB of stuff, or by having a single scene referencing > 4GB of assets exclusively. SECTR could maybe help you in so far as that it splits up your terrain into multiple scenes, so it might even out the asset file size distribution across those scenes. That would heavily depend on how your assets are distributed across the terrain, and that is usually not the reason why you would add terrain streaming. If you have not used terrain streaming in your project yet, I think it is not a good idea to introduce it just to combat that unity filesize issue, as it might make your overall work more complicated and create different issues with gameplay logic, etc.

 

yea but it splits the scene to many scenes wich confuses the resource file limit

 

Unity, why not to add an error or popup about this issue during the build? We spend 2 days researching this issue.
Still, this issue exists in latest Unity without any notifications. We just got several textures corrupted without any reasons. And in Editor all works fine. This is a critical issue that user should be aware of.

 

I spent a month struggling with this issue, blaming the Bakery GPU lightmapper for it, wasting the developer's time with back and forth debugging, redoing and rebaking my scene lighting like 10 times (baking the scene takes hours), and just now I find this thread, describing the exact issue I have. I've had this problem with 2018.3 and the alpha 2019.1. It is 2019 and developers have been on this forum since 2016 asking for a simple error message so that new people like myself encountering this don't waste our project resources trying to figure out what is wrong. What I'm wondering, @karl_jones is if there are plans to at least introduce an error message describing the limitation. It seems obvious that there are no plans to properly fix the issue, so why not at least give your customers that small amount of mercy? Do you not realize how big of a problem the actual debugging process is for people here? If it were an obvious issue, then fine, ignore it, but it takes a lot of time (and money in my case) to figure out what the actual problem is.

 

Holy crap! Is this issue still a thing? Is there still no documentation or warnings about this problem? Is this thread the only source of information after poor devs have wasted days trying to figure out the problem and stumbled upon here trough luck by googling the problem?

Unity... what the hell? I remember how mad i was after i found out about this back in August 2017 and rambled here that you have to clearly tell about this limitation to the users. Otherwise you just waste developers time who stumble upon this and are left wondering what is happening. It is now January 2019 and there is still no message whatsoever? And as a result bunch of people have wasted their time and not knowing what is going on.

For the love of god. Just please put an error or warning message. Do at least something that would let the user know not to go over the limitation. This way we can start look for alternatives and not be left wondering why our builds are looking broken.

To other devs here. 2 years ago i was in the same situation. Luckily in my case i had a friend who already knew (took him several days to figure it out) about this and pointed me to this thread. I can confirm that SECTR stream is a great solution if you want a woraround. Of course it depends on your game but in my case it worked out perfectly.

 

It's even worse. After an upgrade from 2018.2 to 2018.3, my build size went up from 2 GB to a whopping 6 GB, while nothing changed in the game. I am not happy... how am I ever going to solve this?

 

I believe if we move to asset bundles this should work as a fix, you can keep them under 4 gig. Also in the end you dont want to ship a 6 gig game, I think its perferable to ship an exe and download bundles that are required during boot up. This obviously makes a patch much easier due to not downloading the same info again.

The main problem I have is if a game reaches this limit, its surely huge and nightmarish to "convert" to asset bundles (I have over a million objs in my scene). So ultimately I have literally been waiting for years for an asset which can *automatically* change your project to use asset bundles AND keep them small to like 1 gig each or something. Only problem is there is no such asset yet as far as I can see.

I did try this one and was hoping it would work https://assetstore.unity.com/packages/tools/utilities/autoassetbundles-111942 which I think it did but it did not allow me to specify bundle sizes so the exact same issue still came up. Perhaps someone knows one that can automate the process and allow small bundle sizes? I have toiled over this for years, mostly because im too afraid to dive into doing it manually on such a ridiculously large project

PS, I dont want to split my scenes up, I like my 3 cities in one, its very fast to leap between them and I seem to work better when its all in one scene, not to mention most of the core mechanics would need to be duplicated for each scene which leads to problems. Im sure I could do it (they were seperate but i merged them lol).. but I dont want to, I would love to see auto asset bundle asset above fixed and working.

 

I've now spread my build over four assets files of 1,5 GB each by instancing prefebs in scenes that are loaded one by one, but still I wonder why 2018.3 needs three times as much space to store the same information as 2018.2.

And Unity, with all due respect: an export warning is not good enough. We need an asset that spreads your information over files automatically, or we need a bigger file limit. More than half a year has passed and absolutely nothing has been done about this.

Please put this on the very top of your list.

 

Done!

 

@karl_jones Wait. Not done. After 10 minutes, packing the report is still at 1%.

Of course I can just zip everything, but this is taking too many resources for too long. Is there another way to send you what you need, without packing and sending the whole 6 GB of data?

 

Of course. Done!

 

@karl_jones you said in your previous last year posts that we should look at 2018.2 and this is high priority and should be resolved in coming Unity versions,and here we are in 2019 and the 32bit 4gb resource limit issue is still existing. I don't want to use Asset bundles they are slow and have the same 4gb limit and your explained workaround is a pain to say the least, that's not a good solution and what's your answer to us all waiting for it since this thread opened in 2016 when will you guys address and fix this issue, at this point I really do not care how complicated it is, your engine is 64bit and with that all file systems should be as well, this is just not acceptable from a developer point of view. We don't need a workaround but an ETA when this will be addressed and fixed. Split the resource files even more like it was mentioned before during the build process. We needed a solution the moment you labeled your engine 64bit.

 

@karl_jones Asset Bundles, Addressables, I don't think anyone cares which provisionary solution Unity proposes, but can the process at least be automated?

 

This is just terrible!! Even worse now, build report tool ”Total User Assets” well what the hell counts as that? Everything just popped and took even more space? .. None other game engine have some file limits where you need to starts hassling about stuff you dont understand.. Is it really that hard to make it 8gb or even just remove the limit? What is the problem anyway?

Still splitting the scenes and streaming is the best solution!

 

They're working on a fix. It just seems to take longer than expected:
https://forum.unity.com/threads/bug...n-standalone-build.441116/page-2#post-3364498

It's explained here:
https://forum.unity.com/threads/bug-4gb-limit-to-textures-in-standalone-build.441116/#post-2852657

 

Better to gather some guts and do it, because alternative - put a head in the sand or rely on "for programmers by programmers" artists-unfirendly hacks like we have now - leads Unity engine into nowere IMHO. My personal case - I can't upgrade my asset in store ("Real Materials Complete") because of this limitation...

 

hmm, this dosent work...Cant assign anything to the prefabs to hold slot ..

 

Ah okay I see yes, theres alot of work to make it happend! Really appreciate it =) Adnvanced multi scene is something I need to use now and streaming ... but I hit another issue that the File Header takes alot of space, what is that ?

 

when I build, so the File Header takes 38% of the build? What is File Header?

 

hmm, I dont know .. I have only 16 x 16 km terrain in my scene and vegetation studio pro...

 

Easiest way to find out is to fire up a memory profiler and check where it comes from.

Unity's new Memory Profiler (2018.3 and higher)
https://forum.unity.com/threads/new...ew-package-available-for-unity-2018-3.597271/

Unity's old Memory Profiler (5.3 - 2017.4?)
https://forum.unity.com/threads/weve-started-work-on-a-new-memory-profiler.358963/

Heap Explorer (2017.4 and higher)
https://forum.unity.com/threads/wip...ofiler-debugger-and-analyzer-for-unity.527949

 

aaahh perfect! Does it work in build? Didnt find any ”how to” on that, because I need to profile it in build, because even before I see my scene its all black and I get a text in the development console where it says I have exxceeded the 4gb limit...

 

Found my issue... I splitted my 16x16k terrain to 2ks terrain tiles.. And made them all 2k on both texture and height resolutions... That was File Header Size build up..

 

and the memory profiler froze unity..

 

I'm curious if the problem is 4gb limit why can't a second resources folder 3rd fourth etc be implemented that shouldn't need any major restructuring of the unity engine?

 

@karl_jones I am trying the addressable asset system, but running into a problem. I'll take some images so you can see what I'm talking about. I have a lot of textures in my game, and I intend to add even more, so reducing the number of textures isn't an ideal answer for me, but rather finding ways that Unity can support large numbers of textures. My build report is as follows:

Build Report
Uncompressed usage by category (Percentages based on user generated assets only):
Textures --> 6064.1 mb 88.6% <---
Meshes 103.2 mb 1.5%
Animations 13.5 mb 0.2%
Sounds 1.8 mb 0.0%
Shaders 98.5 mb 1.4%
Other Assets 12.0 mb 0.2%
Levels 92.4 mb 1.4%
Scripts 2.3 mb 0.0%
Included DLLs 3.9 mb 0.1%
File headers 450.8 mb 6.6%
Total User Assets 6842.6 mb 100.0%
Complete build size 1174.9 mb

I have arranged most of my scene asset into addressable asset folders per the 'Trash Dash' video. Here you can see the folders.
https://gyazo.com/6ddaa6c4651e175771d9d6f36376de9f

Here you can see the file sizes in my build directory addressables folder: D:\ProjectH\Builds\Build01\Test_Data\StreamingAssets\com.unity.addressables\StandaloneWindows64


https://gyazo.com/35e094a6e1cbd3cecec5b025933abc46

So that looks great, the files are all under 4gb. However, my project still has the same size of the sharedasset files as you can see here:


https://gyazo.com/272d9854ad369a11a415c175047864fd

I am still encountering the texture glitch bug where my textures are bugging out (the thing that made me and others notice this 4gb limit in the first place). Perhaps I'm using the addressables system incorrectly. I followed the information that was available to me. Any ideas on why the build continues to generate the large sharedassets file? Why has sharedassets1.assets.resS stayed the same size despite reorganizing into addressables? Is this where all the shader variations are compiled? My project seems to generate tons of shader variants, which, from reading on several forums, is normal behaviour. I barely use the Resources folder, there is nearly nothing in there, and I'm using addressables as recommended. I am at my wits end here.

 

Ok, i will ask there. All i have done is place the prefabs into my scene and I'm loading the Scene using a simple load level call. It is all just static level geometry. There are no script calls to load anything. I assumed having prefabs in the Scene serves the same function.