Bug: 4GB limit to Textures in standalone build

I have a feeling I won't be able to explain it better to you than a Unity representative along with 6 pages of people discussing the issue.

Exactly, yes.
You summed it up right.

 

To clarify it more: the 4gb limit is for the total size of your assets once loaded in memory on a single scene. (not the file size but the size it takes when the file is loaded. Could vary.)
This restriction shouldn't affect textures, since they are loaded in the video memory, but rather any other asset in your library: 3d models, scripts, audio...

-CORRECTION: this does affect textures too, since the problem is with the actual asset files that unity generates that can't be larger than 4gb because the way they are indexed internally. But keep in mind the files size of an original asset like a texture is not the same as the final size it will have in the build, since that depends on the actual compression applied by unity.
Thanks Peter77 for the clarification.

 

The problem is that a single file, generated by Unity during a build, must not exceed 4gb in Unity 2019.3/.4 and earlier versions.

The project size doesn't really matter, unless you don't split up your content, which I explained here:

• https://forum.unity.com/threads/bug...n-standalone-build.441116/page-5#post-4473202
• https://forum.unity.com/threads/bug...n-standalone-build.441116/page-6#post-5561050

Unity 2020.1 lifts this limitation:
https://forum.unity.com/threads/bug...n-standalone-build.441116/page-5#post-5300130

In Unity 2020.1+, 4gb or larger files should produce a valid build as well. If you don't want or need the benefits that come with splitting up content as described in the linked posts, 2020.1 should work regardless.

 

Thank you!. I have a better understanding of it now!.But this is something that I should concider as a drawback, I would better spend that half of year learning unreal engine and acually make game that has some decent optimizations out of the box and stop updating unity everytime I need a fix for a package that ends up not solving it but completly waisting my time in the process of trying it!?!.I feel like unity never had an update to 5.6 other then the ui skin rework........ .

 

If you're starting game development and you're more art inclined than coding, I would suggest you to go to Unreal Engine.
In it's current state, I can't recommend Unity to anyone just starting now.
Maybe that will change in one or two years. Maybe everything will be wonderful when Unity 2020 releases, but no one knows for sure.
Unity is still my preferred game engine and I love it, but the days of indie friendliness ended a long time ago.
Have you seen the movie "The Money Pit" (1986) ?
Because that's EXACTLY what developing with Unity feels right now.

The thing is, like in the movie, at the end you can end up with something really wonderful.
But be prepared for some MAJOR effort to make it happen.
Also, another VERY important thing from Unity is THE COMMUNITY.
Both in the sense of people like us that will tell you what's wrong and will help you to manage it, and in the sense of the Unity Asset Store, were you will be able to buy EVERYTHING for your game.
Unreal doesn't have that AT ALL.

 

Yes but Unreal has more AAA games then Unity xD I think that is what matters in the end.The capability of an engine to give a AAA game and unity lacks that (exept maybe mobile(UE has aswell fortnite and pubg mobile but not that many mobile games) ).

Edit: And one more thing to mention, If the unity devs work so slow on some of the tools they make maybe they have been preparing for covid19 , 3 years ago?. Kappa(a small joke)

 

If you work on such game, you normally have experts that can handle complicated problems, asset bundles and related limitations become a breeze.

 

I'm not in the AAA business anyway so it's fine for me. lol

 

Karl what is the point of working on a beta version?. I mean yes for testing and nothing else but since I am 1 man do it all I started to feel the need to change something else not the unity version everytime.Unity has become from "the tool that just works" to the tool "that every upgrade brakes things that shoudn't". Like it was with the standar particle shader in 2017, I managed to make them work in 5.6 but unity devs never bothered to give a version of the shaders to the old 5.6.

If i could afford to pay 1k$/month to an expert programmer I woud not bother coming here and complain or ask about it.


Edit: Like someone said "stop upgrade, just create",on a post about having enough of poor unity updates.

 

Just fired up 2020.1b1, this is huge leaps better at least in terms of aesthetic and navigation, those millisecond hangs really grind you down in Unity and can also cause you to slip and dump things into the wrong directories, but this is far more responsive and BIG and SHINY.

 

Is the editor really different than 2019.3.4 though?

 

My last version of 2019.3 i had time to watch crash was 2019.3.0f6 according to HUB, so was all the stuff in 2020.1b1 also in 2019.3.4? This is definitely the most responsive i recall ever seeing Unity in 8 years. And then i loaded it up with a billion trees and made it slow again lol.

 

OK just installed 2019.3.5 to see and yeah that one is still in "old mode", 2020.1b1 is wildly different my dude.

 

Cool. I'm not touching a Unity beta with a stick though.
Regular releases are buggy enough for me.

 

I hear ya, still it's really uplifting to see after 8 year and 50 odd installs from 4.x to now. I installed Unreal recently to see how they were doing and by god i swear they're still in the equivalent of 4.x.

 

I have to say GREAT JOB UNITY TECHNOLOGIES with the 2020.1.beta still a bit glitchy with some UI in Editor errors that cause a endless loop or even in game,(have to research this one a bit more) and 2 times just crashing on me but the 4GB limitation is gone, I was able to upgrade my MEGA 320GB game from 2017.4 gradually so version by version until 2020.1.4b , if you try to update from 2017 or even 2018 it will crash the update at least this was my experience. But I was a big critic about the hole 4GB thing and now using 2020.1 is amazing. So thank you for that version very happy so far @karl_jones

 

I wish I could upgrade past 2020.1.0a12 (which doesnt seem to have the 4 gig fix and has quite a few annoying bugs, but IS usable) but if I do my project always crashes when I tap the gizmo button or play. Unity will not look into it unless I provide my entire game to them, which would be a 200gig .rar file. Seems awkward to send and also worrying to send all my work to them. But im close to working out a way due to desperation, I wonder if you could help somehow @karl_jones ?

 

@radiantboy why don't you upgrade to 2020.1.b7 not the alpha one?

 

I wish I could but as I said, anything after my current version crashes when gizmos is tapped or if I hit play.

 

To anyone reading this: ccsander idea works like a charm. It takes you a few minutes to set up and doesn't require you to touch anything in your current scenes or assets. This man deserves a round of applause.

 

This one?.

 

Yes, post #21 is currently the best solution to this issue.

 

Will these changes be ported to Unity 2019 LTS ?
Because otherwise we don't have any other option.

 

We definitely can't risk using a Beta at this time.

 

@DavidBVal I hear you, but try a backup of your game with 2020.1 it is pretty stable, I'm getting annoying Checking Changes even with disabled and uninstalled Collaboration but play mode, build speed, memory consumption, and the 4GB limit fix are amazing, I would not have used a beta ever for publications but with this one, my game on Steam is pretty stable and is performing great. A very large project over 300 GB, in 2017.4 we had 10GB+ memory consumption now 5-6GB. In my opinion, it's the best Unity version since 2013.

 

Thanks for sharing that, especially coming from a large project (ours is as well). We'll consider an update in a few months.

 

Without changing anything in your project??
What has changed in Untiy 2020.1 to make that ?
Also, post a link of your game on Steam please, it's always cool to check big projects made in Unity.

 

@atomicjoe yeah without changing anything that would improve memory or speed, editor play mode very fast and it drove me crazy in 2017.4 waiting forever, building the game as well very fast, the first time I thought there must be an error but all my scenes have been build. What I did notice is the shader warning error 256 limit that I haven't had before. https://store.steampowered.com/app/745810/SAS/ this is the game that needed the 4GB limit fix so now I can add and have added way more game objects and no issues with performance, I have streamers that play the game and they noticed the performance improvement, I have way more trees and grass. I cannot tell you what changed in Unity 2020.1 I just know my metrics and I'm very happy about it. Of course, you will have the here and there crash in Unity Editor but we all know this from all versions of Unity.

 

Hi, solution you guys came up with worked for me like a health potion on last HP.
I'm having soft launch this week when all of a sudden workaround proposed here earlier stopped working for my project with 7 Gb of textures in one scene. I do use Addressables but still lots of work for a build to handle. Last night I downloaded 2020b.11 and although migration did not worked flawlessly, it's a smallest problem, at least it works now and plays even smoother, despite fear of beta version.
Thanks you.

Although now switching between PC<-> Linux took over 3 hours, I believe this time will be reduced on next platform switch, also could you suggest that feature that loads assets in editor only on demand so it will be working faster I believe? Thanks in advance.

 

will unity fix version 2017 to split asset into different file for texture, asset, etc like you said ? because updating a projet from 2017 to 2020.1 is impossible. Not only it will require the entire code to be revisited and retested because of all the change, but also my asset have completely change between 2019 and 2020 which would also require a lot of redo to remove asset and reimport newer version...

 

thanks for t he answer, so basically all indie studio developping for year on a stable version get abandonned in a situation where they can't publish and can't update... magnificient. Is there a way to get support or paid support to update a project/script in earlier version ?

 

I honestly wanted to give 2020 a shot, but it is extremely buggy and slow on my PC (SSD/AMD 3900x/64gb Ram, 1080ti). Doing anything would cause a significant delay. Not sure if its an AMD thing or what. Def want the ability to use large files though

 

you can still create object array and "cache" all the ressource needed while loading other scene such as splash screen or empty scene you load async. it work for me. ressources are loaded once, so it will help reduce your level size if your asset are loaded somewhere else. very annoying

 

Yeah, Im still a big fan or a simple implementation that they could do.

Simple in my opinion, maybe there is more to it?

If file size > 4gb (static float specifying 4gb file size)...
Split To ...

Or... (say files are 40gb)

File Size /4gb = (10 in this instance)

10 Groupings. Prioritize file groupings by...whatever parameters from a drop down selection if doing it manually or have Unity add in their own priority parameters.

If unity detects more than 1 grouping...

Process Mats for Grouping X ...
Process Objects/FBX for Grouping X
Process Sprites for Grouping X
Process PNGs for Grouping X
etc etc.

If File Size of individual object > 4gb

Octree process based on # of groupings.
If Groupings > 8 <=16
{
do whats needed with octrees
}
if Groupings > 16 < 24
{
do whats needed with octrees
}

Ill look into it more at a later date. I couldnt finish several of my projects so I've been writing my own system to handle these types of things that rely heavily on outside files (outside processors, outside file databases etc etc).

 

Just create an array and put the object you want to cache. Put all the large object and espacially large texture

Code (CSharp):

1.     public Object[] Texture_Ground;
2.  

assets, images, prefab are only bundle once in the first level they appear so you can cache many file in your initial splash screen

 

Hi,
Correct me if i'm wrong but, it looks like that, if we want to solve this problem by using the AssetBUndle, we need to re create completely our scene, by changing the textures, one by one, considering that we exceed the 4GB limit...

But it's gonna take ages ?

Cheers
weex

 

Hello Peter,

Are you trying to say that this 4GB limit, is not longer a problem anymore, on the unity versions UPPER to 2019.3, and .4 ???
May be I missunderstood this one...

Cheers,
weex

 

Yes, if you want to move away from the Resources folder, you have to completely change and complicate the way you organise your game.

 

The 4gb limit is no longer a problem in 2020.1 and newer:
https://forum.unity.com/threads/bug...n-standalone-build.441116/page-5#post-5300130

 

Is there somewhere that I can go to learn more about file structures of Unity Projects that could help with this type of problem? (for example like the "Editor" folder).

For instance it seems that you are implying that if I use a "resource" folder that unity wont process the data the same?

 

I would like to try 2020, however, everytime that I've used it it causes my entire PC system to lag. For instance Google Chrome will lag if Unity is open (empty project in 2020). I will wait until they fix these types of issues.

 

https://docs.unity3d.com/Manual/SpecialFolders.html

 

Thanks Peter77, for the confirmation.

Its a shame that Unity reacted so lately... Because some works, are more stable in 2018, 2019 versions...
To move the all project for us to a 2020 version, will need - however - a lot of work.

Good luck to you all !
weex