Search Unity

  1. Unity 2020.2 has been released.
    Dismiss Notice
  2. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Bug: 4GB limit to Textures in standalone build

Discussion in 'General Graphics' started by ShilohGames, Nov 14, 2016.

  1. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    946
    I have a feeling I won't be able to explain it better to you than a Unity representative along with 6 pages of people discussing the issue.
    Exactly, yes.
    You summed it up right.
     
    Fenixake likes this.
  2. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    946
    To clarify it more: the 4gb limit is for the total size of your assets once loaded in memory on a single scene. (not the file size but the size it takes when the file is loaded. Could vary.)
    This restriction shouldn't affect textures, since they are loaded in the video memory, but rather any other asset in your library: 3d models, scripts, audio...

    -CORRECTION: this does affect textures too, since the problem is with the actual asset files that unity generates that can't be larger than 4gb because the way they are indexed internally. But keep in mind the files size of an original asset like a texture is not the same as the final size it will have in the build, since that depends on the actual compression applied by unity.
    Thanks Peter77 for the clarification.
     
    Last edited: Mar 12, 2020
    Fenixake likes this.
  3. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,477
    The problem is that a single file, generated by Unity during a build, must not exceed 4gb in Unity 2019.3/.4 and earlier versions.

    The project size doesn't really matter, unless you don't split up your content, which I explained here:
    Unity 2020.1 lifts this limitation:
    https://forum.unity.com/threads/bug...n-standalone-build.441116/page-5#post-5300130

    In Unity 2020.1+, 4gb or larger files should produce a valid build as well. If you don't want or need the benefits that come with splitting up content as described in the linked posts, 2020.1 should work regardless.
     
    atomicjoe likes this.
  4. Fenixake

    Fenixake

    Joined:
    Jan 25, 2014
    Posts:
    139
    Thank you!. I have a better understanding of it now!.But this is something that I should concider as a drawback, I would better spend that half of year learning unreal engine and acually make game that has some decent optimizations out of the box and stop updating unity everytime I need a fix for a package that ends up not solving it but completly waisting my time in the process of trying it!?!.I feel like unity never had an update to 5.6 other then the ui skin rework........ .
     
  5. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    946
    If you're starting game development and you're more art inclined than coding, I would suggest you to go to Unreal Engine.
    In it's current state, I can't recommend Unity to anyone just starting now.
    Maybe that will change in one or two years. Maybe everything will be wonderful when Unity 2020 releases, but no one knows for sure.
    Unity is still my preferred game engine and I love it, but the days of indie friendliness ended a long time ago.
    Have you seen the movie "The Money Pit" (1986) ?
    Because that's EXACTLY what developing with Unity feels right now.

    The thing is, like in the movie, at the end you can end up with something really wonderful.
    But be prepared for some MAJOR effort to make it happen.
    Also, another VERY important thing from Unity is THE COMMUNITY.
    Both in the sense of people like us that will tell you what's wrong and will help you to manage it, and in the sense of the Unity Asset Store, were you will be able to buy EVERYTHING for your game.
    Unreal doesn't have that AT ALL.
     
    radiantboy, BradZoob and Fenixake like this.
  6. Fenixake

    Fenixake

    Joined:
    Jan 25, 2014
    Posts:
    139
    Yes but Unreal has more AAA games then Unity xD I think that is what matters in the end.The capability of an engine to give a AAA game and unity lacks that (exept maybe mobile(UE has aswell fortnite and pubg mobile but not that many mobile games) ).

    Edit: And one more thing to mention, If the unity devs work so slow on some of the tools they make maybe they have been preparing for covid19 , 3 years ago?. Kappa(a small joke)
     
    Last edited: Mar 12, 2020
  7. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    4,456
    Have you tried 2020.1? This versions should fix the issue for you.
     
  8. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,477
    If you work on such game, you normally have experts that can handle complicated problems, asset bundles and related limitations become a breeze.
     
    atomicjoe likes this.
  9. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    946
    I'm not in the AAA business anyway so it's fine for me. lol
     
    Fenixake likes this.
  10. Fenixake

    Fenixake

    Joined:
    Jan 25, 2014
    Posts:
    139
    Karl what is the point of working on a beta version?. I mean yes for testing and nothing else but since I am 1 man do it all I started to feel the need to change something else not the unity version everytime.Unity has become from "the tool that just works" to the tool "that every upgrade brakes things that shoudn't". Like it was with the standar particle shader in 2017, I managed to make them work in 5.6 but unity devs never bothered to give a version of the shaders to the old 5.6.

    If i could afford to pay 1k$/month to an expert programmer I woud not bother coming here and complain or ask about it.


    Edit: Like someone said "stop upgrade, just create",on a post about having enough of poor unity updates.
     
  11. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    4,456
    The point is that 2020.1 lifts the limitation that you are having issues with. You won't need to create a workaround or change your workflow. If your project is still in early days then you can switch to 2020.1 and settle on the LTS as you get closer to release.
    If upgrading is breaking your project then please file a bug report.
     
  12. BradZoob

    BradZoob

    Joined:
    Feb 12, 2014
    Posts:
    51
    Just fired up 2020.1b1, this is huge leaps better at least in terms of aesthetic and navigation, those millisecond hangs really grind you down in Unity and can also cause you to slip and dump things into the wrong directories, but this is far more responsive and BIG and SHINY.
     
  13. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    946
    Is the editor really different than 2019.3.4 though?
     
  14. BradZoob

    BradZoob

    Joined:
    Feb 12, 2014
    Posts:
    51
    My last version of 2019.3 i had time to watch crash was 2019.3.0f6 according to HUB, so was all the stuff in 2020.1b1 also in 2019.3.4? This is definitely the most responsive i recall ever seeing Unity in 8 years. And then i loaded it up with a billion trees and made it slow again lol.
     
    Fenixake likes this.
  15. BradZoob

    BradZoob

    Joined:
    Feb 12, 2014
    Posts:
    51
    OK just installed 2019.3.5 to see and yeah that one is still in "old mode", 2020.1b1 is wildly different my dude.
     
    atomicjoe likes this.
  16. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    946
    Cool. I'm not touching a Unity beta with a stick though.
    Regular releases are buggy enough for me.
     
  17. BradZoob

    BradZoob

    Joined:
    Feb 12, 2014
    Posts:
    51
    I hear ya, still it's really uplifting to see after 8 year and 50 odd installs from 4.x to now. I installed Unreal recently to see how they were doing and by god i swear they're still in the equivalent of 4.x.
     
    atomicjoe likes this.
  18. truefx001

    truefx001

    Joined:
    Jan 30, 2013
    Posts:
    126
    I have to say GREAT JOB UNITY TECHNOLOGIES with the 2020.1.beta still a bit glitchy with some UI in Editor errors that cause a endless loop or even in game,(have to research this one a bit more) and 2 times just crashing on me but the 4GB limitation is gone, I was able to upgrade my MEGA 320GB game from 2017.4 gradually so version by version until 2020.1.4b , if you try to update from 2017 or even 2018 it will crash the update at least this was my experience. But I was a big critic about the hole 4GB thing and now using 2020.1 is amazing. So thank you for that version very happy so far @karl_jones
     
    radiantboy, karl_jones and atomicjoe like this.
  19. radiantboy

    radiantboy

    Joined:
    Nov 21, 2012
    Posts:
    1,141
    I wish I could upgrade past 2020.1.0a12 (which doesnt seem to have the 4 gig fix and has quite a few annoying bugs, but IS usable) but if I do my project always crashes when I tap the gizmo button or play. Unity will not look into it unless I provide my entire game to them, which would be a 200gig .rar file. Seems awkward to send and also worrying to send all my work to them. But im close to working out a way due to desperation, I wonder if you could help somehow @karl_jones ?
     
  20. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    4,456
    Does the crash happen in a smaller or new project?
    Do you have a crash dump? A crash dump may help to see where it's crashing but won't necessarily tell us why.


    It's not really that we won't look into it but more we can't. If we don't have a reproduction or some way to identify why it's crashing then there's not a lot we can do. There's so many different things that could go wrong entering playmode or tapping the gizmo button.
    We have ways to accept large project bug reports and they are safe with us.
     
  21. truefx001

    truefx001

    Joined:
    Jan 30, 2013
    Posts:
    126
    @radiantboy why don't you upgrade to 2020.1.b7 not the alpha one?
     
  22. radiantboy

    radiantboy

    Joined:
    Nov 21, 2012
    Posts:
    1,141
    I wish I could but as I said, anything after my current version crashes when gizmos is tapped or if I hit play.
     
  23. DavidBVal

    DavidBVal

    Joined:
    Mar 13, 2017
    Posts:
    134
    To anyone reading this: ccsander idea works like a charm. It takes you a few minutes to set up and doesn't require you to touch anything in your current scenes or assets. This man deserves a round of applause.
     
    Fenixake and atomicjoe like this.
  24. Fenixake

    Fenixake

    Joined:
    Jan 25, 2014
    Posts:
    139
    This one?.
     
  25. DavidBVal

    DavidBVal

    Joined:
    Mar 13, 2017
    Posts:
    134
    Yes, post #21 is currently the best solution to this issue.
     
    Fenixake likes this.
  26. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    4,456
    Is this because you are unable to use 2020.1.beta or you prefer this solution over the changes we made?
     
  27. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    946
    Will these changes be ported to Unity 2019 LTS ?
    Because otherwise we don't have any other option.
     
  28. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    4,456
    No definitely not! This is a feature and too big to backport safely.
     
  29. DavidBVal

    DavidBVal

    Joined:
    Mar 13, 2017
    Posts:
    134
    We definitely can't risk using a Beta at this time.
     
  30. truefx001

    truefx001

    Joined:
    Jan 30, 2013
    Posts:
    126
    @DavidBVal I hear you, but try a backup of your game with 2020.1 it is pretty stable, I'm getting annoying Checking Changes even with disabled and uninstalled Collaboration but play mode, build speed, memory consumption, and the 4GB limit fix are amazing, I would not have used a beta ever for publications but with this one, my game on Steam is pretty stable and is performing great. A very large project over 300 GB, in 2017.4 we had 10GB+ memory consumption now 5-6GB. In my opinion, it's the best Unity version since 2013.
     
    glenneroo, Lex4art and karl_jones like this.
  31. DavidBVal

    DavidBVal

    Joined:
    Mar 13, 2017
    Posts:
    134
    Thanks for sharing that, especially coming from a large project (ours is as well). We'll consider an update in a few months.
     
  32. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    946
    Without changing anything in your project??
    What has changed in Untiy 2020.1 to make that ?
    Also, post a link of your game on Steam please, it's always cool to check big projects made in Unity. :)
     
    Lex4art likes this.
  33. truefx001

    truefx001

    Joined:
    Jan 30, 2013
    Posts:
    126
    @atomicjoe yeah without changing anything that would improve memory or speed, editor play mode very fast and it drove me crazy in 2017.4 waiting forever, building the game as well very fast, the first time I thought there must be an error but all my scenes have been build. What I did notice is the shader warning error 256 limit that I haven't had before. https://store.steampowered.com/app/745810/SAS/ this is the game that needed the 4GB limit fix so now I can add and have added way more game objects and no issues with performance, I have streamers that play the game and they noticed the performance improvement, I have way more trees and grass. I cannot tell you what changed in Unity 2020.1 I just know my metrics and I'm very happy about it. Of course, you will have the here and there crash in Unity Editor but we all know this from all versions of Unity.
     
    Lex4art, atomicjoe and radiantboy like this.
  34. Macode

    Macode

    Joined:
    Nov 28, 2013
    Posts:
    18
    Hi, solution you guys came up with worked for me like a health potion on last HP.
    I'm having soft launch this week when all of a sudden workaround proposed here earlier stopped working for my project with 7 Gb of textures in one scene. I do use Addressables but still lots of work for a build to handle. Last night I downloaded 2020b.11 and although migration did not worked flawlessly, it's a smallest problem, at least it works now and plays even smoother, despite fear of beta version.
    Thanks you.

    Although now switching between PC<-> Linux took over 3 hours, I believe this time will be reduced on next platform switch, also could you suggest that feature that loads assets in editor only on demand so it will be working faster I believe? Thanks in advance.
     
  35. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    4,456
    Thats good. 2020.1 should be out of beta in a few weeks so hopefully you can release with a more stable version.

    This is something we have been working on for some time. Asset Database V2 was the first step towards it. I think there may even be a Unite presentation where we demo it from a few years ago.
     
  36. Unplug

    Unplug

    Joined:
    Aug 23, 2014
    Posts:
    248
    will unity fix version 2017 to split asset into different file for texture, asset, etc like you said ? because updating a projet from 2017 to 2020.1 is impossible. Not only it will require the entire code to be revisited and retested because of all the change, but also my asset have completely change between 2019 and 2020 which would also require a lot of redo to remove asset and reimport newer version...
     
  37. karl_jones

    karl_jones

    Unity Technologies

    Joined:
    May 5, 2015
    Posts:
    4,456
    No this is a huge change and would never be allowed into an LTS version. It also builds on features that are not in 2017.4 so backporting would be a significant task and not safe to do to an LTS. 2020.1 is the earliest version we can offer this feature.
     
  38. Unplug

    Unplug

    Joined:
    Aug 23, 2014
    Posts:
    248
    thanks for t he answer, so basically all indie studio developping for year on a stable version get abandonned in a situation where they can't publish and can't update... magnificient. Is there a way to get support or paid support to update a project/script in earlier version ?
     
  39. Rickmc3280

    Rickmc3280

    Joined:
    Jun 28, 2014
    Posts:
    110
    I honestly wanted to give 2020 a shot, but it is extremely buggy and slow on my PC (SSD/AMD 3900x/64gb Ram, 1080ti). Doing anything would cause a significant delay. Not sure if its an AMD thing or what. Def want the ability to use large files though :(
     
  40. Unplug

    Unplug

    Joined:
    Aug 23, 2014
    Posts:
    248
    you can still create object array and "cache" all the ressource needed while loading other scene such as splash screen or empty scene you load async. it work for me. ressources are loaded once, so it will help reduce your level size if your asset are loaded somewhere else. very annoying
     
  41. Rickmc3280

    Rickmc3280

    Joined:
    Jun 28, 2014
    Posts:
    110
    Yeah, Im still a big fan or a simple implementation that they could do.

    Simple in my opinion, maybe there is more to it?

    If file size > 4gb (static float specifying 4gb file size)...
    Split To ...

    Or... (say files are 40gb)

    File Size /4gb = (10 in this instance)

    10 Groupings. Prioritize file groupings by...whatever parameters from a drop down selection if doing it manually or have Unity add in their own priority parameters.

    If unity detects more than 1 grouping...

    Process Mats for Grouping X ...
    Process Objects/FBX for Grouping X
    Process Sprites for Grouping X
    Process PNGs for Grouping X
    etc etc.

    If File Size of individual object > 4gb

    Octree process based on # of groupings.
    If Groupings > 8 <=16
    {
    do whats needed with octrees
    }
    if Groupings > 16 < 24
    {
    do whats needed with octrees
    }

    Ill look into it more at a later date. I couldnt finish several of my projects so I've been writing my own system to handle these types of things that rely heavily on outside files (outside processors, outside file databases etc etc).
     
  42. Unplug

    Unplug

    Joined:
    Aug 23, 2014
    Posts:
    248
    Just create an array and put the object you want to cache. Put all the large object and espacially large texture

    Code (CSharp):
    1.     public Object[] Texture_Ground;
    2.  
    assets, images, prefab are only bundle once in the first level they appear so you can cache many file in your initial splash screen
     
    atomicjoe likes this.
  43. Deleted User

    Deleted User

    Guest

    Hi,
    Correct me if i'm wrong but, it looks like that, if we want to solve this problem by using the AssetBUndle, we need to re create completely our scene, by changing the textures, one by one, considering that we exceed the 4GB limit...

    But it's gonna take ages ?

    Cheers
    weex
     
    radiantboy likes this.
  44. Deleted User

    Deleted User

    Guest


    Hello Peter,

    Are you trying to say that this 4GB limit, is not longer a problem anymore, on the unity versions UPPER to 2019.3, and .4 ???
    May be I missunderstood this one...

    Cheers,
    weex
     
  45. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    7,375
    Yes, if you want to move away from the Resources folder, you have to completely change and complicate the way you organise your game.
     
    atomicjoe likes this.
  46. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    5,477
    Rickmc3280 and karl_jones like this.
  47. Rickmc3280

    Rickmc3280

    Joined:
    Jun 28, 2014
    Posts:
    110
    Is there somewhere that I can go to learn more about file structures of Unity Projects that could help with this type of problem? (for example like the "Editor" folder).

    For instance it seems that you are implying that if I use a "resource" folder that unity wont process the data the same?
     
  48. Rickmc3280

    Rickmc3280

    Joined:
    Jun 28, 2014
    Posts:
    110
    radiantboy likes this.
  49. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    7,375
    https://docs.unity3d.com/Manual/SpecialFolders.html
     
    Rickmc3280 likes this.
  50. Deleted User

    Deleted User

    Guest

    Thanks Peter77, for the confirmation.

    Its a shame that Unity reacted so lately... Because some works, are more stable in 2018, 2019 versions...
    To move the all project for us to a 2020 version, will need - however - a lot of work.

    Good luck to you all !
    weex
     
    Peter77 likes this.
unityunity