Official Feedback Wanted: Global Texture Import Size/Compression Overrides

1. Yes useful to have!
2. Yes or project settings
3. Extra option - Only compress textures used in build ( not all in project folder )

I would use the faster way during development and only compress at full slow speed quality when starting the optimization phase.

 

Yes. Uncompressed but half res would be useful too (I guess uncompressed and something like max 512 is fine). The use case is "I just want to make a build NOW to check something unrelated / I need to show something at a presentation, and half of my textures aren't compressed yet". It happens surprisingly often.

A Warning in the console when you're building with those overrides would be also prudent, since you also probably don't want to release something with these settings.

(and / or make sure we can turn this off by code so when we click our "Prepare for final build" button we can make sure these are turned off)

Yes.

Another dropdown where I can override the compression format for iOS (like you can for Android).

 

Currently in quality settings there is also the option the half the texture resolution, just to keep in mind that it may be better to have all in one place.

Perhaps a new quality setting for fast exports?

 

I like the "force fast compressor" option idea, as it only effects quality. The problem with setting a max texture size it is may kill LUT tables, and break all kinds of things that use them and expect them to contain exactly sized data. (This is, btw, one of my main issues with all of unity's texture scale settings, and why I never use them - there are issues with MicroSplat that happen if the user sets the project to half res textures and try's to pack the texture array- it ends up having one less mip with white in the last mip map). Could a texture have an "do not resize" flag? That would fix that issue both here and for my current use cases as well.

Ideally Unity would be able to do things like this in sweeps on a background thread- so bring in all textures as fast as possible as a first pass, then go up a mip or two and do a second pass. The real problem here is the blocking nature of the editor, if this was fixed I wouldn't care if all textures were standard checkerboard patterns until compression was done, but I realize that's a larger task. But it is the right one to solve.

 

I believe the one in Quality setting still adds the full res texture in the build (and ram at runtime), but forces everything to ignore LOD0 and start at LOD1. (unless something changed)

 

Yes good good points Jason!
The blocking is what stops us from doing any work during the compression stage.

@AcidArrow Thanks I did not know that

 

As an example lets say I have a project with 100 GB uncompressed textures coming from various assets packs in order to kitbash together a prototype.

Lets say I made a scene that uses 1 GB of textures coming from these packs.

I set compression at import to none.

What is taking the most time:

1. compressing 100 GB of textures to a format that I will not use in the final
build ( because default texture compression settings )

2. Manually setting the right compression format for each texture

3 Blocking the editor during compression

4 Finding that one texture that is set at 8k exr and eating up all the storage in the build.

@Peter77 Had a nice tool for 2 and 4, texture overview. I wish Unity would make a texture overview tool native so we can easy change many texture settings at once and easy sort by properties.

 

+All the textures from demo scenes in downloaded assets that I have spent so much time compressing over the years and they have never made it in a build.

I do think this is an Asset Database feature/issue though, which is probably outside of the scope of what Aras is doing (and it's a bummer that we semi-recently had Asset Database v2 introduced, and really not that much has changed? all the issues and quirks are still there).

 

My main problem with texture compression is that it blocks the entire editor like Jason said. If it could be done in an asynchronous way that alone could be a massive workflow improvement for me.

I think if you guys had some sort of texture group settings similar to what Unreal has where you can specify global settings for the textures in that group would give us a little more control. Plus you could also do what you are proposing with the global settings you mentioned.

For example, if I wanted to set a certain texture mipmap bias I currently have to write an asset importer and somehow identify the specific textures I want to modify whereas if I have a centralized location to specify a configuration for a group of textures kinda solves the problem neatly.

 

Actually just the copy into the library folder is enough to make this time consuming.

Having worked on projects with 18 hour import times (on a meticulously clean data tree, mind you), and contracted with companies with 8-12 hour import times, I can honestly say that in my opinion making this not a blocking operation is the #1 quality of life improvement that could be made to Unity. I hate having to charge someone a day's work because my computer is just sitting there importing a project filled will assets that they aren't even using, and I hate even more trying to convince the team that it's worth their time to clean up their project instead of focusing on whatever they are trying to get done. (Though to be honest I'd still be doing that if texture import times were not an issue).

 

In theory one day some combination of UAS rule changes and the package manager would allow us to move those outside of the main packages, making them optional, which would be nice to decrease this burden.

To be fair, this is because mipMapBias is not exposed on the Texture Import settings, which just seems like a funny oversight.

But yes, Unity's texture importing/scaling stuff is anemic for most real world cases, as it will destroy LUTs and lacks any kind of fine detailed control. So no real project I've worked on has ever used it, and I think a lot of the hobby developers just use it because they don't understand performance and think they should half the texture res of everything "to increase framerate". It basically treats mobile as if we're all on iPhone 1's or 2's, not the wide ranging ecosystem it is today.

So what happens to a 2048 texture which is set to max1024 with these new settings, and half res textures in the quality settings? Does it compress a 1024 version then drop the highest mip for rendering? What does its .mipMapCount and width/height return? I do a lot of texture packing in MicroSplat, and the texture size settings are on my configs, so it would scale the input to 2048 and output a texture array at that size (in a scriptable object, mind you, not the new texture array stuff added to 2021). Ideally, I'd prefer if my editor code could work with the original, uncompressed texture data, not from the data in the library folder. Then I would be unaffected by any of these settings, which can already cause issues that are hard for users to reason about.

 

True enough but that detail has always bugged me so much and complicated things more than it should have.

 

This drop down has plenty of space, but it could also be part of Advanced Project Settings?



Also this is slightly off topic, but the "Compress Assets on Import" preference should be changeable from the hub as well, since if it's On, I either have to suffer the whole project compressing, or I have to create a new dummy project (or have one around), go to preferences, close that, open the project I actually want.

 

Yes it is a welcome option!

 

I haven't tried on Metal, but bias definitely works on GL ES 2.0 and above (not sure about 1.0, and I'm fairly sure it works on metal, but I haven't actually tried it). Also... maybe on platforms that the settings does nothing, then it does nothing, not sure why that prevents something from being in the importer

 

A "nice to have" feature would be if using Force Uncompressed would use a dynamically calculated per texture max size to prevent exceeding the memory usage of the intended compressed settings for that texture. So, for example, a 512x512 DXT5 or ASTC 4x4 would end up as a 256x256 RGBA32, 512x512 DXT1 or (RGB) ASTC 6x6 would be a 128x128 RGB24. Not perfectly like for like, but on memory constrained platforms or projects pushing limits, this would be exceedingly more useful than global clamps. Especially since it wouldn't impact any already uncompressed textures. Those generally don't require significant import times (ignoring convolved cube maps, which maybe need a separate option?).

 

I guess my vote then would be to add the texture compression quality option and not the max size option, if anything at all. I guess it depends on the timeline; I hate putting in stuff that's just going to go away, because I'm always working across 2-3 years of unity versions, and then you end up with tons of #if UNITY_X all over the place to handle stuff.

 

It would be nice to have some way to set this as a profile. One thing that's important for me is to make it easy to kick out a build with production settings, and not have to re-make the build because you forgot you had the wrong texture settings.

This might be a larger change, though. It'd just be nice to be able to define a build profile that had arbitrary modifications to Project Settings.
- Debug (Mono-512textures)
- Production (IL2CPP-maxcompression)

 

One thing I'd like is a "Force Disable Crunch" option. In a large project I work on we've gone through multiple project cycles where we turn crunch compression on for several thousand textures when we first start making final release builds for testing "1.0" of that game, and then we turn it back off when starting production of the next iteration of the game because import times would skyrocket if we left it on outside of the weeks/months of bug fixing time. We also don't want texture quality to be worse in the editor than when we ship the game so we probably wouldn't use a 'force fast / low quality compression' option.

It sounds like this sort of override feature would be a good fit for that situation, where we could just turn it off without messing with thousands of texture import settings / meta files - ideally in an 'override settings' file that can be put into version control along with the rest of the project.

Maybe there could be a project-wide override and then a user-specific override which takes precedence, similar to the Cache Server / Accelerator enable settings.

 

This suits me.

 

To be honest, marginally. Considering the first time you're importing any large project is going to take a lot of time anyway its not really for that case, we suggest everyone does that overnight since its a one-time-thing.

So the use case is when a lot of textures are comitted, which for us at least doesnt really happen all that often. An artist might commit a lot of textures at once initially, and then iterates on one texture individually.

I personally dont think i spend that much time waiting for texture imports after updating SVN, there are plenty of other, imo more urgent, bottlenecks. Such as the long time it takes to enter playmode, script compilation being slow, general startup times etc. I do these actions far more often than i have to import textures so even a smaller gain there has more impact. Luckily a lot of that is being worked on.

 

For us, the time this would be useful is when we get somebody external on the project (usually sound person) that needs to open it to work on something that's unrelated to graphics. In that case, it would be nice if they could specify on their first import that they're fine with low quality textures.

It gets old to tell them "hey, you need to leave your laptop in a useless state for like 12+ hours in order to be able to work on our project".

That being said, that stops being a problem once on-demand imports of assets becomes a thing, so the usefulness of the feature very much depends on how long it takes before on-demand imports arrive.


Would this be relevant for builds at all? I guess if you're creating a build for a different platform?

 

This would only be useful if it is a big time saver.
As I understand for example in our case Texture importer is bottlenecked by the PNG reader.

But for people who bottlenecked by the texture compression the best would be a mode that:
- Has a simple checkbox (on/off)
- Doesn't change the texture size (that can break things)
- Doesn't change the texture format (that can break things)

What it could change is:
- Fall back to the fastest resize/mipmap filter available
- Disable crunch (as somebody already suggested above)
- Use a super fast version of the texture compressor

All texture compressors can be sped up at least an order of magnitude if you don't want the best quality.
Most of these should be already available in libraries or relatively trivial to write.

 

Just found this on PNG decoding speed:
https://github.com/putara/pngbench
Not sure how useful this is for Unity as you need a cross platform solution (I only use the editor on Windows so a Windows only speedup would be very welcome...)

 

It will be good to have texture format override per platform, e.g. on iOS and Android I want all textures to be ASTC compressed not ETC or PVRTC. Option to override compression from something to ASTC (without touching uncompressed textures, just touch compressed with another format)

 

Yes it need for iOS too and for android I can only set ASTC but can not set witch ASTC to use, and better to have few options like for ui use ASTC 4x4 for everyting else ASTC 6x6

Another thing is to have one format for all Arm platforms (ASTC) and another one for all other platform (DXT) just because today many project want to use ASTC LDR for all ARM platforms (dont want to set same texture setting for every platform) and may be even dont create build for older phones.

Minimal help can be just checkbox to use Android texture setting for iOS so we can just setup Android platform settings and iOS will take it.

 

I just need really high quality for some VR mobile devices which means more ASTC options. As I understand it, the current compressor / setup isn't as high quality as it could be...

 

Back when I was doing builds I couldn't specify the block sizes I wanted and ended up with a slightly uglier image vs the standalone compressor. This was a year ago and much could have changed. I will get back to you once I've done current-day tests.

Anyone else chime in with ASTC?

 

I've had it crash Unity a lot on the mac, at least when compressing Texture Arrays. But this reminds me, now that there is a proper texture array workflow coming, is it possible to compress into a texture array and not have unity complain about the format? Essentially, the old workflow is to compress texture arrays and save them into a scriptable object- and if your, say, targeting a platform that uses ETC2, you'll get a massive stream of errors complaining that the format is not supported - because it isn't for the PC your compressing them on.

 

ok, let me give the complete picture. MicroSplat takes a list of textures, extracts all the data from them and then packs them into custom layouts stored in texture array scriptable objects. Essentially, most users don't understand texture packing, and the best formats are rather funky anyway, so I handle it all for them. This has a few issues. Imagine we’re on Android and compressing to ETC2:

1. since the editor is on windows, the textures are likely in DXT format. So we end up loading the DXT compressed source instead of an uncompressed texture, with all the DXT crappiness.
2. After we create the texture in memory, we call the editors compress function to compress it into ETC2 format. Since the editor is on windows, it throws a warning about not supporting ETC2 for every texture we are creating as we copy them into the arrays.


So basically, I’d love a way around both of these, so I could get higher quality source data without having to muck with the import settings of 100 textures, get access to the higher compression/mip map options of the importer, and also not spam the console with warnings. Preferably I also wouldn’t litter the project with a bunch of files, either, and would properly handle storing this data in the library in platform format.

 

@Aras I just saw this feature come up in Unity 2021.2.0a11, not sure if I should be giving feedback here, or if the experimental forum would have been more appropriate.

In any case, it seems like a desired feature for larger projects for a variety of reasons and I can imagine a few cases where it will be useful for me as well.

However, I think there is also a risk here. There are already quite a few methods of overriding or editing import settings of assets, and in its current implementation this is yet another version. This could make it harder to understand for users why the asset is imported as a certain size. I suggest at least making it visible in the asset importer inspector that such an override is enabled, otherwise I guarantee that we'll be chasing ghosts every now and then trying to figure out why an asset isn't imported as its displayed to be.


screenshot for context, actual image is 512x512:

 

Man, this thread is so exciting to follow. Awesome job so far.

PS. Are you planning to allow changing these settings from project's Advanced Project Settings page in Unity Hub or only planning to add command line arguments to change these parameters from outside?

PPS. I'm mostly interested in "Force Uncompressed" option which is a per-project version of "Compress Assets on Import". Is there a way to achieve the "Force Uncompressed" behaviour on older Unity versions (2019 LTS) without setting "Compress Assets on Import" to true (i.e. without affecting other projects)? I'm OK with pretty much anything, be it reflection, AssetPostprocessor and etc.