Improving performance of 3D textures (using texture arrays?)

I have a bunch of volume rendering (ray marching) shaders in my scene which use 3D textures as their input. Right now the game runs fine when the meshes with these volume rendering shaders are small/far away but as they take up more of the FOV (as you approach them) the fps drops dramatically to unarguably unplayable lows. I changed the texture format of the 3D texture to "BC4" (compressed, 4 bit, single channel) and it lead to a doubling of fps which leads me to believe the problem is with the memory bandwidth.

Unfortunately this optimization isn't sufficient, but the 3D textures don't seem to give nearly as many compression and performance/fidelity tuning options as 2D textures (they don't have a "compression quality" setting for instance)... My thinking was to use 2D texture arrays in place of the 3D textures in my ray marching Shaders to to preserve the added control and compression that Unity offers 2D textures... Could this work? Are there any other ways to reduce the size of 3D textures besides just changing their texture format? Thanks!

 

Also changing texture resolution in QualitySettings doesn't seem to result in any performance changes... Is this because 3D textures aren't effected by Quality Settings? Or I just don't know the real cause of the fps drop.

 

If changing to BC4 helped, then yes, it's probably a memory bandwidth issue.

As far as I know, a Texture2DArray and Texture3D are actually the same thing as far as a GPU is concerned. The difference is a Texture2DArray is a Texture3D without blending between the layers. Each layer is still stored as a 2D texture though either way.

Unity doesn't support directly importing 3D textures, only constructing them from script, so if you have something that's doing that it's not an official Unity thing. But there's no reason why a 3D texture couldn't be any format Unity currently supports for 2D textures, it just requires modifying the asset when it's being built from script.

That said, I wonder what format your 3d textures currently are. If you're using alpha and targeting PC, your best option is DXT5, or really BC7. Both of them have the same memory usage. And both are twice the size of the RGB DXT1 or single channel BC4. Unfortunately there aren't any other options on PC that are any better that DXT1 or BC7 when it comes to RGB or RGBA textures. Basically if you want to reduce your memory usage, you'll need to shrink the textures yourself.

Quality settings work on any texture asset generated by the editor's existing importers. Since 3D textures are not something officially imported, they are not affected by Quality settings.

One way to go about that would be to have all of the slices of your 3D texture be normal 2D texture assets and construct / assign the 3D texture in script at runtime. That way you'll have more control over the format, and the existing quality settings will "just work". The downside is you have to manage all of those textures and make sure they stay the same size & format.

 

I think it depends on the GPU. I have vague memories of some Nvidia GPUs not doing any reordering and the memory layouts being totally identical between arrays and 3d textures. The example I'm remembering was someone had a 3D texture and when sampling it from different orientations resulted in significant performance differences showing signs that it wasn't being morton-ordered in 3D.

But I was more referring to the data chunks themselves being snippets of the same 2D texture blocks regardless of them being 3D textures or arrays, and ignoring block swizzling. That's all handled by the drivers and GPU.

Really, using a Texture2DArray isn't likely to be a good option anyway as it means you don't get hardware filtering between Z layers, and otherwise either won't give any performance benefits or potentially be worse due to the block swizzling helping data coherency.

 

I finally re-implemented my Shader using a 2D texture array and sure enough it didn't have any performance benefits, in fact it almost halved my fps...

However, it did seem to increase the fidelity of the final image-- not in terms of resolution obviously-- but it made images crisper at sharp angles (with identical aniso levels) and resolved the images fading out with distance (I think because 2D texture arrays actually apply the "mip maps preserve coverage" correction).

And again. it seems that QualitySettings have no effect on either 3D-textures or texture-arrays.

 

Sorry for hijacking the thread, but that works? I have tried creating a Texture3D-Object with DXT compression before.
In fact, if I create a Texture3D with a format of BC4 right now I get an exception saying:

Texture3D does not support compressed formats (format 102)

About your problem: If bandwidth isn't the reason for bad performance, there needs to be some issue in the shader. Have you checked how many samples each ray is taking? How often do you evaluate your distance function?

 

That's weird, i just set the texture format of my 3D texture in script as I was creating it (and before I saved it as an asset).

Code (CSharp):

1. Texture3D initTex3D = new Texture3D(
2.             slices[0].width, //width of texture
3.             slices[0].height, //height of texture
4.             slices.Length, //z resolution of texture
5.             TextureFormat.RHalf, //texture format
6.             true); //mip textures

I'm not actually using signed distance functions in my Shader as I'm just rendering predetermined volume textures (not simulating geometries using texture data or anything fancy like that if that's what you mean). For me probably my main performance increases would come by taking most of the transformation maths I do in the frag Shader (and even in the ray-marching loop D: ) out and into the vertex shader or c# scripts.

Texture size does seem to be one of the main bottlenecks, but decreasing the samples of course improves performance (but decreases image fidelity). Currently in my frag shader each ray is sampling the texture roughly once every texel, so if my texture is 256x256 and my UVW coords go from 0-1. I move 1/256 units every iteration, I keep marching until the ray is definitely out of the mesh, in a cubes case this would be achieved by marching by the hypotenous of one of the faces of the cube (magnitude(256, 256)) = ~362 marches/sample-iterations.

Not sure if any of that helps, shrug*, but I tried!

 

That does not look like you're creating it as BC4 texture though. I assume that is just how you currently do it and simply changed RHalf to BC4? That does not work for me unfortunately... I have to hand in my thesis in two weeks and getting a nice bandwidth-boost would be so nice!

Unity 2019.3 even added

Texture3D.SetPixelData

for the very reason of creating compressed Texture2DArrays and Texture3Ds. Doesn't make much sense if it won't let us create compressed Texture3Ds...

Right, should have thought of that! Doing some transformations in the fragment shader should be fine as they are nothing compared to the raymarch loop. That one should be as simple as possible though.

Have you looked at Preintegration? That technique really works wonders! It's basically taking your transfer function and turning it into a lookup table which stores the answer to the question "If I had a density of X at the last sample and now have a density of Y, what color would I get if I had more samples in between?".

I could dig out some code which takes a transfer-function and preintegrates it for you if you want.

Look at the difference (same sample count):

Depending on the volume Empty-Space-Leaping can be worth trying. For that you need to precompute a low res volume which stores "how important" the spot is in the main volume. If everything is just air, there is no need to do any sampling. If it's all just the same density (or color), you can get away with sampling only once, etc.

 

Yeah I've switched to using RHalf, I think they have the same performance/memory profile but I just found it more convenient to use in my Shader. I could take a look at some of your code and see if where it differs from mine to try to work out the issue if you want, because at least for me it did make a huge difference.

As for the Empty-Space-Leaping you talk about that would be amazing and I would really appreciate it if you could dig up some code, Shader performance is proving my main issue right now (I'm also working with medical data) and is cause for over 95% of my low framerate.

Thanks!

 

It would be really great if you could do that. I've been trying to get compressed textures to work for months!

BC4 seems to be the same size as BC1 but stores only greyscale, which makes it use 0.5 bytes per pixel. (http://www.reedbeta.com/blog/understanding-bcn-texture-compression-formats/)
RHalf on the other hand stores data as 16-bit floating point numbers, which come at 2 bytes per pixel. They allow to store negative values though.

I'm currently using RHalf myself for my distance fields. Packing them using BC4 or some other compression algorithm would quarter their size!

This is the code I've tried to create a BC4 Texture3D with:

Code (CSharp):

1.        
2. Texture3D tex = new Texture3D(64, 64, 64, GraphicsFormat.R_BC4_UNorm, TextureCreationFlags.None);
3. Texture3D tex2 = new Texture3D(64, 64, 64, TextureFormat.BC4, true);
4.  

Both do not work and result in:

Texture3D does not support compressed formats (format 102)

I have also tried other BCn formats with no luck

Unfortunately I don't have code laying around for that (Only for preintegration) since that part is taken care of by the distance field for me. I recommend getting into compute shaders, if you're not already familiar with them, which are really handy to run computations on volumes.

Here's how it could work (I have never actually implemented this myself, but this is how I understood it):

1. Create a 3D RenderTexture of 1/8 the size of your volume (might go even lower). One voxel of that volume is one "section".
   
2. In a compute shader, sample all voxels of the original volume within a section and figure out how much variance is between them. Higher variance means more importance.
   
3. Return that variance value from the compute shader to store it in the RenderTexture.

The RenderTexture now holds your Importance-data.

In your Raycasting shader, modify the step size according to the importance value.

 

@ataulien What version of Unity / platform / hardware are you running? With Direct3D 11, all formats that are supported for 2D textures and texture arrays are supported by 3D textures too, and Vulkan, OpenGL Core and Metal should also all support the same formats for all texture types too. You mention using GraphicsFormat so I'm assuming you're using some version of Unity 2019, so maybe there's a bug there if @forteller is using 2018?

The other thing to check is if your 2D textures support BC4, as well as if your system supports it using:
https://docs.unity3d.com/ScriptReference/SystemInfo.SupportsTextureFormat.html

I'd be rather surprised if it wasn't though, unless your project settings are for a mobile device, in which case unless you're using an Nvidia Shield is unlikely to support most BCTC formats.

 

I'm using a recent alpha version, 2019.3.0a11 on Windows with a GTX 780 using D3D11. This version added

Texture3D.setPixelData

specifically made to copy raw data into the texture in case it was compressed: https://docs.unity3d.com/2019.3/Documentation/ScriptReference/Texture3D.SetPixelData.html

Getting compressed texture data into a Texture3D would be hard without that method.

I tested the following Unity versions without luck:

• 2017.2.0f3
• 2018.3.5f1
• 2019.3.0a11
  
• 2019.3.0a12

By the way, I can load a compressed 3d texture in a native plugin and force it into a D3D11 texture slot just before rendering just fine.

SystemInfo.SupportsTextureFormat

also reports true for BC4.

Note that any other compressed format does not work for me either.

 

Most curious.

Using the CopyTexture function worked just fine for copying compressed data in previous versions of Unity.
https://docs.unity3d.com/2019.3/Documentation/ScriptReference/Graphics.CopyTexture.html

I've definitely seem to remember using compressed 3D textures in the past, but maybe not? It's been a few years since I need one.

 

I just ran into the problem too. I don't see a reason why Unity would not support compressed Texture3D's. The Direct3D11 documentation does not mention any texture format restriction on 3d textures at least.

I've submitted a bug-report for this:
(Case 1208832) 2019.3: Texture3D does not support compressed formats

If Unity QA thinks it's a bug worth fixing, the report is going to be available (and can be tracked) in the public issue tracker at: https://issuetracker.unity3d.com/product/unity/issues/guid/1208832

Spoiler: Bug Report

Creating a Texture3D with a compressed format outputs the following error:
Texture3D does not support compressed formats (format 109)

Reproduce

• Open attached project
• Click from the main menu "BugReport > Create Texture3D"

Code (CSharp):

1. using UnityEngine;
2. using UnityEditor;
3. static class TestCode
4. {
5.     [MenuItem("BugReport/Create Texture3D")]
6.     static void DoMenuItem()
7.     {
8.         var format = TextureFormat.BC7;
9.         Debug.LogFormat("SupportsTextureFormat BC7 = {0}", SystemInfo.SupportsTextureFormat(format));
10.  
11.         var tex3d = new Texture3D(256, 256, 4, format, true);
12.         Texture3D.DestroyImmediate(tex3d);
13.     }
14. }

Actual
Unity outputs an error.

Expected
Graphics.CopyTexture supports compressed formats.

Note #1
Texture2D and Texture2DArray for example both support compressed formats.

Note #2
There is no indication in the Direct3D documentation that a Texture3D would not support compressed formats.

• https://docs.microsoft.com/en-us/windows/win32/api/d3d11/nf-d3d11-id3d11device-createtexture3d
• https://docs.microsoft.com/en-us/windows/win32/api/d3d11/ns-d3d11-d3d11_texture3d_desc

Note #3
Using compressed formats for a Texture3D reduces memory and bandwidth.

 

Are you using mipmaps with your 3D textures? You want to reduce cache misses on those as much as possible, and mipmaps will help. Also, If you are using mipmaps, disable trilinear filtering! Having it enabled cuts your throughput in half!

 

This is pretty shocking to me, I'm soon going to be wanting to optimise my shaders and one thought was to pack multiple textures into an array (AFAIK array and tex3d are same internally) so not having compressed 3D textures .... wouldn't this be AWFUL for perf? It would basically just force you to have 32x32x32 or suffer horrible perf.

 

Texture2DArray does support various compressed formats in Unity too, perhaps that one works for you and you don't need a Texture3D. See my signature if you're looking for a texture array import pipeline.

 

FYI, there's a twitter thread from @Aras on this specific topic from not too long ago.
https://twitter.com/aras_p/status/1220326405571063810

Short version, for no apparent reason OpenGL & GLES don't seem to actually support 3D textures using DXT formats, even though it should. It supports BC7 or ASTC without an issue, just not DXT1 or DXT5 (aka BC1 & BC3) 3D textures, and the hardware does it fine as it works via Direct3D, Vulkan, or Metal on the same hardware.

 

Here is hoping Unity won't limit us all to the most ridiculous API (looking at you, ES2) just because some Unity users aren't really comfortable having options.

I'd be perfectly happy supporting DX11+ and Vulkan only for my games, indeed I don't think devices lower than this would perform well enough, and ES3 should support fine from what I can see.

 

The issue seems to be fixed in 2020.1 https://issuetracker.unity3d.com/issues/unable-to-create-a-texture3d-with-a-compressed-format

But how do you compress a 3D texture into BC7? I have an uncompressed RGBA 3D Texture. I can convert it easily to Rhalf with SetPixels, but how do you compress into BC7 or DXT?

 

Where does the 3D texture come from? Is it something you're generating in code at runtime? If so, then there's not really any way to convert it to BC7. You can convert it to DXT1 or DXT5 by copying all of the individual slices out to a new

Texture2D

, calling

tex.Compress()

, and creating a new

Texture3D

using DXT1 or DXT5 as its format to copy those layers into. The compress function automatically chooses the appropriate format based on if the source texture is an RGB or RGBA format.
https://docs.unity3d.com/ScriptReference/Texture2D.Compress.html

In the editor, the process is the same, but you can use

EditorUtility.CompressTexture()

to do the same thing as above, but to any format you want, and at much higher quality.
https://docs.unity3d.com/ScriptReference/EditorUtility.CompressTexture.html

Alternatively, in 2020.2 you can import in a texture atlas and set the Shape to 3D to have it import directly to a Texture3D.

 

Thanks @bgolus I understand now. This texture I have came with an Asset Store package as ARGB32. I'm only testing with it, what I'm trying to do in the end is to convert noises from Fast Noise into 3D Textures.