RGBA or RGB for texture array?

I am developing a framework to create shaders. For some parts I use texture arrays and therefore need to determine the texture compression.

Consider these two scenarios for PC:

1. We use DXT1: Albedo on texture1, normal in RG and heightmap in B of texture 2, smoothness map in R of texture 3.
We will have 3 * 4 = 12 bits per pixel and we need 3 samples in shader.

2. We use DXT5: Albedo on RGB and heightmap in A of texture 1, normal in RG and smoothness map in B of texture 2.
We will have 2 * 8 = 16 bits per pixel and we need 2 samples in shader.

The framework should support any of unity platforms so no specific target is considered. What would be a general better choice in terms of performance and visual quality?

 

I would recommend using DXT5 over DXT1. The alpha channel of DXT5 is significantly higher quality than any single color channel from DXT1 (or DXT5). You may also want to put the smoothness or a channel of your normal in the second texture's alpha to make use of that higher quality.

By default, on PC, Unity's normal maps are stored in DXT5 with the x (red) of the normal map in the alpha, and the y (green) in the green. This was a very common practice in the industry until BC5 replaced it (which uses DXT5 style alpha compression for R and G).

The alternative you could use is two arrays. Keep the albedo and data textures separate texture arrays with different compression.

Performance wise, sampling a DXT5 is slightly slower than a DXT1. But sampling a DXT1 array 3 times (especially with a single sampler) is going to be slower than sampling a DXT5 array twice!

 

Thank you very much. That answers my question and more.