Floating point textures?

Hello.

I'm converting heightmaps to normals but I get these stepping issues. My texture is 16bit floating point but Unity doesn't seem to read it that way by default. How can I set up my texture so it reads as floating point in shaders?

 

You'll need to convert them into normal maps before importing them into Unity. Unity's texture to bump map conversion will always convert textures into an 8 bit greyscale image before converting them into a normal map. It's also really, really bad it it and should never be used because it'll always be horribly stair stepped like this.

Use the old Nvidia or xNormal photoshop plugins

 

The problem is that the heightmaps are created semi-procedurally so they never repeat. So the normals have to be created in the shader depending on what kind of current heightmap has been created.

 

So then I think I misunderstood what you’re doing. Are you generating these height maps in Unity and looking to derive the normals in a shader?

 

I'm making a terrain by blending various heightmap textures together with different scales and offsets. The blending operations work on heightmaps but they mess up the normals because of the colors and rotation of the normalmaps has to stay the same.

 

Why not rotate the normals in the normal maps?

 

Really, without seeing more of your setup I'm not sure what else I can say. If you're sampling the texture in the shader, it should be getting the real 16 bit value. If you're not then I would expect the texture you're passing to it isn't actually an RHalf (the 16 bit floating point format).

 

Rotating the normals messes up the angles its supposed to show. Is there an extra setting I have to apply to the texture inside Unity? The heightmaps are 16bit pngs maybe it needs to be another format other than pngs?

 

The problem is Unity's handling of >8 bit texture assets is kind of terrible.

Unity doesn't properly support 16 bit png files. It'll read them, but it quantizes them down to 8 bit before the editor uses them. So that means as far as the shader is concerned, it's an 8 bit texture you're handing it.

You can use a 32 bit float tiff (Unity doesn't handle 16 or 24 bit channels from tiff), or an exr file. Both of those will properly retain the float values and not be quantized down to 8 bit. They'll default to a BC6H file, which is okay quality, but there will be some artifacts in the resulting normal. Ideally you'd want to override the format to use R16 or RHalf, but the R16 format strangely only works when importing 8 bit textures (which makes them worthless), and RHalf isn't displayed as an option (even though its in the enum the importer uses), only ARGBHalf. That that means if you can channel pack your data, it's not a bad option. Otherwise your height maps now use 4x times more memory.

Unity will also completely mangle the float data if your project is using Gamma space rendering as it will always imports float image data with a forced gamma curve.

Which brings me back to normal maps.

Yes, which is why you want to counter rotate the normal vector by how much you rotated the normal map UVs.
https://forum.unity.com/threads/sol...-working-but-lighting-response-is-not.452242/

 

Hello,
Is there a way to get fix this? I am using "AssetDatabase.CreateAsset" and it works in gamma space fine. But the file size even more uncompressed texture. For example 512x512 RGBAfloat =4mb
But asset size is 8mb.Why??

I tried using bundles but there are a lot of problems with them. I just want to store the top-view with a depth buffer for water once and read it from the texture. And also I use VAT(vertex animation textures) for fluid/particles simulation using float pixel as position in world space. But with gamma correction it's incorrect.

 

Right now I use this code

Code (CSharp):

1.  
2. /******
3. * The MIT License (MIT)
4. *
5. * Copyright (c) 2016 Bunny83
6. *
7. * Permission is hereby granted, free of charge, to any person obtaining a copy of
8. * this software and associated documentation files (the "Software"), to deal in
9. * the Software without restriction, including without limitation the rights to
10. * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
11. * of the Software, and to permit persons to whom the Software is furnished to do
12. * so, subject to the following conditions:
13. *
14. * The above copyright notice and this permission notice shall be included in all
15. * copies or substantial portions of the Software.
16. *
17. * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
18. * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
19. * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
20. * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
21. * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
22. * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
23. * SOFTWARE.
24. *
25. * Original source:
26. * https://www.dropbox.com/s/rqctkisgq178fba/Texture2DExtension.cs?dl=0
27. *******/
28.  
29. using UnityEngine;
30.  
31.  
32. public static class Texture2DExtension
33. {
34.     public enum DataFormat
35.     {
36.         NONE = 0,
37.         ARGBFloat = 1,
38.         ARGBUShort = 2,
39.     }
40.     #region ARGBFloat
41.     private static void SaveARGBFloatUncompressed(Texture2D aTex, System.IO.BinaryWriter aWriter)
42.     {
43.         int w = aTex.width;
44.         int h = aTex.height;
45.         Color[] colors = aTex.GetPixels();
46.         aWriter.Write((uint)DataFormat.ARGBFloat);
47.         aWriter.Write(w);
48.         aWriter.Write(h);
49.         for (int i = 0; i < colors.Length; i++)
50.         {
51.             Color c = colors[i];
52.             aWriter.Write(c.a);
53.             aWriter.Write(c.r);
54.             aWriter.Write(c.g);
55.             aWriter.Write(c.b);
56.         }
57.     }
58.     private static void ReadARGBFloatUncompressed(Texture2D aTex, System.IO.BinaryReader aReader)
59.     {
60.         int w = aReader.ReadInt32();
61.         int h = aReader.ReadInt32();
62.         Color[] colors = new Color[w * h];
63.         for (int i = 0; i < colors.Length; i++)
64.         {
65.             Color c;
66.             c.a = aReader.ReadSingle();
67.             c.r = aReader.ReadSingle();
68.             c.g = aReader.ReadSingle();
69.             c.b = aReader.ReadSingle();
70.             colors[i] = c;
71.         }
72.         aTex.Resize(w, h);
73.         aTex.SetPixels(colors);
74.         aTex.Apply();
75.     }
76.     #endregion ARGBFloat
77.     #region ARGBUShort
78.     private static void SaveARGBUShortUncompressed(this Texture2D aTex, System.IO.BinaryWriter aWriter)
79.     {
80.         int w = aTex.width;
81.         int h = aTex.height;
82.         Color[] colors = aTex.GetPixels();
83.         aWriter.Write((uint)DataFormat.ARGBUShort);
84.         aWriter.Write(w);
85.         aWriter.Write(h);
86.         for (int i = 0; i < colors.Length; i++)
87.         {
88.             Color c = colors[i];
89.             aWriter.Write((ushort)(c.a * 65535));
90.             aWriter.Write((ushort)(c.r * 65535));
91.             aWriter.Write((ushort)(c.g * 65535));
92.             aWriter.Write((ushort)(c.b * 65535));
93.         }
94.     }
95.     private static void ReadARGBUShortUncompressed(Texture2D aTex, System.IO.BinaryReader aReader)
96.     {
97.         int w = aReader.ReadInt32();
98.         int h = aReader.ReadInt32();
99.         Color[] colors = new Color[w * h];
100.         for (int i = 0; i < colors.Length; i++)
101.         {
102.             Color c;
103.             c.a = aReader.ReadUInt16() / 65535f;
104.             c.r = aReader.ReadUInt16() / 65535f;
105.             c.g = aReader.ReadUInt16() / 65535f;
106.             c.b = aReader.ReadUInt16() / 65535f;
107.             colors[i] = c;
108.         }
109.         aTex.Resize(w, h);
110.         aTex.SetPixels(colors);
111.         aTex.Apply();
112.     }
113.     #endregion ARGBUShort
114.  
115.     #region Extensions
116.     public static void SaveUncompressed(this Texture2D aTex, System.IO.Stream aStream, DataFormat aFormat)
117.     {
118.         using (var writer = new System.IO.BinaryWriter(aStream))
119.         {
120.             if (aFormat == DataFormat.ARGBFloat)
121.                 SaveARGBFloatUncompressed(aTex, writer);
122.             else if (aFormat == DataFormat.ARGBUShort)
123.                 SaveARGBUShortUncompressed(aTex, writer);
124.         }
125.     }
126.     public static void ReadUncompressed(this Texture2D aTex, System.IO.Stream aStream)
127.     {
128.         using (var reader = new System.IO.BinaryReader(aStream))
129.         {
130.             var format = (DataFormat)reader.ReadInt32();
131.             if (format == DataFormat.ARGBFloat)
132.                 ReadARGBFloatUncompressed(aTex, reader);
133.             else if (format == DataFormat.ARGBUShort)
134.                 ReadARGBUShortUncompressed(aTex, reader);
135.         }
136.     }
137.  
138. #if !UNITY_WEBPLAYER && !UNITY_WEBGL
139.     // File IO versions
140.     public static void ReadUncompressed(this Texture2D aTex, string aFilename)
141.     {
142.         using (var file = System.IO.File.OpenRead(aFilename))
143.         {
144.             aTex.ReadUncompressed(file);
145.             file.Close();
146.         }
147.     }
148.     public static void SaveUncompressed(this Texture2D aTex, string aFilename, DataFormat aFormat)
149.     {
150.         using (var file = System.IO.File.Create(aFilename))
151.         {
152.             aTex.SaveUncompressed(file, aFormat);
153.             file.Close();
154.         }
155.     }
156. #endif
157.  
158. #endregion Extensions
159. }
160.  

In this case I also can use only R/RG/RGB channels and custom compression.

 

I wrote the new script that allows me to use compression and the original texture format. Also it's x10 times faster (or more) and textures are always in linear space with gamma/linear rendering.

Code (CSharp):

1. using System;
2. using System.Collections.Generic;
3. using System.IO;
4. using System.IO.Compression;
5. using UnityEngine;
6.  
7. public static class Texture2DExtensions
8. {
9.     public static Texture2D ReadTextureFromFile(string filename)
10.     {
11.         Texture2D tex;
12.         if (!File.Exists(filename + ".gz")) return null;
13.         using (var fileStream = File.Open(filename + ".gz", FileMode.Open))
14.         {
15.             using (GZipStream decompressionStream = new GZipStream(fileStream, CompressionMode.Decompress))
16.             {
17.                 using (MemoryStream stream = new MemoryStream())
18.                 {
19.                     decompressionStream.CopyTo(stream);
20.                     stream.Position = 0;
21.                     var rawTextureDataWithInfo = stream.ToArray();
22.                     {
23.                         var format = (TextureFormat) BitConverter.ToInt32(rawTextureDataWithInfo, 0);
24.                         int width = BitConverter.ToInt32(rawTextureDataWithInfo, 4);
25.                         int height = BitConverter.ToInt32(rawTextureDataWithInfo, 8);
26.  
27.                         var rawTextureData = new byte[rawTextureDataWithInfo.Length - 12];
28.                         Array.Copy(rawTextureDataWithInfo, 12, rawTextureData, 0, rawTextureData.Length);
29.  
30.                         tex = new Texture2D(width, height, format, false, true);
31.                         tex.LoadRawTextureData(rawTextureData);
32.                         tex.Apply();
33.                     }
34.                 }
35.             }
36.             fileStream.Close();
37.         }
38.  
39.         return tex;
40.     }
41.  
42.     public static byte[] Combine(byte[] first, byte[] second)
43.     {
44.         byte[] bytes = new byte[first.Length + second.Length];
45.         Buffer.BlockCopy(first, 0, bytes, 0, first.Length);
46.         Buffer.BlockCopy(second, 0, bytes, first.Length, second.Length);
47.         return bytes;
48.     }
49.  
50.     public static void SaveToFile(this Texture2D tex, string filename)
51.     {
52.         using (var fileToCompress = File.Create(filename + ".gz"))
53.         {
54.             int w = tex.width;
55.             int h = tex.height;
56.             var rawTextureData = tex.GetRawTextureData();
57.             var textureInfo = new List<byte>();
58.  
59.             textureInfo.AddRange(BitConverter.GetBytes((uint) tex.format));
60.             textureInfo.AddRange(BitConverter.GetBytes(w));
61.             textureInfo.AddRange(BitConverter.GetBytes(h));
62.             rawTextureData = Combine(textureInfo.ToArray(), rawTextureData);
63.  
64.             using (GZipStream compressionStream = new GZipStream(fileToCompress, CompressionMode.Compress))
65.             {
66.                 compressionStream.Write(rawTextureData, 0, rawTextureData.Length);
67.             }
68.  
69.             fileToCompress.Close();
70.         }
71.     }
72. }
73.  

I use it for saving depth texture to R32 bit.

Code (CSharp):

1.  
2. var currentRT = RenderTexture.active;
3.  
4. var tempRT = RenderTexture.GetTemporary(depthRT.width, depthRT.height, 0, RenderTextureFormat.RFloat, RenderTextureReadWrite.Linear);
5. Graphics.Blit(depthRT, tempRT);
6. RenderTexture.active = tempRT;
7.  
8. var tex = new Texture2D(depthRT.width, depthRT.height, TextureFormat.RGBAFloat, false, true);
9. tex.ReadPixels(new Rect(0, 0, depthRT.width, depthRT.height), 0, 0);
10.  tex.Apply();
11. tex.SaveToFile(path);
12.  
13. RenderTexture.active = currentRT;
14.  

Unfortunately I can't save individual channels because readpixel requires all 4 channels

"Unsupported texture format for ReadPixels - needs to be RGBA32, ARGB32, RGB24, RGBAFloat or RGBAHalf"

I tried to use follow code for avoid this problem

Code (CSharp):

1.  
2. var newTex = new Texture2D(depthRT.width, depthRT.height, TextureFormat.RFloat, false, true);
3. Graphics.ConvertTexture(tex, newTex);
4.  

In this case I have new texture2D with only red channel.
But newTex.GetRawTextureData is always returns an array with all "0". I don't know how to get rawTextureData in this case.
how can this be solved?

 

I feel like I've managed to copy single channel render textures before using ReadPixels, but maybe I'm misremembering.

A hacky work around would be to create an ARGBFloat render texture and blit the contents of the RFloat into it before reading back to an RGBAFloat Texture2D.

 

I don't quite understand how this works, because texture2d in your case "RGBAFloat" but I need raw data only for one channel.

 

Right, but if Unity isn't letting you copy data from an RFloat RenderTexture to an RFloat Texture2D, you can use Blit() to copy the RFloat RenderTexture to an ARGBFloat RenderTexture and copy it to an RGBAFloat Texture2D. From there if you wish you can copy the data from the single channel you want back to an RFloat Texture2D to save. Like I said, it's a hacky work around.

 

of course that's exactly what i did first
I used this code

Code (CSharp):

1.  
2. var tempRT = RenderTexture.GetTemporary(depthRT.width, depthRT.height, 0, RenderTextureFormat.RGBAFloat, RenderTextureReadWrite.Linear);
3. //var tempRT = RenderTexture.GetTemporary(depthRT.width, depthRT.height, 0, RenderTextureFormat.RFloat, RenderTextureReadWrite.Linear);
4. //I tested both formats and no difference
5. Graphics.Blit(depthRT, tempRT);
6. RenderTexture.active = tempRT;
7. var tex = new Texture2D(depthRT.width, depthRT.height, TextureFormat.RGBAFloat, false, true);

tex.ReadPixels works with any render target format, but texture2D must have "rgba32/half/float"

Right now I don't know how to copy data from texture2D(RGBA) to texture2D(R) without bug.
As you can see above I use this code for that

Code (CSharp):

1. var newTex = new Texture2D(depthRT.width, depthRT.height, TextureFormat.RFloat, false, true);
2. Graphics.ConvertTexture(tex, newTex);

I can see the new texture with R channel in the editor
but when I use newTex.GetRawTextureData<byte>(); I see only empty array of bytes = 0.

 

I'm not entirely sure how the

Graphics.ConvertTexture()

function works. There's not a ton of information on exactly what it does. There is a cryptic comment in the documentation about needing the destination texture to "correspond" to a supported render texture format that makes me think it does a

Blit()

to render texture and

ReadPixels()

back to the target

Texture2D

... Which since the whole reason you're doing this is because using

ReadPixels

to an

RFloat

format

Texture2D

isn't working probably means neither is this.

Honestly if I were you I'd report it as a bug, because either it should work, or

ConvertTexture

should return an error message for single channel formats.

I think you're going to have to do this manually. Either use

RFloatTex.SetPixels(RGBAFloatTex.GetPixels())

, or call

GetRawTextureData()

and manually copy the first 4 of every 16 bytes to a new array that you assign to the texture.

 

Big thanks! It works.

 

Related to this for anyone else, you can work directly with 16-bit floating point channels using GetRawTextureData through "C# Half-precision data type" and the additional notes below.
https://gist.github.com/vermorel/1d5c0212752b3e611faf84771ad4ff0d

From that just delete everything down from line with "half_tests.cs" in it. Then add this to the top under the namespace..

Code (CSharp):

1. public struct HalfColor
2. {
3.   public Half r;
4.   public Half g;
5.   public Half b;
6.   public Half a;
7. }

You create a texture using the format like this..

Code (CSharp):

1. Texture2D tex = new Texture2D(w, h, TextureFormat.RGBAHalf);

You get the NativeArray like this (don't forget

using namespace SystemHalf;

)..

Code (CSharp):

1. NativeArray<HalfColor> pixels = tex.GetRawTextureData<HalfColor>();

You set a pixel like this (apply as usual when all done with

tex.Apply();

)..

Code (CSharp):

1. pixels[index] = new HalfColor { r = (Half)0.5f, g = 0, b = 0, a = 1 };

I had almost given up on the format just as I got it working ..hopefully this Half data type is cross-platform compatible? There must be a reason it's not part of Unity to begin with.. ?

 

It is. It's part of the Mathematics package.
https://github.com/Unity-Technologi...42c6a4f647cf290/src/Unity.Mathematics/half.cs

However the reason why it's not included with default c# is kind of mentioned in the comments for the SystemHalf you linked above.

It's really slow.

 

Thanks for that info - I had no idea, for me this is needing more precision than RGBA32/Color32 has to offer for something I'm doing with a Sample Texture 2D LOD node in Shader Graph. Since the texture will be processed on the GPU I'm hoping the "really slow" drawback to halfs isn't also the case with shaders

 

Isn't a problem with shaders ... because

half

is always just a full single precision float!*

* Except for a small handful of mobile devices.

For the most part you should assume GPUs are doing all math as single precision floats, and that half precision only exists as a storage format. There’s no reason not to use

half

precision in Shader Graph if you are targeting mobile, for the few devices that do benefit from it. Just don’t expect any kind of performance improvement anywhere else.

 

Excellent - yeah so far everything seems to be working great. Integrating use of halfs turned out to be worthwhile because of 1024x1024 "data textures" being 16MB using floats vs. 8MB using halfs. I can now actually use a bit lower resolution as well because of more precision vs. RGBA32/"quarter precision" (which was all I was using previously for anything similar). I could even have been okay with 1000 values vs. 65536 but there's nothing in between. So anyway, I'm down to 3.1MB at 640x640 hoping there won't be any NPOT hit (512 isn't high enough, 1024 is overkill) - I'm not rendering it just sampling from it. Time will tell if it needs another look, so far I'm thrilled - thanks again

 

Here because I also came across RFloat texture format issues. The error I get is due to "size of texture was larger" than my SetPixels array.