Get external Normal Map at runtime

Hello everyone

Our Project need import external normal map at runtime.

But it's not a correct result when normal map direct import.

I found and tried some method like :

here and here

I tried to exchange order of RGBA, but it's never as correct as same normal map in editor.

Is this method work in 2017 or later version?

Hope someone help.

Thanks

 

Platform is Windows

Below is screenshot:


Left plane is normal map from Resource
Right plane is normal map from external
notice position of sun

Thank you

 

have you tried using modified shader?
https://forum.unity.com/threads/cre...aps-using-rendertotexture.135841/#post-924587

 

You meaning (1, y, 0, x)=(r, g, b, a) or (a, r, g, b)?

 

Ya, I tried to modified shader with "o.Normal = tex2D(_BumpMap, IN.uv_BumpMap) * 2 - 1;"

It's still skew like below:

 

Thanks Aleksandrk

I tried your suggest as :
for (int x = 0; x < m_NMT3.width; x++)
{
for (int y = 0; y < m_NMT3.height; y++)
{
theColour.r = 1;
theColour.g = m_NMT3.GetPixel(x, y).g;
theColour.b = 0;
theColour.a = m_NMT3.GetPixel(x, y).r;
normalTexture.SetPixel(x, y, theColour);
}
}

and result is:

The horizontal position of light point is correct, but not vertical position

 

How are you creating the Texture2D? If you’re just calling new Texture2D(x, y); then you’re creating a texture with sRGB color space. You want the texture to be using linear color space.
https://docs.unity3d.com/ScriptReference/Texture2D-ctor.html

 

Here is my test code:

Code (CSharp):

1. public void PixelRGB_Trs()
2.     {
3.         m_Plane = GameObject.Find("Plane);
4.        m_Plane = m_Plane.GetComponent<MeshRenderer>();
5.        m_pMaterial = Resources.Load<Material>("Material/" + "Default") as Material;
6.        m_Plane_MR.material = m_pMaterial;
7.        WWW NMP = new WWW("file:///D:/NormalMap_Test/" + NMP_Name + ".jpg");
8.         m_NMT = NMP.texture;
9.         Texture2D normalTexture = new Texture2D(m_NMT.width, m_NMT.height, TextureFormat.ARGB32, false);
10.         theColour = new Color();
11.         for (int x = 0; x < m_NMT.width; x++)
12.         {
13.             for (int y = 0; y < m_NMT.height; y++)
14.             {
15.                 theColour.r = 1;
16.                 theColour.g = m_NMT.GetPixel(x, y).g;
17.                 theColour.b = 0;
18.                 theColour.a = m_NMT.GetPixel(x, y).r;
19.                 normalTexture.SetPixel(x, y, theColour);
20.             }
21.         }
22.         normalTexture.Apply();
23.         m_NMT= normalTexture;
24.         m_pMaterial.SetTexture("_BumpMap", m_NMT);
25.     }
26.  

 

Yep, that's creating the Texture2D as an sRGB texture. See the documentation I linked.

 

Thanks bgolus:

I read your link, but don't understand what's the problem

 

Your code is creating a Texture2D with settings for the x and y resolution, the format, and mip maps. But there's one more setting, a bool for linear or sRGB.

The documentation lists three different versions of the Texture2D constructor function, you're using the second one with 4 inputs, you need to be using the third one with 5 inputs.

 

Thank you bgolus:

It's working!!
This problem troubled me for long time.

Thank you very much!!

 

Additional tip - if you haven't used normal map in your material and you want to add it during runtime you must call

Code (CSharp):

1. material.EnableKeyword("_NORMALMAP");

to have bumpmapping effect. If your material already have some other normal map and you just changing it to other at runtime then it's not necesary because shader already have this keywoard enabled.

 

Thank you arzezniczak,

I just want to ask for this.
And now, I guess all issue about normal map is clear.

Thank you everyone

 

Hi,

After importing external normal map then I adjusted bump value.
But the highlights are moved as follow :

How to sloved it?

 

@CoilSpot Your video is set to private so none of us can see it. Also, what do you mean by "bump value"? If you mean the bumpiness in the texture import settings, then this thread is completely unrelated to your issue and you're importing a texture normally, possibly with the wrong settings. If you're adjusting the normal map scale on a Standard shader, then one would expect a highlight to move. If you have a "flat" normal map and wondering why adjusting the scale causes the highlights to move, that's because a normal map cannot actually store a flat normal, the generic "127, 127, 255" is slightly offset from flat, so scaling the normal map will shift the highlight.

 

Thanks for reply bgolus

Everyone can watch the video now.
And it's just like your description.
But it's no problem if the normal map imported to Asset folder.

What's the different?

And another question about textures
I assign a external texture into Standard Shader "_Metallicglossmap" slot
It's looking well in play mode, but not working after build
"_METALLICGLOSSMAP" is enable

Am I missing something?

 

Please read this thread then. The solution is already detailed.

 

Thank you bgolus

I read this thread again
My code is:

Code (CSharp):

1.         //Read NormalMap
2.         FileStream NormalMap = new FileStream(path, FileMode.Open, FileAccess.Read);
3.         NormalMap.Seek(0, SeekOrigin.Begin);
4.         byte[] bytes = new byte[NormalMap.Length];
5.         NormalMap.Read(bytes, 0, (int)NormalMap.Length);
6.  
7.         NormalMap.Close();
8.         NormalMap.Dispose();
9.         NormalMap = null;
10.  
11.         Texture2D Tex = new Texture2D(4, 4);
12.         Tex.LoadImage(bytes);
13.  
14.         //Transform NormalMap
15.         Texture2D NormalTexture = new Texture2D(Tex.width, Tex.height, TextureFormat.ARGB32, false, true);
16.         Color theColour = new Color();
17.         for (int x = 0; x < Tex.width; x++)
18.         {
19.             for (int y = 0; y < Tex.height; y++)
20.             {
21.                 theColour.r = Tex.GetPixel(x, y).r;
22.                 theColour.g = Tex.GetPixel(x, y).g;
23.                 theColour.b = Tex.GetPixel(x, y).b;
24.                 theColour.a = Tex.GetPixel(x, y).a;
25.                 NormalTexture.SetPixel(x, y, theColour);
26.             }
27.         }
28.         NormalTexture.Apply();
29.  
30.         material.EnableKeyword("_NORMALMAP");
31.         material.SetTexture("_BumpMap", NormalTexture);

I'm not sure which part I missing.
Or I can't avoid highlight movement issues?

 

There’s nothing super obviously wrong with your code. Inefficient, but not obviously wrong. What I'm seeing in the video is very clearly a normal map that's using sRGB instead of being set to linear, but your code above doesn't show why. Try this instead:

Code (csharp):

1. //Read NormalMap
2. FileStream NormalMap = new FileStream(path, FileMode.Open, FileAccess.Read);
3. NormalMap.Seek(0, SeekOrigin.Begin);
4. byte[] bytes = new byte[NormalMap.Length];
5. NormalMap.Read(bytes, 0, (int)NormalMap.Length);
6.  
7. NormalMap.Close();
8. NormalMap.Dispose();
9. NormalMap = null;
10.  
11. Texture2D NormalTexture = new Texture2D(1, 1,  TextureFormat.R8, true, true); // created set to linear
12. NormalTexture.LoadImage(bytes); // overrides resolution and format, but doesn't touch the sRGB or mipmap setting
13.  
14. material.EnableKeyword("_NORMALMAP");
15. material.SetTexture("_BumpMap", NormalTexture);

The whole process of creating a new texture and copying the pixels over is totally unnecessary. I'm not really sure where the practice started.

I used this script locally with an image file that has it's extention renamed .bytes and it worked properly for me. It's replacing the normal map of a material that already has a normal map for simplicity.

Code (csharp):

1. using UnityEngine;
2.  
3. public class LoadTextureNormalMapTest : MonoBehaviour
4. {
5.     // Load a .jpg or .png file by adding .bytes extensions to the file
6.     // and dragging it on the imageAsset variable.
7.     public TextAsset imageAsset;
8.     public Texture2D tex;
9.     public void Start()
10.     {
11.         tex = new Texture2D(1, 1, TextureFormat.R8, true, true);
12.         tex.LoadImage(imageAsset.bytes);
13.         GetComponent<Renderer>().material.SetTexture("_BumpMap", tex);
14.     }
15. }

The build is stripping the shader variant with

_METALLICGLOSSMAP

enabled since it's not used by any assets at build time. You need the base material to either have a dummy normal map already applied, or use a Shader Variant Collection to let the build system know there are shader variants needed outside of those in the scene.
https://docs.unity3d.com/Manual/OptimizingShaderLoadTime.html

 

Thank for your help bgolus

I tried your code, the movement of highlight is smaller now.
According to the previous description:

Is this unavoidable?

 

You could use 16 bit per channel normal maps, which require the normal maps you generate / bake are 16 bit, and then you import them in as 16 bit RGHalf or BC6H. Unfortunately, Unity's handling of >8 bpc formats is wonky. 16 bpc PNG files get converted to 8 bpc on import either through the texture importer or via LoadImage(). You can use .exr files which Unity can import and retain their floating point values properly, but LoadImage() doesn't work with those, only normal texture imports, and if you're using a gamma color space project Unity will apply gamma correction to the image no matter what you do, which breaks them. Some people have written tools to read 16 bpc or 32 bpc textures from disk and manually decode them to work around this.

... ...

Other alternatives would be to use a custom shader that tweaks the data a little while unpacking it. For example the default UnpackNormal looks like this (or rather, this is the function UnpackNormal calls on desktop):

Code (CSharp):

1. // Unpack normal as DXT5nm (1, y, 1, x) or BC5 (x, y, 0, 1)
2. // Note neutral texture like "bump" is (0, 0, 1, 1) to work with both plain RGB normal and DXT5nm/BC5
3. fixed3 UnpackNormalmapRGorAG(fixed4 packednormal)
4. {
5.     // This do the trick
6.    packednormal.x *= packednormal.w;
7.  
8.     fixed3 normal;
9.     normal.xy = packednormal.xy * 2 - 1;
10.     normal.z = sqrt(1 - saturate(dot(normal.xy, normal.xy)));
11.     return normal;
12. }

With this code the important part is that

normal.xy = packednormal.xy * 2 - 1;

, which is where the “0.5” of the texture is considered “0.0” in the normal xy. That line can be tweaked so that 127/255 (0.498) is 0.0 instead of the impossible 0.5 (127.5/255).

Code (csharp):

1. normal.xy = packednormal.xy * (255/127) - 1;

One side effect of this is the normal map is slightly skewed messing with some rounded surfaces, and if you have any normal maps that have been flipped or baked with a tool that outputs 128/255 as “flat” it’ll be even more offset that it would have been before.

The next option is to make it so anything between 127 and 128 is considered flat.

Code (csharp):

1. normal.xy = packednormal.xy * 2 - 1;
2. normal.xy = saturate(1 - ((1 - abs(normal.xy)) * (127.5/127))) * (normal.xy < 0 ? -1 : 1);

That’s a bit more code, and it’ll potentially cause some small details to get crushed, but the normal map will be flat. There’s probably a slightly more efficient way to do the above too.

 

Thank bgolus, you are a expert!
It looks complicated, but I will try it.

And about the _MetallicGlossMap
I make a Shader Variants Collection as follow:

But still not working.
Which one I missing?

 

Hello,

I'd have an additional question about this: how would you save the normal map in your assets, at runtime, so that it's then available upon exiting play mode?
More precisely, how to save via script the texture while making sure its type is set to "Normal map".



I use File.WriteAllBytes to save non Normal map textures to my assets, but how would that fit into the solution provided by bgolus above, if I want to do that for normal maps? I mean, should I then re-convert the Normal map into a byte array and use File.WriteAllBytes? If so, how?

If not, should AssetDatabase.CreateAsset be used? (I tried without success though.. and I'm confused because I thought UnityEditor functions couldn't be used at runtime, is that right?)

Thanks a lot!

 

Correct, you cannot make assets at runtime.

AssetDatabase

is a

UnityEditor

class that is not accessible during runtime. And you can't set a texture created from script to even have a type since that's an editor only thing set by the

TextureImporter

class, which is also editor only. And the warning you see in the editor when you use a non-normal map texture in a normal map texture property is also editor only. It's only checking to see if the type exists and is set to normal map, which for runtime created textures they can't be regardless of if they're properly setup to be a normal map or not.

However, if you're in the editor and in play mode, and you want to save a normal map to disk that you can use as an asset, that is something that's possible.

Code (csharp):

1. string savePath = // file path you want to save the texture to, should start with "Assets/"
2. Texture2D normalMap = // texture with the normal map in it, presumably rendered to a render texture and then copied via ReadPixels()
3.  
4. byte[] bytes = normalMap.EncodeToPNG();
5. File.WriteAllBytes(savePath, bytes);
6. DestroyImmediate(normalMap);
7.  
8. AssetDatabase.ImportAsset(savePath);
9. Texture2D newTexture = AssetDatabase.LoadAssetAtPath<Texture2D>(savePath);
10. if (newTexture != null)
11. {
12.   TextureImporter importer = (TextureImporter)AssetImporter.GetAtPath(savePath);
13.   importer.textureType = TextureImporterType.NormalMap;
14.   importer.SaveAndReimport();
15. }

But, again, you can only do this if you're running in the editor. If you try to make a standalone build with this it won't compile. For runtime you'd need to read the png file from disk directly, at which point there's no way to, and no need to set it to be a "normal map" type texture because the actual game and shader don't actually care. It just needs to be a linear space texture as discussed above.

 

Thanks a lot for your answer!
Ok, since in my case it's all happening in scripts, I don't care about the texture "type", got it!
Just one more (very basic, sorry) question: since I want to be able to make a standalone build, I'll avoid UnityEditor functions. Can I then use File.WriteAllBytes on a linear space texture (i.e. a texture that is supposed to be used as a normal map) and expect that to work? Actually I tried and it seems the answer is 'no', (it produces a file that can't b read) but how can I do it then?

Thanks again!

EDIT: I should be more precise, this is what I do and fails. I guess GetRawTextureData is not what is needed:

Code (CSharp):

1.  
2. //Read NormalMap
3. FileStream NormalMap = new FileStream(fileNames[i], FileMode.Open, FileAccess.Read);
4. NormalMap.Seek(0, SeekOrigin.Begin);
5. byte[] nbytes = new byte[NormalMap.Length];
6. NormalMap.Read(nbytes, 0, (int)NormalMap.Length);
7.  
8. NormalMap.Close();
9. NormalMap.Dispose();
10. NormalMap = null;
11.  
12. Texture2D NormalTexture = new Texture2D(1, 1, TextureFormat.R8, true, true); // created set to linear
13. NormalTexture.LoadImage(nbytes); // overrides resolution and format, but doesn't touch the sRGB or mipmap setting
14. byte[] nbytes2 = NormalTexture.GetRawTextureData();
15. string normalMapAssetPath = Directory.GetParent(currentDir).Parent.FullName + "/Assets/Resources/Default Materials/Textures/" + normalTexName;
16. File.WriteAllBytes(normalMapAssetPath, nbytes2);

EDIT2: Ok, I found how to do it: I had to use

byte[] nbytes2 = NormalTexture.EncodeToPNG();

But now I have another question
I want that when I exit play mode my new material is saved in assets, along with its textures. And the only way I found to do that (to keep the 'link' between the material and the textures) is to assign the textures to the material this way:

assignedMaterial.SetTexture("_BumpMap", Resources.Load<Texture2D>("Default Materials/Textures/" + normalTexNameNoSuffix));

But in order to be able to use Resources.Load, well the asset needs to be previously saved.. or at least I found out that I needed to do a call to AssetDatabase.Refresh() before using Resources.Load<Texture2D>. But as said above, I can't use any UnityEditor function.. so how can I do that ?

Thank you!

 

The same rules apply as before.

If you need something that can work in a standalone build, you cannot use anything related to

AssetDatabase

because you cannot save assets to disk at runtime. Similarly you cannot use

Resources.Load

to load files saved to disk at runtime, because they're not assets.

If you're doing something in the editor to create new assets you can do this kind of thing. If you're doing something for runtime, then you'll need to be recreating the assets, be they textures, materials, or anything else, manually from script every time and load the texture data or material properties manually each time.

The only reason why your script sometimes works is because you're in the editor, saving the PNGs to a folder that is then getting processed by the editor when you drop out of play mode. None of this will work at runtime outside of the editor.

 

Thanks a lot!