Generate mipmaps at runtime for a texture loaded with UnityWebRequest?

How can I generate mipmaps for a texture loaded from UnityWebRequest at runtime? Please note while I use 'www' as a reference, it is not using the WWW class.

I'm currently loading a texture from a UnityWebRequest using this code:

Code (CSharp):

1. IEnumerator DownloadImage()
2.    {
3.        string url = videoThumbnailURL;
4.        UnityWebRequest www = UnityWebRequestTexture.GetTexture(url);
5.        DownloadHandler handle = www.downloadHandler;
6.  
7.        yield return www.SendWebRequest();
8.        if (www.isHttpError || www.isNetworkError)
9.        {
10. //do my error stuff
11.        }
12.        else
13.        {
14.            //Load Image
15.            Texture2D texture2d = DownloadHandlerTexture.GetContent(www);
16.  
17.            Sprite sprite = null;
18.            sprite = Sprite.Create(texture2d, new Rect(0, 0, texture2d.width, texture2d.height), Vector2.zero);
19.  
20.            if (sprite != null)
21.            {
22.                videoThumbnailImage.sprite = sprite;
23.            }
24.        }
25.    }

Which loads the texture but obviously doesn't generate mipmaps. I've tried using

Code (CSharp):

1. texture2d.SetPixels(texture2d.GetPixels(0, 0, texture2d.width, texture2d.height));
2. texture2d.Apply(true);

.. per an old Unity forum discussion but that didn't seem to work. Any thoughts?

 

If you do the "texture2d.Apply(true);" it should generate mipmaps... or at least that was my understanding.

 

I've tried adding that below line 15 like this but it didn't work.

Code (CSharp):

1. //Load Image
2. Texture2D texture2d = DownloadHandlerTexture.GetContent(www);
3. texture2d.SetPixels(texture2d.GetPixels(0, 0, texture2d.width, texture2d.height));
4. texture2d.Apply(true);

 

Hmm, as far as I knew that should do it, but perhaps there is more to it. Here is a relevant unity answers question you might check out:
https://answers.unity.com/questions/10292/how-do-i-generate-mipmaps-at-runtime.html

 

Try adding this line after the GetContent() call:

Code (csharp):

1. Debug.Log("MipMap Count = " + texture2d.mipmapCount);

If that prints 0 into the log, then the Texture2D that GetContent is creating does not have mip maps enabled, so nothing you do after that will make it work as there's no way to enable mipmaps post creation of the Texture2D object. You'll likely need to do something like this:

Code (csharp):

1. Texture2D wwwTex = DownloadHandlerTexture.GetContent(www);
2. Texture2D newTex = new Texture2D(wwwTex.width, wwwTex.height);
3. newTex.SetPixels(wwwTex.GetPixels(0));
4. newTex.Apply(); // default is to update mipmaps

You could also use newTex.Resize(wwwTex.width, wwwTex.height); and reuse the "newTex" Texture2D rather than creating a new one every time.

 

loadedTexture = new Texture2D(wwwTexture.width, wwwTexture.height, wwwTexture.format, true);

loadedTexture.LoadImage(uwr.downloadHandler.data);

I use dynamic load texture with mipmap in our main project and this works on my side.

wwwTexture is the texture you get from UnityWebRequest uwr, and loadedTexture is a new Texture2D, which is also the result texture with mipmap.

SetPixel should not be needed, at least not on my side. You may want to call Apply().

 

Thank you all for the help - this solution worked for me!

 

Hey, please note, there is Graphics.CopyTexture API which is much more effective than Get\SetPixels or LoadImage, when supported.

Here is an example:

Code (CSharp):

1. if (SystemInfo.copyTextureSupport != CopyTextureSupport.None)
2. {
3.     Graphics.CopyTexture(wwwTexture, 0, 0, mipTexture, 0, 0); // copies mip 0
4. }
5. else
6. {
7.     mipTexture.LoadImage(www.downloadHandler.data);
8. }
9.  
10. mipTexture.Apply(true); // generates all other mips from mip 0

In my case, Get\SetPixels() took ~64k ticks, LoadImage took ~34k ticks, where Graphics.CopyTexture took only ~8k ticks.

 

Ok, whoa. Your blowing my mind right now. I went with a compute shader because it performed way better than getpixel and setpixel apply methods - but if what you say is true, then compatibility would be way better across more devices. I'm going to experiment with this, so thanks so much for sharing!

 

OK @cdytoby 's solution looks good but Unity why is there no DownloadHandlerTexture.GetContent(UnityWebRequest www, bool generateMipMaps) ?

It seems such an obvious option!

 

@codestage you're explicitly copying mip 0. Do you have to deal with copying each individual mip if you want the final texture to support them?

 

@bgolus Am I correct that the DownloadHanderTexture only supports jpg and png and therefore does not support mipmaps at all?
Which in case of low performance devices is really bad because LoadImage has a huge impact on performance and using it continuously is not an option.
So in essence, there is no way you get a mipmapped texture out of a download unless you spend CPU on it, am I right?

 

Yes, you need to copy all mips to the final texture and do not call Apply after that on such texture. Here is a quote from Texture2D.Apply() API reference:

And from Graphics.CopyTexture API:

 

AFAIK, this is correct.

No, it's not something you should be using continuously even on desktop. It's something to use every so often to load a texture once. After that you use the Texture2D asset it creates. You don't need to call it every frame to use a single if that's what you're thinking. Also decompressing jpg and png files is surprisingly slow, even on high end PCs. These are formats that were designed around reducing the file size as much as possible, not necessarily for real time usage.

If you're looking to load .jpg or .png files from a web address, yes. You can pack all of the mipmaps into a single texture, like this example from the wikipedia article on mipmaps.

But you'd still need to use

Graphics.CopyTexture()

to copy the sections of the image that are for each mip into the new texture. Similar to the examples from @codestage but using a single texture rather than one for each mip.

However I think the real answer you might be looking for is don't use

DownloadHanderTexture

at all. Use

DownloadHandlerAssetBundle

and create asset bundles with pre-compressed texture asset with mips already in it. That'll be far more efficient to load.

 

@bgolus Thanks for your awesome answer. I thought so. It's actually not that easy Right now we are faking sharing a browser screen from one device to the others by reading its texture continuously and sending them over the internet. the clients basically start loading the next just after the previous image was finished. This is totally insane, I know, but for now its fine until we have a real streaming solution.

 

Hi, I'm trying to use this approach to create a texture with mipmaps at runtime, but when I call the LoadRawTextureData method I've the following error.

Code (CSharp):

1. UnityException: LoadRawTextureData: not enough data provided (will result in overread).

I guess that's due to the different size of textures, the destination one having room for mipmaps, the source one not.

Moreover, shouldn't the Apply method be called on the destination texture rather than on the source, like this?

Code (CSharp):

1. //copy to GPU and generate mips
2. textureDestination.Apply(true)

Anyway, the problem is the error on the LoadRawTextureData, have you some hints about it?

 

Thank you for your answer.

In fact, it's the source the one with no mipmaps, not the destination. Anyway I tried to get the subarray and load it into the destination texture like this:

Code (CSharp):

1.  //texture with mips
2. textureDestination = new Texture2D(width,height, format, true);
3.  
4. //copy on the CPU
5. textureDestination.LoadRawTextureData(textureSource.GetRawTextureData<byte>().GetSubArray(0, width*height));
6.  
7. //copy to GPU and generate mips
8. textureDestination.Apply(true);

But I get the same error as before:

Code (CSharp):

1. UnityException: LoadRawTextureData: not enough data provided (will result in overread).

So far, of all the proposed methods, the only one that worked for me is:

Code (CSharp):

1. textureSource = DownloadHandlerTexture.GetContent(webRequest);
2. textureDestination = new Texture2D(textureSource.width, textureSource.height, textureSource.format, true);
3. textureDestination.SetPixels32(textureSource.GetPixels32(0), 0);
4. textureDestination.Apply(true);

Of course, at the price of a noticeable hiccup.

It would be great to have a solid, fast and smooth way to create a texture with mipmaps at runtime, that's a really important feature for a game engine.

Any further ideas?

 

Just wanted to chime in with my solution.

As @RiccardoAxed already pointed out LoadRawPixelData does not work when the source texture does not already have mipmaps.

You can fix that by filling the source texture's data into the destination NativeArray though:

Code (CSharp):

1. var mmTexture = new Texture2D(texture.width, texture.height, texture.format, true);
2. var dst = mmTexture.GetRawTextureData<byte>();
3. var src = texture.GetRawTextureData<byte>();
4. NativeArray<byte>.Copy(src, dst, src.Length);
5. mmTexture.LoadRawTextureData(dst);
6. mmTexture.Apply(true, true);

But, at least in my use-case, the actual winner is this one:

Code (CSharp):

1. var mmTexture = new Texture2D(texture.width, texture.height, texture.format, true);
2. mmTexture.SetPixelData(texture.GetRawTextureData<byte>(), 0);
3. mmTexture.Apply(true, true);

I've set up a little benchmark loading 110 images, most of them around 1920x1080, a mix of PNG (ARGB32) and JPG (RGB24).

The SetPixelData one overall reproducibly runs juuust a little faster with a ratio of about 1.1

Sometimes the LoadRawPixelData one outperforms the SetPixelData one though. I did not find a pattern why this is. I could not relate it to the images' resolution or format. I also played around with the order in which the two different methods get executed but the SetPixelData one always is a tad faster overall.

Edit: One important distinction: Be sure to use the generic overload

texture.GetRawTextureData<byte>()

as the non-generic method returns byte[] and performs measurably worse for me (ratio of 0.8).

 

Hi @AljoshaD , I am trying to re-use the super rad jobyfied example you posted for Set_Pixel_Data_Burst_Parallel, but I can't get it to work. Can you please give me some guidance what I may be doing wrong?

I am trying to change the alpha for the pixels of a texture that are black. This is the original image RGBA32 image: https://drive.google.com/file/d/1mpJOnI804121PFuxlhtZ4VFJBAhzJp-v/view?usp=sharing

And this is the output image: all grays were changed to white and some yellow was added. Black was untouched. https://drive.google.com/file/d/1K48VjWJLNdj_nNrXmdupB0XfEZdG1wXO/view?usp=sharing

This is what I did:

- I solve for those pixels with R=0 in RGB to set its alpha=0, but debug tells me the R in RGB for black pixels is RGBA(255, 0, 0, 0) (which photoshop tells me this RGB is red?!). So I solved to change the alpha for those pixels with G=0 in RGB.

- According to the debug, all other pixels that are not G=0 in RGB are RGBA(255, 255, 255, 255), and in theory those should not have changed. However as you can see the grays also changed to white.

- Not sure if the debug is right - log says Unsupported string.Format for Burst for debug.log

This is the code I used trying to keep it as close as I can to the one you posted. I call UpdateColorRangeTransparency():

Code (CSharp):

1. UpdateColorRangeTransparency(int TexWidth, ColorRange.x, ColorRange.y);
2.  
3. void UpdateColorRangeTransparency(int TexWidth, float bottomColorRange, float topColorRange)
4.         {
5.             Unity.Collections.NativeArray<UnityEngine.Color32> data = m_Texture.GetRawTextureData<UnityEngine.Color32>();
6.  
7.             var job = new UpdateTextureColorsIntoNativeArrayBurstParallel()
8.             {
9.                 data = data,
10.                 _TexWidth = TexWidth,
11.                 bottomColorRange = bottomColorRange,
12.                 topColorRange = topColorRange
13.             };
14.             job.Schedule(TexWidth, 1).Complete();
15.         }
16.  
17.         [BurstCompile]
18.         struct UpdateTextureColorsIntoNativeArrayBurstParallel : IJobParallelFor
19.         {
20.             [NativeDisableParallelForRestriction] public NativeArray<UnityEngine.Color32> data;
21.  
22.             public int _TexWidth;
23.             public float bottomColorRange;
24.             public float topColorRange;
25.             public void Execute(int y)
26.             {
27.                 int _pixelPos = 0;
28.                 var idx = y * _TexWidth;
29.                 for (var x = 0; x < _TexWidth; ++x)
30.                 {
31.                     _pixelPos = idx++;
32.                  
33.                     if (data[_pixelPos].g >= bottomColorRange && data[_pixelPos].g < topColorRange)//bottomColorRange is 0; topColorRange is 25
34.                     {
35.                         data[_pixelPos] = new UnityEngine.Color(data[_pixelPos].r, data[_pixelPos].g, data[_pixelPos].b, 0);
36.                         Debug.Log("INRANGE - Data is "+ data[_pixelPos] + " vs range bottom " + bottomColorRange + " top range "+ topColorRange);
37.                     }
38.                     else
39.                     {
40.                         data[_pixelPos] = new UnityEngine.Color(data[_pixelPos].r, data[_pixelPos].g, data[_pixelPos].b, 255);
41.                         Debug.Log("OUTSIDERANGE - Data is " + data[_pixelPos] + " vs range bottom " + bottomColorRange + " top range " + topColorRange);
42.                     }
43.                 }
44.             }
45.         }

 

Thanks so much for this method... truly a saver!

 

Hi all,

Observation: for generating mipmaps and enabling compression, using "DownloadHandlerBuffer" is resulting in less(>50%) Memory usage and GC than the recommended "DownloadHandlerTexture".


Note: The above logs are in reverse order(top to bottom is the most recent to oldest) to what is usually seen in Unity Editor.

But as per documentation:

Finally, DownloadHandlerTexture only allocates managed memory when finally creating the Texture itself, which eliminates the garbage collection overhead associated with performing the byte-to-texture conversion in script.​

Here are the codes for both the methods:

• Code to download image using "DownloadHandlerTexture" - GetImageUsingUWRTexture
• Code to download image using "DownloadHandlerBuffer" and enable mipmaps, compression - GetImageUsingUWRBufferAndEnableMipmapsAndCompression

Any thoughts on why this could be happening?