Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Texture2D.LoadImage too slow, anyway to use threading to speed it up ?

Discussion in 'Getting Started' started by narf03, Nov 24, 2016.

  1. narf03

    narf03

    Joined:
    Aug 11, 2014
    Posts:
    223
    I have images coming in from the network, they are in jpg format(bytes), average 100-200 image per second. In order to display them on the screen i must convert them into texture first using Texture2D.LoadImage, but the problem is that function can only be called from the main thread, it takes a long time to convert even 1 image, around 20-50 ms per image, so each second the main thread can only do up to ~30 images.

    Problem can be solved if i able to move most of the work to other thread, assuming the system has multiple cores, im wondering if there is a way i can convert the byte streams into raw texture data so it can help speed things up ? Or are there any other methods ?

    thanks.
     
  2. narf03

    narf03

    Joined:
    Aug 11, 2014
    Posts:
    223
    The following code works, it will take a long time in threading to process the image file, but will load almost instantly in the main thread. Just wandering if there are better alternatives ?

    Code (CSharp):
    1. using UnityEngine;
    2. using System.Collections;
    3. using System.IO;
    4. using System.Diagnostics;
    5. using System.Drawing;
    6. using System.Threading;
    7.  
    8. public class clsLoadImage : MonoBehaviour {
    9.     MeshRenderer mr;
    10.     Texture2D tx;
    11.     Thread threadProcessImage;
    12.     Image img;
    13.     byte[] bytTx;
    14.  
    15.     void Awake() {
    16.         img = Image.FromFile(@"c:/temp/3.jpg");
    17.        
    18.         mr = GetComponent<MeshRenderer>();
    19.     }
    20.  
    21.     void OnGUI() {
    22.         if(GUI.Button (new Rect(0,0,100,100), "Process Image")) {
    23.             threadProcessImage = new Thread(fct_ProcessImage);
    24.             threadProcessImage.Start();
    25.         }
    26.         if (GUI.Button(new Rect(100, 0, 100, 100), "Load Image")) {
    27.             fct_LoadImage();
    28.         }
    29.     }
    30.  
    31.     void fct_ProcessImage() {
    32.         Stopwatch sw = new Stopwatch();
    33.         sw.Start();
    34.  
    35.         MemoryStream msBMP = new MemoryStream();
    36.         img.Save(msBMP, System.Drawing.Imaging.ImageFormat.Bmp);
    37.         UnityEngine.Debug.Log("save to bitmap = " + sw.ElapsedMilliseconds + " miliseconds");
    38.  
    39.         byte[] bytBMP = msBMP.ToArray();
    40.         int intPointer = 54;
    41.         MemoryStream msTx = new MemoryStream();
    42.         for (int cnt = 0; cnt < img.Width * img.Height; cnt++) {
    43.             msTx.WriteByte(0);
    44.             msTx.Write(bytBMP, intPointer, 3);
    45.             intPointer += 3;
    46.         }
    47.         msTx.Position = 0;
    48.         bytTx = msTx.ToArray();
    49.  
    50.         UnityEngine.Debug.Log("Process " + img.Width + "x" + img.Height + " image = " + sw.ElapsedMilliseconds + " miliseconds");
    51.     }
    52.  
    53.     void fct_LoadImage() {
    54.         Stopwatch sw = new Stopwatch();
    55.         sw.Start();
    56.  
    57.         tx = new Texture2D(img.Width, img.Height, TextureFormat.ARGB32, false);
    58.         tx.LoadRawTextureData(bytTx);
    59.         tx.Apply();
    60.         mr.material.SetTexture("_MainTex", tx);
    61.  
    62.         UnityEngine.Debug.Log("Loading " + img.Width + "x" + img.Height + " image = " + sw.ElapsedMilliseconds + " miliseconds");
    63.     }
    64.  
    65. }
    66.  
    save to bitmap = 32 miliseconds
    Process 1216x1216 image = 717 miliseconds
    Loading 1216x1216 image = 3 miliseconds
     
  3. emathew

    emathew

    Joined:
    Jul 31, 2017
    Posts:
    13
    Were you able to find an asn
    Were you able to an answer to your question? I am having a similar issue that I am tackling...
     
  4. nsmith1024

    nsmith1024

    Joined:
    Mar 18, 2014
    Posts:
    870
    me too, whole thing freeze for 5 seconds while loading and processing images using WWW
     
    TokyoWarfareProject likes this.
  5. TokyoWarfareProject

    TokyoWarfareProject

    Joined:
    Jun 20, 2018
    Posts:
    815
    Hello, I've also been facing this issue. WWW. is ridiculously slow. Also I tried the new substitution for www, unity "
    using UnityEngine.Networking;" namespace, same ressults, I'm sure there is a bug because it can take like 10 seconds and some other times a minute or so, or even almost "fast" for very same scene loading very same tank skin, its all pretty random, I've been reading, some people say is a network adapter issue in unity blablabla, dunno, whatever, I found the way to fast load textures.

    In some other threads people suggest texture2d.LoadImage(yourByetArray), but while this is faster, it is still some some seconds freeze, the problem is not in the reading the
    fileData = File.ReadAllBytes(fullpath);
    the delay takes place in converting the byte array into a texture, so, the tex.LoadImage(byte[]).

    Some peopla said that that .loadRawTextureData(byte[]) was the fastest so I had tro try, and yes it is VERY fast. I think, from hwat I did read that you can load even a non raw saved textures by striping parts of the byte[] but that I've not yet tested. I've an skin editor and hwat I do is that before saving the texture, witch is already a texture2D type

    Code (CSharp):
    1.  
    2.                 if (rt)
    3.                 {
    4.                     Texture2D tx = new Texture2D(rt.width, rt.height, TextureFormat.ARGB32, false);
    5.                     RenderTexture.active = rt;
    6.                     tx.ReadPixels(new Rect(0, 0, rt.width, rt.height), 0, 0);
    7.                     tx.Apply();
    8.  
    9.                     //byte[] bytes = tx.EncodeToJPG(95);
    10.                     //byte[] bytes = tx.EncodeToPNG();
    11.                     byte[] bytes = tx.GetRawTextureData();
    12.  
    13.                 //file
    14.                 string fullPath = dirPath + textName;
    15.                 //Debug.Log("fullPath" + fullPath);
    16.                  System.IO.File.WriteAllBytes(fullPath, bytes);              
    17.                 }
    you can see how I was encodign to jpg or png before, notice ARGB32 texture format, you need same when loading texture. false is I for mipchain, set it true if when loading you're going to set true otherwise it will say it lacks info.

    when loading do this:

    Code (CSharp):
    1.  
    2.     public static Texture2D LoadSkin(string fullpath)
    3.     {
    4.         Texture2D tex = null;
    5.         byte[] fileData;
    6.  
    7.         if (File.Exists(fullpath))
    8.         {
    9.             fileData = File.ReadAllBytes(fullpath);
    10.             tex = new Texture2D(4096, 4096,TextureFormat.ARGB32,false);
    11.             //tex.LoadImage(fileData); //..this will auto-resize the texture dimensions.
    12.             tex.LoadRawTextureData(fileData); //.no longer.this will auto-resize the texture dimensions.
    13.             tex.Apply();
    14.         }
    15.  
    16.         return tex;
    17.     }
    notice you need to feed the texture size, as theese are skins I've hardcoded but you could feed this as a param I guess, also mip generation in this example is false but when I've all running will set to true. In my first attemp I miss put texture format with RGBA32 and colors where glitched, naming so similar :D

    hope this help. This aint ASYNC, but is fast enough. I found an async method posted in forums BUT it relies in too much thir party libs, good luck doing multyplatform with that.
    Further, I was making a mistake, if you are applying theh skin (in my case a tank) to all objects, same texture is a different instance on each part because LoadSkin() is called for each material, this is poorly coded, I've checked and each have a texture with a different id, instead, you should load the skin in a sort of SkinDispatcher with something like a dictionary like <string SkinID, texture2D skin> and you feed the material with that single instance of the skin. I've yet to do that part.

    Hope this helps to others.

    side note, may be textureload now is async behind scenes now, how to read more about this
    https://docs.unity3d.com/Manual/AsyncTextureUpload.html
     
    Last edited: Mar 6, 2019
  6. nsmith1024

    nsmith1024

    Joined:
    Mar 18, 2014
    Posts:
    870
    I have similar problem, I have a player walking around the scene, and depending on where the player is at any given time, i have to DOWNLOAD maybe 30 JPG images from internet, convert them to textures, then apply them to cubes near the player.

    So i have to do all this while the player is walking around without the walk animation getting jittery because of all this texture downloading and rendering. Right now its really horrible, it looks like an earthquake is happening in the scene when i start loading and rendering all these textures because of the extreme jitter.

    So i thought of putting all the image downloading and rendering in a separate thread from the main thread.

    Right now i wrote the thread so that the downloading is fairly fast and smooth without any jitter. The next thing is to avoid using LoadImage because that has to be done on the main thread and its really slow, this LoadImage is whats causing the jitter, i assume because its decoding the JPG in the main thread.

    So i think the solution is to be able to decode the JPG from inside the separate thread, then send the decoded data back to the main thread do LoadRawImageData which is fast.

    So im trying to find the source code for a JPG image decoder written in C# that i can use in a thread, there are several of the out there....

    https://keyj.emphy.de/nanojpeg/

    https://gist.github.com/jandk/3889823

    https://github.com/BitMiracle/libjpeg.net

    Does anybody think this approach will work? I havnt finished working on it yet, still trying to get one of these decoders to work.....

    Another way is probably to save the JPG as a raw bitmap before hand, and use that with the LoadRawImageData instead of the JPG, so that no decoding is necessary so it may eliminate the jitter, but i see that bitmap files can be maybe 6 times as large as a jpg file, which will cause a longer download time.

    Why do we have to go through all this trouble? Why dont Unity just make an asyc version of LoadImage??? instead of constantly creating new features while ignoring this HUGE problem that still exists in all of their versions.

    Thought they supposed to be image experts since the know how to make a GAME ENGINE which should be WAY MORE DIFFICULT than creating a async LoadImage.
     
    Last edited: Mar 6, 2019
  7. TokyoWarfareProject

    TokyoWarfareProject

    Joined:
    Jun 20, 2018
    Posts:
    815
    it is crazy they do not offer something fast, async and multiplatform out of the box... but this is classic unity style.

    So as a follow up I did what I commented above, the skin dispatcher and it works pretty fast.

    Code (CSharp):
    1.   //SKIN DISPATCHER // llamado desde skinManager
    2.     public void SkinDispatcher(string fullPath, Material m)
    3.     {
    4.         //la textura ya exsite
    5.         if (customSkinDic.ContainsKey(fullPath))
    6.         {
    7.             m.SetTexture("_MainTex", customSkinDic[fullPath]);
    8.         }
    9.  
    10.         //texture is not loaded
    11.         else
    12.         {
    13.            Texture2D skin= Utils.LoadSkin(fullPath);
    14.            customSkinDic.Add(fullPath, skin);
    15.            m.SetTexture("_MainTex", customSkinDic[fullPath]);
    16.  
    17.         }
    18.     }
    This avoids same texture be loaded twice.

    Still, it has minimal impact on load times but that hickup is breaking something in my gameplay, sure I've poorly implemented message system etc.... so I may change the dispatcher to load all the tank skins BEFORE theese are instantiated.

    ALSO I've noticed that textures you load with raw data ARE KEPT IN THE GPU unleess you unload them somehow something I've not yet done
     
  8. leebs0117

    leebs0117

    Joined:
    Jul 1, 2013
    Posts:
    7
    After unity 2017, you can try UnityWebRequestTexture.GetTexture.
     
  9. matias-lavik

    matias-lavik

    Joined:
    Sep 9, 2019
    Posts:
    13
    Old thread, but if anyone reading this needs an async texutre importer then I just made one: https://codeberg.org/matiaslavik/unity-async-textureimport

    It uses FreeImage to load the texture and generates mipmaps on a separate thread. You can use it from a coroutine, and yield until loading is done, without freezing the main thread.
     
  10. CPUCPU

    CPUCPU

    Joined:
    Nov 15, 2019
    Posts:
    10
    This is great! What platforms does it support? Looking to get this working on iOS/Android
     
  11. gulanxiu

    gulanxiu

    Joined:
    Mar 4, 2014
    Posts:
    14
    HI Hello, I am a Unity VR developer from China. I use the code you wrote: https://codeberg.org/matiaslavik/unity-async-textureimport. It's really great.



    My code level is very basic, I hope to get your help. I now use freeimage to decode images, and your code uses the tex.LoadRawTextureData(texData.data); method when loading, so that when loading 4K images, there will be freezes.



    I want to upload the GPU program loading directly, you can see the code of this friend: https://github.com/hecomi/UnityCustomTextureUpdate. The problem is that only PNG can be decoded, and many image formats can be decoded with freeimage.



    Can you teach me how to get texture ID and texture binding code with Freeimage? I hope to get your reply, my email: gulanxiu13@126.com. thank you very much.
     
  12. CastryGames

    CastryGames

    Joined:
    Oct 1, 2014
    Posts:
    77
    an alternative is to save the image, and load it directly with "UnityWebRequestTexture.GetTexture", since this function loads images and does not block the main thread.


    public void RecibirDatos(byte[] TexByte)
    {
    File.WriteAllBytes(Application.persistentDataPath + "/xd", TexByte);
    StartCoroutine(trabcou());
    }
    IEnumerator trabcou()
    {
    UnityWebRequest www = UnityWebRequestTexture.GetTexture("file:///" + Application.persistentDataPath + "/xd", true);
    yield return www.SendWebRequest();

    while (!www.isDone)
    {
    yield return www;
    }
    Texture2D x2 = new Texture2D(2, 2, TextureFormat.ARGB32, false);
    if (www.isNetworkError || www.isHttpError)
    {

    }
    else
    {
    x2 = ((DownloadHandlerTexture)www.downloadHandler).texture;
    x2.name = "uwu";
    }
    yield return null;
    File.Delete(Application.persistentDataPath + "/xd");

    //x2 load and not block main thread
    //
    }
     
    Zapan15 likes this.
  13. narf03

    narf03

    Joined:
    Aug 11, 2014
    Posts:
    223
    Just to update my own thread, i actually found a better way to solve this problem once and for all, with a paid asset
    https://assetstore.unity.com/packages/tools/integration/lightbuzz-super-fast-jpeg-123585

    I am not advertising for the asset, im just sharing my solution.

    The delay happen on the conversion, unity cannot use jpeg directly, so it need to convert jpeg to whatever format unity can use(depends on platform), once the conversion is done, loading it is extremely fast, so you will need to move the conversion to other thread(s) and only load the image in the main thread.

    1) Use the asset shown above to decode jpeg into byte array in thread.
    2) Use texture2D.LoadRawTextureData(decodedByteArray) and texture2D.Apply() in main thread.
     
  14. TokyoWarfareProject

    TokyoWarfareProject

    Joined:
    Jun 20, 2018
    Posts:
    815
    I found another way to load async, its not ideal but works. Save theese textures in a resources folder, and then call a resources.loadasync
     
  15. unity_VvCODKknmBJQGw

    unity_VvCODKknmBJQGw

    Joined:
    Aug 13, 2020
    Posts:
    7

    I have just encountered the problem, bought and downloaded that asset and its crashing. I have sent them a message, but until then - do not buy the asset.
     
  16. JSmithIR

    JSmithIR

    Joined:
    Apr 13, 2023
    Posts:
    113
    Will this work in build though? I didn't think builds had access to resources folder?
     
  17. TokyoWarfareProject

    TokyoWarfareProject

    Joined:
    Jun 20, 2018
    Posts:
    815
    It works, but it is not recommended. You may have memory usage issues because using resources folder is a mess in runtime to memory manage. On Xbox I noticed it takes a lot of memory just on start, I don´t know if it just allocates this memory to match the resources folder or what exactl is going on, but I guiess this is why unity does not recommend to use resources folder, despite of how convenient it is.