Search Unity

  1. Improved Prefab workflow (includes Nested Prefabs!), 2D isometric Tilemap and more! Get the 2018.3 Beta now.
    Dismiss Notice
  2. The Unity Pro & Visual Studio Professional Bundle gives you the tools you need to develop faster & collaborate more efficiently. Learn more.
    Dismiss Notice
  3. Improve your Unity skills with a certified instructor in a private, interactive classroom. Watch the overview now.
    Dismiss Notice
  4. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

Texture2D.LoadImage is slow. Here's a workaround. @Alexey

Discussion in 'iOS and tvOS' started by brianchasalow, May 9, 2014.

  1. brianchasalow

    brianchasalow

    Joined:
    Jun 3, 2010
    Posts:
    186
    Hey guys, particularly Alexey. ( bug report case # 605471)
    Texture2D.LoadImage is slow. I've created a method to asynchronously load textures from files on disk on iOS. This is fast as hell and runs circles around Texture2D.LoadImage. Basically, you provide an URL of the image on disk you want to load, and it'll load the file asynchronously. I haven't heavily tested my method that resizes the texture if you ask it to, but resize == false version I tested a bunch.

    usage:
    mediaTexture = FastTexture2D.CreateFastTexture2D(media.url, false, media.w, media.h, GotFastTexture);
    (GotFastTexture is a delegate of Action<FastTexture2D> OnFastTexture2DLoaded; )

    Here's a C# class:
    Code (csharp):
    1. using UnityEngine;
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using System;
    5. using System.IO;
    6. using System.Runtime.InteropServices;
    7.  
    8.  
    9.  
    10.  
    11. public class FastTexture2D : ScriptableObject {
    12. //(c) Brian Chasalow 2014 - brian@chasalow.com
    13.     [AttributeUsage (AttributeTargets.Method)]
    14.     public sealed class MonoPInvokeCallbackAttribute : Attribute {
    15.         public MonoPInvokeCallbackAttribute (Type t) {}
    16.     }
    17.  
    18.     [DllImport ("__Internal")]
    19.     private static extern void DeleteFastTexture2DAtTextureID(int id);
    20.     [DllImport ("__Internal")]
    21.     private static extern void CreateFastTexture2DFromAssetPath(string assetPath, int uuid, bool resize, int resizeW, int resizeH);
    22.    
    23.  
    24.     [DllImport ("__Internal")]
    25.     private static extern void RegisterFastTexture2DCallbacks(TextureLoadedCallback callback);
    26.  
    27.     public static void CreateFastTexture2D(string path, int uuid, bool resize, int resizeW, int resizeH){
    28.         #if UNITY_EDITOR
    29.         #elif UNITY_IOS
    30.         CreateFastTexture2DFromAssetPath(path, uuid, resize, resizeW, resizeH);
    31.         #endif
    32.     }
    33.    
    34.     public static void CleanupFastTexture2D(int texID){
    35.         #if UNITY_EDITOR
    36.         #elif UNITY_IOS
    37.         DeleteFastTexture2DAtTextureID(texID);
    38.         #endif
    39.     }
    40.  
    41.  
    42.     private static int tex2DCount = 0;
    43.     private static Dictionary<int, FastTexture2D> instances;
    44.     public static Dictionary<int, FastTexture2D> Instances{
    45.         get{
    46.             if(instances == null){
    47.                 instances = new Dictionary<int, FastTexture2D>();
    48.             }
    49.             return instances;
    50.         }
    51.     }
    52.  
    53.     [SerializeField]
    54.     public string url;
    55.     [SerializeField]
    56.     public int uuid;
    57.     [SerializeField]
    58.     public bool resize;
    59.     [SerializeField]
    60.     public int w;
    61.     [SerializeField]
    62.     public int h;
    63.     [SerializeField]
    64.     public int glTextureID;
    65.     [SerializeField]
    66.     private Texture2D nativeTexture;
    67.     public Texture2D NativeTexture{ get{ return nativeTexture; }}
    68.  
    69.     [SerializeField]
    70.     public bool isLoaded = false;
    71.  
    72.     public delegate void TextureLoadedCallback(int nativeTexID, int original_uuid, int w, int h);
    73.  
    74.     [MonoPInvokeCallback (typeof (TextureLoadedCallback))]
    75.     public static void TextureLoaded(int nativeTexID, int original_uuid, int w, int h){
    76.         if(Instances.ContainsKey(original_uuid)  nativeTexID > -1){
    77.             FastTexture2D tex = Instances[original_uuid];
    78.             tex.glTextureID = nativeTexID;
    79.             tex.nativeTexture = Texture2D.CreateExternalTexture(w, h, TextureFormat.ARGB32, false, true, (System.IntPtr)nativeTexID);
    80.             tex.nativeTexture.UpdateExternalTexture( (System.IntPtr)nativeTexID);
    81.             tex.isLoaded = true;
    82.             tex.OnFastTexture2DLoaded(tex);
    83.         }
    84.     }
    85.  
    86.  
    87.     private Action<FastTexture2D> OnFastTexture2DLoaded;
    88.  
    89.     protected void InitFastTexture2D(string _url, int _uuid, bool _resize, int _w, int _h, Action<FastTexture2D> callback){
    90.         this.url = _url;
    91.         this.uuid = _uuid;
    92.         this.resize = _resize;
    93.         this.w = _w;
    94.         this.h = _h;
    95.         this.glTextureID = -1;
    96.         this.OnFastTexture2DLoaded = callback;
    97.         this.isLoaded = false;
    98.     }
    99.  
    100.     private static bool registeredCallbacks = false;
    101.     private static void RegisterTheCallbacks(){
    102.         if(!registeredCallbacks){
    103.             registeredCallbacks = true;
    104.             #if UNITY_IOS
    105.             if(Application.platform == RuntimePlatform.IPhonePlayer)
    106.                 RegisterFastTexture2DCallbacks(TextureLoaded);
    107.             #endif
    108.  
    109.         }
    110.     }
    111.  
    112.  
    113.     //dimensions options: if resize is false, w/h are not used. if true, it will downsample to provided dimensions.
    114.     //to create a new texture, call this with the file path of the texture, resize parameters,
    115.     //and a callback to be notified when the texture is loaded.
    116.     public static FastTexture2D CreateFastTexture2D(string url,bool resize, int assetW, int assetH, Action<FastTexture2D> callback){
    117.         //register that you want a callback when it's been created.
    118.         RegisterTheCallbacks();
    119.         //the uuid is the instance count at time of creation. you pass this into the method to grab the gl texture, and it returns the gl texture with this uuid
    120.         int uuid = tex2DCount;
    121.         tex2DCount = (tex2DCount +1 ) % int.MaxValue;
    122.  
    123.         FastTexture2D tex2D = ScriptableObject.CreateInstance<FastTexture2D>();
    124.         tex2D.InitFastTexture2D(url, uuid, resize, assetW, assetH, callback);
    125.         //call into the plugin to create the thing
    126.         CreateFastTexture2D(tex2D.url, tex2D.uuid, tex2D.resize, tex2D.w, tex2D.h);
    127.  
    128.         //add the instance to the list
    129.         Instances.Add(uuid, tex2D);
    130.  
    131.         //return the instance, someone might want it (but they'll get it with the callback soon anyway)
    132.         return tex2D;
    133.     }
    134.    
    135.  
    136.     private void CleanupTexture(){
    137.         isLoaded = false;
    138.  
    139.         //delete the gl texture
    140.         if(glTextureID != -1)
    141.             CleanupFastTexture2D(glTextureID);
    142.         glTextureID = -1;
    143.  
    144.         //destroy the wrapper object
    145.         if(nativeTexture)
    146.             Destroy(nativeTexture);
    147.  
    148.         //remove it from the list so further callbacks dont try to find it
    149.         if(Instances.ContainsKey(this.uuid))
    150.             Instances.Remove(this.uuid);
    151.     }
    152.  
    153.     //to destroy a FastTexture2D object, you call Destroy() on it.
    154.     public void OnDestroy(){
    155.         CleanupTexture();
    156.     }
    157.  
    158. }
    159.  
    and you'll need this code in a .mm (i use it in an ARC-enabled compiled .a plugin, so no [whatever release] is necessary. )
    and you will need this dependency: https://github.com/coryalder/UIImage_Resize

    Code (csharp):
    1.     extern "C" EAGLContext* ivcp_UnityGetContext();
    2.     typedef void (*TextureLoadedCallback)(int texID, int originalUUID, int w, int h);
    3.     static TextureLoadedCallback textureLoaded;
    4.     static GLKTextureLoader* asyncLoader = nil;
    5.     //(c) Brian Chasalow 2014 - brian@chasalow.com
    6.     void RegisterFastTexture2DCallbacks(void (*cb)(int texID, int originalUUID, int w, int h)){
    7.         textureLoaded = *cb;
    8.     }
    9.    
    10.     void CreateFastTexture2DFromAssetPath(const char* assetPath, int uuid, bool resize, int resizeW, int resizeH){
    11.         @autoreleasepool {
    12.             NSDictionary* options = [NSDictionary dictionaryWithObjectsAndKeys:
    13.                                      [NSNumber numberWithBool:YES],
    14.                                      GLKTextureLoaderOriginBottomLeft,
    15.                                      nil];
    16.             //maybe look here?
    17. //            http://stackoverflow.com/questions/16043204/handle-large-images-in-ios
    18.             NSString* assetPathString = [NSString stringWithCString: assetPath encoding:NSUTF8StringEncoding];
    19.            
    20.             if(asyncLoader == nil)
    21.             asyncLoader = [[GLKTextureLoader alloc] initWithSharegroup:[ivcp_UnityGetContext() sharegroup]];
    22.            
    23.             if(resize){
    24.                 UIImage* img = [UIImage imageWithContentsOfFile:assetPathString];
    25.                 __block UIImage* smallerImg = [img resizedImage:CGSizeMake(resizeW, resizeH) interpolationQuality:kCGInterpolationDefault ];
    26.                                
    27.                 [asyncLoader textureWithCGImage:[smallerImg CGImage]
    28.                                         options:options
    29.                                           queue:NULL
    30.                               completionHandler:^(GLKTextureInfo *textureInfo, NSError *outError) {
    31.                                   if(outError){
    32.                                     smallerImg = nil;
    33.                                     NSLog(@"got error creating texture at path: %@. error: %@ ", assetPathString,[outError localizedDescription] );
    34.                                       textureLoaded(-1, uuid, 0, 0);
    35.                                   }
    36.                                   else{
    37.                                       textureLoaded(textureInfo.name, uuid, resizeW, resizeH);
    38.                                   }
    39.                               }];
    40.                
    41.             }
    42.             else{
    43.                 [asyncLoader textureWithContentsOfFile:assetPathString
    44.                                         options:options
    45.                                           queue:NULL
    46.                               completionHandler:^(GLKTextureInfo *textureInfo, NSError *outError) {
    47.                       if(outError){
    48.                           NSLog(@"got error creating texture at path: %@. error: %@ ", assetPathString,[outError localizedDescription] );
    49.                               NSLog(@"returning texID -1 ");
    50.                           textureLoaded(-1, uuid, 0, 0);
    51.                       }
    52.                       else
    53.                       {
    54.                           //this will get returned on the main thread cuz the queue above is NULL
    55.                         textureLoaded(textureInfo.name, uuid, textureInfo.width, textureInfo.height);
    56.                       }
    57.                   }];
    58.             }
    59.         }
    60.     }
    61.    
    62.     void DeleteFastTexture2DAtTextureID(int texID){
    63.         @autoreleasepool {
    64.             GLuint texIDGL = (GLuint)texID;
    65.             if(texIDGL > 0){
    66.                 if(glIsTexture(texIDGL)){
    67. //                    NSLog(@"deleting a texture because it's a texture. %i", texIDGL);
    68.                     glDeleteTextures(1, &texIDGL);
    69.                 }
    70.             }
    71.         }
    72.     }
    and you'll need to inject ivcp_UnityGetContext() into the UnityAppController to get the surface.context, as in
    Code (csharp):
    1. extern "C" EAGLContext* ivcp_UnityGetContext()
    2. {
    3.     return GetAppController().mainDisplay->surface.context;
    4. }
    5.  
    or use any other means of getting the render context that you desire.

    I hope you guys can integrate something along these lines into Unity directly.
    Maybe one day I'll make a plugin and release it on the asset store, but I'd rather you guys just fix it.
    edit: fixed a bug that i inadvertently added as i cleaned up code to post it ;-)
     
    Last edited: Sep 26, 2014
    Gustavo-Quiroz likes this.
  2. Alexey

    Alexey

    Unity Technologies

    Joined:
    May 10, 2010
    Posts:
    1,372
    if i am allowed to whine: you could cut the code quite considerably and get rid of GLKit dependency if you used CVTextureCache we provide (in <TrampolineFolder>/Classes/Unity ;-)
    also, we kinda provide more and more ways to make stuff behind unity's back, e.g. we've added (sorry, i dont remember exact version, so it if is not yet out, sorry) Texture2D.LoadRawTextureData (though it wont do png conversion, sure).
    so for me it is more about giving you tools not implementing everything ;-)
     
  3. brianchasalow

    brianchasalow

    Joined:
    Jun 3, 2010
    Posts:
    186
    You're allowed! :-D So if I'm following correctly, you mean:
    1) loading a UIImage from disk
    2) grabbing its CVImageBufferRef, via http://stackoverflow.com/questions/8138018/convert-uiimage-to-cvimagebufferref
    3) passing it to CVOpenGLESTextureCacheCreateTextureFromImage, and the rest is the same?

    That's not a bad idea, although it's not asynchronous, but would cut out the GLKit dependency which is a big plus. I'll stick with the asynchronous method for now because background image loading is a huge win for this app i'm working on.

    Yeah I hadn't really poked at LoadRawTextureData, and no idea what that's doing behind the scenes.
     
  4. mihakinova

    mihakinova

    Joined:
    Jan 6, 2015
    Posts:
    84
    @brianchasalow Thanks so much for the code. It needed some fixes before I got it to work, but if anyone else is looking for it, here it is:

    The C# class above works as-is.

    You need to put one file in Plugins/iOS:

    FastTex2D.mm
    Note: I commented out the resize part, as i didn't need it, if you want to use it un-comment it, put the includes provided above in the same folder and add the imports at the top of the file.
    Code (CSharp):
    1. //(c) Brian Chasalow 2014 - brian@chasalow.com
    2. // Edits by Miha Krajnc
    3. #import <GLKit/GLKit.h>
    4.  
    5. extern "C" {
    6.  
    7.   typedef void (*TextureLoadedCallback)(int texID, int originalUUID, int w, int h);
    8.   static TextureLoadedCallback textureLoaded;
    9.   static GLKTextureLoader* asyncLoader = nil;
    10.  
    11.   void RegisterFastTexture2DCallbacks(void (*cb)(int texID, int originalUUID, int w, int h)){
    12.       textureLoaded = *cb;
    13.   }
    14.  
    15.   void CreateFastTexture2DFromAssetPath(const char* assetPath, int uuid, bool resize, int resizeW, int resizeH){
    16.       @autoreleasepool {
    17.           NSDictionary* options = [NSDictionary dictionaryWithObjectsAndKeys:
    18.                                    [NSNumber numberWithBool:YES],
    19.                                    GLKTextureLoaderOriginBottomLeft,
    20.                                    nil];
    21.  
    22.           NSString* assetPathString = [NSString stringWithCString: assetPath encoding:NSUTF8StringEncoding];
    23.  
    24.           if(asyncLoader == nil) {
    25.               asyncLoader = [[GLKTextureLoader alloc] initWithSharegroup:[[EAGLContext currentContext] sharegroup]];
    26.           }
    27.  
    28.           if(resize){
    29.               // UIImage* img = [UIImage imageWithContentsOfFile:assetPathString];
    30.               // __block UIImage* smallerImg = [img resizedImage:CGSizeMake(resizeW, resizeH) interpolationQuality:kCGInterpolationDefault ];
    31.               //
    32.               // [asyncLoader textureWithCGImage:[smallerImg CGImage]
    33.               //                         options:options
    34.               //                           queue:NULL
    35.               //               completionHandler:^(GLKTextureInfo *textureInfo, NSError *outError) {
    36.               //                   if(outError){
    37.               //                     smallerImg = nil;
    38.               //                     NSLog(@"got error creating texture at path: %@. error: %@ ", assetPathString,[outError localizedDescription] );
    39.               //                       textureLoaded(-1, uuid, 0, 0);
    40.               //                   }
    41.               //                   else{
    42.               //                       textureLoaded(textureInfo.name, uuid, resizeW, resizeH);
    43.               //                   }
    44.               //               }];
    45.  
    46.           } else {
    47.               [asyncLoader textureWithContentsOfFile:assetPathString
    48.                            options:options
    49.                            queue:NULL
    50.                            completionHandler:^(GLKTextureInfo *textureInfo, NSError *outError) {
    51.                     if(outError){
    52.                         NSLog(@"got error creating texture at path: %@. error: %@ ", assetPathString,[outError localizedDescription] );
    53.                         NSLog(@"returning texID -1 ");
    54.                         textureLoaded(-1, uuid, 0, 0);
    55.                     }
    56.                     else
    57.                     {
    58.                       //this will get returned on the main thread cuz the queue above is NULL
    59.                       textureLoaded(textureInfo.name, uuid, textureInfo.width, textureInfo.height);
    60.                     }
    61.                 }];
    62.           }
    63.       }
    64.   }
    65.  
    66.   void DeleteFastTexture2DAtTextureID(int texID){
    67.       @autoreleasepool {
    68.           GLuint texIDGL = (GLuint)texID;
    69.           if(texIDGL > 0){
    70.               if(glIsTexture(texIDGL)){
    71.                   NSLog(@"deleting a texture because it's a texture. %i", texIDGL);
    72.                   glDeleteTextures(1, &texIDGL);
    73.               }
    74.           }
    75.       }
    76.   }
    77. }
    78.  
    79.  
    After this, select the FastTex2D.mm file in unity, go to "Rarely used frameworks" in the inspector and select GLKit.

    Now you must manually disable Metal as the Graphics API. Go to player settings and uncheck Auto Graphics API and remove metal from the list. Make sure you only have OpenGLES2 in the list.

    That's all the setup you need, then to use the async loader you need to put your image files in a StreamingAssets folder, with the .bytes extension (I used png files), and in your code, you would run something like:

    Code (CSharp):
    1. public class TexLoad : MonoBehaviour {
    2.     void Start () {
    3.         FastTexture2D.CreateFastTexture2D (Application.streamingAssetsPath + "/tex.bytes", false, 0, 0, Callback);
    4.     }
    5.  
    6.     private void Callback(FastTexture2D t){
    7.         GetComponent<SpriteRenderer> ().sprite = Sprite.Create (t.NativeTexture, new Rect (0, 0, t.NativeTexture.width, t.NativeTexture.height), new Vector2 (.5f, .5f));
    8.     }
    9. }
    And voila, async texture loading. I'm stumped why Unity doesn't natively support this yet, but never give up hope!
     
    Last edited: Nov 18, 2015
    MrEsquire likes this.
  5. brianchasalow

    brianchasalow

    Joined:
    Jun 3, 2010
    Posts:
    186
    Great stuff- I'm glad it worked for you.
     
  6. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    23,000
    So what's the reason Unity doesn't natively support it? Is it the classic "there's nobody among our 500+ staff that has time" ? ;)
     
  7. povilas

    povilas

    Unity Technologies

    Joined:
    Jan 28, 2014
    Posts:
    427
    We've optimized slow texture loading around a year ago. After improvement, it's possible to achieve comparable performance using entirely Unity's APIs.
     
  8. mihakinova

    mihakinova

    Joined:
    Jan 6, 2015
    Posts:
    84
    Do you mean with Texture2D.LoadImage? I have a feeling that this method still works slightly faster, but it might not be the case.
    In any way, my problem was that there is no way to asynchronously load a texture with the Unity API. I have a 2D game that uses around 20 2700x1700 images as backgrounds and using unity's Textuer2D.LoadImage has a visible lag of around half a second to a second. As I want to display an animated loading screen, this will not work.

    Using unity's build in system doesn't work for me either. If I set the image quality of the sprite to TrueColor all images end up at around 16MB which adds up to a lot of space on the disk of a mobile device. Setting it to compressed, and putting them in a POT texture / sprite sheet degrades the quality too much.

    Is there another way that you could do this that I am missing?

    EDIT: Another thing: Does any iOS guru know why this isn't working on iPhones. iPads work fine, but when running this code on an iPhone it crashes, because the the EAGLContext from MyAppController is null...
     
    Last edited: Nov 17, 2015
  9. brianchasalow

    brianchasalow

    Joined:
    Jun 3, 2010
    Posts:
    186
    mihakinova: you might be able to get away with [EAGLContext currentContext] instead of getting it from MyAppController.

    and re: Povilas: the question was about Asynchronous texture loading, not just texture loading performance, and Unity's LoadImage is still a synchronous process as far as I am aware (or at least is on the UI thread) and causes visible jitter. Even if its per-second-loading-time performance has improved, that's not good enough for many use cases.
     
  10. mihakinova

    mihakinova

    Joined:
    Jan 6, 2015
    Posts:
    84
    @brianchasalow Sadly it doesn't work. The weird thing though, is that it does work on iPads.... and I could have sworn that it didn't the first time I tried it... But it returns nil on iPhones. Any ideas?
     
  11. mihakinova

    mihakinova

    Joined:
    Jan 6, 2015
    Posts:
    84
    Heureka! The fix is a simple yet stupid one. The reason it wasn't working on my iPhone is that it has iOS 9. And my iPad has iOS 7.
    And so, Unity automatically used Metal as the Graphics API, so of course unity didn't find the context, because there wasn't one. And when i created it and sent the pointer to Texture2D.CreateExternalTexture it broke.

    The solution is simple: Go to your player settings and uncheck Auto Graphics API, then remove Metal. This fix was good enough for my game, as I don't really need Metal.
     
  12. brianchasalow

    brianchasalow

    Joined:
    Jun 3, 2010
    Posts:
    186