Texture2D.LoadImage is slow. Here's a workaround. @Alexey

Hey guys, particularly Alexey. ( bug report case # 605471)
Texture2D.LoadImage is slow. I've created a method to asynchronously load textures from files on disk on iOS. This is fast as hell and runs circles around Texture2D.LoadImage. Basically, you provide an URL of the image on disk you want to load, and it'll load the file asynchronously. I haven't heavily tested my method that resizes the texture if you ask it to, but resize == false version I tested a bunch.

usage:
mediaTexture = FastTexture2D.CreateFastTexture2D(media.url, false, media.w, media.h, GotFastTexture);
(GotFastTexture is a delegate of Action<FastTexture2D> OnFastTexture2DLoaded; )

Here's a C# class:

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3. using System.Collections.Generic;
4. using System;
5. using System.IO;
6. using System.Runtime.InteropServices;
7.  
8.  
9.  
10.  
11. public class FastTexture2D : ScriptableObject {
12. //(c) Brian Chasalow 2014 - brian@chasalow.com
13.     [AttributeUsage (AttributeTargets.Method)]
14.     public sealed class MonoPInvokeCallbackAttribute : Attribute {
15.         public MonoPInvokeCallbackAttribute (Type t) {}
16.     }
17.  
18.     [DllImport ("__Internal")]
19.     private static extern void DeleteFastTexture2DAtTextureID(int id);
20.     [DllImport ("__Internal")]
21.     private static extern void CreateFastTexture2DFromAssetPath(string assetPath, int uuid, bool resize, int resizeW, int resizeH);
22.    
23.  
24.     [DllImport ("__Internal")]
25.     private static extern void RegisterFastTexture2DCallbacks(TextureLoadedCallback callback);
26.  
27.     public static void CreateFastTexture2D(string path, int uuid, bool resize, int resizeW, int resizeH){
28.         #if UNITY_EDITOR
29.         #elif UNITY_IOS
30.         CreateFastTexture2DFromAssetPath(path, uuid, resize, resizeW, resizeH);
31.         #endif
32.     }
33.    
34.     public static void CleanupFastTexture2D(int texID){
35.         #if UNITY_EDITOR
36.         #elif UNITY_IOS
37.         DeleteFastTexture2DAtTextureID(texID);
38.         #endif
39.     }
40.  
41.  
42.     private static int tex2DCount = 0;
43.     private static Dictionary<int, FastTexture2D> instances;
44.     public static Dictionary<int, FastTexture2D> Instances{
45.         get{
46.             if(instances == null){
47.                 instances = new Dictionary<int, FastTexture2D>();
48.             }
49.             return instances;
50.         }
51.     }
52.  
53.     [SerializeField]
54.     public string url;
55.     [SerializeField]
56.     public int uuid;
57.     [SerializeField]
58.     public bool resize;
59.     [SerializeField]
60.     public int w;
61.     [SerializeField]
62.     public int h;
63.     [SerializeField]
64.     public int glTextureID;
65.     [SerializeField]
66.     private Texture2D nativeTexture;
67.     public Texture2D NativeTexture{ get{ return nativeTexture; }}
68.  
69.     [SerializeField]
70.     public bool isLoaded = false;
71.  
72.     public delegate void TextureLoadedCallback(int nativeTexID, int original_uuid, int w, int h);
73.  
74.     [MonoPInvokeCallback (typeof (TextureLoadedCallback))]
75.     public static void TextureLoaded(int nativeTexID, int original_uuid, int w, int h){
76.         if(Instances.ContainsKey(original_uuid)  nativeTexID > -1){
77.             FastTexture2D tex = Instances[original_uuid];
78.             tex.glTextureID = nativeTexID;
79.             tex.nativeTexture = Texture2D.CreateExternalTexture(w, h, TextureFormat.ARGB32, false, true, (System.IntPtr)nativeTexID);
80.             tex.nativeTexture.UpdateExternalTexture( (System.IntPtr)nativeTexID);
81.             tex.isLoaded = true;
82.             tex.OnFastTexture2DLoaded(tex);
83.         }
84.     }
85.  
86.  
87.     private Action<FastTexture2D> OnFastTexture2DLoaded;
88.  
89.     protected void InitFastTexture2D(string _url, int _uuid, bool _resize, int _w, int _h, Action<FastTexture2D> callback){
90.         this.url = _url;
91.         this.uuid = _uuid;
92.         this.resize = _resize;
93.         this.w = _w;
94.         this.h = _h;
95.         this.glTextureID = -1;
96.         this.OnFastTexture2DLoaded = callback;
97.         this.isLoaded = false;
98.     }
99.  
100.     private static bool registeredCallbacks = false;
101.     private static void RegisterTheCallbacks(){
102.         if(!registeredCallbacks){
103.             registeredCallbacks = true;
104.             #if UNITY_IOS
105.             if(Application.platform == RuntimePlatform.IPhonePlayer)
106.                 RegisterFastTexture2DCallbacks(TextureLoaded);
107.             #endif
108.  
109.         }
110.     }
111.  
112.  
113.     //dimensions options: if resize is false, w/h are not used. if true, it will downsample to provided dimensions.
114.     //to create a new texture, call this with the file path of the texture, resize parameters,
115.     //and a callback to be notified when the texture is loaded.
116.     public static FastTexture2D CreateFastTexture2D(string url,bool resize, int assetW, int assetH, Action<FastTexture2D> callback){
117.         //register that you want a callback when it's been created.
118.         RegisterTheCallbacks();
119.         //the uuid is the instance count at time of creation. you pass this into the method to grab the gl texture, and it returns the gl texture with this uuid
120.         int uuid = tex2DCount;
121.         tex2DCount = (tex2DCount +1 ) % int.MaxValue;
122.  
123.         FastTexture2D tex2D = ScriptableObject.CreateInstance<FastTexture2D>();
124.         tex2D.InitFastTexture2D(url, uuid, resize, assetW, assetH, callback);
125.         //call into the plugin to create the thing
126.         CreateFastTexture2D(tex2D.url, tex2D.uuid, tex2D.resize, tex2D.w, tex2D.h);
127.  
128.         //add the instance to the list
129.         Instances.Add(uuid, tex2D);
130.  
131.         //return the instance, someone might want it (but they'll get it with the callback soon anyway)
132.         return tex2D;
133.     }
134.    
135.  
136.     private void CleanupTexture(){
137.         isLoaded = false;
138.  
139.         //delete the gl texture
140.         if(glTextureID != -1)
141.             CleanupFastTexture2D(glTextureID);
142.         glTextureID = -1;
143.  
144.         //destroy the wrapper object
145.         if(nativeTexture)
146.             Destroy(nativeTexture);
147.  
148.         //remove it from the list so further callbacks dont try to find it
149.         if(Instances.ContainsKey(this.uuid))
150.             Instances.Remove(this.uuid);
151.     }
152.  
153.     //to destroy a FastTexture2D object, you call Destroy() on it.
154.     public void OnDestroy(){
155.         CleanupTexture();
156.     }
157.  
158. }
159.  

and you'll need this code in a .mm (i use it in an ARC-enabled compiled .a plugin, so no [whatever release] is necessary. )
and you will need this dependency: https://github.com/coryalder/UIImage_Resize

Code (csharp):

1.     extern "C" EAGLContext* ivcp_UnityGetContext();
2.     typedef void (*TextureLoadedCallback)(int texID, int originalUUID, int w, int h);
3.     static TextureLoadedCallback textureLoaded;
4.     static GLKTextureLoader* asyncLoader = nil;
5.     //(c) Brian Chasalow 2014 - brian@chasalow.com
6.     void RegisterFastTexture2DCallbacks(void (*cb)(int texID, int originalUUID, int w, int h)){
7.         textureLoaded = *cb;
8.     }
9.    
10.     void CreateFastTexture2DFromAssetPath(const char* assetPath, int uuid, bool resize, int resizeW, int resizeH){
11.         @autoreleasepool {
12.             NSDictionary* options = [NSDictionary dictionaryWithObjectsAndKeys:
13.                                      [NSNumber numberWithBool:YES],
14.                                      GLKTextureLoaderOriginBottomLeft,
15.                                      nil];
16.             //maybe look here?
17. //            http://stackoverflow.com/questions/16043204/handle-large-images-in-ios
18.             NSString* assetPathString = [NSString stringWithCString: assetPath encoding:NSUTF8StringEncoding];
19.            
20.             if(asyncLoader == nil)
21.             asyncLoader = [[GLKTextureLoader alloc] initWithSharegroup:[ivcp_UnityGetContext() sharegroup]];
22.            
23.             if(resize){
24.                 UIImage* img = [UIImage imageWithContentsOfFile:assetPathString];
25.                 __block UIImage* smallerImg = [img resizedImage:CGSizeMake(resizeW, resizeH) interpolationQuality:kCGInterpolationDefault ];
26.                                
27.                 [asyncLoader textureWithCGImage:[smallerImg CGImage]
28.                                         options:options
29.                                           queue:NULL
30.                               completionHandler:^(GLKTextureInfo *textureInfo, NSError *outError) {
31.                                   if(outError){
32.                                     smallerImg = nil;
33.                                     NSLog(@"got error creating texture at path: %@. error: %@ ", assetPathString,[outError localizedDescription] );
34.                                       textureLoaded(-1, uuid, 0, 0);
35.                                   }
36.                                   else{
37.                                       textureLoaded(textureInfo.name, uuid, resizeW, resizeH);
38.                                   }
39.                               }];
40.                
41.             }
42.             else{
43.                 [asyncLoader textureWithContentsOfFile:assetPathString
44.                                         options:options
45.                                           queue:NULL
46.                               completionHandler:^(GLKTextureInfo *textureInfo, NSError *outError) {
47.                       if(outError){
48.                           NSLog(@"got error creating texture at path: %@. error: %@ ", assetPathString,[outError localizedDescription] );
49.                               NSLog(@"returning texID -1 ");
50.                           textureLoaded(-1, uuid, 0, 0);
51.                       }
52.                       else
53.                       {
54.                           //this will get returned on the main thread cuz the queue above is NULL
55.                         textureLoaded(textureInfo.name, uuid, textureInfo.width, textureInfo.height);
56.                       }
57.                   }];
58.             }
59.         }
60.     }
61.    
62.     void DeleteFastTexture2DAtTextureID(int texID){
63.         @autoreleasepool {
64.             GLuint texIDGL = (GLuint)texID;
65.             if(texIDGL > 0){
66.                 if(glIsTexture(texIDGL)){
67. //                    NSLog(@"deleting a texture because it's a texture. %i", texIDGL);
68.                     glDeleteTextures(1, &texIDGL);
69.                 }
70.             }
71.         }
72.     }

and you'll need to inject ivcp_UnityGetContext() into the UnityAppController to get the surface.context, as in

Code (csharp):

1. extern "C" EAGLContext* ivcp_UnityGetContext()
2. {
3.     return GetAppController().mainDisplay->surface.context;
4. }
5.  

or use any other means of getting the render context that you desire.

I hope you guys can integrate something along these lines into Unity directly.
Maybe one day I'll make a plugin and release it on the asset store, but I'd rather you guys just fix it.
edit: fixed a bug that i inadvertently added as i cleaned up code to post it ;-)

 

You're allowed! :-D So if I'm following correctly, you mean:
1) loading a UIImage from disk
2) grabbing its CVImageBufferRef, via http://stackoverflow.com/questions/8138018/convert-uiimage-to-cvimagebufferref
3) passing it to CVOpenGLESTextureCacheCreateTextureFromImage, and the rest is the same?

That's not a bad idea, although it's not asynchronous, but would cut out the GLKit dependency which is a big plus. I'll stick with the asynchronous method for now because background image loading is a huge win for this app i'm working on.

Yeah I hadn't really poked at LoadRawTextureData, and no idea what that's doing behind the scenes.

 

@brianchasalow Thanks so much for the code. It needed some fixes before I got it to work, but if anyone else is looking for it, here it is:

The C# class above works as-is.

You need to put one file in Plugins/iOS:

FastTex2D.mm
Note: I commented out the resize part, as i didn't need it, if you want to use it un-comment it, put the includes provided above in the same folder and add the imports at the top of the file.

Code (CSharp):

1. //(c) Brian Chasalow 2014 - brian@chasalow.com
2. // Edits by Miha Krajnc
3. #import <GLKit/GLKit.h>
4.  
5. extern "C" {
6.  
7.   typedef void (*TextureLoadedCallback)(int texID, int originalUUID, int w, int h);
8.   static TextureLoadedCallback textureLoaded;
9.   static GLKTextureLoader* asyncLoader = nil;
10.  
11.   void RegisterFastTexture2DCallbacks(void (*cb)(int texID, int originalUUID, int w, int h)){
12.       textureLoaded = *cb;
13.   }
14.  
15.   void CreateFastTexture2DFromAssetPath(const char* assetPath, int uuid, bool resize, int resizeW, int resizeH){
16.       @autoreleasepool {
17.           NSDictionary* options = [NSDictionary dictionaryWithObjectsAndKeys:
18.                                    [NSNumber numberWithBool:YES],
19.                                    GLKTextureLoaderOriginBottomLeft,
20.                                    nil];
21.  
22.           NSString* assetPathString = [NSString stringWithCString: assetPath encoding:NSUTF8StringEncoding];
23.  
24.           if(asyncLoader == nil) {
25.               asyncLoader = [[GLKTextureLoader alloc] initWithSharegroup:[[EAGLContext currentContext] sharegroup]];
26.           }
27.  
28.           if(resize){
29.               // UIImage* img = [UIImage imageWithContentsOfFile:assetPathString];
30.               // __block UIImage* smallerImg = [img resizedImage:CGSizeMake(resizeW, resizeH) interpolationQuality:kCGInterpolationDefault ];
31.               //
32.               // [asyncLoader textureWithCGImage:[smallerImg CGImage]
33.               //                         options:options
34.               //                           queue:NULL
35.               //               completionHandler:^(GLKTextureInfo *textureInfo, NSError *outError) {
36.               //                   if(outError){
37.               //                     smallerImg = nil;
38.               //                     NSLog(@"got error creating texture at path: %@. error: %@ ", assetPathString,[outError localizedDescription] );
39.               //                       textureLoaded(-1, uuid, 0, 0);
40.               //                   }
41.               //                   else{
42.               //                       textureLoaded(textureInfo.name, uuid, resizeW, resizeH);
43.               //                   }
44.               //               }];
45.  
46.           } else {
47.               [asyncLoader textureWithContentsOfFile:assetPathString
48.                            options:options
49.                            queue:NULL
50.                            completionHandler:^(GLKTextureInfo *textureInfo, NSError *outError) {
51.                     if(outError){
52.                         NSLog(@"got error creating texture at path: %@. error: %@ ", assetPathString,[outError localizedDescription] );
53.                         NSLog(@"returning texID -1 ");
54.                         textureLoaded(-1, uuid, 0, 0);
55.                     }
56.                     else
57.                     {
58.                       //this will get returned on the main thread cuz the queue above is NULL
59.                       textureLoaded(textureInfo.name, uuid, textureInfo.width, textureInfo.height);
60.                     }
61.                 }];
62.           }
63.       }
64.   }
65.  
66.   void DeleteFastTexture2DAtTextureID(int texID){
67.       @autoreleasepool {
68.           GLuint texIDGL = (GLuint)texID;
69.           if(texIDGL > 0){
70.               if(glIsTexture(texIDGL)){
71.                   NSLog(@"deleting a texture because it's a texture. %i", texIDGL);
72.                   glDeleteTextures(1, &texIDGL);
73.               }
74.           }
75.       }
76.   }
77. }
78.  
79.  

After this, select the FastTex2D.mm file in unity, go to "Rarely used frameworks" in the inspector and select GLKit.

Now you must manually disable Metal as the Graphics API. Go to player settings and uncheck Auto Graphics API and remove metal from the list. Make sure you only have OpenGLES2 in the list.

That's all the setup you need, then to use the async loader you need to put your image files in a StreamingAssets folder, with the .bytes extension (I used png files), and in your code, you would run something like:

Code (CSharp):

1. public class TexLoad : MonoBehaviour {
2.     void Start () {
3.         FastTexture2D.CreateFastTexture2D (Application.streamingAssetsPath + "/tex.bytes", false, 0, 0, Callback);
4.     }
5.  
6.     private void Callback(FastTexture2D t){
7.         GetComponent<SpriteRenderer> ().sprite = Sprite.Create (t.NativeTexture, new Rect (0, 0, t.NativeTexture.width, t.NativeTexture.height), new Vector2 (.5f, .5f));
8.     }
9. }

And voila, async texture loading. I'm stumped why Unity doesn't natively support this yet, but never give up hope!

 

Great stuff- I'm glad it worked for you.

 

So what's the reason Unity doesn't natively support it? Is it the classic "there's nobody among our 500+ staff that has time" ?

 

Do you mean with Texture2D.LoadImage? I have a feeling that this method still works slightly faster, but it might not be the case.
In any way, my problem was that there is no way to asynchronously load a texture with the Unity API. I have a 2D game that uses around 20 2700x1700 images as backgrounds and using unity's Textuer2D.LoadImage has a visible lag of around half a second to a second. As I want to display an animated loading screen, this will not work.

Using unity's build in system doesn't work for me either. If I set the image quality of the sprite to TrueColor all images end up at around 16MB which adds up to a lot of space on the disk of a mobile device. Setting it to compressed, and putting them in a POT texture / sprite sheet degrades the quality too much.

Is there another way that you could do this that I am missing?

EDIT: Another thing: Does any iOS guru know why this isn't working on iPhones. iPads work fine, but when running this code on an iPhone it crashes, because the the EAGLContext from MyAppController is null...

 

mihakinova: you might be able to get away with [EAGLContext currentContext] instead of getting it from MyAppController.

and re: Povilas: the question was about Asynchronous texture loading, not just texture loading performance, and Unity's LoadImage is still a synchronous process as far as I am aware (or at least is on the UI thread) and causes visible jitter. Even if its per-second-loading-time performance has improved, that's not good enough for many use cases.

 

@brianchasalow Sadly it doesn't work. The weird thing though, is that it does work on iPads.... and I could have sworn that it didn't the first time I tried it... But it returns nil on iPhones. Any ideas?

 

Heureka! The fix is a simple yet stupid one. The reason it wasn't working on my iPhone is that it has iOS 9. And my iPad has iOS 7.
And so, Unity automatically used Metal as the Graphics API, so of course unity didn't find the context, because there wasn't one. And when i created it and sent the pointer to Texture2D.CreateExternalTexture it broke.

The solution is simple: Go to your player settings and uncheck Auto Graphics API, then remove Metal. This fix was good enough for my game, as I don't really need Metal.

 

In iOS 9.0 there's something called MetalKit which effectively does the same thing as GLKit's Texture loader. Unfortunately you might not want to restrict your users to iOS 9-only.
See MTKTextureLoader :
https://developer.apple.com/library...der/newTextureWithContentsOfURL:options:error:

 

texture2d.loadimage is still terribly slow, and an async method is needed too. Happy freezing while loading textures form my skin editor...