Question regarding pulling data (like textures etc) from WWW

This is going to be used for Mobile Devices:

I wasn't really sure where to post this question, but this seems like the best place. I was wondering how I can pull data efficiently from URLs efficiently without clogging up app memory and causing the app to crash.

An example of what is going on at the moment:
My app will be pulling rows from tables in MySQL and then storing that information as an object of a custom class.
So each "object" will have several variables including URLs that link to images.
Then in the app, those URLs will be accessed in order to store the image as a Texture for that "object".

All this works fine, but once the app gets to a point where its downloading 100s of images, (Currently loading one at a time in order to stop app from hanging) the app will crash (on the device, not in the editor).

The reason why I need to access these images during runtime is because this information will constantly be changing so I cannot hardcode into the app.

I apologize in advance if I did not explain this properly. Please ask me anything that might help you help me.

Thank you!

 

You might want to look at Resources.UnloadUnusedAssets for releasing those textures you no longer need.

 

Andymads is right by using Resources.UnloadUnusedAssets, but also.. Are your WWW requests running through a Coroutine or is it just a regular method? You should use a coroutine. Coroutines run on a separate thread from everything else, so it won't cause a hang. They'll all download in the background. As a rule of thumb, for Unity, you should always put any internet/download logic into a coroutine.

 

Not true. They just normally run after Update.

See http://docs.unity3d.com/Manual/ExecutionOrder.html

 

Yes, I am using a Coroutine and I am also queueing up the calls as well which actually stops a BIG hang when trying to call all the WWW requests at once.

But Resources.UnloadUnusedAssets does not seem to be doing anything. To give a better explanation of whats going on at one point in this app;

Each user can view his prizes that he won inside his profile page, but these prize information is being downloaded from a SQL table

(Technically I guess I can store this data on the device since it will not be changing too often, but this entire issue will appear on a different scene as well, so I'll continue with this example)

So each slot (prize) that is won retrieves a quick call from the database and fills up the object (custom class) with some information. This is not the problem (I assume) The problem comes right after when each of these objects start to load their images from the URL that was given from the SQL call and these are all WWW requests to grab the Texture and add it to the object which is then displayed on screen. I noticed when they were only a few prizes being drawn, there is no issue. But once there were over 50 or maybe 100 (I did not remember to check the amount) During the load of all the Textures from the WWW requests, the app crashes on the device.

 

Let's take a step back. Are you sure that the crash is due to running out of memory? What do you do with the textures once you've downloaded them? Do you save them locally? Do you release the references when you're finished? You can use the profiler (might be Pro only?) to see all the loaded textures. Once you release the references and do Resources.UnloadUnusedAssets you'll see the difference in the profiler.

 

I can confirm that the profiler is a Pro-Only feature... It's kind-of ridiculous that it's pro-only, but hey, I'm not one to complain about free software xP

 

I have Pro Version so thats not problem, But I have a script running that will display all objects/textures/scripts (etc) currently being used so I can keep track of all the textures being loaded.

So basically, each object from this class will have a Texture variable which gets its value (image) from a WWW request. Once it downloads the image and stores it in said object, it leaves it there until the app restarts and it makes the call again. I don't save them to disk they are only kept on that item.

 

Well Resources.UnloadUnusedAssets isn't going to help then as you're holding onto all the textures.

So each time the app runs it will download all the same textures again? Seems like a waste.

I've been working on something that needs to download textures, but I save them locally and then next time I can just read straight from local storage. Once I've done with the texture I can release the reference, and call Resources.UnloadUnusedAssets as required. If I need it again I'll load it back from local storage again.

 

Well I can understand the benefits of storing the textures locally for the prizes (Not 100% sure on a good way to do that, if you can point me in the right direction that would be great)

But as for the other scene, there will be no point in storing these textures locally and reusing them because these items will constantly be changing throughout the day so if they get downloaded at 10am, its possible at 11am all those that were there at 10am are gone from the server and there are now 100 new items to be downloaded.

Also how can I release the reference? Because I tried clearing the List that held all these items and unloading assets but the Texture count remained the same.

 

I use Texture2D.EncodeToPNG and System.IO.File.WriteAllBytes to save out a downloaded texture. To load back in I use System.IO.File.ReadAllBytes, create a new Texture and then use Texture.LoadImage with the bytes.

Yeah, clearing the list should do the trick. What texture count are you referring to?

 

Well I have this script that tells me all the counts of stuff currently running

Code (CSharp):

1. Object[] objects = FindObjectsOfType(typeof (UnityEngine.Object));
2.        
3.         Dictionary<string, int> dictionary = new Dictionary<string, int>();
4.        
5.         foreach(Object obj in objects)
6.         {
7.             string key = obj.GetType().ToString();
8.             if(dictionary.ContainsKey(key))
9.             {
10.                 dictionary[key]++;
11.             }
12.             else
13.             {
14.                 dictionary[key] = 1;
15.             }
16.         }
17.        
18.         List<KeyValuePair<string, int>> myList = new List<KeyValuePair<string, int>>(dictionary);
19.         myList.Sort(
20.             delegate(KeyValuePair<string, int> firstPair,
21.                  KeyValuePair<string, int> nextPair)
22.             {
23.             return nextPair.Value.CompareTo((firstPair.Value));
24.         }
25.         );
26.        
27.         foreach (KeyValuePair<string, int> entry in myList)
28.         {
29.             GUILayout.Label(entry.Key + ": " + entry.Value);
30.         }

And this Texture count stays the same even after clearing the list and unloading unused assets.

But what do you suggest I do for the other scene where it can technically have 100s or 1000s of images being loaded in and being assigned to an object? I cant store them locally. Or is there a way I can load up only a few at a time and then reuse texture space or something?

 

Its definitely a Texture memory issue. I just tried to load over 100 Textures sequentially in the Editor and received this message:

Code (CSharp):

1. Could not allocate memory: System out of memory!
2. Trying to allocate: 2239491B with 32 alignment. MemoryLabel: Texture
3. Allocation happend at: Line:411 in /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Graphics/Texture2D.cpp
4. Memory overview
5.  
6.  
7. [ ALLOC_DEFAULT ] used: 105457832B | peak: 0B | reserved: 155888300B
8.  
9. [ ALLOC_GFX ] used: 71202335B | peak: 0B | reserved: 101711872B
10.  
11. [ ALLOC_CACHEOBJECTS ] used: 4385774B | peak: 0B | reserved: 20971520B
12.  
13. [ ALLOC_TYPETREE ] used: 1486328B | peak: 0B | reserved: 17825792B
14.  
15. [ ALLOC_PROFILER ] used: 176313B | peak: 0B | reserved: 8388608B
16.  
17. UnityEngine.WWW:get_texture()
18. <LoadImage>c__IteratorF:MoveNext() (at Assets/Scripts/Product.cs:106)

 

The memory profiler (detailed view) will tell you what the references are so you'll have to find the textures in there (they might not have a name in the profiler if under Scene Memory). Check your code for anything that references them. Materials perhaps?

Why can't you store them locally? What do you mean by load up a few? From where?

 

Thanks for your help. I played around with my scripts and I believe I figured out my issue and a better way to do what I'm doing. The issue seemed to be that the images were really large, and by making them much smaller I don't have the issue anymore