Bug Texture2D.ReadPixels leaks memory in standalone build

MacBook Pro 13-inch, M1, 2020
Mac OS 13.1（22C65）
UNITY 2021.3.16f1 Personal & UNITY 2021.3.17f1 Personal

In order to create a function to output the game screen as a video, we capture the screen and save a series of still images.
If you build and test Mac Standalone Applications after upgrading to 2021.3.16f1, the memory will increase in the ReadPixels part.
This phenomenon did not occur in 2019.2.3f1.
It doesn't happen in unity editor either.
When I do a standalone development build and check the profiler, the Untracked Memory keeps increasing during the capture process.
Furthermore, it will not be freed when the capture ends.
this is
Texture2D.ReadPixels(Rect, 0, 0);
is occurring in
Destroy(Texture2D);
but it increases regardless.
Moreover, the memory growth rate varies greatly depending on the content of the captured screen.
It seems to increase more on complex screens than on simple screens.
The number of materials and the number of objects may have an effect.
However, I tried to create a simple program that keeps capturing the screen, but the phenomenon that the memory keeps increasing did not occur.
It may be because the displayed object is too simple.
When I put that simple program into the project in question and try to capture a complex screen, the memory still continues to grow.
Since the running program is the same, I think there is no doubt that the pixel part to be referred to has an effect.
If you move the object out of the display range from the screen, the memory will stop increasing.
(memory is not freed)
on Mac
Intel
Apple Silicon
Intel + Apple Silicon
All had the same result.
mono IL2CPP both occur.
I haven't checked on Windows.
I don't know if it's a bug or a specification change, but I spent a long time and couldn't find a way to avoid it.
Please let me know if there is a cause or a solution.
A simple code looks like this:

Code (CSharp):

1. private IEnumerator caputure()
2.     {
3.  
4.         for (float i = 0; i < 1000000000; i += 0.0000001f)
5.         {
6.             int h = Screen.height;
7.             int w = Screen.width;
8.  
9.  
10.  
11.  
12.             Texture2D tex = new Texture2D(w, h, TextureFormat.RGB24, false);
13.  
14.             yield return new WaitForEndOfFrame();
15.  
16.             Rect rect = new Rect(0,0, w, h);
17.             tex.ReadPixels(rect, 0, 0, false);
18.  
19.             tex.Apply();
20.  
21.  
22.             s++;
23.  
24.  
25.             File.WriteAllBytes(Application.streamingAssetsPath + "/moviepng/" + s + ".png",　tex.EncodeToPNG());
26.  
27.             Destroy(tex);
28.  
29.         }
30.    
31.     }

 

Probably won't make a difference but have you tried
- DestroyImmediate instead of Destroy()
- tex.hideFlags = HideFlags.HideAndDontSave;

 

Thank you for your advice.
I changed it as follows and executed it, but it didn't change.
It turns out that the memory starts to decrease when the window size of the game screen is made as small as possible.
However, the decrease stops at about 4 gigabytes.
If you leave the capture continuously with a large window size, it will amplify to tens of gigabytes.
I was wondering if captured data remains, but the fact that it turns to decrease may not mean that past captured data remains.

Code (CSharp):

1.             Texture2D tex = new Texture2D(w, h, TextureFormat.RGB24, false);
2.             tex.hideFlags = HideFlags.HideAndDontSave;
3.             yield return new WaitForEndOfFrame();
4.             Rect rect = new Rect(0,0, w, h);
5.             tex.ReadPixels(rect, 0, 0,false);
6.             tex.Apply();
7.  
8.             s++;
9.  
10.             File.WriteAllBytes(Application.streamingAssetsPath + "/moviepng/" + s + ".png",tex.EncodeToPNG());
11.             DestroyImmediate(tex);
12.  
13.  

 

Looks like a bug to me but maybe you can use ScreenCapture.CaptureScreenshot instead?

 

I thought about using ScreenCapture.CaptureScreenshot, but I didn't because I can't specify the capture range.
But I tried using CaptureScreenshot for testing.
The result is also memory amplification.
However, when I tested it on my iPad, the memory did not increase even with [ReadPixels] and [CaptureScreenshot] and it was normal.
There are many incomprehensible parts in the increase and decrease of memory like the video.
If you make the window smaller, it will decrease, but if you make it smaller, there may be a size that suddenly increases.
Since there was no problem in the 2019 version, I believe that it is likely to be a bug.

 

Could be a bug, but the fact you are creating a new texture every frame might mean the OS is simply grabbing new memory each time, never re-using it. Normally such an approach would reclaim that memory at some point, but perhaps in this case it isn’t, which is still a bug and should be reported.

simple test then. Instead of creating a new texture every frame, create it once at runtime ( add some code to check for changes in dimension, and only destroy/ recreate the texture if that occurs ) then read pixels into the same texture each frame instead. Don’t forget to destroy the texture when quitting the app.

see if that still exhibits the memory leak.

Regardless I’d still be tempted to report the original code as a bug as it seems like the new Mac chips are causing some unexpected behaviour that Unity may be able to address.

 

I tried declaring it outside the loop as follows, declaring it outside the method, executing it in another thread, and various other experiments, but the result was the same. I tried various things such as overwriting the texture size as small as possible, but the result was exactly the same.
If you comment out tex.ReadPixels, the memory will not increase, so I guess that the texture is not destroyed, but the memory used during the ReadPixels method is not released.
When I have time, I will install the same version of Intel and test whether it is related to the M1 chip.

Code (CSharp):

1.     private IEnumerator caputure()
2.     {
3.         int h = Screen.height;
4.         int w = Screen.width;
5.         Texture2D tex = new Texture2D(w, h, TextureFormat.RGB24, false);
6.  
7.         for (float i = 0; i < 1000000000; i += 0.0000001f)
8.         {
9.             yield return new WaitForEndOfFrame();
10.          Rect rect = new Rect(0,0, w, h);
11.          tex.ReadPixels(rect, 0, 0,false);
12.          tex.Apply();
13.             s++;
14.             ScreenCapture.CaptureScreenshot(Application.streamingAssetsPath + "/moviepng/" + s + ".png");
15.             File.WriteAllBytes(Application.streamingAssetsPath + "/moviepng/" + s + ".png",tex.EncodeToPNG());
16.         }
17.         DestroyImmediate(tex);
18.     }
19.  
20.  
21.  
22. //Experiment with another thread
23.  
24.  private IEnumerator screemshots()
25.     {
26.         int h = Screen.height;
27.         int w = Screen.width;
28.         Texture2D tex = new Texture2D(w, h, TextureFormat.RGB24, false);
29.         Rect rect = new Rect(0, 0, w, h);
30.         tex.ReadPixels(rect, 0, 0, false);
31.         tex.Apply();
32.         s++;
33.         File.WriteAllBytes(Application.streamingAssetsPath + "/moviepng/" + s + ".png", tex.EncodeToPNG());
34.         Destroy(tex);
35.         yield return null;
36.  
37.     }
38.     private IEnumerator caputure()
39.     {
40.         for (float i = 0; i < 1000000000; i += 0.0000001f)
41.         {
42.             yield return new WaitForEndOfFrame();
43.             StartCoroutine(screemshots());
44.         }
45.     }
46.  

 

Your sample code is also doing ScreenCapture.CaptureScreenshot() which itself may well be dynamically creating a new texture every time its called. So try again with that line commented out.

You are also still saving the texture data to PNG which is introducing another potential for memory leaks since tex.encodeToPNG() is used. So again comment that code out.

I'm also not a fan of using a coroutine here, there is no point and it can only add to the potential for bugs/issues especially from a memory fragmentation point of view, which from your original code is what I suspect the real cause. i.e. creation/deletion of the intermediate texture caused memory fragmentation preventing it from being reclaimed.

With something like this its important to keep the code to a minimum to be able to focus on what the cause might be. Its also clear you've not posted the entire code as 's' is not defined within the section you have, so who is to say there isn't something elsewhere in the code that is the cause.


In addition there are some concerns with the language/syntax you've used here, which makes it difficult to understand what you've tried exactly.

You cannot of used this in another thread as its using Unity API which on the whole cannot be called from another thread ( though I believe there are a few exceptions ). If you are referring to use of coroutine then that is not another thread, it has nothing to do with threads.

You say commenting out ReadPixels 'the memory will not increase', but in the same sentence you say 'the texture isn't destroyed' and then 'memory used with Readpixels is not released' - which is hard to understand since with ReadPixels commented out, there is no texture in use and Unity cannot be using any memory to perform ReadPixels.


At this point I'm only entertaining the possibility of a bug in Unity over user error is that its on relatively new Mac hardware, where in theory there might be an unexpected change to the metal API or how the gpu's operate that mean memory is not getting released as expected. Igf so then it is vital that Unity are made aware of this, but in order to confirm this we need to have as simple and straightforward test case - hence a bare bones script is needed.

Alternatively to all this, just make a simple test project and submit it to Unity. As long as they can literally run the project with no additional work and its just a few scripts, they will be able to confirm if the bug exists and get on to fixing it.

 

I'm sorry.
"ScreenCapture.CaptureScreenshot" accidentally posted what I wrote for testing without commenting out.
In actual testing, I commented out.
I made a mistake in posting the script.
Also, expressions such as "texture not destroyed" are my English translation mistake. I should have reviewed whether the English machine translation was correct.
I regret that I needed a proper explanation and preparation to get someone to advise me.
I installed "2021.3.17f1 Intel" and wrote it out, and the results were the same.
It was the same with "2022.2.4f1 I Apple Silicon."
I used a memory profiler to find out more.
The profiler increases Unknown memory, while the memory profiler increases Graphics memory.
The image is captured at a moment when memory continues to grow.
I created a simple project that was reproducible, but a few dozen objects and a few dozen textures didn't increases memory.
Even if you run the same program, the "problem project" will increases memory.
Something simple programs don't have.
something present in a runaway program.
I think that's where the cause is, so I'll do more testing and try to pinpoint the source.

 

No problem and don't feel bad about English conversion, it wasn't a criticism, just made it difficult to follow.

From what I can understand from this post, you were unable to cause the problem with a simple project and thus going to do more testing to try and pin down the issue? If so good luck and please do post any discoveries you make.

As I said I feel that this is more likely to be user error than a bug, but cannot completely discount it being some new issue that is only present on newer Macs, so it will be good to get to the source of the memory leak.

If the issue is due to memory fragmentation from user error ( we all make them at some time ) this thread might provide a starting point, though its about using getpixels() on a texture, the fragmentation issue is not specific to it.

 

Thanks.
This gave me a clue and I was able to identify the cause!
One by one, we looked for objects that cost money while deactivating them.
It turned out that the cause was an object with a line renderer.
I tried creating an object with a line render in a simple project, and when I ran ReadPixels, my memory ran out of control.
Now the editor will also experience memory inflation.
But even in the editor, closing the profiler window does not cause memory inflation.
I've uploaded a video of running what I built for Mac, so you can watch it.
It's just a program that puts cube objects and line objects (line renderers) in a scene and does ReadPixels in succession.
Reading Pixels inflates the memory, and hiding the line renderer's objects stops the inflation.
Stopping ReadPixels deflates the memory.
But hiding the line also stops the deflation.
I still think it's a bug, but what do you think?
If this looks like a possible bug, I'd like to try submitting the project to UNITY.

Code (CSharp):

1. using System.Collections;
2. using UnityEngine;
3. using UnityEngine.UI;
4. public class Turn : MonoBehaviour
5. {
6.     private bool capturestart;
7.  
8.     [SerializeField] GameObject line;
9.     [SerializeField] Text buttontext;
10.     void Update()
11.     {
12.  
13.         transform.Rotate(new Vector3(0.3f, 0.3f, 0.3f));
14.         if (capturestart)
15.         {
16.             StartCoroutine(caputure());
17.         }
18.  
19.     }
20.       private IEnumerator caputure()
21.     {
22.  
23.             int h = Screen.height;
24.             int w = Screen.width;
25.  
26.             Texture2D tex = new Texture2D(w, h, TextureFormat.RGB24, false);
27.  
28.             yield return new WaitForEndOfFrame();
29.  
30.             Rect rect = new Rect(0,0, w, h);
31.             tex.ReadPixels(rect, 0, 0, false);
32.  
33.             tex.Apply();
34.  
35.             Destroy(tex);
36.  
37.     }
38.  
39.     public void startbutton()
40.     {
41.         capturestart = true;
42.         buttontext.text = "Capturing";
43.     }
44.  
45.     public void endbutton()
46.     {
47.         capturestart = false;
48.         buttontext.text = "Captur Stopping";
49.     }
50.     public void linebutton()
51.     {
52.         line.SetActive(!line.activeInHierarchy);
53.     }
54.  
55. }
56.