Bug Recorder path-tracing frame accumulation not consistent

I am attempting to record path-traced videos with frame accumulation using the Recorder timeline integration.

The settings are: Apple Prores 4444 XQ, 1080p, 60fps, 15 seconds, 2000 samples.

Sometimes the saved video looks fine in the beginning, but then after a few seconds it turns grainy. Other times the whole video is grainy. I have checked during rendering and all 2000 samples are clearly being accumulated every frame.

Could it be that the Recorder is loosing sync with the timeline, and that it saves to disk at the wrong time, when only a few samples have been accumulated, instead of at the end of the accumulated frame?

It takes forever to test and reproduce, so while I'm on that ... has anyone experienced this?

Unity 2023.2a16, Recorder 4.0.1 (and 4.0.0).

EDIT. It also happens when recording from the Recorder window. And also when outputting as image sequence.

 

I never got Recorder working. As described above, the first frames may accumulate samples just fine, but then after a few seconds only very few samples are accumulated resulting in the video turning grainy. It happens pretty randomly and I never managed to find precise steps to reproduce the bug. My hunch is still that Recorder is somehow loosing sync with the render pipeline end of frame event.

I am now using this simple frame exporter and FFmpeg for conversion to video.

Code (CSharp):

1. using System;
2. using System.Collections;
3. using System.IO;
4. using UnityEngine;
5. using UnityEngine.Events;
6. using UnityEngine.Rendering;
7. using UnityEngine.Rendering.HighDefinition;
8. public class BasicRecorder: MonoBehaviour
9. {
10.     public string _saveDirectoryPath = "D:/Videos/UnityRecordings/MacroA";
11.     [Tooltip("Accumulation samples per frame")] public int sampleCount = 10;
12.     public int frameStart = 0;
13.     public int frameCount = 60;
14.     public UnityEvent<bool> onRecording = new UnityEvent<bool>();
15.  
16.  
17.     void Start()
18.     {
19.         _saveDirectoryPath += "_" + DateTime.Now.ToString( "yyMMdd_HHmmss" ) + "_a" + sampleCount;
20.         if( !Directory.Exists( _saveDirectoryPath ) ) Directory.CreateDirectory( _saveDirectoryPath );
21.  
22.         StartCoroutine( RecordCoroutine() );
23.     }
24.  
25.  
26.     IEnumerator RecordCoroutine()
27.     {
28.         onRecording.Invoke( true );
29.  
30.         HDRenderPipeline renderPipeline = RenderPipelineManager.currentPipeline as HDRenderPipeline;
31.         while( Time.frameCount < frameStart || renderPipeline == null ) {
32.             yield return new WaitForEndOfFrame();
33.             if( renderPipeline == null ) renderPipeline = RenderPipelineManager.currentPipeline as HDRenderPipeline;
34.         }
35.  
36.         Debug.Log( "Begin recording!" );
37.  
38.         for( int f = 0; f < frameCount; f++ )
39.         {
40.             renderPipeline.BeginRecording( sampleCount, 0f, 0.25f, 0.75f );
41.  
42.             Time.timeScale = 0f;
43.  
44.             for( int s = 0; s < sampleCount; s++ ) yield return new WaitForEndOfFrame();
45.  
46.             ScreenCapture.CaptureScreenshot( $"{_saveDirectoryPath }/frame_{ f.ToString("D4") }.png" );
47.  
48.             renderPipeline.EndRecording();
49.  
50.             Debug.Log( (f+1) + " / " + frameCount + Environment.NewLine );
51.  
52.             Time.timeScale = 1f;
53.  
54.             yield return new WaitForEndOfFrame();
55.         }
56.  
57.         Debug.Log( "End recording!" );
58.  
59.         onRecording.Invoke( false );
60.     }
61. }

 

I have encountered the same problem with 2022.3.13f1, I was exporting exr & png and both ways will reproduce the same problem just like the recorder took the frame after denoising.

I wonder if this is also due to GPU memory leak that you mentioned. As it would happen soon if I set the samples high or if I occupy the gpu for other applications

Thanks for your code I will try this.

update: Found one that causes the bug. When I change the display resolution (or moving unity window from one screen to another), it will occur in the next frame capture. Dont know the reason. This is make it difficult in my workflow as i use remote machine shared with other team members, meaning it will always change the window resolution, and occur when i close the remote control.

 

I just reported it
ref: IN-61251