Video Control Unity Recorder from Script

I just want to get some simple information about the Unity Recorder in my scripts, such as whether it's recording or not, what the target framerate is, etc. I can see there's an API but there's no information about where to get the 'current' recorder.

 

Hi EmmetOT

I just stumbled across your post looking for another issue, but I was looking at the same info you are looking for now a while back, and I came up with this solution.

Code (CSharp):

1. using UnityEditor.Recorder;
2.      
3. private RecorderWindow GetRecorderWindow()
4.         {
5.             return (RecorderWindow)EditorWindow.GetWindow(typeof(RecorderWindow));
6.         }

This code snippet will retrieve the Recorderwindow for you, from here you can access all sorts of things, like Start/Stop Recording and all sorts of really cool stuff. The only drawback is that the Recorder window pops up when you do.
I use this for instance when I wanna start recording:

Code (CSharp):

1.                     RecorderWindow recorderWindow = GetRecorderWindow();
2.                     if(!recorderWindow.IsRecording())
3.                         recorderWindow.StartRecording();

or when I want to stop recording:

Code (CSharp):

1.             RecorderWindow recorderWindow = GetRecorderWindow();
2.             if (recorderWindow.IsRecording())
3.                 recorderWindow.StopRecording();

I also manage the recorders in the recorder, where they save to, fileNames etc.

Hope it helps!
-Daniel

 

RebelNature_Daniel

Can you send the code that you use to manage the recorders in the recorder?

 

Hi Bruno,
I have tried with your code, but it always came up with an error:
ArgumentNullException: Value cannot be null.
Parameter name: source
System.Linq.Enumerable.Any[TSource] (System.Collections.Generic.IEnumerable`1[T] source) (at <fbb5ed17eb6e46c680000f8910ebb50c>:0)
UnityEditor.Recorder.RecorderController.StartRecording () (at Library/PackageCache/com.unity.recorder@2.2.0-preview.4/Editor/Sources/RecorderController.cs:133)

Can you help me with this?

Kind regards,
Jun

 

Thanks Bruno, this is working well right now. I should go through the RecorderController.cs before asking you.

 

HI, there is a way to set the path of the file?

because if i not configure the recorder window to .mp4 BEFORE the script it simply do anything

 

Any advise on image capture with the Unity Recorder via scripting. I tried to alter this example and I used "C:/temp/img_b_<frame>" for the value passed to OutputFile, but it just created single image file named "C:/temp/img_b_frame.png" and kept overwriting the image each time it captured a new frame.

Edit:
I figured it out.
I have to grab the FileNameGenerator of the RecorderSettings that we are using and call the AddWildcard(). This method takes two parameters. It take the name of the wildcard and a lambda expression (with a RecordingSession parameter) that returns/generates the sting that I want to replace "<frame>" with.

_recorderSettings.FileNameGenerator.AddWildcard("frame", (sess) => { return sess.frameIndex.ToString(); });

I'm going to have to add some 0 padding to the numbers to keep things clean for my Users, but that code isn't needed for answering the question that I had.

 

Thanks. I was actually wondering about the correct usage of the DefaultWildcard. Unfortunately, I'm going to have to stick to resolving the wildcards myself because it gives my Users greater flexibility when setting the values of the automation that we're doing.

 

Hi Bruno
I need to assign a RenderTexture that's generated at runtime as the Render Texture Asset for the capture source in the Recorder. I can get a handle to the recorder window but I can't use it to change the RenderTexture property

I tried using your code to instantiate a recorder at runtime but there's no UI associated with the object.. Can this be done? Or is there a way to change the UI input RenderTexture from script?

Any suggestions?

Thank you!!

Here's my code:

Code (CSharp):

1. public class TestVideoCapture : MonoBehaviour
2. {
3.     public RenderTexture renderTexture = null;
4.     void Start()
5.     {
6.         Init();
7.         RecorderWindow rw = GetRecorderWindow.Recorder();  // Open up the recorder window
8.     }
9.  
10.     void Init()
11.     {
12.         RecorderOptions.VerboseMode = true;
13.    
14.         var controllerSettings = ScriptableObject.CreateInstance<RecorderControllerSettings>();
15.         var TestRecorderController = new RecorderController(controllerSettings);
16.    
17.         var videoRecorder = ScriptableObject.CreateInstance<MovieRecorderSettings>();
18.         videoRecorder.name = "My Video Recorder";
19.         videoRecorder.Enabled = true;
20.  
21.         videoRecorder.ImageInputSettings = new RenderTextureInputSettings()
22.         {
23.             OutputWidth = renderTexture.width,
24.             OutputHeight = renderTexture.height,
25.             FlipFinalOutput = true,
26.             RenderTexture = renderTexture
27.         };
28.    
29.         videoRecorder.AudioInputSettings.PreserveAudio = true;
30.    
31.         controllerSettings.AddRecorderSettings(videoRecorder);
32.         controllerSettings.SetRecordModeToFrameInterval(0, 59); // 2s @ 30 FPS
33.         controllerSettings.FrameRate = 60;
34.         TestRecorderController.PrepareRecording();
35.     }
36.  
37. }
38.  

 

Hi,

I am using this script to record a series of videos and save them to my HD. I set the name of the file with videoRecorder.OutputFile. It works, it save the files, they have the right length and they contain the audio, but the video is just black. Am I doing something wrong with the renderTexture?

Thanks!

Code (CSharp):

1. void StartVideoRecording(int duration) {
2.  
3.         renderTexture = new RenderTexture(1080 , 1920 , 16);
4.         renderTexture.Create();
5.  
6.         if (renderTexture == null) {
7.             Debug.LogError($"You must assign a valid renderTexture before entering Play Mode");
8.             return;
9.         }
10.  
11.         RecorderOptions.VerboseMode = true;
12.  
13.         var controllerSettings = ScriptableObject.CreateInstance<RecorderControllerSettings>();
14.         var TestRecorderController = new RecorderController(controllerSettings);
15.  
16.         var videoRecorder = ScriptableObject.CreateInstance<MovieRecorderSettings>();
17.         videoRecorder.name = "My Video Recorder";
18.         videoRecorder.Enabled = true;
19.         videoRecorder.OutputFile = "Recordings/PicTarot_" + currentCardId;
20.         videoRecorder.OutputFormat = MovieRecorderSettings.VideoRecorderOutputFormat.MP4;
21.  
22.         videoRecorder.ImageInputSettings = new RenderTextureInputSettings() {
23.             OutputWidth = renderTexture.width ,
24.             OutputHeight = renderTexture.height ,
25.             FlipFinalOutput = true ,
26.             RenderTexture = renderTexture
27.         };
28.  
29.         videoRecorder.AudioInputSettings.PreserveAudio = true;
30.  
31.         controllerSettings.AddRecorderSettings(videoRecorder);
32.         controllerSettings.SetRecordModeToFrameInterval(0 , duration * 30); // 2s @ 30 FPS
33.         controllerSettings.FrameRate = 30;
34.         TestRecorderController.PrepareRecording();
35.         TestRecorderController.StartRecording();
36.  
37.     }

 

Hello,

I'm also using a script to control the recorder and I face some issues.
I'm using a CameraInputSettings to only record the necessary thing. When I use an output width and height of 2560*1440 everything works fine. But the movie is a bit stretched since our video format is 2560*1303 (there is a control bar of 137px height on the footer).

When I try to use this output height, the recorder does not works and throw an "ObjectDisposedException: Cannot access a disposed object. Object name: 'Encoder is disposed or invalid'."

Is there anything I should do with the recorder to make it accept non standard ratios ?

Thanks.

 

Thanks! Yes, I have a camera. I had to dig a little but adding this worked for me then:

mainCamera.targetTexture = renderTexture;

So rather than using the camera output the camera is told to render to the renderTexture that's used for recording video. It's a bit counterintuitive to me, but it works!

 

Hello,
I also wanted to ask something concerning the Unity Recorder. Is there a way to access the Codec Format of the Media File Format by Script? There are options for it in the Recorder Window (for MOV files), but I didn't find anything in the Scripting API.

 

Hello everyone, I wanted to record the depth textures with recorder , how can we do this?

 

I want to record alpha and exclude the background. For this to work I need to use a camera with special tag I guess because grabbing screen doesn't work with alpha as intended. How can I change the source of recorder via script and assign a camera there?

Code (CSharp):

1.    private void SetRecWindowParameters(RecorderWindow _window)
2.     {
3.         var controllerSettings = ScriptableObject.CreateInstance<RecorderControllerSettings>();
4.         var TestRecorderController = new RecorderController(controllerSettings);
5.  
6.         var imageRecorder = ScriptableObject.CreateInstance<ImageRecorderSettings>();
7.         imageRecorder.name = "SingleFrameRecorder";
8.         imageRecorder.Enabled = true;
9.         imageRecorder.CaptureAlpha = false;
10.         imageRecorder.RecordMode = RecordMode.SingleFrame;
11.         imageRecorder.OutputFormat = ImageRecorderSettings.ImageRecorderOutputFormat.PNG;
12.         imageRecorder.OutputFile = "D:\\Recordings\\image_<Take>_<Frame>";      
13.        
14.         imageRecorder.imageInputSettings = new GameViewInputSettings
15.         {
16.             OutputWidth = 512,
17.             OutputHeight = 512
18.         };
19.         controllerSettings.SetRecordModeToFrameInterval(1,3);      
20.         controllerSettings.AddRecorderSettings(imageRecorder);
21.         controllerSettings.FrameRate = 30;
22.      
23.         RecorderOptions.VerboseMode = false;
24.         TestRecorderController.PrepareRecording();
25.         TestRecorderController.StartRecording();
26.        
27.     }

thank you

 

Thank you very much for the help,
these changes work pretty well if anyone needs later on:
Use this function and it will return a series of screenshots with alpha from your camera with the 'Recorder' tag.

Code (CSharp):

1. private void SetRecWindowParameters()
2.     {
3.         var controllerSettings = ScriptableObject.CreateInstance<RecorderControllerSettings>();
4.         var TestRecorderController = new RecorderController(controllerSettings);  
5.        
6.         var imageRecorder = ScriptableObject.CreateInstance<ImageRecorderSettings>();
7.         imageRecorder.name = "SingleFrameRecorder";
8.         imageRecorder.Enabled = true;
9.         imageRecorder.CaptureAlpha = true;
10.         imageRecorder.RecordMode = RecordMode.SingleFrame;
11.         imageRecorder.OutputFormat = ImageRecorderSettings.ImageRecorderOutputFormat.PNG;
12.         imageRecorder.OutputFile = "D:\\Recordings\\image_<Take>_<Frame>";
13.  
14.         imageRecorder.imageInputSettings = new CameraInputSettings
15.         {
16.             Source = ImageSource.TaggedCamera,
17.             RecordTransparency = true,
18.             CaptureUI = false,
19.             FlipFinalOutput = false,
20.             OutputWidth = (int)_outputDimensions.x,
21.             OutputHeight = (int)_outputDimensions.y,
22.             CameraTag = "Recorder"
23.         };
24.  
25.        
26.         controllerSettings.SetRecordModeToFrameInterval(1,3);      
27.         controllerSettings.AddRecorderSettings(imageRecorder);
28.         controllerSettings.FrameRate = 30;
29.      
30.         RecorderOptions.VerboseMode = false;
31.         TestRecorderController.PrepareRecording();
32.         TestRecorderController.StartRecording();
33.        
34.     }

 

Hi,

I'm looking at the new (and more accessible) AOV API in Unity 2020.3, and I'm wondering how it works. For instance, it was mentioned that I can now do things like render/capture ObjectIDs, but the code sample that was provided is a bit lacking. It also mentioned that I can pass in Custom Passes, but when I looked at the part of the code for passing in Custom Passes, it was nothing but an array of (injection-point, render-target). Is this saying, run the Custom Passes that have the combinations of Injection-Point & Render-Target that are in the array?

I also noticed that in the sample code, the after render callback/event-handler is checking the count of textures. Are we able to capture multiple renders on the same camera within a single frame? There's documentation giving us a very terse explanations that are along the lines of "The AOVRequest is an AOVRequest". Which, is better than nothing, but not by much.

Where can I get more information on how to actually use this thing?

 

So, another question.

Is there a way to use the Recorder's simulation management (in Unity 2019.4 LTS) without having the Recorder record anything?

One of the things I noticed about the Unity Recorder (when combined with AOV) is that it doesn't capture negative values even when the output is EXR images. Yet, the Recorder is excellent at getting the camera and game simulation to run the same way across multiple runs. I guess what I'm asking is, how to we replace the render capture with our own capture?

 

Well, the ultimate objective is to extract any data the client wants and capture it, while repeating the simulation in a deterministic fashion, so that we can extract different data, allowing us augment the previous data-set while keeping all the data in sync because the simulation runs in a deterministic fashion. And, if the client wants to be able to extract a different kind of data, then we can just throw that bit of code in without have to re-write any of the previous code.

When I implemented a Custom Pass that displayed the Motion Vectors after post-processing, all the negative values were clamped to zero by the AOV Recorder. It is possible that post-processing did this, but how it did this to something that ran after post is beyond me. The first attempt to get around that was to 01 compress the motion vector values like we do for normals, but 16 bit floating point numbers only hold 11 bits of precision, so "(value / 2.0) + 0.5" causes most values to drop to zero at 120 frame per second.

A quick example, lets say that our register can only hold 4 digits:
0.5 becomes 5.000 * 10^-1
0.00001 becomes 1.000 * 10^-5
0.5 + 0.00001 becomes 5.000 *10^-1 + 0.0001 * 10^-1, but we can only hold 4 digits, so the 1 gets dropped.
0.5 + 0.00001 becomes 5.000*10^-1 + 0.000 * 10^-1
Instead of getting 0.5001, we got 0.5.

That +0.5, was wiping out most of our motion vector data.

The next attempt around this was to just re-render (to a RenderTexture) at the spots the camera stops when the Recorder stops the camera to render, and it sort of worked. But, when we pulled the frame number from Time, it didn't match up with the frame number the recorder was stamping into the images. Also, we didn't have the time to check if the difference in frame number would be consistent across multiple machines. On the machine we tested this work around, the frame number in Time, was 4 frames larger than the one in the Recorder's output images from the same location.

We (well, I) built my code on top of managing the AOV Extension Recorder because the base Recorder didn't have exr output and it wouldn't capture all of the lights and colors when set it to capture a series of images (it worked great for video capture). The AOV Recorder (Extension) plug-in had exr output and it was able to capture the entire output image when set to capturing a series of images. I still need to re-check this now that there's newer versions of the base Recorder plugin. I know the base Recorder can now capture exr for 2020.3, but I haven't check for 2019.4.

Yet, the reason why we're using the Recorder is because one of our projects pre-dates the Timeline tool. It also pre-dates the Standard Surface Shader used in the Standard Render Pipeline, but it can't predate it by too much, because it was using prototype Stand Surface Shaders. Despite getting the project to work in Unity 2019.4 and HDRP 7.5, getting the scene to run under the Timeline tool is more work than I can do with the time constraints.

Yet, one of the things that the Unity Recorder excels at, is getting the simulation to run the same way every time we run the simulation (as long as scene doesn't contain non-deterministic random particles floating around like Morgan in the Heretic VFX scene).

So, one of the things I noticed about the new AOV API, is that it executes a delegate. This made me think, wouldn't it be nice if we could just tell the Recorder to execute this bit of code when it tries to capture an image. So, I guess my question should have been, how does one create an extension plug-in for the Unity Recorder plug-in. I know it's possible (or used to be possible) to create an extension plug-in for the Unity Recorder plug-in because the now defunct AOV Recorder is an extension to the Unity Recorder. The old documentation (actual documentation that tell me how the code interacts with the rest of the code) that I found for the Unity Recorder was for the Recorder that was used in Unity 2017.4. But, the Recorder gained a major performance boost in Unity 2018.4, so I don't think I can trust that old documentation. Also, I lost the link.

 

Sorry about the late replay, I was a bit preoccupied last week. Just wanted to say thank you.

 

I'm on Unity 2019.4.15f1 and Unity Recorder 2.5.5, and I'm not finding an OutputRenderTexture of any kind inside of the the CameraInput.

 

So, I took a closer look. It turns out that this is a

protected internal

property. I can access it if I create my own Input class that inherits from

CameraInput

; from there, I create a method to grab the result for me.

Of course, that raises the question: How do I get

TestRecorder 

to use use

TestCameraInput : CameraInput

instead of

CameraInput

?

---------------------------
Edit:
I figured it out.

First, I need to create a classed named

TestCameraInput

:

Code (CSharp):

1. public class TestCameraInput
2.     : CameraInput
3. {
4.     public RenderTexture OutputRT
5.     {
6.         get { return OutputRenderTexture; }
7.         set { OutputRenderTexture = value; }
8.     }
9.  
10.     public Texture2D ReadbackTex2D
11.     {
12.         get { return ReadbackTexture; }
13.         set { ReadbackTexture = value; }
14.     }
15.  
16.     public Camera TargetCam
17.     {
18.         get { return TargetCamera; }
19.         set { TargetCamera = value; }
20.     }
21.  
22. }

Next, I need to create a class named

TestInputSettings

:

Code (CSharp):

1. public class TestInputSettings
2.     : CameraInputSettings
3. {
4.     protected override Type InputType => typeof(TestCameraInput);
5. }

By overriding the

InputType

getter, I can tell base Recorder class to use my

TestCameraInput

instead of the

CameraInput

that's specified by

CameraInputSettings

.

To finish it off, I tell the

TestRecorderSettings

to use an instance of

TestInputSettings

instead of using an instance of the

CameraInputSettings

class. This way, when the base Recorder enumerates over the items inside the

InputSettings

property that we overloaded inside of the

TestRecorderSettings

class, it will see that it needs to create an instance of

TestCameraInput

. And, because

TestCameraInput

is something that we created, we can add code to look and/or use the protected members of its parent classes.

By doing all of this,

m_Inputs[0]

will contain an instance of

TestCameraInput

.

Update:
So, there might be a bug with the Editor rendering code, or (more likely) there's another component that needs to be defined. But when alter the field containing the TestCameraInputSettings, do this:

[SerializeField] CameraInputSettings m_InputSettings = new TestCameraInputSettings();

When I changed the field type to TestCameraInputSettings, the Editor became unable to draw the input fields for selecting the camera and resolution due to a null reference error.

So,

[SerializeField] TestCameraInputSettings m_InputSettings = new TestCameraInputSettings();

is bad, and

[SerializeField] CameraInputSettings m_InputSettings = new TestCameraInputSettings();

is good when creating your own

TestRecorderSettings

class.

 

@unitybru is there a way to access the already set recorder settings? I just want to be able to edit the filename by scripting, keeping the takes and other settings intact

 

Hi Bruno, so I used this script/method to start and stop the recorder, however I get an opening file error which cuts the recording short.

Opening file failed

Opening file
C:...Start of path.../Library/TempArtifacts/Primary/randomlettertempfilename.resource:
The system cannot find the specified file.

Attached is my code, which primarily uses what you posted above.

Code (CSharp):

1. using UnityEngine;
2. using UnityEditor;
3. using UnityEditor.Recorder;
4. using UnityEditor.Recorder.Input;
5. public class RotateMug : MonoBehaviour
6. {
7.     public float timePeriod = 2;
8.     public float height = 30f;
9.     public float startAngle;
10.     private float timeSinceStart;
11.     private Vector3 pivot;
12.     private string funword = "silly";
13.     private RecorderController TestRecorderController;
14.  
15.     private void Start()
16.     {
17.         pivot = transform.localEulerAngles;
18.         pivot.y = startAngle;
19.         height /= 2;
20.         StartRecorder();
21.         timeSinceStart = 0;
22.     }
23.  
24.     void Update()
25.     {
26.         if (timeSinceStart < timePeriod)
27.         {
28.             Vector3 nextPos = transform.localEulerAngles;
29.             nextPos.y = pivot.y + height * Mathf.Sin(((Mathf.PI * 2) / timePeriod) * timeSinceStart);
30.             timeSinceStart += Time.deltaTime;
31.             transform.localEulerAngles = nextPos;
32.         }
33.         else
34.         {
35.             StopRecorder();
36.         }
37.     }
38.  
39.     private void StartRecorder()
40.     {
41.         var controllerSettings = ScriptableObject.CreateInstance<RecorderControllerSettings>();
42.         TestRecorderController = new RecorderController(controllerSettings);
43.         var videoRecorder = ScriptableObject.CreateInstance<MovieRecorderSettings>();
44.         videoRecorder.name = "My Video Recorder";
45.         videoRecorder.Enabled = true;
46.         videoRecorder.VideoBitRateMode = VideoBitrateMode.High;
47.         videoRecorder.ImageInputSettings = new GameViewInputSettings
48.         {
49.             OutputWidth = 640,
50.             OutputHeight = 480
51.         };
52.         videoRecorder.AudioInputSettings.PreserveAudio = true;
53.         string fileName = RecordingName();
54.         videoRecorder.OutputFile = fileName;
55.         controllerSettings.AddRecorderSettings(videoRecorder);
56.         controllerSettings.SetRecordModeToFrameInterval(0, 59); // 2s @ 30 FPS
57.         controllerSettings.FrameRate = 30;
58.         RecorderOptions.VerboseMode = false;
59.         TestRecorderController.PrepareRecording();
60.         TestRecorderController.StartRecording();
61.     }
62.  
63.     private void StopRecorder()
64.     {
65.         TestRecorderController.StopRecording();
66.     }
67.  
68.     private string RecordingName()
69.     {
70.         return string.Format("{0}/MP4Videos/vid_{1}",
71.             Application.dataPath,
72.             funword
73.         );
74.     }
75. }

 

Saving to a different folder solved the opening file error, but the saved video is still being cut short

Edit:

I solved the issue, I needed to change the recorder mode to manual

 

I've been able to set everything up using the info in this thread but I still have a question. I'm trying to render out a series of video files from one scene and since my computer can play back the animations at really high framerates, I'm wondering if it's possible to record at speeds faster than real time. An example would be rendering many 24fps videos at a rate of 100 fps or higher. I have been able to do using a render texture to an image sequence (Not using the recorder). Would there be some way to do this with the video recorder?

 

@unitybru et al

Thank you very much for the recorder; I find it absolutely fabulous.

Whilst accessing the recorder from script is news to me, I realise there may be parts accessible, which are otherwise not.

Having said so - as I'm working on 8k 120fps output - I have two questions regarding scripting:

1) Would it be possible to append output to an existing MP4 file instead of making a new one, of course those two would have identical settings (i.e. w*h, fps etc.)? If so, how would I do that?

2) Could I force 16:9 output beyond 8192px width? Maybe something like when "superSize" (in ScreenCapture.CaptureScreenshot(string filename, int superSize)) is used?

 

@unitybru

Thank you very much;

I searched a bit and found some software called MP4Joiner that's available for MacOS (and Win) and supposedly meant to do just that, i.e. join two (or more) MP4 files; hopefully it works

 

Hello Daniel, how to change the location of the file saved using script?
recorderWindow.____ ?

 

Hello Bruno, Video recording works fine. However, audio is NOT recorded. Can you please help? (I also tried by inserting microphone)