NatCorder - Video Recording API

Hi!

I'm trying to recreate exactly that but unfortunately I'm failing and could use some help.

I have a camera set up and recording my AR scene with UI on top of that. I'm using CameraInput and this works fine. UI is not being recorder but I've seen your comment about that so that's fine. But - while trying to recreate the setup that you described I've added another camera to the scene and an image that I want to use as a watermark.
Apart from that I've updated CameraInput code to:

cameraInput = new CameraInput(videoRecorder, recordingClock, Camera.allCameras);

to record all cameras on the scene.
Now, in the second camera's preview my watermark is visible, but it's not included in a recorder video.
What am I missing? Should I set the camera or my watermark in any special way? Or maybe Camera.allCameras is not the way to go?

Thank you in advance!

 

If you watermark camera is looking at a UI canvas, then you need to make sure that the UI canvas can get recorded. Make sure that the canvas render mode is not Overlay.

 

Hmm... Right now my canvas is set to Screen Space - Camera, but I tried all modes to be hones.
My camera and watermark are set up like this:
https://postimg.cc/yWhdSptC
but I also tried putting the watermark inside camera object.

Camera itself is set like this:
https://postimg.cc/Ln6PTgcX

Canvas like this:
https://postimg.cc/V54rdLJc

And image like this:
https://postimg.cc/BthbJYzS

I already tried dozens different combinations and parameters, without luck. I'm pretty sure I'm missing something simple here :/

 

Make sure that the depth of your watermark camera is greater than that of your main camera.

 

I know that but I want to check in the code if os won't support I just disable the recording functionality

 

Then SystemInfo is the place to check.

 

Hey
I am receiving an error "NatCorder Error: Failed to record video frame for time ..." every frame while trying to record a video on the iPhone (See it in the XCode console), which eventually doesn't create the video.
If I run the same code on the mac (Unity editor) it works without an issue and creates the video.
What does this error mean??

 

Hey Lanre,

I've got a possible issue where when I launch the app on device (Android for now) where, I think, it's turning on the microphone and then off again.

So when I have my bluetooth headphones connected, when I launch the app, any current audio gets killed, and I can briefly hear static, then my headphones give a notification that a "call has ended" when there was no call. This happens every time I launch the app.

To produce the error, I created a new project, imported the latest NatCorder and built the ReplayCam scene to my devices. I also tried it with an empty scene (just in case the mic initialization in ReplayCam was the issue).

Happens on an up to date Pixel 2 and an old Sony Z3c on Lollipop. Latest NatCorder on Unity 2019.2.10f1.

Any thoughts on this?

 

I think I found the issue. I changed the MP4Recorder file path to a different location. Once changed it back to the original path - the error stopped.
Is there a way to save the file at a different location instead of moving it after the video is complete?

 

Can you share the logs from Xcode? Make sure to use code tags or attach a .txt file.

 

What version of NatCorder are you running on? ReplayCam in NatCorder 1.6.6 will always start the microphone when the scene is opened, regardless of whether you will be recording with microphone audio. This is because it makes Unity's microphone audio more stable.

 

Do not change NatCorder internals, especially the recording path. You would need to get an understanding of platform-specific file system differences.

 

trying record camera with Post Processing. Noticed it fails to record HDR colors into texture, what should be done to do this ?

 

How do you mean "it fails to record HDR colors into texture". CameraInput will record to an RGBA32 RenderTexture (no HDR) because native codecs don't accept HDR pixel data.

 

What is the most efficient way to record from a RenderTexture in the most recent version of Natcorder?
Previously, we would use the AquireFrame() interface, then do a Graphics.Blit.

ie.
var frame = natcorder.AcquireFrame();
Graphics.Blit(GetRenderTarget(), frame);
natcorder.CommitFrame(frame, time);

Now that AquireFrame is deprecated, what approach will yield the best performance?
We tried ReadPixels(), but it appears to be noticeably slower than the old way of doing it.

 

AsyncGPUReadback.Request.

 

Not supported on the Android Phones we're testing on.

 

I am using 1.6.6. I've realised that it isn't a NatCorder thing so don't worry about it.

Just for reference though, it seems like it's something with unity 2019+. I create a new project, create a script and put "Microphone.GetPosition(null);" in a new method. I don't even have to attach the script to anything. Then when I run it on the device my bluetooth headphones indicate that a call has been ended. It doesn't happen with 2018.

 

Does this plugin support recording audio to separate files? If so, does it support multiple simultaneous audio captures?

I would like to do a capture where the export is 1 video file, and 3 separate audio files. For three separate tracks of audio.

This would be so I can combine and edit them in premier separately

 

Yes I noticed. I'm using Post Process Bloom with combination with material/shader , containing emission with intensive HDR color for nice volume effect, but video recorder do not catch this. So HDR must be converted to LDR. Could your ReadableTexture readback handle with HDR and convert to LDR ? Or other fixes are welcome.
Im quite new to Unity so maybe there are other solutions?

 

AsyncGPUReadback is supported on Android devices running Vulkan. You might need to enable it in Player Settings. The `CameraInput` script uses this for recording.

 

NatCorder currently doesn't have an audio-only recorder. In the NatCorder 1.7.0 update, we have added a simple WAVRecorder for recording to waveform audio files.

 

NatCorder 1.6.6 doesn't use ReadableTexture. Make sure you are using 1.6.6 as we use Unity-provided functions for performing the readback. You can then investigate and tweak it to get the effect you want (I have no experience with HDR rendering in Unity).

 

Is there an expected release for the 1.7.0 update?

 

Planning on sometime next week.

 

Hi, interested in NatCorder as my game required recoding gameplay for about 5 min (User highlight saving so that he can save the video file for upload or personal use) I just want to check the following before I make any payments.

1. Does it support HDRP? (2019.3f6, 7.18) I've seen that it does, but that was about a year ago, and I want to know if it supports the latest version and that it will be supported throughout the HDRP cycle.

2. Can the saved video be played back in Unity?

3. Can the video recording stop, pause, resume at any time without issues?

Thanks!

 

Yes, there's nothing that prevents it from working with HDRP. All NatCorder needs to do is to render to a texture,

Yes, using the VideoPlayer component.

From the NatCorder perspective, there is no such thing as pausing or resuming recording. If you have a recording session active (you've created a recorder) and you don't commit frames to it, then you could say you are in a paused state. There is a constraint that for mobile platforms (and maybe on macOS and Windows too, I haven't tried), if the app gets suspended, then recording will automatically fail.

 

Lanre, I just implemented NatCorder to my game and it is working great!

But I have one last concern,

it looks like mp4encoder encodes the videos using H264 codec, right?

And my game allows users to make videos of the game content and allows access to the video files (stored as mp4)

Will I run into any license issues regarding H264?
I read this article here
https://www.engadget.com/2010/05/04/know-your-rights-h-264-patent-licensing-and-you/

and it looks like the commercial use of H264 is not free.
Could you kindly clarify?

 

AFAIK if you aren't selling H.264-encoded video content, then you don't have to pay any royalties. See this.

 

I read through the post you linked, and it looks like the users of software (my client) don't have to pay, but software developers (you and I) have to. Here is the exert.

I am very worried about this situation and that the licensing of NatCorder doesn't cover this.

I think you should really contact MPEG LA, and get written confirmation that natCorder and its uses are covered. Unless of course, I understood the whole thing wrong.

But, provided that I am wrong, you might have to contact your clients (those who bough NatCorder) to make sure they are aware of this and that all this is straightened out - before it becomes a clusterfk of all ages....

 

Using an integrated encoder (AVFoundation on macOS and iOS, MediaCodec on Android, MediaFoundation on Windows) subverts the need to acquire a license. This makes the most sense, as it would not be realistic to expect every app out there that does some form of video recording to pay licensing fees to MPEG LA.

 

Hi, I have been trying to get this to work for a short while now and seem to be running into a few bugs. I searched through this thread but couldn't seem to find out what to do. (I apologize if this has been answered millions of times.)

I have a coroutine that should render a video from a render texture, but it freezes about one second in every time. Any help would be appreciated, thanks.

Code (CSharp):

1.  
2. private IEnumerator RenderRoutine(float length)
3. {
4.     float fps = 60;
5.     int audioSamples = 44100;
6.     int channels = 2;
7.     Camera renderCam = CreateRenderingCamera();
8.     RenderTexture tex = renderCam.targetTexture;
9.     MP4Recorder recorder =
10.         new MP4Recorder(tex.width, tex.height, fps, audioSamples, channels, s => { Debug.Log(s); });
11.     FixedIntervalClock clock = new FixedIntervalClock(fps);
12.  
13.     for (int frame = 0; frame < length * fps; frame++)
14.     {
15.         yield return new WaitForEndOfFrame();
16.         Texture2D fTex = RenderTextureToTexture2D(tex);
17.         recorder.CommitFrame(fTex.GetPixels32(), clock.Timestamp);
18.         clock.Tick();
19.         DestroyImmediate(fTex);
20.     }
21.  
22.     recorder.Dispose();
23. }
24.  

 

You are setting a non-zero sample rate and channel count without actually committing any audio. On Windows, this causes weird behaviour (app starts to lag, no clue why). Set those arguments to zero.

Minor note: a fixed interval clock will auto-tick by default when Timestamp is accessed, so no need to manually tick. You can disable this in the constructor.

 

Thanks for the reply! That helped immensely!
Another less pressing question, Am I able to specify the output path? Or should I just move it after its created? Thanks.

 

Move it after it is created. Don't try modding the sources to change the output path.

 

Hi!

I have been using NatCorder for quite some time now. It was working fine on both my Mac, Android, and iPhone. But post the update, I'm unable to hear any sound in Android devices. I am yet to check on iPhones. I checked this forum and tried all possible things that i could do but still am unable to get the audio recording working. Can you please help with this?

Note that the audio recording is working fine in the unity editor (mac)

Spec:
NacCorder - 1.6.6
Unity - 2019.3
Android Version - 10

Any help would be appreciated.
Thanks

 

Can you share the unfiltered logs from logcat in a .txt file?

 

Hi,

Thanks for replying.
Attached the log file

Thanks

 

Hi Lanre,

What is the best way to record a cropped screen? The newest iPhone have a 19.5:9 screen ratio, but I want to record a 16:9, so chop off a rectangle from the top and bottom. I've read that there was an example called 'GreyWorld' that used a custom material, but can't find it in the current NatCorder.

Can you advice how to do this? THanks!

 

Can you share your recording code? Are you recording from a microphone AudioSource or an AudioListener?

 

You could just create a recorder with a 16:9 aspect, and Unity will render your app for the recorder like it is being displayed on a screen with a 16:9 aspect.

 

The outcome of doing that is that the content of the 19.5:9 ratio is crammed in the 16:9 video, and the content looks distorted. See screenshot attached. Recording and displaying in the app looks good, but when saving that video to the camera roll show the wierd result.

 

Are you rendering the app with a CameraInput. You should never get squishing with CameraInput, as Unity will always re-render to match the size of the recorder.

 

Yes, this is the code :

Code (CSharp):

1. recordingClock = new RealtimeClock();
2.             HEVCVideoRecorder = new HEVCRecorder(_videoWidth, _videoHeight, 30, AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode, OnReplay);
3.             cameraInput = new CameraInput(HEVCVideoRecorder, recordingClock, Camera.main);
4.            
5.             // Start the microphone
6.             audioDevice = AudioDevice.GetDevices()[0];
7.             audioDevice.StartRecording(AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode, this);
8.         }
9.        
10.         public void OnSampleBuffer(float[] sampleBuffer, int sampleRate, int channelCount, long timestamp)
11.         {
12.             HEVCVideoRecorder.CommitSamples(sampleBuffer, recordingClock.Timestamp);
13.         }

 

How is your UI setup? If you change the aspect of the game view in the editor, does it come out squished?

 

Hi,

I am using the Audiosource as a microphone and attached is the code that I've used.

 

Check that you have granted microphone permissions to your app.

 

Hi,

I recently updated to the newest version of NatCorder because of a null error fix.
In the process i lost the CameraInput.OnPreRender(); and CameraInput.OnPostRender(); which was made to hide UI elements in the recording. Is there a new alternative for this?

Thanks,
Anders

 

No there isn't. Use a second game camera instead (with layer masks to hide certain game objects), and record from that camera.

 

Yes, the permissions are all enabled.