Search Unity

  1. Click here to see what's on sale for the "Best of Super Sale" on the Asset Store
    Dismiss Notice
  2. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

NatCorder - Video Recording API

Discussion in 'Assets and Asset Store' started by Lanre, Nov 18, 2017.

  1. omerfarukozdemir

    omerfarukozdemir

    Joined:
    Feb 22, 2013
    Posts:
    8
    Can you share the basic project for this? You can just hide the save button. That's enough. Can you?
     
  2. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    I won't have time to do this anytime soon. This is not a NatCorder feature; it is a Unity feature. Refer to Unity's manual on layer masks for more info.
     
  3. omerfarukozdemir

    omerfarukozdemir

    Joined:
    Feb 22, 2013
    Posts:
    8
    Ok. Thats work now. Great performance. Thanks.
     
    Lanre likes this.
  4. jiraphatK

    jiraphatK

    Joined:
    Sep 29, 2018
    Posts:
    70
    Hi, has anyone ever record a video with Unity's microphone as audio input? (Unity's default Microphone Class)
    We are encountering this weird popping and cracking sound in IOS
    It only occurs on IOS. The problem comes up when Force IOS Speakers when recording in Project Setting is true.
    We need this setting to true because otherwise, IOS will lower game sound when we started the Microphone.
    (We want both mic and game sound)

    https://drive.google.com/drive/folders/1Qdz3CaAWLeKsLO4mRt8rp9sF0JXZGkc0?usp=sharing
    Here's a script that we use to record audio. It worked fine in Android and Editor.
    upload_2020-11-2_10-8-23.png

    We use AudioSource input.
     
    Last edited: Nov 2, 2020
  5. sharangsharma09

    sharangsharma09

    Joined:
    Apr 27, 2018
    Posts:
    3
    Hi,
    I am trying to record the AR feed using ARFoundation. The problem I am facing is that if the AR object is too far from the ARcamera, the Preview screen doesn't render any AR object. When the AR object is too close to the camera, the AR objects gets stuck to the camera view. Has anyone facing the problem?

    I am using Unity 2019.4.6
     
  6. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    This is a big reason why we introduced NatMic, our native microphone API. It has since been succeeded by NatDevice, which supports both the camera and microphone.
     
  7. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    I haven't faced this issue before, but check your camera's clipping distances.
     
  8. dieghobonilla

    dieghobonilla

    Joined:
    Sep 4, 2019
    Posts:
    5
    Hello, how do you record just the ar camera using ar foundation and URP? I've been trying different options with stacking cameras but the video either is black or renders everything
     
    Last edited: Nov 2, 2020
  9. ghasedak3411

    ghasedak3411

    Joined:
    Aug 25, 2015
    Posts:
    22
    hi,
    What could this error be for?
     

    Attached Files:

  10. PSST_Adam

    PSST_Adam

    Joined:
    Aug 26, 2020
    Posts:
    5
    @Lanre Would be good to know if you can think of a way this can be done? Would be useful in our project.
     
    Lanre likes this.
  11. Emile-Swain

    Emile-Swain

    Joined:
    Jun 23, 2015
    Posts:
    18
    Hi,
    I'm trying to find a definitive example of recording game audio and microphone audio using NatDevice and NatCorder.
    With the code below i get stuttering mic audio. So i figured its perhaps a samplerate issue. But i'm unsure as to what to change.

    You've mentioned in the docs not to use multiple Inputs of the same type. So i'm unsure if audioDevice.StartRunning is considered another input to the AudioInput.

    I'm currently looking to see if the issue is related to the sampleRates being different between the device(mic) and the output sampleRate. I'm unsure as to what i should do here.

    EDIT: So i disabled the _audioInput and instead of stuttering i got super fast playback. I think i read that thats clock related somewhere....

    FIXED:
    My continued search found an example in the forum to use Mixer device here.
    https://forum.unity.com/threads/natdevice-media-device-api.374690/page-46#post-5648608

    phew!

    Would be great to have this in the help docs.

    [Additional fix]
    Further searching fixed an issue with iOS. https://forum.unity.com/threads/natdevice-media-device-api.374690/page-50
    Destination array was not long enough. Check destIndex and length, and the array's lower bounds
    Edit MixerDevice.cs to increase buffer size .
    var copyBuffer = new float[16384];

    Code (CSharp):
    1.  
    2.         // Microphone audio.
    3.         var query = new MediaDeviceQuery(MediaDeviceQuery.Criteria.AudioDevice);
    4.         audioDevice = query.currentDevice as IAudioDevice;
    5.  
    6.  
    7.         Debug.Log("SampleRate: Device: "+audioDevice.sampleRate);
    8.         Debug.Log("SampleRate: AudioSettings: "+AudioSettings.outputSampleRate);
    9.        /// SampleRate: Device: 44100
    10.        /// SampleRate: AudioSettings: 48000
    11.  
    12.  
    13.         // Recorder setyop
    14.         var clock = new RealtimeClock();
    15.         recorder = new MP4Recorder((int) prod.x, (int) prod.y, 30, AudioSettings.outputSampleRate, 2);
    16.         cameraInput = new CameraInput(recorder, clock, Camera.main);
    17.  
    18.         /// Game + Mic Audio
    19.         _audioInput = new AudioInput(recorder, clock, Camera.main.gameObject.GetComponent<AudioListener>());
    20.         audioDevice.StartRunning((sampleBuffer, timestamp) => recorder.CommitSamples(sampleBuffer, clock.timestamp));
    21.        

    Code (CSharp):
    1. // Microphone audio.
    2.         var query = new MediaDeviceQuery(MediaDeviceQuery.Criteria.AudioDevice);
    3.         var microphone = query.currentDevice as IAudioDevice;
    4.         audioDevice = new MixerDevice(microphone, Camera.main.gameObject.GetComponent<AudioListener>());
    5.  
    6.         Debug.Log("SampleRate: Device: "+audioDevice.sampleRate);
    7.         Debug.Log("SampleRate: AudioSettings: "+AudioSettings.outputSampleRate);
    8.         // Recorder setyop
    9.         var clock = new RealtimeClock();
    10.         recorder = new MP4Recorder((int) prod.x, (int) prod.y, 30, AudioSettings.outputSampleRate, 2);
    11.         cameraInput = new CameraInput(recorder, clock, Camera.main);
    12.  
    13.         /// Game + Mic Audio
    14.         //_audioInput = new AudioInput(recorder, clock, Camera.main.gameObject.GetComponent<AudioListener>());
    15.         audioDevice.StartRunning((sampleBuffer, timestamp) => recorder.CommitSamples(sampleBuffer, clock.timestamp));
     
    Last edited: Nov 5, 2020
  12. Claude_V

    Claude_V

    Joined:
    Apr 16, 2013
    Posts:
    16
    Hi Again,

    The Black screen on on the Blackview android happens sometimes but on the Samsung S9 I have an issue that appears very often.
    The camera seems to halt when the device comes back from being switched "off" (pause).
    The logcat (attached here) states :

    CameraService: CameraService::connect evicting conflicting client for camera ID 0

    As I call (to test) a query.Advance(); I get this just after the "conflicting"

    NatSuite: NatDevice Error: Camera 0 failed to stop preview

    Steps to reproduce : run the app with the camera preview playing; Press the "on/off" button of the phone so the phone goes to sleep. Wait a bit and press the on/off button again, the phone will unpause (in my case ask the passcode) and then show the app again... with the camera not moving anymore. When I than call the "StopRunning" the script crashes on the failed to stop preview.

    Thanks!
     

    Attached Files:

    Last edited: Nov 5, 2020
  13. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    I haven't used ARF and URP together, so I'll have to check this and get back to you. It seems that the camera stacking feature might interfere with a standard render-to-texture.
     
  14. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    This means that for some reason, the hardware codec is throwing an error. It is impossible to tell why this happens from the exception that is logged.
     
  15. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    The challenge here is in mixing audio, while avoiding drifting. The MixerDevice implementation is a reference implementation, but it still falls into the synchronization issue. It will be removed in the next NatDevice update.
    Basically, don't commit video from more than one source; and don't commit audio from more than one source. You can think of recording as a FIFO queue: the frames you put in, in the sequence you put them in, "come out" in the final recording. If you recorded say audio using both an audio device and an audio input, your "queue" will have two disparate audio streams sliced together. Think of this like a flickering video. What you want to do is to mix the audio from both sources, then commit them.
    The MixerDevice implementation is a reference; it isn't intended to be used in production. It is being removed as a result. Mixing two audio streams is very much an open problem.
     
  16. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    Looks like you posted on the wrong thread. This thread is for NatCorder issues; you'll want to follow up on the NatDevice thread.
    It looks like there are two consecutive attempts to open the camera. I'll try to reproduce it on my end.
     
  17. Claude_V

    Claude_V

    Joined:
    Apr 16, 2013
    Posts:
    16
    It happens 1/3 up to 1/5 procedures of putting the app to sleep. I do not re-initialize the camera. Only on initial "Start()" of the app
     
    Lanre likes this.
  18. Emile-Swain

    Emile-Swain

    Joined:
    Jun 23, 2015
    Posts:
    18
    Totally with you on the one input per media.

    Whilst it the mizer was sort of working, it basically needed a bit more testing around the buffer lengths which were throwig up errors.. So i switched out to just using the mic for now. I wouldn't necessarily remove Mixer unless you have an alternative. With some comments that its untested/supported and needs work, i actually felt it helped me understand the overall process of writing to the media streams. Someone might find it usefull is all.

    The main issue i'm facing at the moment is with the stutter when initiating the MP4Recorder. I'm trying different resolutions to see if that helps. I'm wondering if starting in a coroutine would help.
     
  19. bramble-operations

    bramble-operations

    Joined:
    Aug 27, 2019
    Posts:
    5
    I have a problem with the android app crashing when I tap the phone screen after I start recording. It works on iOS, but does not work when I build the APK and put it on my android device. please advise thank you!

    Device: Doogee X10S runs Android 8
    Android API Level target: 30
    Unity version: 2019.4.2f
    NatCorder version: 1.7.3

    Code:
    Code (CSharp):
    1.     public IEnumerator StartRecording ()
    2.     {
    3.         if (recordingState != RecordingState.Stopped)
    4.         {
    5.             Debug.Log("Attempting to start recording when already active");
    6.         }
    7.         Debug.Log("Start Recording");
    8.         yield return InitializeWithPermissions();
    9.         if (!permissionsFailed)
    10.         {
    11.             // Start recording
    12.             var frameRate = 30;
    13.             var sampleRate = recordMicrophone ? AudioSettings.outputSampleRate : 0;
    14.             var channelCount = recordMicrophone ? (int)AudioSettings.speakerMode : 0;
    15.             clock = new RealtimeClock();
    16.             recorder = new HEVCRecorder(Screen.width, Screen.height, frameRate, sampleRate, channelCount);
    17.             CreateInputs();
    18.         }
    19.         else
    20.         {
    21.             Debug.Log("failed to get permissions to record");
    22.         }
    23.     }
    24. private void CreateInputs()
    25.     {
    26.         audioInput = recordMicrophone ? new AudioInput(recorder, clock, microphoneSource, true) : null;
    27.         // Unmute microphone
    28.         microphoneSource.mute = audioInput == null;
    29.         // configure camera inputs
    30.         cameraInput = new CameraInput(recorder, clock, cameras.ToArray());
    31.         recordingState = RecordingState.Active;
    32.     }
    Crash Log:
    2020-11-07 16:25:28.023 5379-5379/? E/InputEventReceiver: Exception dispatching input event.
    2020-11-07 16:25:28.023 5379-5379/? E/MessageQueue-JNI: Exception in MessageQueue callback: handleReceiveCallback
    2020-11-07 16:25:28.026 5379-5379/? E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.letsbramble.bramble, PID: 5379
    java.lang.Error: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    Version '2019.4.2f1 (20b4642a3455)', Build type 'Release', Scripting Backend 'il2cpp', CPU 'armeabi-v7a'
    Build fingerprint: 'DOOGEE/X10S/X10S:8.1.0/O11019/20180717:user/release-keys'
    Revision: '0'
    ABI: 'arm'
    Timestamp: 2020-11-07 03:25:13-0500
    pid: 5379, tid: 5440, name: Thread-19 >>> com.letsbramble.bramble <<<
    uid: 10133
    signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
    r0 00000000 r1 00001540 r2 00000006 r3 00000008
    r4 00001503 r5 00001540 r6 89481fcc r7 0000010c
    r8 00000000 r9 89482019 r10 0000000a r11 89482018
    ip 7f7fd420 sp 89481fb8 lr a9762a5b pc a975c48c

    backtrace:
    #00 pc 0001a48c /system/lib/libc.so (abort+64) (BuildId: 6f041c4782cb6c6ffbcea36ea218aaba)
    #01 pc 00360b3f /system/lib/libart.so (art::Runtime::Abort(char const*)+262) (BuildId: 04658dfd580f58ea5c3323d4dc975b2e)
    #02 pc 0042145d /system/lib/libart.so (android::base::LogMessage::~LogMessage()+452) (BuildId: 04658dfd580f58ea5c3323d4dc975b2e)
    #03 pc 0024f981 /system/lib/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+1232) (BuildId: 04658dfd580f58ea5c3323d4dc975b2e)
    #04 pc 0024fb77 /system/lib/libart.so (art::JavaVMExt::JniAbortF(char const*, char const*, ...)+66) (BuildId: 04658dfd580f58ea5c3323d4dc975b2e)
    #05 pc 00283287 /system/lib/libart.so (art::JNI::GetObjectClass(_JNIEnv*, _jobject*)+670) (BuildId: 04658dfd580f58ea5c3323d4dc975b2e)
    #06 pc 00000f87 /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libNatCorder.so (NCCommitSamples+70) (BuildId: 2567ae6a57fad542c249dc2cc296d227ffcfe4c3)
    #07 pc 010fb088 /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libil2cpp.so (BuildId: cdfa55d4fc734abd8c7e61c2a595291c87af6fb7)
    #08 pc 010fb778 /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libil2cpp.so (BuildId: cdfa55d4fc734abd8c7e61c2a595291c87af6fb7)
    #09 pc 010f9cd4 /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libil2cpp.so (BuildId: cdfa55d4fc734abd8c7e61c2a595291c87af6fb7)
    #10 pc 010fa228 /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libil2cpp.so (BuildId: cdfa55d4fc734abd8c7e61c2a595291c87af6fb7)
    #11 pc 00da6c88 /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libil2cpp.so (BuildId: cdfa55d4fc734abd8c7e61c2a595291c87af6fb7)
    #12 pc 0033c45c /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libil2cpp.so (BuildId: cdfa55d4fc734abd8c7e61c2a595291c87af6fb7)
    #13 pc 002707fc /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libil2cpp.so (BuildId: cdfa55d4fc734abd8c7e61c2a595291c87af6fb7)
    #14 pc 00228641 /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libunity.so (BuildId: 3a891f7d3dec7a88ee0697a98eee5125e85cced4)
    #15 pc 00231c8f /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libunity.so (BuildId: 3a891f7d3dec7a88ee0697a98eee5125e85cced4)
    #16 pc 0011d031 /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libunity.so (BuildId: 3a891f7d3dec7a88ee0697a98eee5125e85cced4)
    #17 pc 00875e0b /data/app/com.letsbramble.bramble-yjB9vZqHnTIlbJ1ni-1XAQ==/lib/arm/libunity.so (BuildId: 3a891f7d3dec7a88ee0697a98eee5125e85cced4)

    at libc.abort(abort:64)
    at libart.art::Runtime::Abort(char const*)(Abort:262)
    at libart.android::base::LogMessage::~LogMessage()(~LogMessage:452)
    at libart.art::JavaVMExt::JniAbort(char const*, char const*)(JniAbort:1232)
    at libart.art::JavaVMExt::JniAbortF(char const*, char const*, ...)(JniAbortF:66)
    at libart.art::JNI::GetObjectClass(_JNIEnv*, _jobject*)(GetObjectClass:670)
    at libNatCorder.NCCommitSamples(NCCommitSamples:70)
    at libil2cpp.0x10fb088(Native Method)
    at libil2cpp.0x10fb778(Native Method)
    at libil2cpp.0x10f9cd4(Native Method)
     
  20. Interfo

    Interfo

    Joined:
    Dec 11, 2017
    Posts:
    3
    ARCore 1.18 + unity3d 2020.1.3f
    Can't record mp4 of ARcore camera.
    how to?
    Your ReplayCam class saving a video file.
    but recorded video image is just stopped when run app android 10
    on PC working well(instant preview).
    only problem in android app.
     
  21. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    I'll move it to GitHub gists.
    Make sure you're using the latest version of NatCorder.
     
  22. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    Can you be more descriptive? I don't understand exactly what your setup is. What does ReplayCam have to do with you recording with ARCore?
     
  23. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    Can you share the full, unfiltered logs from app start to crash in a .txt attachment? The crash is likely either because you are trying to record at screen resolution which is highly discouraged, or because your device doesn't have a HEVC encoder.
     
  24. danidiazr

    danidiazr

    Joined:
    Dec 27, 2016
    Posts:
    15
    Hi there!

    I just installed NatShare and NatCorder from Asset Store to my project. I'm making an AR filter app for iOS & Android. My app crashes when saving the video to the user's camera roll. I use:

    microphoneSource.mute = true;
    audioInput?.Dispose();
    cameraInput.Dispose();
    var path = await recorder.FinishWriting();

    Debug.Log($"Saved recording to: {path}");

    var payload = new SavePayload(Application.productName);
    payload.AddMedia(path);

    await payload.Commit(); // Here the app crashes

    When testing it in Unity I got an error:

    DllNotFoundException: NatShare
    NatSuite.Sharing.SavePayload+<>c__DisplayClass0_0.<.ctor>b__0 (NatSuite.Sharing.Internal.Bridge+CompletionHandler callback, System.IntPtr context) (at Assets/NatSuite/Plugins/Managed/Sharing/SavePayload.cs:22)
    NatSuite.Sharing.Internal.NativePayload..ctor (System.Func`3[T1,T2,TResult] payloadCreator) (at Assets/NatSuite/Plugins/Managed/Sharing/Internal/NativePayload.cs:21)
    NatSuite.Sharing.SavePayload..ctor (System.String album) (at Assets/NatSuite/Plugins/Managed/Sharing/SavePayload.cs:22)
    CameraManager+<get_pathAsync>d__36.MoveNext () (at Assets/Assets/Scripts/CameraManager.cs:304)
    --- End of stack trace from previous location where exception was thrown ---
    System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at <fb001e01371b4adca20013e0ac763896>:0)
    System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__6_0 (System.Object state) (at <fb001e01371b4adca20013e0ac763896>:0)
    UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at <a979f18d51af41179d12b797e8c5be14>:0)
    UnityEngine.UnitySynchronizationContext:ExecuteTasks()

    Help!
     
  25. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    Sharing won't work in the Editor. See this issue.
     
  26. danidiazr

    danidiazr

    Joined:
    Dec 27, 2016
    Posts:
    15
    For sure it won't work in Editor, but in Android build it crash. The app closes without any logs, the crash happends in the commit line. I saw this issue previously and find out something from the source code, how's that?
     
  27. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    Can you share the full unfiltered logs from logcat in a .txt attachment? Does your Android device have an x86 CPU? NatSuite API's only support armeabi-v7a and arm64-v8a architectures on Android.
     
  28. danidiazr

    danidiazr

    Joined:
    Dec 27, 2016
    Posts:
    15
    I attached the log file. I'm using a Samsung Galaxy Note 8.

    I need the app to save the recorded video to the camera roll. I'm sure the app has the user permissions.
     

    Attached Files:

    • log.txt
      File size:
      8.5 KB
      Views:
      3
  29. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    I need the logs from app start to crash. These logs don't contain any information pertinent to this issue. Can you retake them?
     
  30. ghasedak3411

    ghasedak3411

    Joined:
    Aug 25, 2015
    Posts:
    22
    @Lanre
    hi

    Do you have a tutorial for using native mode?

    public void CommitFrame (IntPtr nativeBuffer, long timestamp)
     
  31. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
  32. erikblue88

    erikblue88

    Joined:
    Aug 19, 2013
    Posts:
    6
    Same issue here, I can´t save the video to gallery, the app in android crashes and closes in the commit line.

    I need help please.
     
  33. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    I need to see the logs. Please provide the logs in a .txt attachment.
     
  34. erikblue88

    erikblue88

    Joined:
    Aug 19, 2013
    Posts:
    6
    Don´t worry man, i have solved this issue using the nativeGallery plugin:

    https://github.com/yasirkula/UnityNativeGallery

    NatCorder and nativeGallery result in a very great combination.

    Thanks and Regards!
     
    Lanre likes this.
  35. jiraphatK

    jiraphatK

    Joined:
    Sep 29, 2018
    Posts:
    70
    It would be great if your plugin has support for setting video metadata such as "Rotation", "Author", "Title" etc before recording.

    I have an issue where we need to output video in landscape mode in our portrait-locked app.
    My solution right now is Graphics.Blit() with rotate shader every frame which did not look like a good idea considering it add even more command to the already stressed GPU (from re-rendering the scene) and we HAVE to read pixels after bliting which will cause sync point. No, we can not use Vulkan because we are developing an AR app.

    I have tried using https://github.com/tanersener/mobile-ffmpeg to rotate video metadata after finish recording and it worked like a charm. There's no overhead when recording and it's super fast. Sadly, we can't use this in our product because of the LGPL licensing issue.
     
  36. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    NatCorder won't be adding support for this because of sparse support and the engineering effort. Android doesn't support adding a metadata track until API level 26. And Windows' SinkWriter doesn't support writing metadata.
    This is pretty fair; working from Unity you really don't have many options. Without Vulkan, you can't use async readbacks. You could simply transpose the pixel buffer on the CPU in a worker thread to free up the GPU. If you use unsafe code with pointers, you should get close to the speed you'd have in C/C++.
    As far as I can tell, and from a compatibility standpoint, writing the metadata after the MP4 has been written is your best option. I recommend looking into a way to do this that doesn't rely on FFmpeg.
     
  37. Makio64

    Makio64

    Joined:
    Nov 9, 2014
    Posts:
    22
    @lanrell Hi, Recording in Mp4 the screen with an HDR Camera as CameraInput in a URP doesnt loooks to work, the color are totally incorrect, is there a way to fix it ? PS : the recording integrated to the iphone record the right color.

    - Tested on iPhone 12 pro
    - URP 8.2
    - Latest Natcorder Version
    - HDR pipeline rendering ( HDR camera )
    - Unity 2020.2.0b5
    - DisplayP3 color profile

    Thanks in advance !
     
    Elizeu159 likes this.
  38. bramble-operations

    bramble-operations

    Joined:
    Aug 27, 2019
    Posts:
    5
    Hi Lanre, yes the problem was fixed after changing to MP4, thank you
     
    Lanre likes this.
  39. Makio64

    Makio64

    Joined:
    Nov 9, 2014
    Posts:
    22
    @lanrell Any Idea how to record the HDR colors ? Thanks!
     
  40. Elizeu159

    Elizeu159

    Joined:
    Nov 25, 2016
    Posts:
    7
    I am too having problems using URP. The post-processing is not captured at all by the recording: bloom, HDR colors, etc.

    Also it doesn't record any of the stacked Overlay cameras, the only solution was to set the Canvas to screen space Camera and link the Base camera to it, but that's not a good solution, it breaks other stuff not related to NatCorder. Is there a way to record other cameras when using URP in the current version?
     
  41. Ikaro88

    Ikaro88

    Joined:
    Jun 6, 2016
    Posts:
    131
    It is possible stop device audio while recording?
    Or record only some audiosource and others not?
    Or add multiple audiosiurce mutated to the recording?
     
    Last edited: Nov 24, 2020
  42. Ikaro88

    Ikaro88

    Joined:
    Jun 6, 2016
    Posts:
    131
    Last edited: Nov 24, 2020
  43. Seginus

    Seginus

    Joined:
    Feb 1, 2018
    Posts:
    9
    Is there any root issue that might cause very slight desync between audio and video recording? No error are being generated or the like, just a minor drift in the output file (like maybe a tenth of a second after a minute of recording).
     
  44. Ikaro88

    Ikaro88

    Joined:
    Jun 6, 2016
    Posts:
    131
    My app just freeze everything after 4 seconds, with a script copied from the example...

    It is because you use use 2 audio channels on mp4 recorder!, but in the demo it works..


    this line of code

    Code (CSharp):
    1.             recorder = new MP4Recorder(
    2. Screen.width,                           // Video width
    3. Screen.height,                            // Video height
    4. 30,                             // Video framerate
    5. AudioSettings.outputSampleRate, // Unity's audio sample rate
    6. 2  // Unity's audio channel count
    7.  
    8.  
    9. );
    10.      
    or


    Code (CSharp):
    1.             recorder = new MP4Recorder(
    2. Screen.width,                           // Video width
    3. Screen.height,                            // Video height
    4. 30,                             // Video framerate
    5. AudioSettings.outputSampleRate, // Unity's audio sample rate
    6. (int)AudioSettings.speakerMode  // Unity's audio channel count
    7.  
    8.  
    9. );
    10.      
    totally crash my app after 4 seconds of recordings
    but this one:



    Code (CSharp):
    1.             recorder = new MP4Recorder(
    2. Screen.width,                           // Video width
    3. Screen.height,                            // Video height
    4. 30,                             // Video framerate
    5. AudioSettings.outputSampleRate, // Unity's audio sample rate
    6. 0  // Unity's audio channel count
    7.  
    8.  
    9. );
    10.      
    works but not let me to commit audio that is the reason why my company buy this for


    another strange behavour

    Example:

    this class work


    Code (CSharp):
    1. /*
    2. *   NatCorder
    3. *   Copyright (c) 2020 Yusuf Olokoba
    4. */
    5.  
    6. namespace NatSuite.Examples {
    7.  
    8.     using UnityEngine;
    9.     using UnityEngine.UI;
    10.     using System.Collections;
    11.     using Recorders;
    12.     using Recorders.Clocks;
    13.     using Recorders.Inputs;
    14.     using UnityEngine.Rendering;
    15.  
    16.     public class WebCam1 : MonoBehaviour {
    17.  
    18.         [Header(@"UI")]
    19.  
    20.         public MP4Recorder recorder;
    21.         public IClock clock;
    22.         public bool recording;
    23.         private Color32[] pixelBuffer;
    24.         private AudioSource microphoneSource;
    25.         public static WebCam1 instance;
    26.         public RenderTexture rt;
    27.         public Texture2D TestTexture;
    28.         #region --Recording State--
    29.         CameraInput cameraInput;
    30.         public bool SimpleRecording;
    31.         public void StartRecording () {
    32.    
    33.             clock = new RealtimeClock();
    34.  
    35.             recorder = new MP4Recorder(
    36. Screen.width,                           // Video width
    37. Screen.height,                            // Video height
    38. 30,                             // Video framerate
    39. AudioSettings.outputSampleRate, // Unity's audio sample rate
    40. 2  // Unity's audio channel count
    41.  
    42.  
    43. );
    44.             // Create recording inputs
    45.             if(SimpleRecording)
    46.             cameraInput = new CameraInput(recorder, clock, Camera.main);
    47.             recording = true;
    48.         }
    49.  
    50.         public async void StopRecording () {
    51.             if(SimpleRecording)
    52.             cameraInput.Dispose();
    53.  
    54.             // Stop recording
    55.             recording = false;
    56.             var path = await recorder.FinishWriting();
    57.             // Playback recording
    58.             Debug.Log($"Saved recording to: {path}");
    59.             Handheld.PlayFullScreenMovie($"file://{path}");
    60.         }
    61.         #endregion
    62.  
    63.  
    64.         #region --Operations--
    65.  
    66.         IEnumerator Start () {
    67.             instance = this;
    68.             yield return Application.RequestUserAuthorization(UserAuthorization.WebCam);
    69.             if (!Application.HasUserAuthorization(UserAuthorization.WebCam))
    70.                 yield break;
    71.         }
    72.  
    73.         void Update () {
    74.             // Record frames from the webcam
    75.             if(!SimpleRecording)
    76.                 if (recording)
    77.                 {
    78.                  
    79.                         Debug.Log(rt.width);
    80.                         Debug.Log(rt.height);
    81.  
    82.                         Texture2D myTexture2D = new Texture2D(rt.width, rt.height);
    83.                            RenderTexture.active = rt;
    84.                            myTexture2D.ReadPixels(new Rect(0, 0, rt.width, rt.height), 0, 0);
    85.                            myTexture2D.Apply();
    86.                            Color32[] fanculo = myTexture2D.GetPixels32();
    87.                            recorder.CommitFrame(fanculo, clock.timestamp);
    88.                 }
    89.             }
    90.  
    91.         private IEnumerator InitMicrophone()
    92.         {
    93.             microphoneSource = gameObject.AddComponent<AudioSource>();
    94.             microphoneSource.mute =
    95.             microphoneSource.loop = true;
    96.             microphoneSource.bypassEffects =
    97.             microphoneSource.bypassListenerEffects = false;
    98.             microphoneSource.clip = Microphone.Start(null, true, 10, AudioSettings.outputSampleRate);
    99.             yield return new WaitUntil(() => Microphone.GetPosition(null) > 0);
    100.             microphoneSource.Play();
    101.         }
    102.         #endregion
    103.     }
    104. }
    this one cause the freeze of the app, BUT ARE THE SAME CLASS!!!!

    Code (CSharp):
    1. *
    2. *   NatCorder
    3. *   Copyright (c) 2020 Yusuf Olokoba
    4. */
    5.  
    6. namespace NatSuite.Examples {
    7.  
    8.     using UnityEngine;
    9.     using UnityEngine.UI;
    10.     using System.Collections;
    11.     using Recorders;
    12.     using Recorders.Clocks;
    13.     using Recorders.Inputs;
    14.     using UnityEngine.Rendering;
    15.  
    16.     public class WebCam2 : MonoBehaviour {
    17.  
    18.         [Header(@"UI")]
    19.  
    20.         public MP4Recorder recorder;
    21.         public IClock clock;
    22.         public bool recording;
    23.         private Color32[] pixelBuffer;
    24.         private AudioSource microphoneSource;
    25.         public static WebCam2 instance;
    26.         public RenderTexture rt;
    27.         public Texture2D TestTexture;
    28.         #region --Recording State--
    29.         CameraInput cameraInput;
    30.         public bool SimpleRecording;
    31.         public void StartRecording () {
    32.    
    33.             clock = new RealtimeClock();
    34.  
    35.             recorder = new MP4Recorder(
    36. Screen.width,                           // Video width
    37. Screen.height,                            // Video height
    38. 30,                             // Video framerate
    39. AudioSettings.outputSampleRate, // Unity's audio sample rate
    40. 2  // Unity's audio channel count
    41.  
    42.  
    43. );
    44.             // Create recording inputs
    45.             if(SimpleRecording)
    46.             cameraInput = new CameraInput(recorder, clock, Camera.main);
    47.             recording = true;
    48.         }
    49.  
    50.         public async void StopRecording () {
    51.             if(SimpleRecording)
    52.             cameraInput.Dispose();
    53.  
    54.             // Stop recording
    55.             recording = false;
    56.             var path = await recorder.FinishWriting();
    57.             // Playback recording
    58.             Debug.Log($"Saved recording to: {path}");
    59.             Handheld.PlayFullScreenMovie($"file://{path}");
    60.         }
    61.         #endregion
    62.  
    63.  
    64.         #region --Operations--
    65.  
    66.         IEnumerator Start () {
    67.             instance = this;
    68.             yield return Application.RequestUserAuthorization(UserAuthorization.WebCam);
    69.             if (!Application.HasUserAuthorization(UserAuthorization.WebCam))
    70.                 yield break;
    71.         }
    72.  
    73.         void Update () {
    74.             // Record frames from the webcam
    75.             if(!SimpleRecording)
    76.                 if (recording)
    77.                 {
    78.                  
    79.                         Debug.Log(rt.width);
    80.                         Debug.Log(rt.height);
    81.  
    82.                         Texture2D myTexture2D = new Texture2D(rt.width, rt.height);
    83.                            RenderTexture.active = rt;
    84.                            myTexture2D.ReadPixels(new Rect(0, 0, rt.width, rt.height), 0, 0);
    85.                            myTexture2D.Apply();
    86.                            Color32[] fanculo = myTexture2D.GetPixels32();
    87.                            recorder.CommitFrame(fanculo, clock.timestamp);
    88.                 }
    89.             }
    90.  
    91.         private IEnumerator InitMicrophone()
    92.         {
    93.             microphoneSource = gameObject.AddComponent<AudioSource>();
    94.             microphoneSource.mute =
    95.             microphoneSource.loop = true;
    96.             microphoneSource.bypassEffects =
    97.             microphoneSource.bypassListenerEffects = false;
    98.             microphoneSource.clip = Microphone.Start(null, true, 10, AudioSettings.outputSampleRate);
    99.             yield return new WaitUntil(() => Microphone.GetPosition(null) > 0);
    100.             microphoneSource.Play();
    101.         }
    102.         #endregion
    103.     }
    104. }



    It is useless in this way, waste 1 week of my time and money I am REALLY disapponinted


    This sdk create also more problem on my app.....
     
    Last edited: Nov 25, 2020
  45. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    NatCorder does not support HDR rendering at this time.
     
  46. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    I'm not sure what you mean by stopping device audio while recording. If you want to 'mute' the audio while recording, then you have to send zeros in the audio buffer to the recorder. Recording some audio sources and not others is not trivial; you have to manually mix the audio from the audio sources before sending them to the recorded. I'm not sure what you are asking with your third question.
     
  47. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    I'm not aware of any drifting that happens. What device does this happen on? Can you share your recording code?
     
  48. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    You are specifying an audio format (passing in non-zero numbers for the sample rate and channel count) without actually committing any audio (no audio input, no recorder.CommitSamples). On Windows, this always results in a hang. Don't specify an audio format if you are not committing any audio. Read the documentation because all these notes are listed.
     
  49. Ikaro88

    Ikaro88

    Joined:
    Jun 6, 2016
    Posts:
    131

    I forgot to mention that I commit the audio with another script

    this is used for the example of webcam2



    Code (CSharp):
    1. void OnAudioFilterRead(float[] data, int channels)
    2.     {
    3.         if(record)
    4.         if (data.Length > 0)
    5.                 {
    6.                     if(WebCam2.instance!=null)
    7.                     if (WebCam2.instance.recording)
    8.                     {
    9.                         WebCam2.instance.recorder.CommitSamples(data, WebCam2.instance.clock.timestamp);
    10.              
    11.                     }
    12.                 }
    13.             if (mute)
    14.         {
    15.             System.Array.Clear(data, 0, data.Length);
    16.         }
    17.     }

    And sometimes it also cause some problem with the camera (is a AR project)

    btw I also have a delay in the michropone input too in the scene with the mic recorder, can i compensate it with removing some seconds from the clock.timestamp while recording?

    Could be the unity version? 2019.4.11f?
     
  50. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,445
    Does the ReplayCam example work for you?
     
unityunity