Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

NatCorder - Video Recording API

Discussion in 'Assets and Asset Store' started by Lanre, Nov 18, 2017.

  1. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    NatReader is still pre-release, so keep that in mind (look at the open issues to get a sense of what is outstanding). To use the API, clone it and open it in Unity, export the NatSuite folder as a unitypackage, then import the package into your project.
     
    Mr-Mechanical likes this.
  2. Lordmin

    Lordmin

    Joined:
    Mar 9, 2017
    Posts:
    62
    The Unity version currently supported by NatCorder is 2019.3.11.

    But our project version is 2018.4.17f1.

    Can I download NatCorder to 2019.3.11 and then downgrade to 2018.4.17f1?
     
  3. Venzel

    Venzel

    Joined:
    May 7, 2018
    Posts:
    19
    Hi @Lanre!
    I have an application freeze on recording start. Is it normal? Is there a way to minimize this freeze?
    There is no freeze on pause/resume recording, only on start. Logs are clear, there are no alerts or anything.

    Unity 2019.3
    iOS 13/14, iPhones 7/8/X
    NatCorder 1.7.2
    NatDevice 1.0.2 for sound recording from microphone
    AR Foundation

    Code for start recording:

    Code (CSharp):
    1. var query = new MediaDeviceQuery(MediaDeviceQuery.Criteria.AudioDevice);
    2. device = query.currentDevice as AudioDevice;
    3. clock = new RealtimeClock();
    4. recorder = new MP4Recorder(videoWidth, videoHeight, 30, device.sampleRate, device.channelCount, 4800000);
    5. cameraInput = new CameraInput(recorder, clock, Camera.main);
    6. device.StartRunning((sampleBuffer, timestamp) => recorder.CommitSamples(sampleBuffer, clock.timestamp));
     
  4. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    Yup. You can copy the NatSuite folder over to your 2018 project.
     
  5. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    The hiccup should only happen on the very first recording. In every recording after that, there should be no hiccups. There is currently no way to avoid this.
     
  6. Mr-Mechanical

    Mr-Mechanical

    Joined:
    May 31, 2015
    Posts:
    507
    What is the best way to use Unity VideoPlayer to decode videos frame by frame?
    Thank you so much for your help.
     
  7. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    I've never tried doing this so I wouldn't know. I recommend checking out the VideoPlayer docs.
     
    Mr-Mechanical likes this.
  8. sabanmete

    sabanmete

    Joined:
    Mar 20, 2020
    Posts:
    19
    how can i show the video i recorded in app like snapchat or instagram ? i am using playfullscreen func but i want to show it in app. Thank you.
     
  9. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    You can use Unity's VideoPlayer component. Video playback is out of scope for NatCorder.
     
  10. NaijaNY82

    NaijaNY82

    Joined:
    Mar 7, 2017
    Posts:
    19
    Do you have an On Click () version for RecordButton.cs? I want to record with the same circle fill effect by clicking a start button and a stop button instead of holding the record button down.
     
  11. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    No, the component is built for the example scene only; it is not intended to be used outside of the example scenes.
     
  12. sabanmete

    sabanmete

    Joined:
    Mar 20, 2020
    Posts:
    19
    i did that for my project. maybe this can help you. put this on RecordButton.cs
    Code (CSharp):
    1.  
    2.     using System.Collections;
    3.     using UnityEngine;
    4.     using UnityEngine.UI;
    5.     using UnityEngine.Events;
    6.     using UnityEngine.EventSystems;
    7.  
    8.     [RequireComponent(typeof(EventTrigger))]
    9.     public class RecordButton : MonoBehaviour, IPointerDownHandler, IPointerUpHandler {
    10.  
    11.         public Image button, countdown;
    12.         public UnityEvent onTouchDown, onTouchUp, onTap;
    13.         private bool pressed;
    14.         private const float MaxRecordingTime = 30f; // seconds
    15.         bool recording = false;
    16.         private void Start () {
    17.             Reset();
    18.         }
    19.  
    20.         private void Reset () {
    21.             // Reset fill amounts
    22.             if (button)
    23.                 button.fillAmount = 1.0f;
    24.             if (countdown)
    25.                 countdown.fillAmount = 0.0f;
    26.         }
    27.         void IPointerDownHandler.OnPointerDown (PointerEventData eventData) {
    28.             // Start counting
    29.             StartCoroutine(Countdown());
    30.         }
    31.  
    32.         void IPointerUpHandler.OnPointerUp (PointerEventData eventData) {
    33.             // Reset pressed
    34.             pressed = false;
    35.         }
    36.  
    37.         private IEnumerator Countdown () {
    38.             pressed = true;
    39.             // First wait a short time to make sure it's not a tap
    40.             yield return new WaitForSeconds(0.5f);
    41.             if (!pressed)
    42.                 {
    43.                 if (recording == true){
    44.                     recording = false;
    45.                 }
    46.                 else
    47.                 {
    48.                     onTap?.Invoke(); Debug.Log(onTap);
    49.                 }
    50.                    
    51.                 yield break;
    52.                 }
    53.             // Start recording
    54.             else
    55.             {
    56.                 if (!recording)
    57.                 {
    58.                     onTouchDown?.Invoke();
    59.                     recording = true;
    60.                     // Animate the countdown
    61.                     float startTime = Time.time, ratio = 0f;
    62.                     while (recording && (ratio = (Time.time - startTime) / MaxRecordingTime) < 1.0f)
    63.                     {
    64.                         countdown.fillAmount = ratio;
    65.                         button.fillAmount = 1f - ratio;
    66.                         yield return null;
    67.                     }
    68.                     // Reset
    69.                     Reset();
    70.                     // Stop recording
    71.                     onTouchUp?.Invoke();
    72.                     recording = false;
    73.                 }
    74.  
    75.             }
    76.         }
    77.     }
     
    Lanre likes this.
  13. sinjimonkey

    sinjimonkey

    Joined:
    Jul 31, 2013
    Posts:
    72
    Hi.
    I can't get the audio to record. Am I doing something wrong? The Audio Listener is the listener that's on the Camera and the only Audio Listener in the scene.

    Code (CSharp):
    1. _recorder = new NatSuite.Recorders.MP4Recorder(width, height, 30, 44100, 2);
    2.         _recording = true;
    3.  
    4.         Debug.Log("Audio:  "+AudioListener);
    5.         _audioInput = new AudioInput(_recorder, clock, AudioListener);
    6.         Application.targetFrameRate = 30;
    7.         _cameraInput = new CameraInput(_recorder, clock, cam);
     
  14. sinjimonkey

    sinjimonkey

    Joined:
    Jul 31, 2013
    Posts:
    72
    Also as a seperate question, is there a way to get direcly get the video object that results, or do I have to load it from the drive? I want to use NatShare to save it to the camera roll and social media (which is what I'm already doing with photos)

    Thanks.
     
  15. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    Make sure that on all the AudioSource components in the scene, "Bypass Listener Effects" is disabled. Also, you are hard-coding the sample rate and channel count which is wrong. You must use the values reported by Unity's audio engine if you are recording audio from Unity.
    I'm not sure what you mean by video object. Recorders provide you the path to the recorded media file. NatShare accepts that path, with the `AddMedia` method.
     
    sinjimonkey likes this.
  16. jiraphatK

    jiraphatK

    Joined:
    Sep 29, 2018
    Posts:
    306
    Hi again, Lanre. Could you guide me on how to updated this script?
     
  17. dri_richard

    dri_richard

    Joined:
    Mar 10, 2017
    Posts:
    153
    I'd like to request support for specifying the audio bitrate. Our game is a music game, and right now the user hears high quality music in the game, then they play back the recording and hear the same music at 64kHz. It sounds really bad. We want users to share recordings on social networks, and we worry that the low audio quality is a barrier to that.
     
  18. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    I don't think that script will work with the latest versions of NatCorder because of how the API has changed. @HappyShip did you ever update this script for the latest version of NatCorder?
     
  19. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    This is fair. I'll add it in the next update.
     
    Mr-Mechanical and dri_richard like this.
  20. sinjimonkey

    sinjimonkey

    Joined:
    Jul 31, 2013
    Posts:
    72
    Ah good. I thought I'd have to load it and then pass it to natshare. If it accepts the path then that's perfect.

    Thanks for the tip on the sample rate. That's the part that wasn't obvious.
     
  21. sinjimonkey

    sinjimonkey

    Joined:
    Jul 31, 2013
    Posts:
    72
    Wait. If I bypass listener effects doesn't that mean the audio won't play through the speakers while recording?

    EDIT: OH. Bypass Listener Effects is DISABLED. Yeah. If they were disabled then I wouldn't hear them. It's a good 'is it plugged in? Have you tried turning it off and on again?' pointer. Hopefully it was the hard-coded values. I've changed the script and will test it in a bit.
     
    Last edited: Oct 18, 2020
  22. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    Always make sure to read through the docs. Most questions of this form are already touched on in the docs.
    Bypassing effects means ignoring components that try to do audio filtering (using OnAudioFilterRead). It must be disabled, so that the effects are not bypassed (this is how AudioInput works). It has nothing to do with audio output being muted.
     
    sinjimonkey likes this.
  23. sinjimonkey

    sinjimonkey

    Joined:
    Jul 31, 2013
    Posts:
    72
    Yeah, sorry. I had an old version of NatShare working last year and I kind of expected a lot of this to be similar to before, and I'm rebuilding the architecture for this project. I was sorely tempted to just use the old api when it wasn't working as expected/
     
  24. sinjimonkey

    sinjimonkey

    Joined:
    Jul 31, 2013
    Posts:
    72
    So I'm looking at the docs here;
    https://docs.natsuite.io/natcorder/

    And I don't see any mention of using AudioSettings at all. I've even tried searching for 'AudioSettings'

    This is still not working. I don't see a way to get the channel count in the AudioSettings docs nor is it mentioned in the NatCorder docs.

    Am I supposed to 'switch' based on the SpeakerMode? Other than some of the obvious ones "Stereo and Mono" what do I switch them all to?


    Code (CSharp):
    1.  
    2.         foreach (var listener in Resources.FindObjectsOfTypeAll<AudioSource>())
    3.         {
    4.             listener.bypassListenerEffects = false;
    5.         }
    6. _recorder = new NatSuite.Recorders.MP4Recorder(width, height, 30, AudioSettings.outputSampleRate, 2);
    7.        
     
  25. sinjimonkey

    sinjimonkey

    Joined:
    Jul 31, 2013
    Posts:
    72
    Last edited: Oct 18, 2020
  26. sinjimonkey

    sinjimonkey

    Joined:
    Jul 31, 2013
    Posts:
    72
    My code currently looks like this;

    Incidentally, the Speaker Mode is set to Stereo, so a hardcoded 2 should have worked if it were to work at all.

    Code (CSharp):
    1.       var clock = new RealtimeClock();
    2.         int speakers = 1;
    3.         Debug.Log(AudioSettings.speakerMode);
    4.         switch (AudioSettings.speakerMode)
    5.         {
    6.             case AudioSpeakerMode.Mono:
    7.                 speakers = 1;
    8.                 break;
    9.             case AudioSpeakerMode.Stereo:
    10.             case AudioSpeakerMode.Prologic:
    11.                 speakers = 2;
    12.                 break;
    13.             case AudioSpeakerMode.Quad:
    14.                 speakers = 4;
    15.                 break;
    16.             case AudioSpeakerMode.Surround:
    17.                 speakers = 5;
    18.                 break;
    19.             case AudioSpeakerMode.Mode5point1:
    20.                 speakers = 6;
    21.                 break;
    22.             case AudioSpeakerMode.Mode7point1:
    23.                 speakers = 8;
    24.                 break;
    25.                 break;
    26.             default:
    27.                 throw new ArgumentOutOfRangeException();
    28.         }
    29.      
    30.      
    31.         BypassAllListenerEffects(false);
    32.         _recorder = new NatSuite.Recorders.MP4Recorder(width, height, 30, AudioSettings.outputSampleRate, speakers);
    33.         _recording = true;
    34.  
    35.         Debug.Log("Audio:  "+AudioListener);
    36.         _audioInput = new AudioInput(_recorder, clock, AudioListener);
    37.         Application.targetFrameRate = 30;
    38.         _cameraInput = new CameraInput(_recorder, clock, cam);
    39.         //_cameraInput.frameSkip = 1;
    40.  
    41.     }
    42.  
    43.     private void BypassAllListenerEffects(bool doBypass)
    44.     {
    45.         foreach (var listener in Resources.FindObjectsOfTypeAll<AudioSource>())
    46.         {
    47.             listener.bypassListenerEffects = doBypass;
    48.         }
    49.     }
    50.  
    51.     public  async void StopRecording()
    52.     {
    53.         if (_recording)
    54.         {
    55.  
    56.             _recording = false;
    57.             if (_cameraInput != null)
    58.                 _cameraInput.Dispose();
    59.             if (_audioInput != null)
    60.                 _audioInput.Dispose();
    61.             var path = await _recorder.FinishWriting();
    62.             Debug.Log("Stop Recording Path:  "+path);
    63.         }
    64.  
    65.         if (Application.targetFrameRate != -1)
    66.             Application.targetFrameRate = -1;
    67.     }
     
    Lanre likes this.
  27. sinjimonkey

    sinjimonkey

    Joined:
    Jul 31, 2013
    Posts:
    72
    Does it matter that the Audio output on the VideoPlayer is set to 'Direct'? Ooh it might do if you are using AudioListener effects.

    That's not something I can easily fix now, as these are all AR assets downloaded from a server.
     
  28. sinjimonkey

    sinjimonkey

    Joined:
    Jul 31, 2013
    Posts:
    72
    Yup! That fixed it. Okay. I basically have to add an audio source after instantiating these and changing it over (which is what I just did manually in the editor in 'play' mode to test it)

    Anyway, while I'm here I'd like to suggest an overload method that convers the AudioSpeakerMode to an int so that your users don't have to manually create that switch block I just did.
     
    Lanre likes this.
  29. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    Sounds like you've fixed the issue. If you are building a mobile app, then your channel count will almost always be mono or stereo, in which case you can do:
    Code (CSharp):
    1. int channelCount = (int)AudioSettings.speakerMode;
    This is what the ReplayCam example does when microphone recording is enabled. Adding an overload for this is overkill. Besides, the NatCorder front-end is specifically written to be completely Unity-agnostic. There is only one C# source file in NatCorder that has a `using UnityEngine`, and that is the internal `Utility` class that gets the app's persistent data path. This means that NatCorder's C# sources can be dropped as-is in a Xamarin project, and will just work.
     
    sinjimonkey likes this.
  30. i4mtheone

    i4mtheone

    Joined:
    May 16, 2018
    Posts:
    11
    Hello!

    I have a problem with simultaneously playing a video (through AVProVideo plugin), and recording the screen + microphone.

    The app will then throw:

    Code (csharp):
    1. AVAudioSession.mm:997:-[AVAudioSession setActive:withOptions:error:]: Deactivating an audio session that has running I/O. All I/O should be stopped or paused prior to deactivating the audio session.
    And stop playing the video. It will not crash, I can exit and enter the video screen again, but same thing will happen. I've googled the problem but it's related to native code so I'm not sure if it's fixable from my end

    This only happens on iOS.
     
  31. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    NatCorder does not interfere with the audio session (or microphone) in any way. You'll probably want to report this to the AVPro developers.
     
  32. jiraphatK

    jiraphatK

    Joined:
    Sep 29, 2018
    Posts:
    306
    this can be fixed in project setting,
    https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues/164
     
    i4mtheone and Lanre like this.
  33. omerfarukozdemir

    omerfarukozdemir

    Joined:
    Feb 22, 2013
    Posts:
    8
    hello. i have a problem. i use vuforia. and when i click start recording, my app is close. plese help me? thank you
     
  34. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    Can you share the full logs in a .txt attachment? What device and OS version are you running on?
     
  35. omerfarukozdemir

    omerfarukozdemir

    Joined:
    Feb 22, 2013
    Posts:
    8
    There arent any logs. I cant see any warning message on logcat. I work on redmi note 7. And unity 2019.4.0f1.
     
  36. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    Take the unfiltered logs (simply run "adb logcat" without any flags) and share them. There will always be crash logs generated.
     
  37. omerfarukozdemir

    omerfarukozdemir

    Joined:
    Feb 22, 2013
    Posts:
    8
    Ok. i fixed them. I used natshare for saved my camera roll.

    // Finish recording
    var path = await recorder.FinishWriting();
    // Save the recording to the camera roll
    // We can save it to a specific album in the camera roll
    var payload = new SavePayload("My Awesome App"); // album name here
    payload.AddMedia(path);
    payload.Commit();

    But it crashed for "My Awesome App" path. And when i delete this way it work.
     
  38. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    Can you share the logs?
     
  39. omerfarukozdemir

    omerfarukozdemir

    Joined:
    Feb 22, 2013
    Posts:
    8
    0001.01.01 00:00:00.000 -1 -1 Info --------- beginning of crash
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime FATAL EXCEPTION: SavePayload Commit Thread
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime Process: com.vuforia.engine.VuforiaEngine, PID: 21197
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime java.lang.IllegalArgumentException: Primary directory My Awesome App not allowed for content://media/external/video/media; allowed directories are [DCIM, Movies]
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime at android.database.DatabaseUtils.readExceptionFromParcel(DatabaseUtils.java:170)
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime at android.database.DatabaseUtils.readExceptionFromParcel(DatabaseUtils.java:140)
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime at android.content.ContentProviderProxy.insert(ContentProviderNative.java:481)
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime at android.content.ContentResolver.insert(ContentResolver.java:1844)
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime at api.natsuite.natshare.SavePayload.lambda$commit$0$SavePayload(SavePayload.java:90)
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime at api.natsuite.natshare.-$$Lambda$SavePayload$0wjtSNJuMX77rkaraI8nWQBBikI.run(Unknown Source:2)
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime at android.os.Handler.handleCallback(Handler.java:883)
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime at android.os.Handler.dispatchMessage(Handler.java:100)
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime at android.os.Looper.loop(Looper.java:224)
    2020.10.29 16:07:34.842 21197 21763 Error AndroidRuntime at android.os.HandlerThread.run(HandlerThread.java:67)
     
  40. omerfarukozdemir

    omerfarukozdemir

    Joined:
    Feb 22, 2013
    Posts:
    8
    And how to i hide record button?
     
    dieghobonilla likes this.
  41. PSST_Adam

    PSST_Adam

    Joined:
    Aug 26, 2020
    Posts:
    15
    When using the MP4Recorder and the CameraInput class, is there a way to grab a specific frame as a Texture2D after the recording is done?
     
    dieghobonilla likes this.
  42. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    There's an open issue for this.
     
  43. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    No, NatCorder is not a video decoder.
     
  44. omerfarukozdemir

    omerfarukozdemir

    Joined:
    Feb 22, 2013
    Posts:
    8
  45. dieghobonilla

    dieghobonilla

    Joined:
    Sep 4, 2019
    Posts:
    6
    how to i hide record button?
     
  46. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    You need to create a secondary camera, and use layer masks to make sure that your UI elements (button and other things you don't want recorded) can only be seen by the secondary camera. You can record your main camera (which can't see your UI), but on screen the output of both cameras will be visible.
     
  47. dieghobonilla

    dieghobonilla

    Joined:
    Sep 4, 2019
    Posts:
    6
    Someone asked that before, about grabbing a frame to use as a preview image, you mentioned is not possible, but can the JPGRecorder be used for that purpose (iOS/Android)?
    Alternatively, can you suggest a way to do this?
     
  48. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    Capturing 'screenshots' is out of scope for NatCorder. The JPGRecorder won't help you, because it records directly to a sequence of image files. The recorder is not intended to use on mobile platforms.

    You can take screenshots using Unity's ScreenCapture API. Check Unity's documentation for more info.
     
    dieghobonilla likes this.
  49. omerfarukozdemir

    omerfarukozdemir

    Joined:
    Feb 22, 2013
    Posts:
    8
    Can you share the basic project for this? You can just hide the save button. That's enough. Can you?
     
  50. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,974
    I won't have time to do this anytime soon. This is not a NatCorder feature; it is a Unity feature. Refer to Unity's manual on layer masks for more info.