Search Unity

NatCorder - Video Recording API

Discussion in 'Assets and Asset Store' started by Lanre, Nov 18, 2017.

  1. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    Yeah that's correct, NatMic does not have stuttering.
     
  2. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    807
    Hi! Does NatCorder support playing back an animated gif (recorded with NatCorder)?
     
  3. JackmanGomel

    JackmanGomel

    Joined:
    Jun 3, 2015
    Posts:
    12
    How to record the whole screen without using the second camera and not using the render texture with UI? The second camera increases the drawing calls by 2 times.

    It is on Android with resolution 640*360. I need full screen with this resolution


    How change default path to a video on my path?
    Can I get video in byte array without save to a disk?

    Also falls a lot fps. With 24 to 10
    Disabled

    Enabled
     
    Last edited: Nov 19, 2018
  4. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    Hey there. No it doesn't. But there are packages on the Asset Store that advertise GIF playback.
     
  5. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    I'm confused. Why not just record your primary game camera? You don't need to create a second camera.
     
  6. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    807
    So I like how Nat* seems to be expanding to cover all camera recording/sharing needs - will gif playback / loading be also on the roadmap?
     
  7. JackmanGomel

    JackmanGomel

    Joined:
    Jun 3, 2015
    Posts:
    12
    The main camera only records part of the low resolution screen
    What about others question?
     
  8. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    No, not GIF textures. There are already solutions for this on the Asset Store, so we'd rather not bother with our own implementation.
     
  9. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    If you are trying to record multiple game cameras, you would need to write a custom recorder to do this. Here is a starting point for doing that.
    You can't. Instead, use the System.IO.File functions to move the file to your desired directory.
    No, NatCorder does not support recording to a memory stream. It records directly to the file system.
    We are looking into improving recording performance in the next minor update. That being said, your app already seems to be bottlenecked before adding recording.
     
  10. JackmanGomel

    JackmanGomel

    Joined:
    Jun 3, 2015
    Posts:
    12
    I tried Everyplay plugin and got 16-18 fps
     
  11. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    Good to know. As I mentioned, we are working on improving recording performance.
     
  12. Geneworm

    Geneworm

    Joined:
    Jun 26, 2017
    Posts:
    17
    Hi Lanre, I'm trying to use your audio API, I've added the following to the video recorder:

    Code (CSharp):
    1.         [SerializeField] Camera _camera;
    2.  
    3.         [SerializeField] AudioSource _audioSource;
    4.  
    5.         [Header("Recording")]
    6.         public Container container = Container.MP4;
    7.  
    8.         private CameraRecorder videoRecorder;
    9.         private AudioRecorder audioRecorder;
    10.  
    11.         public static string _videoPath;
    12.         public static event Action<string> OnVideoRecorded;
    13.  
    14.         public void StartRecording()
    15.         {
    16.             _videoPath = null;
    17.  
    18.             var audioFormat = new AudioFormat(AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode);
    19.  
    20.             var width = Screen.width / 2;
    21.             var height = Screen.height / 2;
    22.  
    23.             var framerate = container == Container.GIF ? 10 : 30;
    24.             var videoFormat = new VideoFormat(width, (int)height, framerate);
    25.  
    26.             NatCorder.StartRecording(container, videoFormat, audioFormat, OnShare);
    27.  
    28.             videoRecorder = CameraRecorder.Create(_camera);
    29.             audioRecorder = _audioSource.gameObject.AddComponent<AudioRecorder>();
    30.         }
    31.  
    32.         public void StopRecording(string albumName)
    33.         {
    34.             audioRecorder.Dispose();
    35.             videoRecorder.Dispose();
    36.             NatCorder.StopRecording();
    37.         }
    However every time audio is triggered I get a null reference error NullReferenceException: Object reference not set to an instance of an object at NatCorderU.Core.Recorders.AudioRecorder.OnAudioFilterRead (System.Single[] data, Int32 channels) [0x00008] in #######\Assets\NatCorder\Plugins\Managed\Recorders\AudioRecorder.cs:78

    Am I missing anything in the setup of this? Thanks.
     
  13. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    You have to create the AudioRecorder using `AudioRecorder.Create`, not by using `AddComponent`.
     
  14. JackmanGomel

    JackmanGomel

    Joined:
    Jun 3, 2015
    Posts:
    12
    I get black screen on ios, Iphone 5.On Mac recording normal video. Multithread enabled and disabled
     
  15. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    Can you email me the logs from Xcode in a .txt file? If you're running on an iPhone 5, then your app is probably using OpenGL ES.
     
  16. Geneworm

    Geneworm

    Joined:
    Jun 26, 2017
    Posts:
    17
    Thanks, I've got it recording but there's a lot of crackling in the audio, I have the sample rate at 48000 so it shouldn't be that. My sounds are all normalized but if I lower amplitude it helps a bit however when I get multiple sounds together it's pretty bad. Any ideas?
     
  17. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    You might need to reduce the volume on the AudioSource. Crackling on higher amplitudes usually indicates clipping. NatCorder doesn't modify the audio data; it copies it verbatim from Unity.
     
  18. Geneworm

    Geneworm

    Joined:
    Jun 26, 2017
    Posts:
    17
    Yeah that's done it, strange that it wasn't clipping during play but doesn't matter now thanks.
     
    Lanre likes this.
  19. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    807
    Well, there are also solutions for video/camera on the asset store, as well but they are inferior in ways that caused you to create Nat*?
     
  20. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    Yes, that's correct. But GIF playback is not something we are interested in.
     
  21. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    807
    How do you have AR objects from OpenCVforUnity show in a natcam video recording? The AR objects are on an AR Camera, while the video is from an orthographic main camera
     
  22. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    Sounds like you want to record multiple cameras simultaneously. Check out the MultiCameraRecorder script as a starting point.
     
  23. walaber_entertainment

    walaber_entertainment

    Joined:
    Jul 10, 2017
    Posts:
    14
    Hello. I'm just also writing to report I'm having the same OSX issue as reported by other users.
    Code (CSharp):
    1. DllNotFoundException: NatCorder
    2. NatCorderU.Core.Platforms.NatCorderOSX..ctor () (at Assets/NatCorder/Plugins/Managed/Platforms/NatCorderOSX.cs:42)
    I'm on Unity 2018.3 beta, macOS Mojave 10.14.1.
     
  24. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    I'll be submitting an update tomorrow with the fix for this. If you'd rather not wait, email me with your invoice number.
     
  25. ndream

    ndream

    Joined:
    May 2, 2016
    Posts:
    10
    Builds below Android API level 18 are not possible.
    However, our app has already been commercialized. And we are using API Level 16 minimum.
    If the API level is raised, the previous user can not execute.
    Make it possible to build under API level 18, and below API level 18 we want to implement the feature so that it is not available.
    Is it possible?
     
  26. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    Unfortunately we can't make the minimum any lower than API level 18. NatCorder relies on Android functions that were introduced in API level 18.
     
  27. ndream

    ndream

    Joined:
    May 2, 2016
    Posts:
    10
    We will implement it so that we can use NatCorder only at API Level 18 or higher.
    Can I choose to build it?
     
  28. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    I'm not sure what you're asking. How do you mean "can you choose to build it"?
     
  29. ndream

    ndream

    Joined:
    May 2, 2016
    Posts:
    10
    I hope that there is no limit to the build, and I will implement it myself to avoid using NatCorder.
     
  30. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    I'm not sure what "the build" is. Can you clarify?
     
  31. ndream

    ndream

    Joined:
    May 2, 2016
    Posts:
    10
    Yeah.
    An Android build.
    upload_2018-10-23_20-57-39.png
     
  32. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    As far as I can tell, you can't remove the minimum API requirement for an Android library. Hence, it isn't possible to build to a lower target when a library has a higher requirement. This is a limitation of the Android build system.
     
  33. alex_varela

    alex_varela

    Joined:
    Jul 10, 2018
    Posts:
    2
    Hello, I have a crash when I'm recording and I send the application to the background on Android devices.
     
  34. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    Hi there. I responded to your email. But I'll post it here for anyone else to see:

    This is expected behaviour. You must end recording before the app loses focus; not doing so results in undefined behaviour. This is an OS restriction (because any new app that has primary focus might need exclusive access to the hardware codecs).
     
  35. alex_varela

    alex_varela

    Joined:
    Jul 10, 2018
    Posts:
    2

    Yes, I stop the recording when I send the application to the background, in IOS it behaves well, but in android is where the problems occur. this is what Im using to stop recording

    onAplicationPause (bool a) {
    if (recording && a) {
    StopRecording ();
    }
    }
     
  36. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    Can you upload the full logs from logcat in a .txt file?
     
  37. tevesweb11

    tevesweb11

    Joined:
    Oct 16, 2018
    Posts:
    1
    Hi Lanre,

    I'm having exactly the same problem as reported by FlashyGoblin. You said there's some code signing error going on.
    I just shoot you an email with my invoice.
    Could you please help with that?

    Regards,
    Andre Teves
     
  38. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    I responded to your email.
     
  39. blacktest

    blacktest

    Joined:
    Mar 2, 2014
    Posts:
    3
    finding problems with the system of gif in vuforia in the version of unity 2018.2.18 is not recording the screen, it remains black in the replay and then it comes out with an error that is this:

    My e-mail is miguel.bohorquez890@gmail.com pls help me
     

    Attached Files:

  40. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    Can you share the full logs from logcat in a .txt file? Also, what does your recording code look like?
     
  41. blacktest

    blacktest

    Joined:
    Mar 2, 2014
    Posts:
    3
    I just put the ARcamera and the image but when compiling the modeling it is behind the preview and when it finishes recording it fails and the message of the previous sent image

    Maybe I'm doing something wrong, could you explain or send a simple example of vuforia and natcorder with gif recording? I'm a novice developer yet, thanks.
     

    Attached Files:

    • log.txt
      File size:
      17.1 KB
      Views:
      209
    • nat.txt
      File size:
      30 KB
      Views:
      215
    Last edited: Dec 1, 2018
  42. Geneworm

    Geneworm

    Joined:
    Jun 26, 2017
    Posts:
    17
    Hi Lanre, sorry to pester you again, the video and audio recorded on android are fine yet for some reason on iOS the videos run fine but the audio is 2x speed, I can't understand what's causing it, any idea what might be wrong? Code below:

    Code (CSharp):
    1.                 var audioFormat = new AudioFormat(48000, (int)AudioSettings.speakerMode);
    2.  
    3.                 var width = Screen.width / 2;
    4.                 var height = Screen.height / 2;
    5.  
    6.                 var framerate = 30;
    7.                 var videoFormat = new VideoFormat(width, (int)height, framerate);
    8.  
    9.                 NatCorder.StartRecording(Container.MP4, videoFormat, audioFormat, OnShare);
    10.  
    11.                 videoRecorder = CameraRecorder.Create(_camera);
    12.                 audioRecorder = AudioRecorder.Create(AudioManager.instance.GetComponent<AudioSource>(), AudioManager.mute);
     
  43. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    There isn't a Vuforia example. But you should simply create a CameraRecorder to record the AR game camera (just like the ReplayCam example does to record the main camera).
     
  44. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    Unlike the bitrate in the VideoFormat which denotes the output bitrate, the bitrate in AudioFormat denotes the input bitrate (NatCorder fixes the audio output bitrate to 64kHz I believe). Since you are recording Unity audio, the bitrate should be that of Unity's audio engine (AudioSettings.outputSampleRate).
     
  45. woowee

    woowee

    Joined:
    Jul 1, 2017
    Posts:
    9
    I'm recording mp4 for an ARKit project with NatCorder. Normally my app just takes up to about 300MB memory (view in XCode), but after I call StartRecording the memory usage will soon boost to 1.4GB within 10 seconds and the app crashes (on iOS 12).

    I've commented out audio committing codes and got the same result. Here's my code in LateUpdate:

    Code (CSharp):
    1.  
    2. void LateUpdate() {
    3.         if (NatCorder.IsRecording) {
    4.             var frame = NatCorder.AcquireFrame();
    5.  
    6.             for (int i = 0; i < cameras.Length; i++) {
    7.                 cameras[i].targetTexture = frame;
    8.                 cameras[i].Render();
    9.                 cameras [i].targetTexture = null;
    10.             }
    11.             NatCorder.CommitFrame(frame, clock.CurrentTimestamp);
    12.         }
    13.     }
    14.  
    2018-12-04 12:51:03.211403+0800 marbles[2709:955958] NatCorder: Prepared MP4 video encoder at resolution 750x1334@30Hz with average bitrate 5909759 and keyframe interval 3s

    I'm using NatCorder 1.3f2 and Unity 2017.4. Have I missed something? Please help...


    UPDATE: If I start the app in AR mode the memory will be consumed fast, but in non-AR mode everything seems Ok. Maybe something in Unity ARKit plugin is conflicting with NatCorder
     
    Last edited: Dec 4, 2018
  46. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    What is your app's framerate? If it's 60FPS, then you're committing frames to be recorded at 60FPS. It sounds that this is not feasible when using AR on your device, so only commit frames every other frame (creating a 30FPS video).
     
  47. Mitusov

    Mitusov

    Joined:
    Feb 18, 2016
    Posts:
    1
    Hello. when using NatCorder on Android , NatCorder.StartRecording(...) never called. And after ~10 srconds app crashes. On IOS and in editor it works awesome. What im missing ?
    Thakns.
    Here the code im using
    Code (CSharp):
    1.  
    2. public void pressVideoButton(){
    3.         Debug.Log("Pressed video button");
    4.         if (!NatCorder.IsRecording){
    5.             Debug.Log("Started recording");
    6.             StartRecording();
    7.             videoButtonBig.GetComponent<Image>().color = Color.red;
    8.         }else if (NatCorder.IsRecording){
    9.             Debug.Log("stopping recording");
    10.             StopRecording();
    11.             videoButtonBig.GetComponent<Image>().color = Color.white;
    12.         }
    13.     }
    14.     void OnReplay(string path)
    15.     {
    16.         Debug.Log("Saved recording to: " + path);
    17.         // Playback the video
    18. #if UNITY_IOS
    19.         //Handheld.PlayFullScreenMovie("file://" + path);
    20.         NatShare.SaveToCameraRoll(path);
    21. #elif UNITY_ANDROID
    22.         //Handheld.PlayFullScreenMovie(path);
    23.         Debug.Log(path);
    24.         NatShare.SaveToCameraRoll(path);
    25. #endif
    26.     }
    27.     void StartRecording()
    28.     {
    29.         Debug.Log("start recording method");
    30.         // Start recording
    31.         format = new VideoFormat(960, 540);
    32.         NatCorder.StartRecording(Container.MP4, format, AudioFormat.Unity, OnReplay);
    33.         // Create a camera recorder to record the main camera
    34.         clock = new RealtimeClock();
    35.         videoRecorder = CameraRecorder.Create(mainCamera, clock);
    36.         audioRecorder = AudioRecorder.Create(sourceAudio);
    37.     }
    38.  
    39.     void StopRecording()
    40.     {
    41.         Debug.Log("stop recording method");
    42.         //Destroy the camera recorder
    43.         videoRecorder.Dispose();
    44.         audioRecorder.Dispose();
    45.         // Stop recording
    46.         NatCorder.StopRecording();
    47.  
    48.     }
    49.  
    50.     void OnAudioFilterRead(float[] data, int channels)
    51.     {
    52.         // Check that we are recording
    53.         if (NatCorder.IsRecording)
    54.         {
    55.             NatCorder.CommitSamples(data, clock.CurrentTimestamp);
    56.         }
    57.     }
     
  48. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    I responded to your email. Let's continue the conversation over email.
     
  49. risnandar_unity

    risnandar_unity

    Joined:
    Dec 3, 2018
    Posts:
    1
    How to record with UI screen space - overlay?

    i got some error when running record video on osx editor in macOS Mojave but running well on ios device..

    DllNotFoundException: NatCorder
    NatCorderU.Core.Platforms.NatCorderOSX..ctor () (at Assets/NatCorder/Managed/Platforms/NatCorderOSX.cs:42)
    NatCorderU.Core.NatCorder..cctor () (at Assets/NatCorder/Managed/NatCorder.cs:113)
    Rethrow as TypeInitializationException: An exception was thrown by the type initializer for NatCorderU.Core.NatCorder
    NatCorderU.Examples.ReplayCam.StartRecording () (at Assets/NatCorder/Examples/ReplayCam/ReplayCam.cs:55)
    UnityEngine.Events.InvokableCall.Invoke () (at /Users/builduser/buildslave/unity/build/Runtime/Export/UnityEvent.cs:165)
    UnityEngine.Events.UnityEvent.Invoke () (at /Users/builduser/buildslave/unity/build/Runtime/Export/UnityEvent_0.cs:58)
    UnityEngine.UI.Button.Press () (at /Users/builduser/buildslave/unity/build/Extensions/guisystem/UnityEngine.UI/UI/Core/Button.cs:36)
    UnityEngine.UI.Button.OnPointerClick (UnityEngine.EventSystems.PointerEventData eventData) (at /Users/builduser/buildslave/unity/build/Extensions/guisystem/UnityEngine.UI/UI/Core/Button.cs:45)
    UnityEngine.EventSystems.ExecuteEvents.Execute (IPointerClickHandler handler, UnityEngine.EventSystems.BaseEventData eventData) (at /Users/builduser/buildslave/unity/build/Extensions/guisystem/UnityEngine.UI/EventSystem/ExecuteEvents.cs:50)
    UnityEngine.EventSystems.ExecuteEvents.Execute[IPointerClickHandler] (UnityEngine.GameObject target, UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.EventFunction`1 functor) (at /Users/builduser/buildslave/unity/build/Extensions/guisystem/UnityEngine.UI/EventSystem/ExecuteEvents.cs:261)
    UnityEngine.EventSystems.EventSystem:Update()

    How to fix this issue?
     
  50. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,442
    This is not possible. Please look through the README for more info on this. It is a limitation from Unity Engine.
    Shoot me an email with your invoice number and I'll share an updated build with the fix.