Search Unity

  1. Calling all beginners! Join the FPS Beginners Mods Challenge until December 13.
    Dismiss Notice
  2. It's Cyber Week at the Asset Store!
    Dismiss Notice

NatCorder - Video Recording API

Discussion in 'Assets and Asset Store' started by Lanre, Nov 18, 2017.

  1. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    As far as I've seen, this is a Chrome issue (especially because it works on other browsers). Let's see what they find.
     
  2. zmmround

    zmmround

    Joined:
    Oct 15, 2018
    Posts:
    7
    Thanks. Can you possibly help with the coding - I don't understand :)
     
  3. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    I don't have code for this. Check Unity forums/answers, StackOverflow, or Google.
     
  4. fabiotgarcia

    fabiotgarcia

    Joined:
    Apr 28, 2017
    Posts:
    31
    Frame processing rate has fallen below pre-set threshold

    My app, dev with Unity 2018.4.11 - Vuforia 8.3.8, is crashing when I hit record button. I'm using XCode 11.1
     
  5. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    Can you share the logs and a screenshot of Xcode when the app crashes?
     
  6. fabiotgarcia

    fabiotgarcia

    Joined:
    Apr 28, 2017
    Posts:
    31
    Hi, Mr. Lanre.
    In attachment the entire log in .txt and the screenshot. The app is freezing when we hit the share button.
    Thanks for your help.
    Regards
     

    Attached Files:

  7. jalajshah

    jalajshah

    Joined:
    Mar 5, 2018
    Posts:
    9
    Hello any one facing this kind of issue in ios Unity using nat coder Recording ?

    I am trying to record video with mic Audio using Nat coder Record ,
    Unity version : 2018.4.8
    Nat coder : 1.6.3

    issue : first few time audio record very good with video but after that audio not recording properly.

    i had try option with prepare ios for recoding True , force ios when recording true
     
  8. edee1337

    edee1337

    Joined:
    Apr 10, 2013
    Posts:
    30
    I was getting the same "Main Thread Checker: UI API called on a background thread" error when using NatShare. Don't remember seeing that in earlier builds of my app so not sure what could have changed. Had to switch over to a different share library in the meantime
     
  9. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    This is a NatShare bug, and it has been fixed in the PR I'm working on.
     
    edee1337 and fabiotgarcia like this.
  10. fabiotgarcia

    fabiotgarcia

    Joined:
    Apr 28, 2017
    Posts:
    31
    Thanks! I'll be waiting. Please let me know as soon it available.
     
    Lanre likes this.
  11. LeeLorenzSr

    LeeLorenzSr

    Joined:
    May 3, 2015
    Posts:
    24
    Will the new version of NatCorder be up soon? The Unity Editor shutting down when the recording is stopped is driving me nuts. :(
    At least it seems to be working great on Android, and I also was able to record nice video on iOS.
     
  12. zmmround

    zmmround

    Joined:
    Oct 15, 2018
    Posts:
    7
    Ok - I'll try! Here's a vote to include within the plugin though.
     
  13. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    You can always email me with your invoice number for a build.
     
  14. HeavyArmoredMan

    HeavyArmoredMan

    Joined:
    Apr 9, 2017
    Posts:
    7
    Hi Lanre,

    I have three major issues in my iOS build.

    1. I used the same RealtimeClock for both CameraInput and AudioInput, the code looks like:
    Code (CSharp):
    1. _clockReplay = new RealtimeClock();
    2. _recorderReplay = new MP4Recorder(1280, 720, 30, 44100, 2, SaveReplay, 3600000);
    3. _cameraInputReplay = new CameraInput(_recorderReplay, _clockReplay, _cameraSelected);
    4. _audioInputReplay = new AudioInput(_recorderReplay, _clockReplay, _audioListener);
    It records normally in Unity Editor. But in the iOS build, the audio playback speed is way faster than the picture. Please check the two links. One is recorded in the Editor and the other one is exported from iOS.



    2. The output from iOS does not follow the parameters I set, e.g., in my code, I set the audio channels to 2, but the actual recorded soundtrack will be 1. Please check the attached screenshot for the different outputs with the same code.

    3. In the iOS build, if I stop recording when the RealtimeClock is set to paused(
    _clockReplay.Paused = true;
    ), the app crashes.

    My environments:
    Unity: 2018.4.11f1
    NatCorder: 1.6.3
    iOS: 13.1.2
     

    Attached Files:

    Last edited: Oct 18, 2019
  15. SSUAR

    SSUAR

    Joined:
    Dec 19, 2017
    Posts:
    5
    I have updated the plugin to the latest available and refactored my code to suit. It keeps crashing in the editor any time i attempt to record a video now unfortunately! This is happening on editor versions 2018.4.6f1 and 2019.2.9f1. Any help is much appreciated!
     
  16. HeavyArmoredMan

    HeavyArmoredMan

    Joined:
    Apr 9, 2017
    Posts:
    7
    And for the Android build, I have tested on a Sony Z5 Premium running Android 7.1.1 and a Xiaomi 8SE with Android 8.0. Audio recorded by the Sony Z5 Premium is perfectly synchronized with the picture, but the Xiaomi 8SE has the same issue as iPhone - audio is way faster than the picture.

    In an Android device, stopping the recorder when RealtimeClock is paused does not crash the app, but I got the error message as follow:

    Code (CSharp):
    1. NullReferenceException: Object reference not set to an instance of an object.
    2.  
    3. UnityEngine.GlobalJavaObjectRef.op_Implicit (UnityEngine.GlobalJavaObjectRef obj) (at <00000000000000000000000000000000>:0)
    4. UnityEngine.AndroidJavaObject._Call (System.String methodName, System.Object[] args) (at <00000000000000000000000000000000>:0)
    5. NatCorder.Internal.Readback.GLESReadback.Dispose () (at <00000000000000000000000000000000>:0)
     
    Last edited: Oct 18, 2019
  17. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    You are passing in 44100 and 1 as the sample rate and channel count respectively, but there is no guarantee that this is the sample rate and channel count of the audio data you are recording (because you are recording Unity audio). Refer to Unity's AudioSettings class for the correct values to pass in.
    This can't be possible. NatCorder passes the values of sample rate and channel count in verbatim.
    Can you upload a crash log and a screenshot of the Xcode window when the app crashes?
     
  18. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    Fix coming to the Asset Store soon.
     
  19. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    Can you share your pause and stop recording code?
     
  20. viku_99

    viku_99

    Joined:
    Dec 14, 2016
    Posts:
    5
    Getting black screen issue in natcorder 1.6.2 plugin in moto g6 plus
     
  21. HeavyArmoredMan

    HeavyArmoredMan

    Joined:
    Apr 9, 2017
    Posts:
    7
    Referring to Unity's AudioSettings class works. Issue 1 solved! Appreciate your timely help.

    After further digging, I believe Issue 2 has nothing to do with NatCorder. The video is compressed by the app that transfers files from my iOS device to PC, causing the parameters to change.

    For issue 3, my new code solved the problem. It looks like:
    Code (CSharp):
    1.    void StartRecorder()
    2.     {
    3.         isRecording = true;
    4.         clockReplay = new RealtimeClock();
    5.         recorderReplay = new MP4Recorder(1280, 720, 30, AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode, SaveReplay, 3600000);
    6.         cameraInputReplay = new CameraInput(recorderReplay, clockReplay, cameraSelected);
    7.         audioInputReplay = new AudioInput(recorderReplay, clockReplay, audioListener);
    8.     }
    9.  
    10.     void StopRecorder()
    11.     {
    12.         if (!clockReplay.Paused)
    13.         {
    14.             audioInputReplay.Dispose();
    15.             cameraInputReplay.Dispose();
    16.         }
    17.         recorderReplay.Dispose();
    18.         isRecording = false;
    19.     }
    20.  
    21.     void PauseRecorder()
    22.     {
    23.         //clockReplay.Pause();    //NatCorder1.5.1
    24.         clockReplay.Paused = true;
    25.         audioInputReplay.Dispose();
    26.         cameraInputReplay.Dispose();
    27.     }
    28.  
    29.     void ResumeRecorder()
    30.     {
    31.         //clockReplay.Resume();    //NatCorder1.5.1
    32.         clockReplay.Paused = false;
    33.         cameraInputReplay = new CameraInput(recorderReplay, clockReplay, cameraSelected);
    34.         audioInputReplay = new AudioInput(recorderReplay, clockReplay, audioListener);
    35.     }
    36.  
    In my old code, the AudioInput and CameraInput are disposed without checking if they have been disposed in the PauseRecorder method. The old code:
    Code (CSharp):
    1. void StopRecorder()
    2.     {
    3.         audioInputReplay.Dispose();
    4.         cameraInputReplay.Dispose();
    5.         recorderReplay.Dispose();
    6.         isRecording = false;
    7.     }
    But the old code won't crash the app in NatCorder 1.5.X.
    Attached is the screenshot of the XCode window when the app crashes.
     

    Attached Files:

    Lanre likes this.
  22. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    Can you upload the unfiltered logs from logcat in a .txt file? Make sure that your app is not rendering with Vulkan (Player Settings > Auto Graphics API).
     
  23. tonyhan

    tonyhan

    Joined:
    Dec 29, 2015
    Posts:
    10
    Natcorder 1.6.3. I found that my app will crash sometimes when the recording stopped, and if I remove Vulkan from Graphis APIs list( with OpenGles2 left), the game will not crash any more. Is there any other suggestions?
     
  24. leoapp

    leoapp

    Joined:
    Mar 29, 2018
    Posts:
    2
    Hi @Lanre

    I record my AR scene using your ReplayCam example. I updated to the latest version and I see a couple of black frames at the end of the recorded videos. What could be the reason?
     
  25. CCRush

    CCRush

    Joined:
    Feb 2, 2017
    Posts:
    1
    How can I know if MP4Recorder is available on current device?
     
  26. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    NatCorder doesn't support Vulkan; only OpenGL ES 3.
     
  27. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    Not sure why this would happen. What platform/device are you running on? Can you share one such video?
     
  28. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    Every platform that NatCorder supports (Android, iOS, macOS, Windows, WebGL) support the MP4Recorder.
     
  29. Packedbox

    Packedbox

    Joined:
    Jun 20, 2013
    Posts:
    17
    Hi, is ArmV7 or 7s supported ?
     
  30. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    I presume you are asking about iOS. If so, then no. Only arm64 is supported.
     
  31. thesanketkale

    thesanketkale

    Joined:
    Dec 14, 2016
    Posts:
    26
    Hi Lanre,

    After days of banging my head with Google Chrome support about the issue, they finally allowed me to raise it on the Chromium Bugs repository. Now they seem to have found the problem with the video and said:

    Fails audio decoding; at least one of the initial audio packets is corrupt. You can check this with ffmpeg by running

    ffmpeg -err_detect explode -i <file> -acodec pcm_f32le -vn -f wav -y /dev/null​

    And you can see my bug report for Chrome browser here - https://bugs.chromium.org/p/chromium/issues/detail?id=1016081#c3

    Now that we know that the issue is with some of the initial audio packets being corrupt, can you fix this?
     
  32. arielfel

    arielfel

    Joined:
    Mar 19, 2018
    Posts:
    32
    Hi Lanre, thanks for creating this awesome asset.
    I do have one important question before purchasing... is it work on Android 10?
     
  33. DrSharky

    DrSharky

    Joined:
    Dec 7, 2016
    Posts:
    8
    Hello,
    I've searched through the documentation, the Medium articles, and this forum thread, and I'm a bit lost still.
    My ultimate goal here is to record audio from game objects with audio sources in a scene, as well as microphone audio, and include that in a video capture.
    In my quest to do this, I've tried starting small first, and attempted to record 1 video that includes audio with the Replay example. So far, I've managed to get a video with the audio from regular game objects by creating the AudioInput with the audioListener as a parameter. When using any audio sources as a parameter, I've gotten videos with no audio. Following the audio workflows article has gotten it to crash Unity when stopping recording. I'm curious to know if that article is just outdated, and there's something else I'm missing here. But through the documentation, examples, and forum thread, I haven't figured out what I need to do to include recorded microphone audio in the video capture.
    According to this, it seems like I'm able to do what I describe, I'm just not figuring it out. Even a point in the right direction would help out, thanks.
     
  34. edee1337

    edee1337

    Joined:
    Apr 10, 2013
    Posts:
    30
    Updated NatCorder from 1.6.3 to 1.6.4 and got the latest NatMic, everything works in editor (start and stop recording without Unity crashing), but on device (iPhone X with iOS 13.1.3) recording video freezes my entire UI. Video does get recorded tho.

    Not sure yet if this is an error on my part, I'll have more time in the next day or two to investigate, just posting here for now to see if anyone else is getting the same.
     
  35. visualmusicdesign

    visualmusicdesign

    Joined:
    Oct 16, 2019
    Posts:
    4
    Hi, just purchased NatRecorder 1.6.4 -

    The ReplayCam Example script is completely crashing the Editor for me on StopRecording() in Unity 2019.2.9f1 on MacOS Catalina 10.15.

    Unity is giving me the option to report a crash bug to them, but I'm not getting normal MacOS crash log, that I can see.


    I checked that my Player Settings are set to use Metal.

    Seems like this could still be the issue people were experiencing on the last page of the forum, with version 1.6.3?


    Would really like to get this working...
    Thanks,
    Peter
     
  36. Araj-Tejani

    Araj-Tejani

    Joined:
    Sep 25, 2015
    Posts:
    4
    NetCorder Video recording not work in 64bit apk in pie device it can be crash after half video capture
     
  37. DrSharky

    DrSharky

    Joined:
    Dec 7, 2016
    Posts:
    8
    After searching around some more, I've found the example of using NatCorder + NatMic together is a maybe possibly better solution to what I'm doing, but the example provided is using a very outdated version of NatCorder. Now I'd be willing to go and get NatMic to just get this functionality done and over with, $50 is worth not having the headache of building this from scratch myself, which I don't have the time to do that anyway. But if the only documentation is obsolete and can't be used, that's more wasted money, and more wasted time on top of that trying to figure out if it still works together.
    I don't doubt that these assets are good, I just can't seem to figure out how they're supposed to work. So a suggestion on where to start and how would be awesome. Thanks.
     
  38. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    I got your email. I'll check this out this weekend.
     
  39. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    Yup. There's nothing different about Android 10.
     
    arielfel likes this.
  40. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    The challenge with doing this is that you'll need some way to mix audio from the game with audio from the microphone. We tried rolling out our own implementation in NatMic but it didn't work properly due to timestamp synchronization issues, so we're removing the functionality.
    Make sure that the audio source does not bypass effects or listener effects. This is mentioned in the README.
    Fixed in 1.6.4. Grab from the Asset Store.
    The easiest thing to do is to record only the microphone. The expectation is that the microphone hears game audio, in addition to audio from the outside world, hence achieving your goal.
     
    DrSharky likes this.
  41. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    Not sure what this might be. Are you using a custom rendering setup (AR, postfx, and so on)?
     
  42. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    1.6.4 fixed the crashing bug when stop recording is called. Click 'preview' in the bug report window that pops up then upload the crash logs.
     
  43. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    Can you upload the full logs from logcat in a .txt?
     
  44. visualmusicdesign

    visualmusicdesign

    Joined:
    Oct 16, 2019
    Posts:
    4
    Thanks - even though I just purchased yesterday, maybe I still somehow got 1.6.3, because reimporting the Asset fixed the crash on StopRecording() for me. So good to go there, and awesome to see a first recording in the example!

    - However, whenever I try to Build and Run the ReplayCam example, nothing seems to be happening after the recording finishes - no file is saved. Not sure how to debug this, since it now works perfectly in the Editor. Is there an additional Privacy request I need to be making in actual builds on MacOS, in order for the onReplay function to save the file?


    For anyone else who didn't immediately figure this out:


    The relevant function in Replay Cam is the OnReplay callback, and here's some code I wrote to save the capture to the user's choice of location on desktop platforms.
    It only opens the file in the Editor after recording, because that's what's written at the end of the original Asset function.


    private void OnReplay (string path) {

    string savePath = "";

    if (Application.platform == RuntimePlatform.WindowsPlayer || Application.platform == RuntimePlatform.WindowsEditor || Application.platform == RuntimePlatform.OSXPlayer || Application.platform == RuntimePlatform.OSXEditor)
    {
    string name = string.Format("SeeMusicVideo_{0}.mp4", DateTime.Now.ToString("yyyy_MM_dd_HH_mm_ss"));

    savePath = EditorUtility.SaveFilePanel(
    "Save Video",
    "",
    name,
    "mp4");

    if (savePath.Length != 0)
    {
    FileUtil.ReplaceFile(path, savePath);
    FileUtil.DeleteFileOrDirectory(path);
    }
    }


    Debug.Log("Saved recording to: "+path);
    // Playback the video
    #if UNITY_EDITOR
    if (savePath != "")
    {
    EditorUtility.OpenWithDefaultApp(savePath);
    }
    #elif UNITY_IOS
    Handheld.PlayFullScreenMovie("file://" + path);
    #elif UNITY_ANDROID
    Handheld.PlayFullScreenMovie(path);
    #endif
    }
     
    Last edited: Oct 25, 2019
  45. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    Can you share the logs in a .txt file?
     
  46. DrSharky

    DrSharky

    Joined:
    Dec 7, 2016
    Posts:
    8
    Definitely didn't have bypass effects or listener effects on. Maybe I just did something wrong, I really have no idea without more info.

    I believe I have this working almost how I am intending, thanks. I think my volume levels were just off. When creating audioInput with the audioListener parameter, it seems to work as intended, but I had to lower the game volume clip to be able to hear the microphone input. Also I was testing a lot in the editor, I should've expected some things to not function 100% as intended until it's compiled... I did run several tests on the device (iOS), but compiling for iOS every test gets tiresome.

    The one thing I'm coming across now is that the volume of the game audio dips down really far when I am recording. I'm not changing the volume levels when starting the recording, I just set the volume of the clip to lower in the editor before the scene starts. So I'm confused as to how the volume is getting lowered further during recording, then it goes back to normal once the recording is done. Would you happen to know if this is a built-in thing with the Unity Microphone API?

    EDIT: Answered my own question on this one.
    If anyone else runs into what I describe in the above paragraph, it appears this is an iOS thing. going into Project Settings > Other Settings, and then checking the option "Force iOS Speakers when recording" will fix this.
     
    Last edited: Oct 25, 2019
    Lanre likes this.
  47. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    I'm confused. Are you saying that it works in the editor but not on a standalone build on macOS? According to this, there should be a Player.log file.
     
  48. Packedbox

    Packedbox

    Joined:
    Jun 20, 2013
    Posts:
    17
    Same problem here on iPhone XR ios 13.1.3 on Unity 2018.4.10 but not on 2019.2.10

    Another problem I migrated from everyplay, my app was armv7 and ARM64, but with Natcorder I cannot compile for ARMV7 anymore.

    When uploading to AppStoreConnect, I receive this warning:

    "This bundle is invalid. The key UIRequiredDeviceCapabilities in the Info.plist may not contain values that would prevent this application from running on devices that were supported by previous versions."

    That because now I'm requesting ARM64 only, can you provide a stub librairie for armv7 that will allow to compile even if it has no code behind the functions and I will disable recording for 32 bits iPhones

    https://developer.apple.com/library/archive/qa/qa1623/_index.html
     
  49. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,657
    NatCorder is compiled for iOS 11, and iOS 11 dropped support for armv7, and the article suggest that targeting a more recent version of iOS can prevent this issue. So you simply have to increase your app's deployment target to iOS 11.
     
  50. Packedbox

    Packedbox

    Joined:
    Jun 20, 2013
    Posts:
    17
    Our app is targeted at kid so we want to have the largest possible range of devices and specially old ones. The video capture is is a feature that we can turn off in code on our side if the requirements are not met. It will allow us to to support more devices, back to iOS9 ArmV7 OpenGL 2, compared to iOS11 + ARM64 + Metal, for us it's very important for our business.