Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

NatCorder - Video Recording API

Discussion in 'Assets and Asset Store' started by Lanre, Nov 18, 2017.

  1. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Actually nothing changed on iOS between 1.5 and 1.6. Committing both video and audio will block the calling thread. We decided to optimize NatCorder for offline recording (recording in a for-loop, especially useful for transcoding applications). So when you call CommitSamples/CommitFrame, said frame is guaranteed to have been sent to the encoder when the call returns. The only exception to this is Android.
     
  2. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Big thanks for this. Your workaround looks pretty solid.
     
  3. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    We removed the MixerDevice because we've been facing timing issues (trying to get syncing to be perfectly aligned). We are trying to see if we can get it to work close enough, so we can add it back.
     
  4. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    There's some bug specific to the Pixel 3 that causes this. I've been looking for how to fix it.
     
  5. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Upload full logs from logcat in a .txt.
     
  6. Bongo_Studios

    Bongo_Studios

    Joined:
    Jan 26, 2019
    Posts:
    33
    Hi, I recently bought your plugin but I'm facing two problems:

    First one, everthing works on windows but on android doesn't.
    At the moment when I instantiate the MP4Recorder I get the following error:

    Code (CSharp):
    1. 07-28 19:13:32.359: V/Unity(403): NatCorder: Preparing MP4 video encoder with format: {color-format=2130708361, i-frame-interval=3, mime=video/avc, width=1334, bitrate=1800000, frame-rate=30, height=750}
    2. 07-28 19:13:32.658: V/Unity(403): NatCorder: Preparing MP4 audio encoder with format: {channel-mask=16, sample-rate=48000, mime=audio/mp4a-latm, channel-count=1, aac-profile=2, bitrate=64000, max-input-size=8192}
    3. 07-28 19:13:32.816: D/Unity(403): Uploading Crash Report
    4. 07-28 19:13:32.816: E/Unity(403): AndroidJavaException: java.lang.NullPointerException: Attempt to invoke virtual method 'boolean android.os.Handler.post(java.lang.Runnable)' on a null object reference
    5. 07-28 19:13:32.816: E/Unity(403): java.lang.NullPointerException: Attempt to invoke virtual method 'boolean android.os.Handler.post(java.lang.Runnable)' on a null object reference
    6. 07-28 19:13:32.816: E/Unity(403):     at com.yusufolokoba.natcorder.MP4Recorder.encodeSamples(MP4Recorder.java:171)
    7. 07-28 19:13:32.816: E/Unity(403):   at UnityEngine.AndroidJNISafe.CheckException () [0x0008c] in /Users/builduser/buildslave/unity/build/Runtime/Export/Android/AndroidJNISafe.cs:25
    8. 07-28 19:13:32.816: E/Unity(403):   at UnityEngine.AndroidJNISafe.CallVoidMethod (System.IntPtr obj, System.IntPtr methodID, UnityEngine.jvalue[] args) [0x00011] in /Users/builduser/buildslave/unity/build/Runtime/Export/Android/AndroidJNISafe.cs:358
    9. 07-28 19:13:32.816: E/Unity(403):   at UnityEngine.AndroidJavaObject._Call (System.String methodName, System.Object[] args) [0x00038] in /Users/builduser/buildslave/unity/build/Runtime/Export/Android/AndroidJavaImpl.cs:266
    10. 07-28 19:13:32.816: E/Unity(403):   at UnityEngine.AndroidJavaObject.Call (System.String methodName, System.O
    Tested on Xiaomi Redmi Note 4, Android 9 with the following config:
    Code (CSharp):
    1. NatCorderMP4Recorder = new MP4Recorder(
    2.             1334,
    3.             750,
    4.             30,
    5.             AudioSettings.outputSampleRate,
    6.             (int)AudioSettings.speakerMode,
    7.             OnVideo,
    8.             1800000
    9.         );
    The second one happens trying to compile the iOS project:
    Code (CSharp):
    1. + [armv7] Linking Guitaring...
    2. ld: warning: ignoring file Libraries\NatCorder\Plugins\iOS\libNatCorder.a , file was built for archive which is not the architecture being linked (armv7): Libraries\NatCorder\Plugins\iOS\libNatCorder.a
    3. Undefined symbols for architecture armv7:
    4.   "_NCCreateGIFRecorder", referenced from:
    5.       GIFRecorder_ctor_m7F756D46A8AB83D626C712C794E693E32B75170D in Assembly-CSharp4.cpp.obj
    6.       _MediaRecorderBridge_CreateGIFRecorder_m864CC320596FA68C452631EA94E0E4F121A322CD in Assembly-CSharp4.cpp.obj
    7.   "_NCEncodeSamples", referenced from:
    8.       _MediaRecorderBridge_EncodeSamples_m7E9330CCF63AEE98059BBB4011CE869F33018C9B in Assembly-CSharp4.cpp.obj
    9.       _MediaRecorderiOS_CommitSamples_mAD80F4AA75288D90EC1CE69DAD0F29FB7D6EDB13 in Assembly-CSharp4.cpp.obj
    10.   "_NCStopRecording", referenced from:
    11.       _MediaRecorderBridge_StopRecording_m2F1E61E1673BB0A94C7D37617DDC55FE8300A6D1 in Assembly-CSharp4.cpp.obj
    12.       _MediaRecorderiOS_Dispose_m676D703FE863268A36BEF41F7B4DEC5E3C0D8057 in Assembly-CSharp4.cpp.obj
    13.   "_NCStartRecording", referenced from:
    14.       _MediaRecorderBridge_StartRecording_m16086725960E6789DC668280C83105900432F4F1 in Assembly-CSharp4.cpp.obj
    15.   "_NCCreateMP4Recorder", referenced from:
    16.       MP4Recorder_ctor_mBA3A1FEBEB319598577761EEB49C6DC4B73AAC1E in Assembly-CSharp4.cpp.obj
    17.       _MediaRecorderBridge_CreateMP4Recorder_m24CA59D164C1B5A1DE7C383E320B9070427093EA in Assembly-CSharp4.cpp.obj
    18.   "_NCCreateHEVCRecorder", referenced from:
    19.       HEVCRecorder_ctor_mCFFD4AD82AE3A4B8C9F7E193184439CA87AB0A0D in Assembly-CSharp4.cpp.obj
    20.       _MediaRecorderBridge_CreateHEVCRecorder_mA29C1ABBA2640FE1A4850274B7029FD833A1D763 in Assembly-CSharp4.cpp.obj
    21.   "_NCEncodeFrame", referenced from:
    22.       _MediaRecorderBridge_EncodeFrame_mD122667E89A002540F902029824AB8A5BF26C800 in Assembly-CSharp4.cpp.obj
    23. ld: symbol(s) not found for architecture armv7
    Thanks for your time and help with this problem.
     
    Unityzkcmkj likes this.
  7. SteveDET

    SteveDET

    Joined:
    Jan 23, 2018
    Posts:
    10
    Hi there. We are using NatCorder 1.5.1 in a recently launched app for iOS and Android. It works great on most devices, but there are more than a few crashes reported on the Play Store and in Unity Crash Reports.

    I was using the app on a Asus Nexus 7 tablet and got the following crash when starting a recording.

    Code (CSharp):
    1. 07-29 10:22:20.234: W/Unity(19011): >>>> MP4Recorder (Constructor) - recordingPath: /storage/emulated/0/Android/data/au.com.coles.littleshop2/files/recording_2019_07_29_10_22_20_230.mp4
    2. 07-29 10:22:20.234: W/Unity(19011): UnityEngine.Logger:LogFormat(LogType, String, Object[])
    3. 07-29 10:22:20.234: W/Unity(19011): UnityEngine.Debug:LogWarningFormat(String, Object[])
    4. 07-29 10:22:20.234: W/Unity(19011): NatCorder.MP4Recorder:.ctor(Int32, Int32, Int32, Int32, Int32, Action`1, Int32, Int32)
    5. 07-29 10:22:20.234: W/Unity(19011): Recorder:StartRecording()
    6. 07-29 10:22:20.234: W/Unity(19011):
    7. 07-29 10:22:20.234: W/Unity(19011): (Filename: ./Runtime/Export/Debug.bindings.h Line: 45)
    8. 07-29 10:22:20.280: V/Unity(19011): NatCorder: Preparing video encoder with format: {height=768, width=480, bitrate=3000000, mime=video/avc, frame-rate=30, i-frame-interval=3, color-format=2130708361}
    9. 07-29 10:22:20.318: I/OMXClient(19011): Using client-side OMX mux.
    10. 07-29 10:22:20.360: I/MediaCodec(19011): MediaCodec will operate in async mode
    11. 07-29 10:22:20.361: E/ACodec(19011): [OMX.qcom.video.encoder.avc] storeMetaDataInBuffers (output) failed w/ err -1010
    12. 07-29 10:22:20.362: W/ACodec(19011): do not know color format 0x7f000789 = 2130708361
    13. 07-29 10:22:20.365: I/ACodec(19011): setupVideoEncoder succeeded
    14. 07-29 10:22:20.366: W/ACodec(19011): do not know color format 0x7f000789 = 2130708361
    15. 07-29 10:22:20.398: V/Unity(19011): NatCorder: Preparing audio encoder with format: {max-input-size=8192, bitrate=64000, aac-profile=2, mime=audio/mp4a-latm, channel-count=2, channel-mask=12, sample-rate=48000}
    16. 07-29 10:22:20.412: I/OMXClient(19011): Using client-side OMX mux.
    17. 07-29 10:22:20.457: E/OMXMaster(19011): A component of name 'OMX.qcom.audio.decoder.aac' already exists, ignoring this one.
    18. 07-29 10:22:20.489: I/MediaCodec(19011): MediaCodec will operate in async mode
    19. 07-29 10:22:20.500: I/Unity(19011): >>>> MediaRecorderAndroid (Constructor) - recordingPath: /storage/emulated/0/Android/data/au.com.coles.littleshop2/files/recording_2019_07_29_10_22_20_230.mp4
    20. 07-29 10:22:20.543: V/Unity(19011): NatCorder: Video encoder changed output format: {height=768, width=480, csd-1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], what=1869968451, mime=video/avc, csd-0=java.nio.ByteArrayBuffer[position=0,limit=17,capacity=17]}
    21. 07-29 10:22:20.547: I/MPEG4Writer(19011): limits: 4294967295/0 bytes/us, bit rate: -1 bps and the estimated moov size 3195 bytes
    22. 07-29 10:22:20.548: V/Unity(19011): NatCorder: Audio encoder changed output format: {what=1869968451, mime=audio/mp4a-latm, csd-0=java.nio.ByteArrayBuffer[position=0,limit=2,capacity=2], channel-count=2, sample-rate=48000}
    23. 07-29 10:22:20.549: I/MPEG4Writer(19011): setStartTimestampUs: 0
    24. 07-29 10:22:20.549: I/MPEG4Writer(19011): Earliest track starting time: 0
    25. 07-29 10:22:20.549: E/MediaMuxer(19011): addTrack() must be called after constructor and before start().
    26. 07-29 10:22:20.589: E/AndroidRuntime(19011): FATAL EXCEPTION: NatCorder Audio Encoding Thread
    27. 07-29 10:22:20.589: E/AndroidRuntime(19011): Process: au.com.coles.littleshop2, PID: 19011
    28. 07-29 10:22:20.589: E/AndroidRuntime(19011): java.lang.Error: FATAL EXCEPTION [NatCorder Audio Encoding Thread]
    29. 07-29 10:22:20.589: E/AndroidRuntime(19011): Unity version     : 2018.3.7f1
    30. 07-29 10:22:20.589: E/AndroidRuntime(19011): Device model      : asus Nexus 7
    31. 07-29 10:22:20.589: E/AndroidRuntime(19011): Device fingerprint: google/razor/flo:6.0.1/MOB30X/3036618:user/release-keys
    32. 07-29 10:22:20.589: E/AndroidRuntime(19011): Caused by: java.lang.IllegalStateException: Failed to add the track to the muxer
    33. 07-29 10:22:20.589: E/AndroidRuntime(19011):     at android.media.MediaMuxer.nativeAddTrack(Native Method)
    34. 07-29 10:22:20.589: E/AndroidRuntime(19011):     at android.media.MediaMuxer.addTrack(MediaMuxer.java:294)
    35. 07-29 10:22:20.589: E/AndroidRuntime(19011):     at com.yusufolokoba.natcorder.MP4Recorder$5.onOutputFormatChanged(MP4Recorder.java:274)
    36. 07-29 10:22:20.589: E/AndroidRuntime(19011):     at android.media.MediaCodec$EventHandler.handleCallback(MediaCodec.java:1571)
    37. 07-29 10:22:20.589: E/AndroidRuntime(19011):     at android.media.MediaCodec$EventHandler.handleMessage(MediaCodec.java:1505)
    38. 07-29 10:22:20.589: E/AndroidRuntime(19011):     at android.os.Handler.dispatchMessage(Handler.java:102)
    39. 07-29 10:22:20.589: E/AndroidRuntime(19011):     at android.os.Looper.loop(Looper.java:148)
    40. 07-29 10:22:20.589: E/AndroidRuntime(19011):     at android.os.HandlerThread.run(HandlerThread.java:61)
    41. 07-29 10:22:20.619: V/Unity(19011): NatRender: Created ES3 GLRenderContext
    42. 07-29 10:22:24.895: W/Unity(19011): Timeout while trying to pause the Unity Engine.
    43. 07-29 10:22:32.984: W/Unity(19011): Timeout while trying detaching primary window.
    44.  
    This appears to be due to onOutputFormatChanged() in the videoCodecCallback starting the muxer, but then it gets onOutputFormatChanged() in the audioCodecCallback triggered. This causes the MediaMuxer to throw a fatal exception which causes the app to die.

    I see that 1.6.0 was released earlier this month, but the risks associated with updating now are just too great, especially seeing some of the other issues above.
     
  8. Elizeu159

    Elizeu159

    Joined:
    Nov 25, 2016
    Posts:
    9
    The issue was with the permissions to record the mic on android.
    Totally forgot to implement the permission request, now it's working fine!
     
    Lanre likes this.
  9. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    This bug has been brought to my attention. I'm working on a fix now.
    NatCorder cannot be compiled for armv7. It uses a minimum deployment target of iOS 11, which formally dropped support for armv7.
     
  10. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    We delay the video's onOutputFormatChanged so that this never happens. Also, it seems that you decompiled our code. Please desist from doing so. In any case, I'll look into this.
     
  11. ZAUBAR

    ZAUBAR

    Joined:
    Jun 18, 2019
    Posts:
    53
    Okay, when do you think it will be fixed and released? This is a seriously important feature.
     
  12. Bongo_Studios

    Bongo_Studios

    Joined:
    Jan 26, 2019
    Posts:
    33
    I want to know, when is going to be fixed?

    Thanks
     
  13. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    I'm going to submit NatMic 1.3.1 tomorrow.
     
  14. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    In a day or two.
     
  15. CorstiaanMediamonks

    CorstiaanMediamonks

    Joined:
    Sep 14, 2015
    Posts:
    21
    Hi Lanre, we're using your amazing MP4Recorder to record our runtime scene to MP4 but whenever I try to enable audio arguments to anything else then 0 (zero) the callback is never called and the MP4 file, although it 's written, stays 0 bytes. If audio arguments applied as 0 all works fine but no audio of course.
    Any Idea?
     
  16. CorstiaanMediamonks

    CorstiaanMediamonks

    Joined:
    Sep 14, 2015
    Posts:
    21
    Oh correction; it works on Android it just fails in-editor
     
  17. CNGameDev01

    CNGameDev01

    Joined:
    Feb 1, 2016
    Posts:
    6
    Hi,

    Does anyone getting Dll not found exception: Unable to load DLL 'NatCamRenderDispatch'?
    This happens in android ARM64 build.
    I have attached log image as well.

    Does anyone encounter this issue or know any solution?
     

    Attached Files:

  18. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Windows or macOS? Does the ReplayCam example work?
     
  19. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    You are using a much older version of the API. Upgrade to 1.6 on the Asset Store.
     
  20. Aidan-Wolf

    Aidan-Wolf

    Joined:
    Jan 6, 2014
    Posts:
    59
    Any reason setting a custom framerate and clock time works fine on Mac but doesn't work at all on Android?
     
  21. alonlevi

    alonlevi

    Joined:
    Apr 22, 2019
    Posts:
    11
    is there a way to use Natcorder to record a scene camera with specific resolution that does not scale but crop? for example saving to standard 1080p (1920x1080) will squeeze image on most mobiles...
     
  22. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Custom framerate practically doesn't matter on any platform.
    Are you using a fixed interval clock? If you are using a realtime clock, then the recording framerate is explicitly tied to the framerate of your app, which may differ from editor to android device.
     
  23. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    You can achieve this easily by creating a custom CameraInput that records with a shader (like GreyWorld) to do the aspect fit/fill. There is no provision for this out of the box.
     
  24. HEROTECH70

    HEROTECH70

    Joined:
    May 24, 2017
    Posts:
    74
    using NatCorder won't allow you to upload app to google play store due to missing 64bit libraries.

    Checking inside the .aab file in the armv7 folder "libNatCamRenderDispatch.so" and "libNatCamFastRead.so" are present but not in the arm64 folder.
     
  25. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    None of these libraries exist anymore. Looks like you are using a much older version of NatCorder.
     
  26. HEROTECH70

    HEROTECH70

    Joined:
    May 24, 2017
    Posts:
    74
    No, but I tried removing it completely and it would still show up, the NatCorder folder just reinstalls from itself upon building, i had to remove and quit unity several times before I successfully deleted it.

    These libraries were present even after upgrading to 1.6, I deleted the folder, quit unity, reopened it and updated it.
    it still included those libraries.
     
  27. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Do you have NatCam in your project? If so, what version?
     
  28. Flag74

    Flag74

    Joined:
    May 31, 2017
    Posts:
    128
    After upgrade I cannot record at 25 fps anymore on my ZenPad S8.
    Using RenderTextureInput helper class It records at 15fps.
    Also my tablet doesn't support AsyncGPUReadback.

    @Lanre Why older NatCorder was much faster than the newer? How did u manage to have that good performance while copying texture content from GPU to CPU?
     
  29. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    I'm still working on the RenderTextureInput class. It isn't complete. Once it is, performance will be comparable to the previous version. My priority right now is iOS.
     
  30. Bongo_Studios

    Bongo_Studios

    Joined:
    Jan 26, 2019
    Posts:
    33
    Hello,

    I would like to know if there is already a new version available.

    Thanks
     
  31. Bongo_Studios

    Bongo_Studios

    Joined:
    Jan 26, 2019
    Posts:
    33
    Greetings,

    I have been experiencing some trouble with the screen resolution when recording. Previously when I used Natcorder v1.3,
    when recording the video automatically scaled the RenderTexture to the desired resolution. But now in v1.6 the flow for recording
    forces me to create a RenderTexture with the desired recording resolution of the screen, and then apply it to the main camera, but doing
    this causes the UI go all over the place, resulting in a bad recording of the game.

    How can I obtain the same automatic scaling effect that was implemented in v1.3 in the current v1.6?
    Thanks for your time and help with this issue.
     
  32. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    No, it'll be on the Asset Store tomorrow or Tuesday (depends on how fast Unity reviews it)
    Are you using CameraInput? You need to either update your UI to properly layout depending on the aspect ratio of the current display (phone screens come in different sizes) or record at a resolution that matches that of the screen (which I don't recommend, because screen resolutions are really high these days).
     
  33. Bongo_Studios

    Bongo_Studios

    Joined:
    Jan 26, 2019
    Posts:
    33
    Thanks for your answer.
    Could you please specify what needs to be done to obtained the desired results?

    The UI has always used Canvas to properly adapt to many resolutions and has been tested in many devices, so the layout depending on aspect ratio isn't the issue.

    I am using CameraInput, which to my understanding is basically the same as the previous CameraRecorder, with the only difference being that the former uses byte[] and the latter uses RenderTexture, which worked fine in past versions. Is there anything specific that needs to be done with it to get good recordings?

    Thanks for your time.
     
  34. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    I have no clue what you mean by "desired results". CameraInput works the exact same as CameraRecorder. It renders the camera to a RenderTexture. The size of the RenderTexture is defined by the recording resolution (passed to the constructor of a recorder). And Unity will always change the size of the camera view based on what it is rendering onto, whether that's the screen or a RenderTexture.
     
  35. Bongo_Studios

    Bongo_Studios

    Joined:
    Jan 26, 2019
    Posts:
    33
    Although both work in the same way, before when using CameraRecorder a RenderTexture was sent to NatCorder, but this RenderTexture had the same dimentions as the the Canvas, which isn't the same as the recording resolution, but still worked because the image was scaled to match the desired video resolution.
    Now with CameraInput, the RenderTexture has the dimentions of the video that is being recorded, which causes the recored image to be cut, provoking that some parts of the image are left outside the video.
    In the provided examples, the first image corresponds to the result recording in v1.3 and the second in v1.6. It can be seen how in the second image the elements at the sides are cut out. Both recordings where generated with the resolution of an iPhoneX.

    Using NatCorder 1.3


    Using NatCorder 1.6
     
  36. shacharoz

    shacharoz

    Joined:
    Jul 11, 2013
    Posts:
    98
    hi lanre,
    maybe its better if i post here.
    once we update the code from the asset store thats fine, but can i suggest you to update the namespace NatCorderU from the article? it confuses not only me :)

    and then, please write there the right code that we should use.

    thank you for this wonderful plugin
     
  37. mathias_unity633

    mathias_unity633

    Joined:
    Jul 17, 2018
    Posts:
    35
    What is the recommended way to record on iOS and Android?
    We're doing this, but experiencing a lot of stuttering. It is running ARKit or ARCore meanwhile, but it's pretty severe still.

    Code (CSharp):
    1. _recorder = new MP4Recorder(Screen.width / 2, Screen.height / 2, 24, sampleRate, channelCount, OnVideoFinish);
    I didn't really find a mention of MP4Recorder in the documentation or what the recommended video sample rate should be, but I might have looked in the wrong places.
     
    shacharoz likes this.
  38. CorstiaanMediamonks

    CorstiaanMediamonks

    Joined:
    Sep 14, 2015
    Posts:
    21
    Hi Lanre, I'm running on Windows but I'll have to come back to you on running the example. We had already removed all the examples from our project and after downloading the current version to retrieve the examples I noticed that you've completely refactored in such a thorough way that everything stopped working.

    [UPDATE] Tried latest version in empty project; all works fine.
     
    Last edited: Aug 6, 2019
  39. vice39

    vice39

    Joined:
    Nov 11, 2016
    Posts:
    108
    I just bought this asset and the documentation seems to be totally wrong. Was there a recent breaking update or am I missing something? Where can I find updated docs on how to use this asset?
     
  40. shacharoz

    shacharoz

    Joined:
    Jul 11, 2013
    Posts:
    98
    i have trouble with building for android. works fine on the PC but on the android build it seems that clicking the record button freezes the app.
    anyone has any ideas for what i am doing wrong?

    could the fact that i am recording also the microphone be an issue?
    is the microphone of the pc and the one of android accessible from different classes?

    thanks
     
  41. SteveDET

    SteveDET

    Joined:
    Jan 23, 2018
    Posts:
    10
    I spent some time trying to update our project to the 1.6 version of NatCorder (update in-place and also delete the previous version and install new). The app works in the editor, but on Android device it crashes hard on a Google Pixel 3 when the video recording is started. This is the logcat output from the time of creating the MP4 video recorder to the time of death (0.5s).

    Code (CSharp):
    1. 08-07 16:28:07.971: V/Unity(15100): NatCorder: Preparing MP4 video encoder with format: {color-format=2130708361, i-frame-interval=3, mime=video/avc, width=480, bitrate=2500000, frame-rate=60, height=960}
    2. 08-07 16:28:07.978: I/OMXClient(15100): IOmx service obtained
    3. 08-07 16:28:07.979: I/OMXMaster(892): makeComponentInstance(OMX.qcom.video.encoder.avc) in android.hardwar process
    4. 08-07 16:28:07.986: I/OMX-VENC(892): Component_init : OMX.qcom.video.encoder.avc : return = 0x0
    5. 08-07 16:28:07.989: E/OMX-VENC(892): Enable/Disable allocate-native-handle allowed only in secure session
    6. 08-07 16:28:07.989: E/OMXNodeInstance(892): setParameter(0xf4dabf04:qcom.encoder.avc, OMX.google.android.index.allocateNativeHandle(0x7f00005d): Output:1 en=0) ERROR: UnsupportedSetting(0x80001019)
    7. 08-07 16:28:07.989: E/OMX-VENC(892): set_parameter: metamode is valid for input port only
    8. 08-07 16:28:07.989: W/OMXNodeInstance(892): [0xf4dabf04:qcom.encoder.avc] component does not support metadata mode; using fallback
    9. 08-07 16:28:07.989: W/OMXUtils(15100): do not know color format 0x7fa30c06 = 2141391878
    10. 08-07 16:28:07.989: W/OMXUtils(15100): do not know color format 0x7fa30c04 = 2141391876
    11. 08-07 16:28:07.990: W/OMXUtils(15100): do not know color format 0x7fa30c00 = 2141391872
    12. 08-07 16:28:07.990: W/OMXUtils(15100): do not know color format 0x7fa30c09 = 2141391881
    13. 08-07 16:28:07.990: W/OMXUtils(15100): do not know color format 0x7fa30c08 = 2141391880
    14. 08-07 16:28:07.990: W/OMXUtils(15100): do not know color format 0x7fa30c07 = 2141391879
    15. 08-07 16:28:07.990: W/OMXUtils(15100): do not know color format 0x7f000789 = 2130708361
    16. 08-07 16:28:07.991: E/OMX-VENC: venc_dev(892): Unsupported eColorFormat 0x7f000789
    17. 08-07 16:28:07.992: I/ACodec(15100): setupAVCEncoderParameters with [profile: Baseline] [level: Level1]
    18. 08-07 16:28:07.992: E/OMX-VENC(892): Failed to update the profile_level
    19. 08-07 16:28:07.993: I/ACodec(15100): [OMX.qcom.video.encoder.avc] cannot encode HDR static metadata. Ignoring.
    20. 08-07 16:28:07.993: I/ACodec(15100): setupVideoEncoder succeeded
    21. 08-07 16:28:07.993: W/OMXUtils(15100): do not know color format 0x7f000789 = 2130708361
    22. 08-07 16:28:07.993: E/OMX-VENC(892): ERROR: unsupported index 1870659589
    23. 08-07 16:28:07.993: E/OMXNodeInstance(892): getConfig(0xf4dabf04:qcom.encoder.avc, ConfigLatency(0x6f800005)) ERROR: UnsupportedIndex(0x8000101a)
    24. 08-07 16:28:07.998: V/Unity(15100): NatCorder: Preparing MP4 audio encoder with format: {channel-mask=12, sample-rate=48000, mime=audio/mp4a-latm, channel-count=2, aac-profile=2, bitrate=64000, max-input-size=8192}
    25. 08-07 16:28:08.013: D/CCodec(15100): allocate(c2.android.aac.encoder)
    26. 08-07 16:28:08.015: I/CCodec(15100): setting up 'default' as default (vendor) store
    27. 08-07 16:28:08.017: V/C2Store(892): in init
    28. 08-07 16:28:08.017: V/C2Store(892): loading dll
    29. 08-07 16:28:08.018: I/CCodec(15100): Created component [c2.android.aac.encoder]
    30. 08-07 16:28:08.018: D/CCodecConfig(15100): Query domain & kind failed => BAD_INDEX
    31. 08-07 16:28:08.018: D/CCodecConfig(15100): read media type: audio/raw
    32. 08-07 16:28:08.018: D/CCodecConfig(15100): read media type: audio/mp4a-latm
    33. 08-07 16:28:08.020: I/CCodecConfig(15100): query failed after returning 7 values (BAD_INDEX)
    34. 08-07 16:28:08.020: D/CCodecConfig(15100): c2 config is Dict {
    35. 08-07 16:28:08.020: D/CCodecConfig(15100):   c2::u32 coded.bitrate.value = 64000
    36. 08-07 16:28:08.020: D/CCodecConfig(15100):   c2::u32 coded.pl.level = 0
    37. 08-07 16:28:08.020: D/CCodecConfig(15100):   c2::u32 coded.pl.profile = 8192
    38. 08-07 16:28:08.020: D/CCodecConfig(15100):   c2::u32 input.buffers.max-size.value = 2048
    39. 08-07 16:28:08.020: D/CCodecConfig(15100):   string input.media-type.value = "audio/raw"
    40. 08-07 16:28:08.020: D/CCodecConfig(15100):   string output.media-type.value = "audio/mp4a-latm"
    41. 08-07 16:28:08.020: D/CCodecConfig(15100):   c2::u32 raw.channel-count.value = 1
    42. 08-07 16:28:08.020: D/CCodecConfig(15100):   c2::u32 raw.sample-rate.value = 44100
    43. 08-07 16:28:08.020: D/CCodecConfig(15100): }
    44. 08-07 16:28:08.021: D/CCodecConfig(15100): no c2 equivalents for channel-mask
    45. 08-07 16:28:08.021: D/CCodecConfig(15100): no c2 equivalents for aac-profile
    46. 08-07 16:28:08.021: D/CCodecConfig(15100): no c2 equivalents for encoder
    47. 08-07 16:28:08.021: D/CCodecConfig(15100): c2 config is Dict {
    48. 08-07 16:28:08.021: D/CCodecConfig(15100):   c2::u32 coded.bitrate.value = 64000
    49. 08-07 16:28:08.021: D/CCodecConfig(15100):   c2::u32 coded.pl.level = 0
    50. 08-07 16:28:08.021: D/CCodecConfig(15100):   c2::u32 coded.pl.profile = 8192
    51. 08-07 16:28:08.021: D/CCodecConfig(15100):   c2::u32 input.buffers.max-size.value = 4096
    52. 08-07 16:28:08.021: D/CCodecConfig(15100):   string input.media-type.value = "audio/raw"
    53. 08-07 16:28:08.021: D/CCodecConfig(15100):   string output.media-type.value = "audio/mp4a-latm"
    54. 08-07 16:28:08.021: D/CCodecConfig(15100):   c2::u32 raw.channel-count.value = 2
    55. 08-07 16:28:08.021: D/CCodecConfig(15100):   c2::u32 raw.sample-rate.value = 48000
    56. 08-07 16:28:08.021: D/CCodecConfig(15100): }
    57. 08-07 16:28:08.021: W/Codec2Client(15100): query -- param skipped. index = 1107298332
    58. 08-07 16:28:08.021: D/CCodec(15100): setup formats input: AMessage(what = 0x00000000) = {
    59. 08-07 16:28:08.021: D/CCodec(15100):   int32_t channel-count = 2
    60. 08-07 16:28:08.021: D/CCodec(15100):   int32_t max-input-size = 8192
    61. 08-07 16:28:08.021: D/CCodec(15100):   string mime = "audio/raw"
    62. 08-07 16:28:08.021: D/CCodec(15100):   int32_t sample-rate = 48000
    63. 08-07 16:28:08.021: D/CCodec(15100):   int32_t channel-mask = 12
    64. 08-07 16:28:08.021: D/CCodec(15100): } and output: AMessage(what = 0x00000000) = {
    65. 08-07 16:28:08.021: D/CCodec(15100):   int32_t bitrate = 64000
    66. 08-07 16:28:08.021: D/CCodec(15100):   int32_t channel-count = 2
    67. 08-07 16:28:08.021: D/CCodec(15100):   int32_t level = 0
    68. 08-07 16:28:08.021: D/CCodec(15100):   int32_t max-bitrate = 64000
    69. 08-07 16:28:08.021: D/CCodec(15100):   string mime = "audio/mp4a-latm"
    70. 08-07 16:28:08.021: D/CCodec(15100):   int32_t profile = 2
    71. 08-07 16:28:08.021: D/CCodec(15100):   int32_t sample-rate = 48000
    72. 08-07 16:28:08.021: D/CCodec(15100): }
    73. 08-07 16:28:08.022: E/OMX-VENC(892): Does not handle dataspace request
    74. 08-07 16:28:08.022: E/OMXNodeInstance(892): getConfig(0xf4dabf04:qcom.encoder.avc, ??(0x7f000062)) ERROR: UnsupportedSetting(0x80001019)
    75. 08-07 16:28:08.023: D/GraphicBufferSource(892): setting dataspace: 0x104, acquired=0
    76. 08-07 16:28:08.023: D/GraphicBufferSource(892): requesting color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M))
    77. 08-07 16:28:08.334: D/qc_adm(782): ns 1567432 > expected_ns 1000000 (skipped 727)
    78. 08-07 16:28:08.364: D/qc_adm(782): ns 1555033 > expected_ns 1000000 (skipped 727)
    79. 08-07 16:28:08.395: E/OMXNodeInstance(892): !!! Observer died. Quickly, do something, ... anything...
    80. 08-07 16:28:08.395: E/OMX-VENC: venc_dev(892): async_venc_message_thread interrupted to be exited
    81. 08-07 16:28:08.395: V/C2Store(892): in ~ComponentModule
    82. 08-07 16:28:08.395: V/C2Store(892): unloading dll
    83. 08-07 16:28:08.396: W/InputDispatcher(1185): channel '6ae97eb au.com.#####.###########/com.unity3d.player.UnityPlayerActivity (server)' ~ Consumer closed input channel or an error occurred.  events=0x9
    84. 08-07 16:28:08.396: E/InputDispatcher(1185): channel '6ae97eb au.com.#####.###########/com.unity3d.player.UnityPlayerActivity (server)' ~ Channel is unrecoverably broken and will be disposed!
    85. 08-07 16:28:08.396: I/OMX-VENC(892): Component Deinit
    86. 08-07 16:28:08.399: I/ActivityManager(1185): Process au.com.#####.########### (pid 15100) has died: fore TOP
    87. 08-07 16:28:08.400: I/WindowManager(1185): WIN DEATH: Window{6ae97eb u0 au.com.#####.###########/com.unity3d.player.UnityPlayerActivity}
    88.  
    Similar crash happened on an Asus Nexus 7 tablet too. I took out the AsyncGPU code to use the stalling ReadPixels, and I get this on the Nexus 7.

    Code (CSharp):
    1. 08-07 17:23:20.698: V/Unity(9707): NatCorder: Preparing MP4 audio encoder with format: {max-input-size=8192, bitrate=64000, aac-profile=2, mime=audio/mp4a-latm, channel-count=2, channel-mask=12, sample-rate=48000}
    2. 08-07 17:23:20.701: I/OMXClient(9707): Using client-side OMX mux.
    3. 08-07 17:23:20.725: D/mm-camera-intf(195): mm_stream_read_msm_frame: Stoping poll on stream 0xb33376d4 type :5
    4. 08-07 17:23:20.725: D/mm-camera-intf(195): mm_stream_read_msm_frame: Stopped poll on stream 0xb33376d4 type :5
    5. 08-07 17:23:20.726: E/OMXMaster(9707): A component of name 'OMX.qcom.audio.decoder.aac' already exists, ignoring this one.
    6. 08-07 17:23:20.726: D/mm-camera-intf(195): mm_stream_qbuf: Starting poll on stream 0xb33376d4 type :5
    7. 08-07 17:23:20.726: D/mm-camera-intf(195): mm_stream_qbuf: Started poll on stream 0xb33376d4 type :5
    8. 08-07 17:23:20.750: E/OMX-VENC-720p(195): ERROR: get_parameter: unknown param 6f800004
    9. 08-07 17:23:20.751: E/OMX-VENC-720p(195): ERROR: get_parameter: unknown param 6f800004
    10. 08-07 17:23:20.751: E/OMXNodeInstance(195): getParameter(21:qcom.encoder.avc, ParamConsumerUsageBits(0x6f800004)) ERROR: UnsupportedIndex(0x8000101a)
    11. 08-07 17:23:20.759: D/mm-camera-intf(195): mm_stream_read_msm_frame: Stoping poll on stream 0xb33376d4 type :5
    12. 08-07 17:23:20.759: D/mm-camera-intf(195): mm_stream_read_msm_frame: Stopped poll on stream 0xb33376d4 type :5
    13. 08-07 17:23:20.759: D/mm-camera-intf(195): mm_stream_qbuf: Starting poll on stream 0xb33376d4 type :5
    14. 08-07 17:23:20.759: D/mm-camera-intf(195): mm_stream_qbuf: Started poll on stream 0xb33376d4 type :5
    15. 08-07 17:23:20.815: E/OMX-VENC-720p(195):  Allocated virt:0xac9c1000, FD: 172 of size 589824 at index: 0
    16. 08-07 17:23:20.818: E/OMX-VENC-720p(195):  Allocated virt:0xac931000, FD: 174 of size 589824 at index: 1
    17. 08-07 17:23:20.833: V/Unity(9707): NatCorder: MP4 video encoder changed output format: {height=768, width=480, csd-1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], what=1869968451, mime=video/avc, csd-0=java.nio.ByteArrayBuffer[position=0,limit=17,capacity=17]}
    18. 08-07 17:23:20.838: V/Unity(9707): NatCorder: MP4 audio encoder changed output format: {what=1869968451, mime=audio/mp4a-latm, csd-0=java.nio.ByteArrayBuffer[position=0,limit=2,capacity=2], channel-count=2, sample-rate=48000}
    19. 08-07 17:23:20.842: I/MPEG4Writer(9707): limits: 4294967295/0 bytes/us, bit rate: -1 bps and the estimated moov size 3195 bytes
    20. 08-07 17:23:20.848: I/MPEG4Writer(9707): setStartTimestampUs: 0
    21. 08-07 17:23:20.849: I/MPEG4Writer(9707): Earliest track starting time: 0
    22. 08-07 17:23:20.871: E/CRASH(9707): signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 9fea1f3c
    23. 08-07 17:23:20.871: E/CRASH(9707): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    24. 08-07 17:23:20.871: E/CRASH(9707): Build type 'Release', Scripting Backend 'il2cpp', CPU 'armeabi-v7a'
    25. 08-07 17:23:20.871: E/CRASH(9707): Build fingerprint: 'google/razor/flo:6.0.1/MOB30X/3036618:user/release-keys'
    26. 08-07 17:23:20.871: E/CRASH(9707): Revision: '0'
    27.  
     
  42. vice39

    vice39

    Joined:
    Nov 11, 2016
    Posts:
    108
    Is there a way to control the MP4 encoding parameters? Quality, bitrate, etc.
     
  43. andrewdSuper77

    andrewdSuper77

    Joined:
    Jul 12, 2019
    Posts:
    15
    Hello! does Natcorder support recording VisionlibAR? We seem to be having a problem where our Visionlib fails once Natcorder begins. Seems to record properly, but it's interference stops Visionlib entirely
     
  44. andrewdSuper77

    andrewdSuper77

    Joined:
    Jul 12, 2019
    Posts:
    15
    I should note that this only happens on device as well, I have had success with it in the editor
     
  45. shacke

    shacke

    Joined:
    Oct 4, 2018
    Posts:
    2
    Hello, i recently bought this plugin and it works great in a simple scene, but when i try to record a vuforia ARCamera it gives me this error and i dont really know what could be:

    Cannot get pixels when webcam is not running
    UnityEngine.WebCamTexture:GetPixels32()

    I shearched in all the pages of this post ans didnt found the solution so here am i
     
  46. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    What CameraRecorder did in NatCorder 1.3 was somewhat of a problem because it always stretched the camera view onto the recording RenderTexture. We've moved away from that. Again, your UI should be able to properly scale. We don't have similar issues in our own apps (do you use CanvasScaler)?
     
  47. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    How do you mean? We've renamed the namespace to NatCorder (and done similar things to all our API's).
     
  48. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Apparently the Nexus encoder doesn't support a setting of yours:
    Code (CSharp):
    1. E/OMXNodeInstance(892): getConfig(0xf4dabf04:qcom.encoder.avc, ??(0x7f000062)) ERROR: UnsupportedSetting(0x80001019)
    2.  
    Try recording at a standard resolution (1280x720, 1920x1080, 640x480).
    These logs don't show the cause of the crash. Can you upload the full logs in a .txt from app start to crash?
     
  49. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    See MP4Recorder constructor.
     
  50. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    I'm not at all familiar with Visionlib. You might want to reach out to its support. NatCorder works entirely independently; it doesn't interfere with other system processes except using the hardware encoders.