Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

NatCorder - Video Recording API

Discussion in 'Assets and Asset Store' started by Lanre, Nov 18, 2017.

  1. UDN_7452c628-7c91-4b1d-b7eb-6b100da4e893

    UDN_7452c628-7c91-4b1d-b7eb-6b100da4e893

    Joined:
    Mar 2, 2017
    Posts:
    6
    Hello,
    I have an issue, video record is working but not audio, only video show but not audio recording how can i solve this problem? thank you..

    Code (CSharp):
    1.    
    2.         public AudioListener audioListener;
    3.         public Camera vuforiaCam;
    4.  
    5.         public void StartRecording () {
    6.             // Start recording
    7.             recordingClock = new RealtimeClock();
    8.             videoRecorder = new MP4Recorder(
    9.                 videoWidth,
    10.                 videoHeight,
    11.                 30,
    12.                 AudioSettings.outputSampleRate, 1,
    13.                 OnReplay
    14.             );
    15.  
    16.  
    17.            
    18.             // Create recording inputs
    19.             cameraInput = new CameraInput(videoRecorder, recordingClock, vuforiaCam);
    20.             audioListener = vuforiaCam.GetComponent<AudioListener>();
    21.             audioInput = new AudioInput(videoRecorder, recordingClock, audioListener);
    22.  
    23.  
    24.             if (recordMicrophone) {
    25.                 StartMicrophone();
    26.                 audioInput = new AudioInput(videoRecorder, recordingClock, audioListener);
    27.             }
    28.         }
    29.  
     
    Last edited: Jul 17, 2019
  2. Slawomir-Sokol

    Slawomir-Sokol

    Joined:
    Dec 10, 2013
    Posts:
    12

    Try this, it works for me:
    Code (CSharp):
    1. private RenderTexture gameplayRecorderTexture;
    2. private byte[] pixelBuffer;
    3. private Texture2D copyTexture;
    4.  
    5. public void StartRecording()
    6. {
    7. videoRecorder = new MP4Recorder(1280, 720, 30, AudioSettings.outputSampleRate, (int) AudioSettings.speakerMode, OnRecordingDone, Mathf.CeilToInt(5909760 * 0.5f));
    8. this.pixelBuffer = new byte[videoRecorder.pixelWidth * videoRecorder.pixelHeight * 4];
    9. copyTexture = new Texture2D(gameplayRecorderTexture.width, gameplayRecorderTexture.height, TextureFormat.RGBA32, false, false);
    10. }
    11.  
    12. private void Update()
    13.         {
    14.             if (videoRecorder == null)
    15.                 return;
    16.            
    17.             RenderTexture.active = gameplayRecorderTexture;
    18.             copyTexture.ReadPixels(new Rect(0, 0, gameplayRecorderTexture.width, gameplayRecorderTexture.height), 0, 0, false);
    19.             copyTexture.GetRawTextureData<byte>().CopyTo(pixelBuffer);
    20.             videoRecorder.CommitFrame(pixelBuffer, recordingClock.Timestamp);
    21.         }
     
    Lanre likes this.
  3. Slawomir-Sokol

    Slawomir-Sokol

    Joined:
    Dec 10, 2013
    Posts:
    12
    Hi

    @Lanre
    I'm experiencing an interesting error on some of the devices
    Code (CSharp):
    1. E/Unity: AndroidJavaException: java.lang.NoSuchMethodError: no non-static method with name='address' signature='()J' in class Ljava.nio.DirectByteBuffer;
    2.     java.lang.NoSuchMethodError: no non-static method with name='address' signature='()J' in class Ljava.nio.DirectByteBuffer;
    3.         at com.unity3d.player.ReflectionHelper.getMethodID(Unknown Source)
    4.         at com.unity3d.player.UnityPlayer.nativeRender(Native Method)
    5.         at com.unity3d.player.UnityPlayer.c(Unknown Source)
    6.         at com.unity3d.player.UnityPlayer$e$1.handleMessage(Unknown Source)
    7.         at android.os.Handler.dispatchMessage(Handler.java:98)
    8.         at android.os.Looper.loop(Looper.java:148)
    9.         at com.unity3d.player.UnityPlayer$e.run(Unknown Source)
    10.       at UnityEngine._AndroidJNIHelper.GetMethodID (System.IntPtr jclass, System.String methodName, System.String signature, System.Boolean isStatic) [0x00041] in <46e4e6680eb4499aa3462989631a406c>:0
    11.       at UnityEngine.AndroidJNIHelper.GetMethodID (System.IntPtr javaClass, System.String methodName, System.String signature, System.Boolean isStatic)
    So far we have found this issue on Samsung Galaxy A5 and Google Nexus 7 Tablet (both have Android 6.0.1) and Samsung Galaxy Tab 4 10.1 (Android 5.1.1)

    Any ideas?
     
  4. Menion-Leah

    Menion-Leah

    Joined:
    Nov 5, 2014
    Posts:
    189
    Hi @Lanre

    I'm using NatCorder 1.6.0 on several Android devices without any issue.
    Unfortunately, I just tried it on a OnePlusOne and it fails to record, giving this exception every frame:

    Code (CSharp):
    1. java.lang.NoSuchMethodError: no non-static method with name='address' signature='()J' in class Ljava.nio.DirectByteBuffer;
    2. com.unity3d.player.ReflectionHelper.getMethodID(Unknown Source)
    3. com.unity3d.player.ReflectionHelper.nativeProxyInvoke(Native Method)
    4. com.unity3d.player.ReflectionHelper.a(Unknown Source)
    5. com.unity3d.player.ReflectionHelper$1.invoke(Unknown Source)
    6. java.lang.reflect.Proxy.invoke(Proxy.java:393)
    7. $Proxy16.onReadback(Unknown Source)
    8. com.yusufolokoba.natrender.AsyncGPUReadback.dispose(AsyncGPUReadback.java:90)
    9. com.yusufolokoba.natrender.AsyncGPUReadback.access$300(AsyncGPUReadback.java:13)
    10. com.yusufolokoba.natrender.AsyncGPUReadback$1.run(AsyncGPUReadback.java:108)
    11. android.os.Handler.handleCallback(Handler.java:739)
    12. android.os.Handler.dispatchMessage(Handler.java:95)
    13. android.os.Looper.loop(Looper.java:148)
    14. android.os.HandlerThread.run(HandlerThread.java:61)
    15. UnityEngine._AndroidJNIHelper.GetMethodID (System.IntPtr jclass, System.String methodName, System.String signature, System.Boolean isStatic) (at <930eb16cce8646f1bc56ac8d2bdbfbdd>:0)
    16. UnityEngine.AndroidJNIHelper.GetMethodID (System.IntPtr javaClass, System.String methodName, System.String signature, System.Boolean isStatic) (at <930eb16cce8646f1bc56ac8d2bdbfbdd>:0)
    17. UnityEngine._AndroidJNIHelper.GetMethodID[ReturnType] (System.IntPtr jclass, System.String methodName, System.Object[] args, System.Boolean isStatic) (at <930eb16cce8646f1bc56ac8d2bdbfbdd>:0)
    18. UnityEngine.AndroidJNIHelper.GetMethodID[ReturnType] (System.IntPtr jclass, System.String methodName, System.Object[] args, System.Boolean isStatic) (at <930eb16cce8646f1bc56ac8d2bdbfbdd>:0)
    19. UnityEngine.AndroidJavaObject._Call[ReturnType] (System.String methodName, System.Object[] args) (at <930eb16cce8646f1bc56ac8d2bdbfbdd>:0)
    20. UnityEngine.AndroidJavaObject.Call[ReturnType] (System.String methodName, System.Object[] args) (at <930eb16cce8646f1bc56ac8d2bdbfbdd>:0)
    21. NatCorder.Internal.AsyncGPUReadbackES+AsyncGPUReadbackRequestES.CopyTo (System.Byte[] pixelBuffer) (at D:/Angrymals Development/Game Client/AngrymalsGame/Assets/NatCorder/Plugins/Managed/Internal/AsyncGPUReadback.cs:61)
    22. Angrymals.Game.Battlefield.Core.VideoRecorder+<>c__DisplayClass87_0.<CommitFrame>b__1 (NatCorder.Internal.AsyncGPUReadbackES+AsyncGPUReadbackRequestES request) (at D:/Angrymals Development/Game Client/AngrymalsGame/Assets/Angrymals/Scripts/Battlefield/Replays/VideoRecorder.cs:709)
    23. NatCorder.Internal.AsyncGPUReadbackES+AsyncGPUReadbackRequestES.onReadback (UnityEngine.AndroidJavaObject nativeBuffer) (at D:/Angrymals Development/Game Client/AngrymalsGame/Assets/NatCorder/Plugins/Managed/Internal/AsyncGPUReadback.cs:72)
    24. System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at <1cc141e219f7479c97cfdd69a39cd92f>:0)
    25. Rethrow as TargetInvocationException: NatCorder.Internal.AsyncGPUReadbackES+AsyncGPUReadbackRequestES.onReadback(UnityEngine.AndroidJavaObject)
    26. UnityEngine.AndroidJavaProxy.Invoke (System.String methodName, System.Object[] args) (at <930eb16cce8646f1bc56ac8d2bdbfbdd>:0)
    27. UnityEngine.AndroidJavaProxy.Invoke (System.String methodName, UnityEngine.AndroidJavaObject[] javaArgs) (at <930eb16cce8646f1bc56ac8d2bdbfbdd>:0)
    28. UnityEngine._AndroidJNIHelper.InvokeJavaProxyMethod (UnityEngine.AndroidJavaProxy proxy, System.IntPtr jmethodName, System.IntPtr jargs) (at <930eb16cce8646f1bc56ac8d2bdbfbdd>:0)

    I'm using your AsyncGPUReadback implementation as in CameraInput example.

    Do you have any advice on that?

    - edit: I just refreshed and saw last post, seems that I'm facing the same issue as @Slawomir-Sokol
     
    Last edited: Jul 17, 2019
  5. Luchunpen_0

    Luchunpen_0

    Joined:
    Aug 7, 2014
    Posts:
    58
    Hi.
    I'm using NatCorder with NatMic and my questions is:
    1) Can I record multiscene video/audio and, if I can, what I need to do for this (if it not solve by couple of strings, could you write an example)?
    2) How to check when Natcorder in recording process ?
     
    Last edited: Jul 17, 2019
  6. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    Can you share your recording code? The error on Android is because your sample rate is invalid (22Hz).
     
  7. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    This must be a Unity issue. I'll reach out to them. There shouldn't be any platform-specific difference in our recording code. If it works on one platform, it should work on all platforms.
     
  8. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    You are hard coding a channel count of 1. For a normal Unity project, this is incorrect (unity uses stereo). See AudioSettings.speakerMode.
     
  9. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    See CameraInput.cs. Either use AsyncGPUReadback or Texture2D.ReadPixels. And as for the null reference exception, check your code (you should be able to debug trivial issues like this in the editor).
     
  10. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    This is already what CameraInput does.
    See CameraInput.cs for how we handle Android specifically.
    Nope, we've moved away from this with very good reason.
     
  11. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    Email me for updated build with fix. This issue seems to affect random devices.
     
  12. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    This entirely depends on your setup. NatCorder doesn't have anything to do with what scene is open. It simply expects you to provide video (and audio) data for encoding. How you handle scenes and what gets recorded is entirely up to you.
    When you create a recorder, you are implicitly recording. And when you Dispose the recorder, you have implicitly stopped recording. There isn't some boolean flag to track this; it is implied.
     
  13. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    Can you upload the logs from logcat in a .txt file?
     
  14. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    Because a lot of devs are facing issues with committing video from a RenderTexture, we are adding a general `RenderTextureInput` helper class.

    I'll have a working build ready next week Monday. Hope y'all can wait till then :)

    EDIT: Build is ready, email me with your invoice number and I'll share it with you.
     
    Last edited: Jul 17, 2019
    Menion-Leah likes this.
  15. alonlevi

    alonlevi

    Joined:
    Apr 22, 2019
    Posts:
    11
    I've been using Natcorder to record an AR screen with audio. The audio is sometimes broken (shutter). how can I fix this? another thing - what are the benefits of using NatMic instead of just using the microphone input as used in ReplayCam example?
     
    ina likes this.
  16. MichalHuuuge

    MichalHuuuge

    Joined:
    May 8, 2019
    Posts:
    4
    Thank you for your answer. You were right, we checked it on the 2019.1.10f and it works just fine.
     
    Lanre likes this.
  17. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    This is because Unity's Microphone API produces glitchy audio. NatMic provides clean microphone audio with no glitching.
     
  18. GuyInception

    GuyInception

    Joined:
    Jun 12, 2019
    Posts:
    4
    Thanks!
    Once I used AudioSettings.outputSampleRate it worked on Windows.
    On Android it freezes.
     
  19. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    Something must be going on, because AudioSettings.outputSampleRate cannot report 22. Debug.Log it in Unity to check. If it does return that value, then file a bug report with Unity because that value is invalid.
     
  20. GuyInception

    GuyInception

    Joined:
    Jun 12, 2019
    Posts:
    4
    Oh sorry for the mix-up.
    I manually set it to 22. Once changed to AudioSettings.outputSampleRate - it is not 22 anymore and working.

    Also, when skipping frames it did not crash on the Android.
     
    Lanre likes this.
  21. alonlevi

    alonlevi

    Joined:
    Apr 22, 2019
    Posts:
    11
    thanks... will it be able to record from the main mic although ARcore is using the webcam?
     
  22. Packedbox

    Packedbox

    Joined:
    Jun 20, 2013
    Posts:
    20
    Hi Lanre,
    On some Android device I us to test my app, the video output is out of sync with the audio, the video stream is playing slowly and stops before last recorded image, sound has the right duration and speed though.

    I have the problem with these phones:
    - Huawey Y5 2018 Android 8.1
    - Samsung Galaxy S7 Android 8.0 )

    But it's working fine on a Xiaomi MI A2 LITE Android 9.0

    I'm using Unity 2018.4.4f1 LTS
    Building with Minimum API 21 and Target API Level 28

    I record the camera and the audio listener, here is my recording and stop code code :

    Code (CSharp):
    1.     private MP4Recorder videoRecorder;
    2.     private IClock recordingClock;
    3.     private CameraInput cameraInput;
    4.     private AudioInput audioInput;
    5.  
    6.     [Header("Recording")]
    7.     public int videoMaxWidth = 1280;
    8.  
    9.     void StartRecordingNatCorder()
    10.     {
    11.  
    12.         int w = Mathf.Min(videoMaxWidth, Screen.width / 2);
    13.         int h = w * Screen.height / Screen.width;
    14.  
    15.         Debug.Log("Recording resolution : " + w + "x" + h);
    16.  
    17.         // Start recording
    18.         recordingClock = new RealtimeClock();
    19.         videoRecorder = new MP4Recorder(
    20.             w,
    21.             h,
    22.             30,
    23.             AudioSettings.outputSampleRate,
    24.             1,
    25.             OnRecordingDone
    26.         );
    27.  
    28.         var audioListener = GetComponent<AudioListener>();
    29.  
    30.         // Create recording inputs
    31.         cameraInput = new CameraInput(videoRecorder, recordingClock, Camera.main);
    32.         audioInput = new AudioInput(videoRecorder, recordingClock, audioListener);
    33.  
    34.     }
    35.  
    36.     void StopRecordingNatCorder()
    37.     {
    38.         // Stop recording
    39.         audioInput.Dispose();
    40.         cameraInput.Dispose();
    41.         videoRecorder.Dispose();
    42.     }
     
  23. jasminepark

    jasminepark

    Joined:
    Aug 21, 2018
    Posts:
    3
    Thanks Lanre for the reply. Also would like to ask whether it is possible to ask for permission to access gallery/photos at the start of the app instead of when im saving the video? For iOS. thank you so much!
     
  24. unity_J3kjkRR5rO8l3w

    unity_J3kjkRR5rO8l3w

    Joined:
    Oct 11, 2017
    Posts:
    1
    Hi Have you find solution of this issue?, thene plase tell me what is the solution
     
  25. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    I haven't tested this. I don't know if ARCore uses the hardware microphone.
     
  26. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    Might be related to an issue on Android with PBO's. I'm looking into it.
     
  27. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    NatCorder doesn't have anything to do with the gallery. It saves videos to the app's private documents directory.
     
  28. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    Have you tried the latest build of NatCorder?
     
  29. unitydevstudio

    unitydevstudio

    Joined:
    Jun 24, 2017
    Posts:
    24
    Hi @Aidan-Wolf

    I am having exactly same logs in Xcode logs when I do pause and resume recording.
    So when I record without pause/resume action, recording was good.
    If I do pause/resume while recording, after pause and resume, I get this error.

    Failed to record video frame for time 16370000 with error: Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-16364), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x280a20c00 {Error Domain=NSOSStatusErrorDomain Code=-16364 "(null)"}}

    2019-07-14 23:07:04.676143+0800 TapestryAR[2379:354037] NatCorder Error: Failed to record video frame for time 20357000 due to invalid recorder state: 3
    ....


    What was your solution?

    Best regards.
     
  30. unitydevstudio

    unitydevstudio

    Joined:
    Jun 24, 2017
    Posts:
    24
    Hi @Lanre

    This is Song.
    I sent email with logs.txt of pause/resume recording problem. but didnt get reply yet.
    Any solution for not working pause/resume?

    Best regards.
    Song
     
  31. alonlevi

    alonlevi

    Joined:
    Apr 22, 2019
    Posts:
    11
    Hi, I'm doing it myself using the really nice (and free) Native Gallery plug-in - one the video file is saved i'm moving it o the gallery in the success callback... just for your information ;)
     
    Lanre likes this.
  32. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    I am looking into this. I was able to reproduce it.
     
  33. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    I am looking into this. I was able to reproduce it.
     
  34. alonlevi

    alonlevi

    Joined:
    Apr 22, 2019
    Posts:
    11
    After a few days updating to 1.6 I've disocvered the preformance was really downgraded from previous version. I'm recording an AR elements composed on video (ARCORE) using a camera (something similar to replayCam demo) at double default bitrate (around 10mb/s). the performance went down from 30fps to around unstable 17-18 fps.... reverting back to previous version fixed it...
     
    Last edited: Jul 23, 2019
  35. Alond

    Alond

    Joined:
    Dec 9, 2013
    Posts:
    12
    Hi,
    Great Asset! I've already used it in a few projects and it really does the job.
    Right now, I have an issue with the output video and I was hoping to get some help here...
    I'm trying to record the game-camera view which includes a RawImage containing a video clip (with Unity VideoPlayer) on top of another RawImage containing a WebCamTexture.
    The problem is that the saved video is very flickering...
    This is my code:

    Code (CSharp):
    1. void StartRecording()
    2.     {
    3. videoWidth = Screen.width;
    4.         videoHeight = Screen.height;
    5.         // Start recording
    6.         recordingClock = new RealtimeClock();
    7.         videoRecorder = new MP4Recorder(
    8.             videoWidth,
    9.             videoHeight,
    10.             30,
    11.             0,
    12.             0,
    13.             OnReplay
    14.         );
    15.      
    16.         // Create recording inputs
    17.         cameraInput = new CameraInput(videoRecorder, recordingClock, mainCamera);
    18.     }
    19.  
    20. void StopRecording()
    21.     {
    22.         // Stop the recording inputs
    23.         cameraInput.Dispose();
    24.         videoRecorder.Dispose();
    25.         videoRecorder = null;
    26.     }  
    Thanks in advance!
     
  36. Aidan-Wolf

    Aidan-Wolf

    Joined:
    Jan 6, 2014
    Posts:
    59
    @Lanre the new NatCorder update unfortunately returns the recording lag to iOS, and I can't improve it with a background thread this time. Worse, it affects the quality of the video as the first 0.5s is a few stuttering frames.

    https://www.dropbox.com/s/m67ntqjo4a070sr/img_1072.trim.mov?dl=0

    :/

    Addition: just realized with AirPods in, the lag is upwards of 2 seconds... what happened?
    Addition 2: Audio is out of sync as well
     
    Last edited: Jul 22, 2019
  37. UDN_7452c628-7c91-4b1d-b7eb-6b100da4e893

    UDN_7452c628-7c91-4b1d-b7eb-6b100da4e893

    Joined:
    Mar 2, 2017
    Posts:
    6
    i have an issue, when i clicked record button every thing on the screen is stack and after hold up my finger screen capture preview working perfectly and i can save camera roll up, this is a big problem because i cannot see what im recording on game screen.
    how can fix this?
     
  38. UDN_7452c628-7c91-4b1d-b7eb-6b100da4e893

    UDN_7452c628-7c91-4b1d-b7eb-6b100da4e893

    Joined:
    Mar 2, 2017
    Posts:
    6
    Actually when i start recording all screen is freezing after i hold up my finger everything great, how can fix this freezing i am working xsmax.
     
  39. AlexeySeverin

    AlexeySeverin

    Joined:
    Oct 6, 2016
    Posts:
    12
    First of all thank you for this wonderful asset, I've really waiting for something like this to be able to easily record videos on both iOS and Android.
    It used to work smoothly, but after upgrading Unity I ran into some issues that I struggle to resolve.

    I am using NatCorder with Vuforia. The plugin used to work fine, but after I upgraded from Unity 2017.3.0p3 to 2018.1.9f2 and from Vuforia 7.0.43 to 7.1.35 I started to get very jittery video recordings on iOS on older devices (including e.g. iPhone 7 that is not THAT old after all...). In app everything looks good, but when you watch the video afterwards it is very jittery with frames jumping back and forth (resembling the effect in Twin Peaks season 3, when Cooper meets the Eyeless Woman in the Purple Room...). Tried to reproduce the issue in an empty project but it works fine. It seems that the issue is somehow related to Vuforia or maybe to the complexity of the scene overall.

    The thing is before the upgrade the video was always pretty smooth. The content of the project itself didn't change, the only thing changed were versions of Unity (2017.3.0p3 to 2018.1.9f2) and Vuforia (7.0.43 to 7.1.35). I also switched from XCode 9.x to 10.x (don't remember the exact versions right now).
    The NatCorder version I am using is 1.2f2 and it remained the same.

    Do you have any idea of why this could be happening and how I could fix this? Thank you!
     
    yongvits likes this.
  40. Aidan-Wolf

    Aidan-Wolf

    Joined:
    Jan 6, 2014
    Posts:
    59
    Luck
    Luckily I had 1.5.1 sitting in another project on my computer. Everything is back to normal.

    It seems that audio input in 1.6 is thread blocking. For example, the NatMic AirPod delay is still 2s, but in 1.5.1 the video will have smoothly recorded 2 seconds of video before the audio starts. In 1.6, it will freeze the entire main thread for 2s and cause all kinds of havoc in memory.

    1.6 iOS issues
    - Unavoidable recording delay
    - Recording delay is in the video (first frame takes up first 0.25-2s of video depending)
    - Freezes main thread causing memory havoc, breaks ARKit tracking completely
    - Audio will be out of sync for the length of the delay, rather than simply starting late like in 1.5.1
     
  41. prankard

    prankard

    Joined:
    Mar 24, 2010
    Posts:
    19
    Thought I'd post here. I've been using NatCorder for iOS with the LWRP.
    Sadly it's broken at the moment, but it's a super easy fix.

    The LWRP side is all great, and the plugin still works fine. but the dispatcher fails to dispatch. Which means you never get the message that your file has finished recording and where your file is located.

    The Dispatcher class handles taking the thread from the video recording back to the Unity main thread:
    This line now doesn't work in LWRP:
    Code (CSharp):
    1. Camera.onPostRender += DequeueMain;
    So I handled it in a different way, not the best but still working to get it up and running before a true fix:
    Just use this dispatcher instead in the file 'MediaRecorderDispatcher.cs':
    Code (CSharp):
    1. /*
    2. *   NatCorder
    3. *   Copyright (c) 2019 Yusuf Olokoba
    4. */
    5.  
    6. namespace NatCorder.Internal {
    7.  
    8.     using AOT;
    9.     using UnityEngine;
    10.     using System;
    11.     using System.Collections;
    12.     using System.Collections.Generic;
    13.     using System.Runtime.InteropServices;
    14.  
    15.     public class DispatcherMonoBehaviour : MonoBehaviour
    16.     {
    17.         public void Awake()
    18.         {
    19.             DontDestroyOnLoad(gameObject);
    20.         }
    21.  
    22.         public void LateUpdate()
    23.         {
    24.             for (; ; )
    25.                 lock ((Dispatcher.mainQueue as ICollection).SyncRoot)
    26.                     if (Dispatcher.mainQueue.Count > 0)
    27.                         Dispatcher.mainQueue.Dequeue()();
    28.                     else
    29.                         break;
    30.         }
    31.     }
    32.  
    33.     public static class Dispatcher {
    34.  
    35.         #region --Client API--
    36.      
    37.         public enum Target {
    38.             MainThread = 0, RenderThread
    39.         }
    40.  
    41.         public static void Dispatch (Target target, Action workload) {
    42.             switch (target) {
    43.                 case Target.MainThread:
    44.                     lock ((mainQueue as ICollection).SyncRoot)
    45.                         mainQueue.Enqueue(workload);
    46.                     break;
    47.                 case Target.RenderThread:
    48.                     GL.IssuePluginEvent(renderDelegateHandle, ((IntPtr)GCHandle.Alloc(workload, GCHandleType.Normal)).ToInt32());
    49.                     break;
    50.             }
    51.         }
    52.         #endregion
    53.  
    54.  
    55.         #region --Operations--
    56.  
    57.         [RuntimeInitializeOnLoadMethod]
    58.         public static void CreateDispatcher()
    59.         {
    60.             GameObject go = new GameObject("Dispatcher");
    61.             go.hideFlags = HideFlags.HideInHierarchy;
    62.             go.AddComponent<DispatcherMonoBehaviour>();
    63.         }
    64.  
    65.         public static readonly Queue<Action> mainQueue;
    66.         private static readonly IntPtr renderDelegateHandle;
    67.  
    68.         [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
    69.         private delegate void UnityRenderingEvent (int context);
    70.  
    71.         static Dispatcher () {
    72.             // Setup main dispatch
    73.             mainQueue = new Queue<Action>();
    74.             //Camera.onPostRender += DequeueMain;
    75.             // Setup render dispatch
    76.             renderDelegateHandle = Marshal.GetFunctionPointerForDelegate((UnityRenderingEvent)DequeueRender);          
    77.         }
    78.  
    79.         private static void DequeueMain (Camera _) {
    80.             for (;;)
    81.                 lock ((mainQueue as ICollection).SyncRoot)
    82.                     if (mainQueue.Count > 0)
    83.                         mainQueue.Dequeue()();
    84.                     else
    85.                         break;
    86.         }
    87.  
    88.         [MonoPInvokeCallback(typeof(UnityRenderingEvent))]
    89.         private static void DequeueRender (int context) {
    90.             GCHandle handle = (GCHandle)(IntPtr)context;
    91.             Action target = handle.Target as Action;
    92.             handle.Free();
    93.             target();
    94.         }
    95.         #endregion
    96.     }
    97. }
    Hope that helps people.
     
    Lanre likes this.
  42. ZAUBAR

    ZAUBAR

    Joined:
    Jun 18, 2019
    Posts:
    53
    @Lanre Using NatMic, how can I mix Game Audio (AudioSource on GameObject) with microphone audio, all synced up, during video recording? Thanks!
     
  43. Marcin-Priachin

    Marcin-Priachin

    Joined:
    Dec 5, 2012
    Posts:
    2
    Hi,
    I'm using NatCoder 1.6.0 with Unity 2019.1.8f1 and when I try to record video with audio on Android device (Google Pixel 3 XL with Android 9) app crashes, recording video without audio works great. I tried run empty project with just example scene and run into same issue. Are there some project setting, or permissions I should have changed. I attach error log
     

    Attached Files:

    • err.txt
      File size:
      201.7 KB
      Views:
      852
    Lanre likes this.
  44. Aidan-Wolf

    Aidan-Wolf

    Joined:
    Jan 6, 2014
    Posts:
    59
    There seems to be something wrong with your audio hardware. Try a different audio device (I think the Pixel 3 XL has 2 microphones)
     
    Lanre likes this.
  45. Elizeu159

    Elizeu159

    Joined:
    Nov 25, 2016
    Posts:
    9
    The plugin was working just fine in Mono builds, now i tried bulding in IL2CPP and the app freezes when i start recording.
    This only happens if the record microphone option is on.
     
    Last edited: Jul 26, 2019
  46. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    I'm working on improvements for this. The performance cost is in getting pixel data from the GPU to send to the encoder.
     
  47. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    What platform/device does this happen on?
     
  48. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    It would seem that Unity's AsyncGPUReadback implementation is buggy. Literally nothing else has changed since 1.5 apart from this (for all platforms except Android). I'm working on an update that reverts to Texture2D.ReadPixels which we used in 1.5. Not the best performance, but it will fix all the issues people are facing until we roll out our own async readback implementation on iOS.
    Likely has nothing to do with NatCorder. iOS might be switching audio routes.
    Fixed in build I'm working on right now.
     
  49. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    I'm have a fix for this. Email me on Monday for a build.
     
  50. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,969
    I saw your email. You are using a much older version of NatCorder. Upgrade to 1.6.