Search Unity

NatCorder - Video Recording API

Discussion in 'Assets and Asset Store' started by Lanre, Nov 18, 2017.

  1. QQMatt

    QQMatt

    Joined:
    Sep 29, 2016
    Posts:
    10
    Hi just to follow up on this, I was reading this and it seems as though you could add a check that the API level is 18+ other wise do nothing (or other fallback). I imagine what nDream is asking for and what I would find useful is for the Natcorder manifest to have at lower minimum API level, but have a fallback or do nothing. This would allow me to do a check and if the version is 18+ I'll use natcorder, if lower I can then call a different implementation that will work. This would be useful as it means I have to manage less APKs when distributing.

    This may not be possible, but I thought I'd mention in case it is of any use.
     
  2. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Unfortunately, we don't have this option. We can't build the AAR without specifying the minimum as 18 because the functions we use were only introduced in 18.
     
  3. unityCZH

    unityCZH

    Joined:
    Nov 12, 2018
    Posts:
    16
    Hello, I reported the following error when using IOS device. Could you please tell me how to solve this problem?Error:A call to a standard soundcard driver failed, which could possibly mean a bug in the driver or resources were missing or exhausted.
     
  4. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    I've never seen any error like this (especially because iOS devices don't have conventional soundcards). Can you share a screenshot of the error?
     
  5. unityCZH

    unityCZH

    Joined:
    Nov 12, 2018
    Posts:
    16
    12-07 09:24:26.359 757 - E : ReportException: UnityLogError Starting microphone failed: "A call to a standard soundcard driver failed, which could possibly mean a bug in the driver or resources were missing or exhausted. " (57) ********* OM.PB.ReplayCamManager.StartMicrophone() OM.PB.ReplayCamManager.StartRecording(Action`2) RReplayManager.StartRecord() UnityEngine.EventSystems.ExecuteEvents.Execute(GameObject, BaseEventData, EventFunction`1) OM.PictureBooks.UI.CustomInputModule.ProcessTouchPress(PointerEventData, Boolean, Boolean) OM.PictureBooks.UI.CustomInputModule.ProcessTouchEvents() OM.PictureBooks.UI.CustomInputModule.Process() *********
     
  6. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    It looks like the microphone won't start for some reason. I have no clue what it is. And from what I can tell it is not related to NatCorder, because NatCorder doesn't do anything with the microphone. Consider filing a bug report with Unity.
     
  7. TobiasW

    TobiasW

    Joined:
    Jun 18, 2011
    Posts:
    78
    While trying to fix a weird recording bug with NatCam 2.0f1 / NatCorder on iOS (https://www.dropbox.com/s/zpq3e4ybubx5md9/test_video_1.MOV?dl=0 - happens only when using Metal), I realized that I was recording every LateUpdate() and not just in NatCam.OnFrame. However, switching from calling AcquireFrame, Blit and CommitFrame in Android to calling it in NatCam.OnFrame made it crash on Android: The camera display (or the whole app, I'm not sure) just freezes, and then it crashes after a few seconds.The only relevant log entry that I found was "AudioFlinger: RecordThread: buffer overflow", and it sure reminds me of the crash I experienced months ago when I didn't write any audio data. I think it's just a symptom though, and not the cause - I think the app freezes and then can't read audio anymore. I didn't change anything in the way the audio records.

    Debug.Log()s in NatCam.OnFrame trigger, so it's not like it isn't calling it.

    Do you have any what might be happening here? (And maybe any insight into the weird iOS video glitch? It looks to me like the texture is overwritten while written to video? I'm using a field pointing to NatCam.Preview in Graphics.Blit.)

    edit:
    After more research, I have a tiny bit more information. It seems that if I call AcquireFrame, Blit and CommitFrame in OnFrame, OnFrame is called multiple times within a Unity frame (Debug.log(Time.frameCount) outputs the same number). If I called AcquireFrame, Blit and CommitFrame in LateUpdate, OnFrame is almost always just called once per Unity frame.

    edit2:
    Regarding the iOS-Video-Bug: That only happens in the video made out of NatCam.Preview frames. The RawImage component we use as a display of NatCam.Preview looks fine.

    edit3:
    More weirdness: Once I start recording with AcquireFrame, Blit and CommitFrame in OnFrame, I get 20 OnFrame calls in the same Unity Frame. If I only call AcquireFrame, Blit and CommitFrame in the first one and just do nothing in the other 19, everything is okay afterwards - the video records and I don't get multiple OnFrame per Unity frame.
     
    Last edited: Dec 7, 2018
  8. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Can you upload the full logs from logcat in a .txt file?
    Are you using NatCorder 1.3f2 from the Asset Store, or a different build? The video looks like frames are incomplete. I'll need to know what version of NatCorder you are using to reason about it (we keep updating how we handle encoding on Metal to make things faster). We will be dropping OpenGL ES support on iOS in the next update.
    This would suggest that the camera is running at a higher framerate than Unity. So NatCam is calling back into Unity several times.
    This shouldn't be the case. The multiplier should be at most 2 (camera running at 60FPS, Unity running at 30FPS).
     
  9. TobiasW

    TobiasW

    Joined:
    Jun 18, 2011
    Posts:
    78
    Will do later!

    It's NatCorder 1.3b3 that you sent to me on 2018-11-26. I couldn't use the NatCorder build from the Asset Store because of the OSX DllNotFoundException problem.

    It would seem that way, except that before starting the recording (but already previewing), at most one OnFrame per Unity frame is called, and if I skip out of that 20 OnFrame per Unity frame madness it also goes back to one OnFrame per Unity frame.

    Yeah, I assumed so. First I thought that new items are added to the dispatcher queue while it's handled, but after reading
    MainDispatcher.Update I can't see how that would happen. Another idea would be that the main thread stalls for some reason so the camera thread adds too many OnFrame queue actions, but I have no idea why would that only happen when calling AcquireFrame, Blit and CommitFrame in OnFrame.
     
  10. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Okay perfect. This flickering is a known issue and I'm already testing a fix.
    Wait, so you're saying that you only get 20 OnFrames while recording? I don't fully understand.
    Can you share your recording code?
     
  11. look001

    look001

    Joined:
    Mar 23, 2017
    Posts:
    44
    Hi, i realy like this asset. It's straight forward and seems to be easy to use. However i have a problem with it.
    I set the VideoFormat to 30fps with 500.000bps and 200 keyframes like this:
    Code (CSharp):
    1. VideoFormat videoFormat = new VideoFormat(renderTexture.width, renderTexture.height, Settings.framerate, Settings.bitrate * 1000, 200);
    2. NatCorder.StartRecording(Container.MP4, videoFormat, AudioFormat.None, OnRendered);
    3. // Settings.framerate=30; Settings.bitrate=500

    However my output file has 30.34 fps and 26 kbit/s and the video quality is poor (blury). I don't understand what Keyframes exactly do, i didn't find information about it

    Also i manualy set the Frames in the Update function like this:
    Code (CSharp):
    1. frameMul = Mathf.RoundToInt((1 / Settings.framerate) * Mathf.Pow(10, 9)); // multiplier to get the time nanosecounds from the framecount
    2.  
    3. void UpdateTime(int f) // called from Update. t is the current framecount
    4.     {
    5.         if (NatCorder.IsRecording)
    6.         {
    7.             Debug.Log(f); // counts up from 0 to 91
    8.             var frame = NatCorder.AcquireFrame();
    9.             frame.timestamp = (f * frameMul);
    10.  
    11.             Graphics.Blit(renderTexture, frame);
    12.             NatCorder.CommitFrame(frame);
    13.  
    14.             if (f > Settings.frameCount)
    15.             {
    16.                 StopRecording();
    17.             }
    18.         }
    19.     }

    But if i set the frameCount to 90 (3 Sek.) the video will be 2 Sek. only. Can you please help me with this?
    Thank you for this asset you are the best :)
     
    Last edited: Dec 8, 2018
  12. TobiasW

    TobiasW

    Joined:
    Jun 18, 2011
    Posts:
    78
    Awesome!

    I can't share my complete code, but it maybe this pseudocode makes it clearer:

    Scenario A: RecordFrame() is called for every NatCam.OnFrame when we're recording.
    Code (CSharp):
    1. void OnNatCamFrame()
    2. {
    3.     if (NatCorder.IsRecording)
    4.     {
    5.         Debug.Log("NatCam.OnFrame in Unity frame " + Time.frameCount + " while recording");
    6.         RecordFrame();
    7.     }
    8.     else
    9.     {
    10.         Debug.Log("NatCam.OnFrame in Unity frame " + Time.frameCount + " while not recording");
    11.     }
    12. }
    13.  
    14. void RecordFrame()
    15. {
    16.     var frame = NatCorder.AcquireFrame();
    17.     Graphics.Blit(natCamPreviewTexture, frame);
    18.     NatCorder.CommitFrame(frame, recordingClock.CurrentTimestamp);
    19. }
    Example debug log output:
    Code (csharp):
    1. NatCam.OnFrame in Unity frame 1222 while not recording
    2. NatCam.OnFrame in Unity frame 1224 while not recording
    3. NatCam.OnFrame in Unity frame 1226 while not recording
    4. NatCam.OnFrame in Unity frame 1228 while not recording
    5. NatCam.OnFrame in Unity frame 1230 while not recording
    6. NatCam.OnFrame in Unity frame 1232 while not recording
    7. NatCam.OnFrame in Unity frame 1234 while not recording
    8. NatCam.OnFrame in Unity frame 1236 while not recording
    9. [Recording starts, screen freezes]
    10. NatCam.OnFrame in Unity frame 1237 while recording
    11. NatCam.OnFrame in Unity frame 1237 while recording
    12. NatCam.OnFrame in Unity frame 1237 while recording
    13. [Repeats for 10+ times]
    14. NatCam.OnFrame in Unity frame 1238 while recording
    15. NatCam.OnFrame in Unity frame 1238 while recording
    16. NatCam.OnFrame in Unity frame 1238 while recording
    17. [Repeats for 10+ times]
    18. NatCam.OnFrame in Unity frame 1239 while recording
    19. NatCam.OnFrame in Unity frame 1239 while recording
    20. NatCam.OnFrame in Unity frame 1239 while recording
    21. [Repeats until crash]
    Scenario B: Exactly the same as Scenario A, except now we're reducing NatCam.OnFrame calls to a single call per Unity frame.
    Code (CSharp):
    1. void OnNatCamFrame()
    2. {
    3.    if (lastUnityFrame == Time.frameCount)
    4.    {
    5.        Debug.Log("Skipped NatCam.OnFrame in Unity frame " + Time.frameCount + " because we already had one this Unity frame");
    6.        return;
    7.    }
    8.  
    9.    lastUnityFrame = Time.frameCount;
    10.  
    11.    if (NatCorder.IsRecording)
    12.     {
    13.         Debug.Log("NatCam.OnFrame in Unity frame " + Time.frameCount + " while recording");
    14.         RecordFrame();
    15.     }
    16.     else
    17.     {
    18.         Debug.Log("NatCam.OnFrame in Unity frame " + Time.frameCount + " while not recording");
    19.     }
    20. }
    Example debug log output:
    Code (csharp):
    1.  
    2. NatCam.OnFrame in Unity frame 1083 while not recording
    3. NatCam.OnFrame in Unity frame 1084 while not recording
    4. NatCam.OnFrame in Unity frame 1087 while not recording
    5. NatCam.OnFrame in Unity frame 1088 while not recording
    6. [Recording starts, everything works fine]
    7. NatCam.OnFrame in Unity frame 1089 while recording
    8. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    9. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    10. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    11. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    12. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    13. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    14. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    15. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    16. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    17. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    18. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    19. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    20. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    21. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    22. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    23. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    24. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    25. Skipped NatCam.OnFrame in Unity frame 1089 because we already had one this Unity frame
    26. NatCam.OnFrame in Unity frame 1090 while recording
    27. Skipped NatCam.OnFrame in Unity frame 1090 because we already had one this Unity frame
    28. NatCam.OnFrame in Unity frame 1091 while recording
    29. NatCam.OnFrame in Unity frame 1092 while recording
    30. NatCam.OnFrame in Unity frame 1093 while recording
    31. NatCam.OnFrame in Unity frame 1095 while recording
    32. NatCam.OnFrame in Unity frame 1096 while recording
    33. NatCam.OnFrame in Unity frame 1098 while recording
    34. NatCam.OnFrame in Unity frame 1099 while recording
    35. NatCam.OnFrame in Unity frame 1101 while recording
    I'm sending the full logcat logs to you via mail. I don't want to post them publicly.
     
    look001 likes this.
  13. QQMatt

    QQMatt

    Joined:
    Sep 29, 2016
    Posts:
    10
    I was wondering if you had any idea why the apk built is failing to install on an actual device. If I remove Natcorder it installs no problem?
    • Min SDK is set to 18
    • I am using Native Gallery and Unity IAP, don't know if that would cause any conflicts?
    • Split CPU architectures (building for all 3)
    • Using graddle with proguard minify
     
  14. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Your bitrate is too low and your keyframe interval is too high. The bitrate setting is in bits per second. For reference, a 1080p video on YouTube is 8Mbps, or 8,000,000 bps. As for keyframes, VideoFormat takes a keyframe interval in seconds. So setting 200 means that the encoder will insert a keyframe every 200 seconds or 3+ minutes. This is extremely high. For reference, the default setting is 3 (which is also really high).
    I recommend not specifying these parameters if you don't fully understand their effects. Instead, construct your VideoFormat like so:
    Code (CSharp):
    1. var videoFormat = new VideoFormat(width, height, framerate);
    The other settings like bitrate and keyframe interval will be initialized to reasonable defaults.
    I'm not too sure why this happens (can you upload one such recorded video?), but your timestamp calculation is a bit bothersome. Because the timestamp is in nanoseconds, it takes on really large values. Don't use Mathf to compute your `frameMul`. Instead, compute it to double precision and cast the result to a long (and make sure `frameMul` is a long):
    Code (CSharp):
    1. frameMul = (long)(1e+9 / Settings.framerate);
     
  15. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Okay this is super weird. Have you tried NatCam 2.1? This might be a bug in how we handle callbacks from Unity's graphics thread (when both API's are used together in the same project).

    EDIT: I recommend that you try NatCam 2.1. I don't know how you handle NatCam state, but it might be that your OnNatCamFrame delegate is subscribed to NatCam.OnFrame several times.
     
    Last edited: Dec 9, 2018
  16. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Can you share the Editor logs in a .txt file? The cause of the failure will likely be stated there.
     
  17. QQMatt

    QQMatt

    Joined:
    Sep 29, 2016
    Posts:
    10
    https://we.tl/t-hVQ8OApL3A

    Cheers.
     
  18. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Can you describe exactly what you mean by the APK failing to install on the device? The editor logs indicate that the build is successful:
    Code (CSharp):
    1. *** Completed 'Build.Player.AndroidPlayer' in 434 seconds (433702 ms)
    2.  
    3. Build completed with a result of 'Succeeded'
     
  19. walaber_entertainment

    walaber_entertainment

    Joined:
    Jul 10, 2017
    Posts:
    14
    Just checking in, I haven't seen an update in the Asset Store yet, so I think the MacOS issue still exists, no? I see you referencing a "version 2.1", is that available somewhere?
     
  20. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    That's correct. The NatCorder update isn't live yet because we're still making changes. Version 2.1 refers to our native camera API, NatCam.
     
  21. unityCZH

    unityCZH

    Joined:
    Nov 12, 2018
    Posts:
    16
    Hello, I have a question here.There will be serious noises after recording on all platforms. How can you solve this problem
     
  22. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Hey there. What do you mean by recording noises? Are you referring to video or audio? Can you upload sample media (image, video) showing what you are referring to?
     
  23. unityCZH

    unityCZH

    Joined:
    Nov 12, 2018
    Posts:
    16
    After video is recorded, the sound will be noisy
     
  24. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Are you recording game audio or microphone audio?
     
  25. unityCZH

    unityCZH

    Joined:
    Nov 12, 2018
    Posts:
    16
  26. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
  27. unityCZH

    unityCZH

    Joined:
    Nov 12, 2018
    Posts:
    16
    private void StartMicrophone()
    {
    #if !UNITY_WEBGL || UNITY_EDITOR
    if (audioSource.clip == null)
    {
    audioSource.clip = Microphone.Start(null, false, ConstValues.RecordMaxTime, 48000);
    while (Microphone.GetPosition(null) <= 0) ;
    }
    audioSource.timeSamples = Microphone.GetPosition(null);
    audioSource.loop = true;
    audioSource.Play();
    #endif
    }
     
  28. TobiasW

    TobiasW

    Joined:
    Jun 18, 2011
    Posts:
    78
    I just tried it. The MiniCam example crashes in less than a second with a signal 11 (SIGSEGV) on my Sony Xperia Z3 Compact, even faster and much more reproducible than 2.0f2 did :(
     
  29. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Just as I thought. The glitchy audio you are experiencing is from Unity's Microphone API. That's why we made our own microphone API, NatMic.
     
  30. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Can you send me the full logs from logcat? At the very least, it should be a different thing that's causing the crash.
     
  31. QQMatt

    QQMatt

    Joined:
    Sep 29, 2016
    Posts:
    10
    Android Log

    The build works and produces an APKs, but they are not able to be installed on an actual device. I have included the logcat log when I try to install it, it looks like it fails to extract the native libraries (line 320). I'm going to try removing the proguard minify and see it that does anything, though I'm not sure at the moment, do I need to add something to my proguard user file for the plugin?

    Edit:
    Tried turning off minify and no change. Tried using internal build system instead of graddle and it installed, but crashes immediately.
     
    Last edited: Dec 10, 2018
  32. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    What device are you trying to install on? Have you tried building without splitting ABI's?
     
  33. QQMatt

    QQMatt

    Joined:
    Sep 29, 2016
    Posts:
    10
    If I turn of splitting it is now installing (though due to file size I'd prefer splitting), but does not actually work in the build. I tried as was and got this log.
    I also tried adding "-keep class com.yusufolokoba.natcorder.* { *; }" to the user proguard file, but that resulted in a crash when trying to record. Log

    I am installing it on Galaxy A5 (2017) running Android 8.0.0
     
  34. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Both crashes are happening because of missing classes. Are you using the Asset Store version of NatCorder (1.3f2)? The internal build system without splitting should work. As for proguard, you need to keep "com.yusufolokoba.natcorder.*" and "com.yusufolokoba.natcam.*" (and in the next update, "com.yusufolokoba.natrender.*").
     
  35. unityCZH

    unityCZH

    Joined:
    Nov 12, 2018
    Posts:
    16
    The combination of NatCorder and NatMic will slow down the synthesis speed of video. Do you have a good solution
     
  36. rroylance

    rroylance

    Joined:
    May 1, 2018
    Posts:
    8
    Hi @Lanre, assets seems to be working well so far except for an issue on Android. Video recording is working just fine, but when switch over to GIF recording it seems to be compressed really heavily. Nothing changed in my config except for changing the Container to GIF. Any idea why?

    Code (CSharp):
    1.     public void SetRecordingVideo(bool startRecording) {
    2.         if (startRecording) {
    3.             // Start recording
    4.             NatCorder.StartRecording(Container.MP4, VideoFormat.Screen, AudioFormat.None, OnRecording);
    5.             // Create a camera recorder for the main cam
    6.             cameraRecorder = CameraRecorder.Create(recordingCamera);
    7.         } else {
    8.             // Stop recording
    9.             cameraRecorder.Dispose();
    10.             NatCorder.StopRecording();
    11.         }
    12.     }
    13.  
    14.     public void SetRecordingGIF(bool startRecording) {
    15.         if (startRecording) {
    16.             // Start recording
    17.             NatCorder.StartRecording(Container.GIF, VideoFormat.Screen, AudioFormat.None, OnRecording);
    18.             // Create a camera recorder for the main cam
    19.             cameraRecorder = CameraRecorder.Create(recordingCamera);
    20.         } else {
    21.             // Stop recording
    22.             cameraRecorder.Dispose();
    23.             NatCorder.StopRecording();
    24.         }
    25.     }
     
  37. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Can you provide more information on this? How do you mean it will slow down the synthesis speed of video?
     
  38. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    On Android we use uniform quantization with a fixed palette and global color table. As a result, the GIF's don't look as high-quality as on other platforms. I'll look into whether we can move to a higher quality solution, but I can't guarantee that we will be implementing changes due to performance constraints.
     
    rroylance likes this.
  39. rroylance

    rroylance

    Joined:
    May 1, 2018
    Posts:
    8
    Thanks for the quick response!

    So it's supposed to be that way at this time. Good to know.

    Maybe you can put in full quality/ definable quality settings as an option; so users that only plan on supporting high end devices, for example, can tweak the settings for their chosen supported devices and get the best possible quality to performance they are ok with?
     
  40. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    We only have the one implementation in the NatCorder source code. The difference isn't small enough where you can choose a different method on high-end devices. A 'proper' quantization scheme like K-means could take hundreds of milliseconds for a given image (even on high-end devices). And since different parts of the GIF encoding pipeline already take a lot of time, it could mean that the user is left waiting for many seconds for the recording callback to get invoked--in other words, for recording to actually finish after StopRecording is called. I'll profile it to get hard values, but I don't expect it to be feasible enough to replace our current implementation.
     
    rroylance likes this.
  41. rroylance

    rroylance

    Joined:
    May 1, 2018
    Posts:
    8
    Really like the way you explain things and how much info you present, much better than many plugin devs...

    I completely understand your stance, and is good to know. I was only planning on video recording anyway, once I saw GIF support I just figured I'd give it a shot. Thanks!
     
  42. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Thank you :). Even though the sources are closed, I'm happy to provide any insight into the inner workings that'll be helpful.
    I'm definitely all for improving GIF recording performance on Android. Currently, there are bottlenecks littered all around the process going from a single frame to an encoded chunk on the file system. I've just been so swarmed with the more in-demand aspects of NatCorder (most notably, MP4 recording on the resource-constrained iOS and Android platforms) that I haven't been able to go back to the Android GIF stuff. The upcoming version of NatCorder should push performance on these platforms to the absolute best we can get it, so I can focus on other things.
     
  43. QQMatt

    QQMatt

    Joined:
    Sep 29, 2016
    Posts:
    10
    I am using the graddle build system (I found this more stable when distributed than internal) without splitting. I've attached my user proguard file and android log as I still get a crash when I try to use Natcorder. I am using version 1.3f2.

    Android Log
    User Proguard

    Edit 1: I did try and it seems to work with the internal build system. Which I can fallback if needed, though it's not ideal.

    Edit 2: It also works when I turn of minify. I did try removing "{ *; }" from the proguard file as well, just to make sure I was matching what you put exactly, but that did not work either. For the time being I'll likely use gradle with no minify, I assume I must be doing something wrong with my user proguard file.

    Edit 3: Targeting a single architecture(I tried ARM64 and then ARMv7) using gradle (no minify) fails to install, I think it is "Failed to extract native libraries". Tried ARM64 with the internal build and ARM64 + x86 to get the same problem. The only time I am able to install the apk is when I target all 3 architectures, I'm wondering if the issue is related to this?
     
    Last edited: Dec 12, 2018
  44. maxzerrr

    maxzerrr

    Joined:
    Jan 25, 2016
    Posts:
    5
    Hello,
    there is a very bad lag spike when I call StartRecording. It's much worse on Android! Is there a way to avoid this? Could I run it in a separate thread, so that the UI doesn't freeze?
     

    Attached Files:

  45. blacktest

    blacktest

    Joined:
    Mar 2, 2014
    Posts:
    3
    Code (CSharp):
    1. public void StartRecording()
    2.         {
    3.             tiempo = Time.time;
    4.             Debug.Log("tiempo1 " + tiempo);
    5.  
    6.  
    7.            
    8.             // Create recording configurations // Clamp video width to 720
    9.             var width = 720;
    10.             var height = width * Screen.height / Screen.width;
    11.             var framerate =30;
    12.             var videoFormat = new VideoFormat(width, (int)height, framerate);
    13.             var audioFormat = new AudioFormat(AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode);
    14.             // Create a recording clock for generating timestamps
    15.             recordingClock = new RealtimeClock();
    16.             // Start recording
    17.             NatCorder.StartRecording(Container.MP4, videoFormat, audioFormat, OnReplay);
    18.             videoRecorder = CameraRecorder.Create(Camera.main, recordingClock);
    19.             audioRecorder = AudioRecorder.Create(Camera.main.GetComponent<AudioListener>(), recordingClock);
    20.                    
    21.  
    22.         }
    23. void OnAudioFilterRead (float[] data, int channels)
    24.         {
    25.             Debug.Log("entra a grabar sonido");
    26.             // Check that we are recording
    27.             if (NatCorder.IsRecording)
    28.             {
    29.                 Debug.Log("entra a grabar sonido2");
    30.                 // Commit the frame
    31.                 NatCorder.CommitSamples(data, recordingClock.CurrentTimestamp);
    32.             }
    33.         }
    34.  
    35. public void StopRecording()
    36.         {
    37.                                
    38.             // Stop the recording
    39.             videoRecorder.Dispose();
    40.             audioRecorder.Dispose();
    41.             NatCorder.StopRecording();
    42. }
    Hi, I'm trying to get the audio from the augmented reality camera but the sound is not recorded. thanks
     
  46. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    I'm not too sure, since it doesn't seem to log exactly why it fails to extract the native libs. Did you try what was suggested in this link? You'll have to do it on your end (for building the application APK), not on ours (building the library AAR).
     
  47. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    You can call StartRecording and StopRecording from any arbitrary thread. Same goes for IsRecording and CommitSamples. CommitFrame though must be called from the Unity main thread.
     
  48. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    You are creating an AudioRecorder but also committing audio samples in OnAudioFilterRead. You have to pick one workflow instead of using both.
     
  49. QQMatt

    QQMatt

    Joined:
    Sep 29, 2016
    Posts:
    10
    A universal build targeting all 3 architectures works with both internal and gradle (without minify), which is useful as it means I can get a usable build. Ideally I would want to split the builds by cpu architecture is this an issue on my end, Unity's or Natcorder's?
     
  50. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,440
    Hey all!

    If you'd like to test the upcoming NatCorder 1.4.0 update, email me your invoice number.

    We want to pass around the current build of the upcoming update for performance and stability testing. We only have so many testing devices so we'd like this to be more community driven. This update introduces the most aggressive performance optimizations we'll ever be able to achieve on Android and iOS. NatCorder.CommitFrame will now take 0ms on the Unity main thread and the Unity render thread, so literally no slowing down when recording is started. I might publish a technical whitepaper explaining just what we've done. Anyway, here's the current changelog:

    NatCorder 1.4.0
    + Greatly improved recording stability and performance on iOS Metal and Android.
    + Dropped support for OpenGL ES on iOS. NatCorder will only use Metal on iOS. As a result, NatCorder now requires a minimum of iOS 8.
    + Added support for the new Light Weight Render Pipeline (LWRP) and High Definition Render Pipeline (HDRP) with `CameraRecorder`.
    + Update iOS and macOS backend to generate `.mp4` instead of `.mov` file when recording MP4.
    + Added `FixedIntervalClock` for generating timestamps to maintain a constant framerate in recorded videos.
    + Added aspect fitting in `CameraRecorder`. This means that your videos will not appear stretched in the case of app autorotation.
    + Added a dedicated GIF recording example called Giffy.
    + Updated `CameraRecorder.Create` to now take in the `VideoFormat` used to start recording.
    + Fixed `BufferOverflowException` when recording with audio on some Android devices.
    + Fixed `DllNotFoundException` when running on macOS.
    + Deprecated `VideoFormat.Screen` property. Manually create a video format with your intended resolution.
    + Deprecated `CameraRecorder.recordingMaterial` property. Use Image Effects instead.
    + Removed GIF recording from ReplayCam.

    ___

    We hope to add official support for linear rendering and greatly improve Android GIF recording performance in time for the actual Asset Store release. But this is currently in the air.