Search Unity

NatCorder - Video Recording API

Discussion in 'Assets and Asset Store' started by Lanre, Nov 18, 2017.

  1. edee1337

    edee1337

    Joined:
    Apr 10, 2013
    Posts:
    30
    Just want to add that I'm getting the app freeze on on 2018.4.4
     
  2. NicoWakatoon

    NicoWakatoon

    Joined:
    Jul 30, 2015
    Posts:
    2
    I ended up defining my own "stub" in .mm file for iOS:

    Code (CSharp):
    1. #if !CGFLOAT_IS_DOUBLE
    2. extern "C" {
    3.  
    4. void NCEncodeFrame(){}
    5. void NCEncodeSamples(){}
    6. void NCStartRecording(){}
    7. void NCStopRecording(){}
    8. void NCCreateGIFRecorder(){}
    9. void NCCreateHEVCRecorder(){}
    10. void NCCreateMP4Recorder(){}
    11.  
    12. }
    13.  
    14. #endif
    15.  
    And check for possible natcorder support in C# :

    Code (CSharp):
    1.  
    2.     public static bool is64bits
    3.     {
    4.         get
    5.         {
    6.             return System.IntPtr.Size > 4;
    7.         }
    8.     }
    9.  
    10.     public static bool isVideoRecordingSupported
    11.     {
    12.         get
    13.         {
    14.  
    15. #if UNITY_IPHONE && !UNITY_EDITOR
    16.             if (SystemInfo.graphicsDeviceType != GraphicsDeviceType.Metal){
    17.                 return false;
    18.             }
    19.  
    20.             if(!is64bits){
    21.                 return false;
    22.             }
    23.          
    24.             var ver = UnityEngine.iOS.Device.systemVersion;
    25.             string[] v = ver.Split('.');
    26.             var major = v[0];
    27.             int maji = 9;
    28.             int.TryParse(major,out maji);
    29.             if(maji < 11 ){
    30.                 Debug.Log("Video Recording disabled iOS < 11 Detected version = " + maji);
    31.                 return false;
    32.             }
    33. #endif
    34.  
    35.             return true;
    36.         }
    37.     }
    38.  
    39.  
     
  3. CNGameDev01

    CNGameDev01

    Joined:
    Feb 1, 2016
    Posts:
    4
    Hi, we've upgraded from 1.6.2 to 1.6.3. On our android build test, the recorded video color is significantly darker than normal. Is there any way to solve this?
    (see attached videos for comparison)
    Note:
    - 1.6.2 works fine on android and provides correct color, but recorded with inverted color in ios (the reason why we update to 1.6.4).
     

    Attached Files:

    cparki77 likes this.
  4. CNGameDev01

    CNGameDev01

    Joined:
    Feb 1, 2016
    Posts:
    4
    Correction: we upgraded from 1.6.2 to 1.6.4 (latest)
     
  5. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    Sorry for the late response. We can't compile NatCorder with an iOS deployment target lower than iOS 11. So defining custom stubs is your best option.
     
  6. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    Make sure that your app doesn't use linear rendering. NatCorder doesn't support linear rendering on mobile platforms.
     
  7. sucko

    sucko

    Joined:
    Jun 17, 2016
    Posts:
    5
    I updated NatCorder to version 1.6.4. and I'm having troubles on IOS (>11.0).
    The Application crashes after recording the clip, when Handheld.PlayFullScreenMovie("file://"+path); is invoked in the OnReplay() function. (in ReplayCam.cs)

    This worked fine in 1.6.3.


    Here is the error message from XCode:

    2019-10-29 17:54:55.887689+0100 RabbitEye[1289:363307] NatCorder: MP4 recorder finishing
    Saving to Pictures: RabbitEye_2019_10_29_17_54_55_911.mp4
    NativeGallery:SaveToGalleryInternal(String, String, MediaType, MediaSaveCallback)
    NativeGallery:SaveToGallery(String, String, String, MediaType, MediaSaveCallback)
    NatCorder.CamRecorder:OnPreview(String)
    System.Action`1:Invoke(T)
    NatCorder.Internal.<>c__DisplayClass16_0:<OnRecording>b__0()
    System.Action:Invoke()
    NatCorder.Internal.<Dispatch>d__5:MoveNext()
    UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
    (Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)



    2019-10-29 17:54:59.356295+0100 RabbitEye[1289:363307] [core] "Error returned from daemon: Error Domain=com.apple.accounts Code=7 "(null)""

    2019-10-29 17:54:59.364108+0100 RabbitEye[1289:363307] [Common] _BSMachError: port 11a03; (os/kern) invalid capability (0x14) "Unable to insert COPY_SEND"

    2019-10-29 17:54:59.364948+0100 RabbitEye[1289:363307] [Common] _BSMachError: port 11503; (os/kern) invalid capability (0x14) "Unable to insert COPY_SEND"

    2019-10-29 17:54:59.365053+0100 RabbitEye[1289:363307] [Common] _BSMachError: port 11503; (os/kern) invalid capability (0x14) "Unable to insert COPY_SEND"

    -> applicationWillResignActive()
     
  8. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    Can you post a screenshot of the Xcode window when the crash occurs?
     
  9. The_Mean_Fiddler

    The_Mean_Fiddler

    Joined:
    Nov 27, 2014
    Posts:
    13
    Hi, Natcorder 1.6.4 works really well on Android. Two questions please:

    1. If the user kills the game while it's recording, Does the Media recorder stay in memory?

    2. I can't call MediaRecorder.Dispose() in the OnDestroy() function because the recording gets saved and I miss the callback. I end up with a lot of old recordings in the directory. That's OK I can workaround it, but am I missing something?
     
  10. Yode_group

    Yode_group

    Joined:
    Apr 5, 2016
    Posts:
    6
    Hi, please tell me how we can compress a video by using your asset?

    What are the compression options?
     
  11. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    When the user kills a process, everything gets closed. There's no way to keep anything in memory.
    You'll have to engineer your app so that this is not a problem. One suggestion would be keeping a manager that doesn't get destroyed, so it can always receive the recording callback.
     
    The_Mean_Fiddler likes this.
  12. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    The MP4 and HEVC recorders allow you to specify the recording bitrate and keyframe interval.
     
  13. visualmusicdesign

    visualmusicdesign

    Joined:
    Oct 16, 2019
    Posts:
    4
    Sorry - I thought I had deleted the post this one is replying to. I updated my previous post with a code solution implementing saving after recording. The original ReplayCam code callback was indeed written to only open the file in the default application when using the Editor, not when running in standalone.

    Thanks!
     
    Lanre likes this.
  14. sucko

    sucko

    Joined:
    Jun 17, 2016
    Posts:
    5
    (Un)fortunately I cannot replicate that bug, the application is working fine now, it could have been some different issue due to working on collab, I can't really tell what it was that caused the bug. If I run into it again, I will post a screenshot. For now thanks for your effort.
     
    Lanre likes this.
  15. jowild25

    jowild25

    Joined:
    Nov 14, 2012
    Posts:
    3
    Hi,

    great plugin. I want to record multiple cameras. I am setting up my cameraInput with multiple cameras. But how do I save the multiple recordings? On the recordingCallback is saving only the last camera. Can I iterate through the recordings on recordingCallback somehow?
     
  16. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    You need to create multiple recorders for this. You have to check if your target device supports having multiple recorders; most Android devices don't support this, and this might also be the case on iOS.
     
  17. wechat_og5za0tFFf2J8t1HshoEiwPgzFfk

    wechat_og5za0tFFf2J8t1HshoEiwPgzFfk

    Joined:
    Feb 24, 2018
    Posts:
    1
    How to solve the problem of black screen and flower screen when I use Demo on android device
     
  18. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    Make sure that your app is not using Vulkan for rendering.
     
  19. Araj-Tejani

    Araj-Tejani

    Joined:
    Sep 25, 2015
    Posts:
    4
     

    Attached Files:

  20. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    These logs don't contain a lot of information. Can you take the logs from app start?
     
  21. thesanketkale

    thesanketkale

    Joined:
    Dec 14, 2016
    Posts:
    26
    Hi Lanre,
    Did you happen to solve this?
     
  22. hromhromhrom

    hromhromhrom

    Joined:
    Oct 29, 2019
    Posts:
    1
  23. Araj-Tejani

    Araj-Tejani

    Joined:
    Sep 25, 2015
    Posts:
    4
     

    Attached Files:

  24. rensiyao

    rensiyao

    Joined:
    Nov 6, 2019
    Posts:
    3
    我使用了这个插件,该插件通常可以在PC和android上录制和播放。但是,当我将其上传到ar眼镜(morningstar G2)时,可以看到录像预览框,但在阅读时视频是黑色的,这是眼镜的兼容性还是我的设置有问题?希望及时回答!

    您能给我一种设置和录制音频的确切方法吗?
     
  25. kkiruk

    kkiruk

    Joined:
    Sep 7, 2015
    Posts:
    6
    I'm using
    Code (CSharp):
    1. videoRecorder.preserveAspectRatio = false;
    at 1.3f version.

    Do the ver. 1.6.4 have same function?
     
  26. rensiyao

    rensiyao

    Joined:
    Nov 6, 2019
    Posts:
    3
    Hello, thank you very much for your reply, but I did not find the attribute you mentioned
     
  27. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    I got your email. I haven't been able to just yet. I'll prioritize it for the rest of this week/next week.
     
  28. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    These logs have no mention of NatCorder (or Unity). You can check the logs to make sure that there are mentions of Unity or NatCorder before sending them.
     
  29. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    Your AR glasses probably don't support recording.
    Check the README and let me know if you have further questions.
     
  30. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    No, 1.6.4 doesn't. You have to make sure that your game displays properly on 'screens' of different aspect ratios.
     
  31. sergioeith

    sergioeith

    Joined:
    Nov 8, 2019
    Posts:
    3
    Hello I'm really new on this Ive bought the natcorder hoping that I can film ar with my note 10 I've instal the asset but cant find a way to make it work. Any help would be appreciated

    Thanks
     
  32. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    Can you be more descriptive? How are you not able to make it work?
     
  33. sergioeith

    sergioeith

    Joined:
    Nov 8, 2019
    Posts:
    3
    When I create the APK file there's not record or other option to record videos in vuforia app on Android. I bought the app thinking that I will have an option or button start and stop recording option on my phone when I'm using it to to see some augmented reality trough my android phone.
     
  34. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    A recording button doesn't magically appear. You have to write code to start and stop recording, then tie it up to your UI. You can follow the ReplayCam example for guidance.
     
  35. sergioeith

    sergioeith

    Joined:
    Nov 8, 2019
    Posts:
    3
    I'm really noob on this what and where I have to paste the code paying 60$ for an app and still have to write the . I'm just a 3d graffiti writer trying to bring something crazy to the game .. but code its to wild for my brain
     
  36. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    I recommend following the README and included examples for guidance. NatCorder is an Application Programming Interface (API), which means that it is entirely code, and is meant to be used by writing your own code. Let me know if you have other specific questions.
     
  37. Pixel2015

    Pixel2015

    Joined:
    Mar 10, 2016
    Posts:
    7
    Hi in the description you say it is possible to record the whole screen instead of just a camera, how can this be achieved? There is no example of doing this, as we need to get the stuff that is in canvas - overlay to be captured as well.
     
  38. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    Overlay canvases cannot be recorded as they are outside the traditional Unity render pipeline (offscreen cameras and render textures can't see them, so we can't record them). This is mentioned in the README.
     
  39. Pixel2015

    Pixel2015

    Joined:
    Mar 10, 2016
    Posts:
    7
    Ok no problem, just don't say it can render camera as well as the screen in the description just say it can render the camera only. BTW I found a way how to record it anyway in latest Unity.
     
  40. olli-3dbear

    olli-3dbear

    Joined:
    Sep 14, 2016
    Posts:
    7
    Hi Lanre! Thanks for making an awesome suite of recording tools. There’s two iOS issues we’ve been running into for a couple of weeks now. Today I managed to sit down and made a simple repro for them.

    We’ve seen both of these issues since iOS 13 was released, so my spider sense tells me that it might be related to that. I tested this clean project with an iPad (specs below), but we’ve seen both of these with a wide variety of iOS devices running iOS 13 in our production app.

    

Issue 1 I'm also experiencing the recording freeze issue mentioned by at least Packedbox and edee1337. Steps to reproduce (Unity 2018.4.8f1):

    1. Start with NatMicCorder as the base project https://github.com/natsuite/NatMicCorder-Demo
    2. Import latest NatMic, NatCam and NatCorder from Asset Store
    3. Clone the 'payload-api' branch from https://github.com/natsuite/NatShare-API/tree/payload-api
    4. Copy the Assets/NatShare folder into the NatMicCorder project Assets folder
    5. Fix compilation error in NatMicCorder.cs:69 (should be

    Code (CSharp):
    1. aspectFitter.aspectRatio = previewTexture.width / (float)previewTexture.height;
    )
    6. Build app for iOS (I'm running iOS 13.1.3, iPad Pro 9,7", model MLMP2KN/A)
    7. Run the app and try recording.

    Expected behaviour: I can monitor the camera view while the video being recorded
    Actual behaviour: The UI freezes until I let go of the recording button. The resulting video file is as expected.

    Attached:
    An UI screen capture showing the issue: https://www.dropbox.com/s/lpla87ymj6biz1o/NatCorder Issue1 UI Freeze Screencapture.mp4?dl=1
The recorded video showing that video capture still works correctly https://www.dropbox.com/s/0uqrd2ff88sdjh7/NatCorder Issue1 Recorded Video.mp4?dl=1

    Issue 2 Another (more minor) bug:

    Using the same project as above:

    1. Build for iOS (same Unity version & iPad as above)
    2. Set volume to 0 by repeatedly pressing volume down
    3. Open app and do a recording
    4. Check volume level (for example by pressing volume up or down once)

    Expected behaviour: The volume level stays at 0.
    Actual behaviour: The volume gets reset to 50%.

    Attached:
    An UI screen capture showing the issue (see the volume bar at the top of the screen): https://www.dropbox.com/s/14bq76fq5olnosr/NatCorder Issue2 Volume Screencapture.MP4?dl=1

    For both of these, the Xcode logs were pretty uninteresting. You can find a copy of the logs for a run where I record a short video and both of these issues are apparent here: https://www.dropbox.com/s/dlkpylbmy6l793l/logs.txt?dl=1
     
    edee1337 likes this.
  41. edee1337

    edee1337

    Joined:
    Apr 10, 2013
    Posts:
    30
    I'm also seeing volume set to 50% in my app. Wasn't sure what to pin it on, glad I'm not losing my mind! :)
     
    Lanre likes this.
  42. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    It can render the screen, as we illustrate in the ReplayCam example. The constraint from Unity Engine is that we can't record overlay canvases, so we change the canvas render mode to screen space camera.
     
  43. anthonyjamesgirdler

    anthonyjamesgirdler

    Joined:
    Mar 15, 2018
    Posts:
    8
    Hello,
    I want to make a mobile phone app that uses the phones camera to record video.
    I then want to analyse the video by choosing frames.
    Can NatCord help with that ?
     
  44. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    This is a bug in Unity 2018.4. It is apparently fixed in Unity 2019.2. Interestingly enough, I don't encounter it on Unity 2018.3.
    NatCorder doesn't do anything with the app volume, so it's likely because of something else (video playback? microphone? iOS itself?).
     
  45. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    This is possible with NatCorder. The included examples already show how to record the camera preview using WebCamTexture.
    I don't know what "analyse the video by choosing frames" means, and nevertheless it sounds out of scope for NatCorder. NatCorder does not perform any video editing or compositing functionality; it only records to media files.
     
  46. edee1337

    edee1337

    Joined:
    Apr 10, 2013
    Posts:
    30
    Darn, trying to stay on 2018.4 as long as possible since its the newest version where AR remoting still works. Looks like I'll stick with NatCorder 1.6.3 for now. Good to know a fix is out there

    I'm using NatMic and https://github.com/cbaltzer/UnitySpeakerFix. Looking through the source for UnitySpeakerFix I don't see it changing the system volume. Is NatMic doing anything?

    Really appreciate the hard work!
     
  47. olli-3dbear

    olli-3dbear

    Joined:
    Sep 14, 2016
    Posts:
    7
    I also did a build on Unity 2019.2.12 and kept running into shader compilation issues & pure magenta screen on iOS builds. Probably an Unity side issue rather than NatCorder. I'll keep trying different combinations, but seeing as the LFS version is 2018.4, you might want to consider actively supporting it.

    Without seeing the NatCorder native side sources, it's very difficult for me to track this down. I've narrowed it down to the start of recording. Before starting to record the volume stays at 0. After starting to record and before stopping, the volume has changed.

The recording button simply calls this code:

    Code (CSharp):
    1.     public void StartRecording () {
    2.         var sampleRate = 44100;
    3.         var channelCount = 1;
    4.         // Start recording from the main camera
    5.         recordingClock = new RealtimeClock();
    6.         videoRecorder = new MP4Recorder(videoWidth, videoHeight, 30, sampleRate, channelCount, OnRecording);
    7.         cameraInput = new CameraInput(videoRecorder, recordingClock, Camera.main);
    8.         // Start the microphone
    9.         audioDevice = AudioDevice.GetDevices()[0];
    10.         audioDevice.StartRecording(sampleRate, channelCount, this);
    11.     }
    12.  
    If I had to guess, you most likely start or change the current AVCaptureSession when starting to record. I’d be happy to dig deeper if you’re willing to privately share the iOS native source code.
     
  48. anthonyjamesgirdler

    anthonyjamesgirdler

    Joined:
    Mar 15, 2018
    Posts:
    8
    Maybe frames is the wrong term.
    I want the user to choose 2 points in the video and I want the app to measure the time in between.
    For example they choose 3.9654s and 5.9654s. I want to measure the time between the two chosen points. 5.9654 - 3.9654 = 2s of elapsed time.
     
  49. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    We are considering making the minimum Unity requirement 2019.1. Partly because of this iOS issue, and because there is a new API that allows recording the screen as-is (without manually rendering game cameras and dealing with all the intricacies of doing so). We expect that lots of developers would appreciate a plain ScreenInput recorder input, as somewhat of a hands-off version of CameraInput.
    When AudioDevice.GetDevices() is called, NatMic asks iOS to default to speaker (similar to what UnitySpeakerFix does). Neither NatMic not UnitySpeakerFix does anything regarding volume. iOS on the other hand is free to do whatever it wants, which is why I strongly believe iOS is resetting the volume somewhere.
     
  50. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,666
    Yup definitely a Unity issue here. I'm not sure what Unity's policy is on retroactive bug fixes on LTS, but it might be worth filing a bug report to see if they add a fix there. This bug is entirely out of scope for NatCorder; the problem arises when we render a game camera to a RenderTexture at the end of an application frame (see CameraInput.cs) then perform a readback so we can commit the frame for recording, all using Unity's API's. If you're able to help me test some mods that might fix it, shoot me an email. I'm not able to reproduce it on 2018.4.