Search Unity

  1. New Unity Live Help updates. Check them out here!

    Dismiss Notice

NatCorder - Video Recording API

Discussion in 'Assets and Asset Store' started by Lanre, Nov 18, 2017.

  1. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Happy Holidays everyone! Thank you all for hopping on the NatCorder train so early. In 2018, we expect to build the API up to be the complete solution for everything video recording in Unity.
     
    teexiii likes this.
  2. gunner0304

    gunner0304

    Joined:
    Oct 20, 2017
    Posts:
    1
    Hello! i using everyplay my AR app for recording videos. but difficult to recording audio with mic......
    so i want to test your API.
     
  3. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    That sounds good. PM me your email address and I will share the beta with you.
     
  4. teexiii

    teexiii

    Joined:
    Oct 7, 2015
    Posts:
    14
    Look very potential, I'm curious about how it should work, can you share the beta to me too? Thanks in advanced.
     
  5. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Great! I have shared the beta with you.
     
    teexiii likes this.
  6. teexiii

    teexiii

    Joined:
    Oct 7, 2015
    Posts:
    14
    Just tested on my iPhone, works just great!

    But when I try to build macOS app, it showed some errors:
    upload_2017-12-27_12-3-52.png

    It's also not working on Unity Editor macOS as well, when I press Space, Unity was crashed, can't see the error messages...

    I'm doing some other tests on PC and Android, will keep you posted.

    ## Update 01:
    - Work great so far on my Android phone (XiaoMi 2), the video quality is good.
    - Still figuring out how to make it work on Unity Editor (macOS) and macOS app, no solutions yet.
    - I'm setting up microphone, let see if it can record voice properly, will update soon.
     
    Last edited: Dec 27, 2017
  7. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Excellent! The error on macOS is because in the OnReplay callback, NatCorder plays the recorded video with Handheld.PlayFullScreenMovie which seems to only exist on Android and iOS. For now, surround that line with #if UNITY_IOS || UNITY_ANDROID.
     
  8. matiasjaure

    matiasjaure

    Joined:
    May 27, 2011
    Posts:
    39
    Hi Lanre,

    I wasn't able to integrate the plugin with NatCam, I wanted to record the camera. There's an error because two libraries are colliding, I think you'd need to provide a separate package for this purpose.

    Regards,
    Matias
     
  9. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    This is probably because of the NatCam Render Pipeline (libNatCamRenderPipeline.a); both plugins depend on it. Delete a copy and you should be all good.
     
  10. matiasjaure

    matiasjaure

    Joined:
    May 27, 2011
    Posts:
    39
    But it's packed on a aar library, I tried deleting the content and repacking but didn't work. it's for Android
     
  11. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Oh I thought you were trying to run on iOS. On Android, the same thing applies. NatCorder has NatCorder.aar and NatCamRenderPipeline.aar. Delete the render pipeline aar as NatCam also has it. You should not modify any of the aar packages.
     
  12. matiasjaure

    matiasjaure

    Joined:
    May 27, 2011
    Posts:
    39
    I have the following error now when start recording:

    12-27 20:10:15.529: E/Unity(23788): AndroidJavaException: java.lang.NoClassDefFoundError: Failed resolution of: Lcom/yusufolokoba/natcorder/VideoEncoder;
    12-27 20:10:15.529: E/Unity(23788): java.lang.NoClassDefFoundError: Failed resolution of: Lcom/yusufolokoba/natcorder/VideoEncoder;
    12-27 20:10:15.529: E/Unity(23788): at com.yusufolokoba.natcorder.NatCorder.startRecording(NatCorder.java:39)
    12-27 20:10:15.529: E/Unity(23788): at com.unity3d.player.UnityPlayer.nativeRender(Native Method)
    12-27 20:10:15.529: E/Unity(23788): at com.unity3d.player.UnityPlayer.c(Unknown Source)
    12-27 20:10:15.529: E/Unity(23788): at com.unity3d.player.UnityPlayer$c$1.handleMessage(Unknown Source)
    12-27 20:10:15.529: E/Unity(23788): at android.os.Handler.dispatchMessage(Handler.java:98)
    12-27 20:10:15.529: E/Unity(23788): at android.os.Looper.loop(Looper.java:154)
    12-27 20:10:15.529: E/Unity(23788): at com.unity3d.player.UnityPlayer$c.run(Unknown Source)
    12-27 20:10:15.529: E/Unity(23788): Caused by: java.lang.ClassNotFoundException: com.yusufolokoba.natcorder.VideoEncoder
    12-27 20:10:15.529: E/Unity(23788): at java.lang.VMClassLoader.findLoadedClass(Native Method)
    12-27 20:10:15.529: E/Unity(23788): at java.lang.ClassLoader.findLoadedClass(ClassLoader.java:742)
    12-27 20:10:15.529: E/Unity(23788): at java.lang.ClassLoader.loadClass(ClassLoader.java:362)
    12-27 20:10:15.529: E/Unity(23788): at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
    12-27 20:10:15.529: E/Unity(23788): ... 7 more
    12-27 20:10:15.529: E/Unity(23788): Caused by: java.lang.NoClassDefFoundError: Failed resolu
     
    Last edited: Dec 27, 2017
  13. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Did you delete NatCorder.aar? Try reimporting the NatCorder unitypackage and only deleting NatCamRenderPipeline.aar. That's a weird error (the class is in the same package as NatCorder.java which seems to be what fails to load it).
     
  14. matiasjaure

    matiasjaure

    Joined:
    May 27, 2011
    Posts:
    39
    Yes, it's there.
    Screen Shot 2017-12-27 at 22.24.11.png
     
  15. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Just noticed; you are using an older version of NatCam. You should upgrade to 1.6. You shouldn't delete NatCamRenderPipeline.aar from NatCorder since your version of NatCam doesn't have it. Regarding the ClassNotFoundException, I am not sure what could be causing it since the class that triggers it and the class that is apparently missing are both in the same package. Try opening a blank project, loading just NatCorder, and running the RecordScreen sample. Let me know if the error persists.
     
  16. teexiii

    teexiii

    Joined:
    Oct 7, 2015
    Posts:
    14
    Yea, I figured it out already. However, I am wondering where is the default path of saved video on Unity Editor or macOS app?

    By the way, here is my another update:
    - After hours of scratching my head, I can get the microphone to be recorded on my Android phone, the funny thing is when I set the volume to 1.0f, the app will yell very loud with a lot of noise & echo, but when I set the volume back to 0.1f, the video can be recorded well.
    - However, when recording, you can hear your voice broadcast clearly from the device, I'm still trying to get rid of it. Setting microphone muted or volume 0.0f won't help.

    I'm building another version for iOS, let see if the microphone are recorded as well. Will keep you posted.

    Cheers,
     
  17. teexiii

    teexiii

    Joined:
    Oct 7, 2015
    Posts:
    14
    Ok, another issue, the first time I record the screen with audio, it worked just fine. But when I record the second time, the result video will be frozen at the first frame & the recorded audio start playing, after the audio finished, the video start to play in silence.
     
  18. matiasjaure

    matiasjaure

    Joined:
    May 27, 2011
    Posts:
    39
    Ok now it starts recording but then it starts giving errors dozens times and crashed:

    12-28 09:33:21.896: E/Camera3-OutputStream(900): getBufferLocked: Stream 0: Can't dequeue next output buffer: Broken pipe (-32)
    12-28 09:33:21.896: E/Camera3-Device(900): RequestThread: Can't get output buffer, skipping request: Broken pipe (-32)
    12-28 09:33:21.896: W/Camera2Client(900): notifyError: Received recoverable error 3 from HAL - ignoring, requestId 10000001

    12-28 09:33:21.897: E/Camera3-OutputStream(900): getBufferLocked: Stream 0: Can't dequeue next output buffer: Broken pipe (-32)
    12-28 09:33:21.897: E/Camera3-Device(900): RequestThread: Can't get output buffer, skipping request: Broken pipe (-32)
    12-28 09:33:21.897: E/Surface(900): queueBuffer: error queuing buffer to SurfaceTexture, -32
    12-28 09:33:21.897: E/Camera3-OutputStream(900): returnBufferCheckedLocked: Stream 0: Error queueing buffer to native window: Broken pipe (-32)
    12-28 09:33:21.897: E/Camera3-Device(900): Can't return buffer to its stream: Broken pipe (-32)
    12-28 09:33:21.901: E/Camera2Client(900): notifyError: Error condition 0 reported by HAL, requestId -1
    12-28 09:33:21.899: W/Camera2Client(900): notifyError: Received recoverable error 3 from HAL - ignoring, requestId 10000001
     
  19. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    In the editor, it saves to the project directory (one level up from the Assets directory). On standalone builds, it saves to the directory containing the executable. We are considering moving it to Application.persistentDataPath. Which would you prefer?
    That sounds like a Unity bug. I'm not sure what could be causing this.
    Firstly, are you passing in an AudioSource or an AudioListener to NatCorder? If you are recording only the microphone, then you should use the AudioSource overload (and stream microphone audio through it).

    You've shown a feature that the API should have. When recording from an audio source, there should be a flag for if the audio source is for recording only. If the flag is set to true, then NatCorder will mute the source after it receives the audio samples in the filter callback. This way, NatCorder can record the audio source, but the audio source won't actually contribute to the game audio (preventing you from hearing your voice :) ).
    We already know of this issue; we are tracking it.
     
    tik_tikkers and teexiii like this.
  20. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Please email me the full logs from logcat as a .txt attachment. Also, please include your code.
     
  21. ajsh

    ajsh

    Joined:
    Jan 7, 2017
    Posts:
    1
    wow, this is what i'm looking for the whole year !! I'm so curious about it.

    I have found a tutorial that using Intel INDE but its no longer supported.
    Hope it will became a very best solution for video recording on unity.

    Can you share me your beta? Can't wait to test your awesome plugin ;).
     
    Lanre likes this.
  22. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Hey guys. We have big news:

    NatCorder will include a WebGL backend for recording videos (with optional audio) in WebGL games.

    We have alphas with video recording working, and will be integrating audio recording soon. I expect a new beta with this backend to be up within the next two weeks. NatCorder is the only API for Unity that supports recording on WebGL.
     
    Last edited: Dec 28, 2017
    teexiii likes this.
  23. teexiii

    teexiii

    Joined:
    Oct 7, 2015
    Posts:
    14
    Well, I guess it would be better if we can configure where the recorded video should be saved to, wouldn't it?

    I think it shouldn't broadcast the microphone clip, at the end of the recording, we already have the stream of recorded audio, the thing is how to merge it with the recorded video? ffmpeg maybe?

    ## Test on iOS App:
    Another issue on iOS is when I playback the recorded video, some frames were flipped in vertical, not sure why, I'm still debugging...
     
  24. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Perhaps, but this is not compliant with the INatCorder spec (on platforms like iOS and Android, there might be more considerations). We've changed the Standalone backend to save to Application.persistentDataPath.
    I have added a flag in the Replay.StartRecording overload that takes an AudioSource to allow the user specify whether the audio source is for recording only. When this flag is set to true, then the audio generated by it will be recorded, but will not generate sound in the Unity application.

    The Standalone backend does not support recording audio. On platforms that do support audio (iOS, Android, WebGL), audio is encoded with video in realtime, not deferred and muxed.
    What graphics API are you running on, Metal? Also, do you have antialiasing enabled? Another dev has reported this issue to occur when antialiasing is enabled in Quality Settings.
     
    teexiii likes this.
  25. tcmeric

    tcmeric

    Joined:
    Dec 21, 2016
    Posts:
    182
    I would love to join the Beta. If things work out, I would be interested in helping to add playmaker support (I have written playmaker code for more than a few dozen other assets, including textmesh pro, VRTK, post processing stack, etc).
     
    Lanre likes this.
  26. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    That sounds great! Please PM me your email address and I will share the beta with you.
     
  27. HonorableDaniel

    HonorableDaniel

    Joined:
    Feb 28, 2007
    Posts:
    2,916
    Does this plugin support custom frame rates (e.g. 4k @ 60 FPS) for rendering a scene out?
     
  28. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Yes, NatCorder allows you to specify a resolution and framerate (and other parameters like bitrate) for recording. I should note that mobile devices do not support recording resolutions as high as 4K due to hardware limitations. The safest high resolution for recording will be 1080p on most mobile devices around today.
     
    HonorableDaniel likes this.
  29. HonorableDaniel

    HonorableDaniel

    Joined:
    Feb 28, 2007
    Posts:
    2,916
    I’ve been looking for a good android recorder. I’ll be sure and check this out further.
     
    Lanre likes this.
  30. pjccccc

    pjccccc

    Joined:
    Oct 7, 2015
    Posts:
    43
    I'm currently testing your NatCorder, and now I have two suggestions.

    1. Names are little confusing for begginers. Both classes have same API (NatCam.StartRecording & NatCorder.StartRecording)
    But it's a just a little thing.

    2. Videos saved in root directory on Android. So I have to copy and remove previous one.
    I want to specify path at 'StartRecording' or somewhere. (at least saving in temporary directory, I think nobody wants to mess up his own device.)
     
  31. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Both NatCam and NatCorder offer recording functionality, with the difference being that NatCam records only the camera whereas NatCorder can record anything (including the NatCam preview). NatCam and NatCorder serve different purposes.
    We've fixed this; now, videos are recorded to the app's documents directory (away from plain sight).
     
  32. mitchtbh

    mitchtbh

    Joined:
    Dec 8, 2013
    Posts:
    66
    I’ve been looking for a screen recording asset for my AR app so that I can record up to 30 seconds of video and save it to the gallery. This looks promising - would love to give it a try and see how stable it is!
     
  33. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    NatCorder sounds perfect for what you want to do. I should note that NatCorder doesn't come with a way to save to the gallery. We will consider adding this in. PM me your email address and I will share the beta with you.
     
    Blarp likes this.
  34. mvhgroup

    mvhgroup

    Joined:
    May 18, 2017
    Posts:
    1
    We are also looking for an alternative to Everyplay. Is it possible to check out the NatCorder? Sounds awesome!
     
  35. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Totally! PM me your email address and I'll share the beta with you.
     
  36. henriqueranj

    henriqueranj

    Joined:
    Feb 18, 2016
    Posts:
    144
    Hello @Lanre , is it possible to join the Beta testers for the NatCorder API? I have sent you a PM.
     
  37. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Seen and responded!
     
  38. Archviz3d

    Archviz3d

    Joined:
    Apr 4, 2016
    Posts:
    92
    Hi Lanre,

    Sorry for the delay!! But with all these holydays, just lost track of this!

    So, this is happening with ios, if you enable msaa in the quality settings, and of course are using foward rendering and metal.

    This is also the important part to reproduce the bug:
    you have to have two cameras one for rendering the cube , and another with depth for rendering just the ui, so i can record what the first camera sees but not the UI.

    Try to record for 30 seconds, and then reproduce the video, you will see that after 15 seconds it will start to stutter...

    Could you reproduce it?

    Still didnt try with post processing effects...
     
  39. teexiii

    teexiii

    Joined:
    Oct 7, 2015
    Posts:
    14
    That's exactly what happened over here!
     
    Lanre likes this.
  40. tcmeric

    tcmeric

    Joined:
    Dec 21, 2016
    Posts:
    182
    Relative scripting noob here. Is there a way to check if the path has a valid video file before replay? Or at least a null check?

    Also, what does this bool control?

    NatCorder.Verbose = true;
     
  41. tcmeric

    tcmeric

    Joined:
    Dec 21, 2016
    Posts:
    182
    Here are first few actions for playmaker. So far running well in unity 2017.3 with a build to android 18. I will be making some more for more options.

    https://github.com/dumbgamedev/natCorder

    Includes:
    Start Recording
    Stop Recording
    Playback
    Pause
    Resume
    Check is Recording
    Check is Paused
    Share
    Get Thumbnail Texture





    The code is under MIT, so feel free to use, share, modify, etc.
     
    Last edited: Jan 5, 2018
    Blarp and Lanre like this.
  42. mrmdesign

    mrmdesign

    Joined:
    Apr 18, 2013
    Posts:
    239
    This looks fantastic! Can I still get in on the Beta? I will be testing this on a few AR apps I am working on.
     
    Lanre likes this.
  43. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,203
    i'm curious.... what's the advantage of this over Everyplay from unity? they also record the screen/audio and your face if needed and share all that at a very good fps.
     
  44. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    Thanks for the very detailed information. I'll be working on reproducing it on Monday. I'll email you with my findings or any questions.
     
  45. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    The path returned by NatCorder is guaranteed to be valid. If NatCorder fails to record a video, it will not invoke the save callback. So no need to check for nulls or read-/write-ability.
    This is excellent! Thank you so much. I'll open a pull request in a few days with some changes. We've made some slight updates to the front end API.
     
  46. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    I've responded to your PM. The beta's still open!
     
  47. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,969
    NatCorder has a number of advantages. I go over them in better detail in the Medium post, but I'll quickly reiterate some of them:
    • You can record anything. With NatCorder, you aren't limited to recording the screen. You can record textures, cameras, the screen, or anything that can be put into a RenderTexture.
    • You can record at custom resolutions. With NatCorder, you provide the resolution that you want the video to be recorded at. Hence, you can subsample or supersample whatever you are recording, for free.
    • You can control recording quality because you can provide the recording bitrate and keyframe interval.
    • You are given direct access to the recorded video in device storage. Everyplay absolutely does not do this, and it has been a main frustration with a lot of developers including myself.
    • It is insanely fast. Sharing the same multithreaded rendering pipeline we developed for NatCam, NatCorder will contribute exactly 0ms to your game's frametime.
    • It is very easy to use. The front-end API's are super minimal and straight-to-the-point. You can literally record an entire video with only two calls to NatCorder, as shown in the Medium post. Everyplay cannot boast such simplicity.
    • It is very lightweight. Currently, we have the beta unitypackage at < 900KB.
    • You can record in the Editor. Version 1.0 is coming with an experimental Standalone backend that uses FFmpeg to record.
    • You can record on WebGL. We are currently adding an experimental WebGL backend. I'm pretty excited about this one specifically.
    • You get much much better support. As NatCam (the other API I have worked on) developers know, we provide instant support. Unity, on the other hand, is sluggish at best.
    The only big difference that sets Everyplay apart is that it also has a larger replay sharing/social media network going. This is completely out of the scope of NatCorder. When it comes to simply recording video in Unity, NatCorder is your best option.
     
    bgmulti15a and henriqueranj like this.
  48. creat327

    creat327

    Joined:
    Mar 19, 2009
    Posts:
    1,203
    so 3 things:
    1) Can you also record on windows/windows universal and mac? because that's the main thing missing from everyplay
    2) can you share your videos to different platforms and can you also force it to share to a specific channel? (like the game youtube channel)
    3) can you record front face camera? (so we can place the face of the player on the video) as a different quad so we can also move it around?
     
  49. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    199
    AR recording
    Automatic video sharing to a central youtube profile (dont ask the user)

    These are the two things im super interested in.

    Would love this
     
  50. pjccccc

    pjccccc

    Joined:
    Oct 7, 2015
    Posts:
    43
    I want to start recording with 'AudioClip'.
    Current version of `StartRecording` only accepts `AudioSource` and `AudioListener`.

    or.. I just want to record video with Microphone API.
     
unityunity