Search Unity

NatCorder - Video Recording API

Discussion in 'Assets and Asset Store' started by Lanre, Nov 18, 2017.

  1. lordzeon

    lordzeon

    Joined:
    Jun 1, 2018
    Posts:
    25
    i understand, but fast cinematic could be possible, like a inverse bullet time, depending on current FPS to match the video fps, for example, if the app is running at 120fps, and i want a video in 30 fps, playing the cinematic at 4x the speed (10 seconds played in 2.5sec), in that case, the capture would be ok? the question is, the recording could be configured to record based on frames or it will always record based on time?
     
  2. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    You mentioned wanting to run your app on iOS and Android. This eliminates the possibility of rendering your app at anything higher than 60FPS. NatCorder isn't constrained to record on app frames or time intervals; you can record in a for-loop if you so desired. The challenge is that to record the app, you are constrained to how fast the app can render things.
     
  3. Pirax

    Pirax

    Joined:
    Apr 23, 2014
    Posts:
    2
    Hi!

    I'm trying to recreate exactly that but unfortunately I'm failing and could use some help.

    I have a camera set up and recording my AR scene with UI on top of that. I'm using CameraInput and this works fine. UI is not being recorder but I've seen your comment about that so that's fine. But - while trying to recreate the setup that you described I've added another camera to the scene and an image that I want to use as a watermark.
    Apart from that I've updated CameraInput code to:
    cameraInput = new CameraInput(videoRecorder, recordingClock, Camera.allCameras);

    to record all cameras on the scene.
    Now, in the second camera's preview my watermark is visible, but it's not included in a recorder video.
    What am I missing? Should I set the camera or my watermark in any special way? Or maybe Camera.allCameras is not the way to go?

    Thank you in advance!
     
  4. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    If you watermark camera is looking at a UI canvas, then you need to make sure that the UI canvas can get recorded. Make sure that the canvas render mode is not Overlay.
     
  5. Pirax

    Pirax

    Joined:
    Apr 23, 2014
    Posts:
    2
    Hmm... Right now my canvas is set to Screen Space - Camera, but I tried all modes to be hones.
    My camera and watermark are set up like this:
    https://postimg.cc/yWhdSptC
    but I also tried putting the watermark inside camera object.

    Camera itself is set like this:
    https://postimg.cc/Ln6PTgcX

    Canvas like this:
    https://postimg.cc/V54rdLJc

    And image like this:
    https://postimg.cc/BthbJYzS

    I already tried dozens different combinations and parameters, without luck. I'm pretty sure I'm missing something simple here :/
     
  6. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Make sure that the depth of your watermark camera is greater than that of your main camera.
     
  7. Arjun-Gupte

    Arjun-Gupte

    Joined:
    May 31, 2013
    Posts:
    28
    I know that but I want to check in the code if os won't support I just disable the recording functionality
     
  8. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Then SystemInfo is the place to check.
     
  9. takoy

    takoy

    Joined:
    Jul 2, 2015
    Posts:
    20
    Hey
    I am receiving an error "NatCorder Error: Failed to record video frame for time ..." every frame while trying to record a video on the iPhone (See it in the XCode console), which eventually doesn't create the video.
    If I run the same code on the mac (Unity editor) it works without an issue and creates the video.
    What does this error mean??
     
  10. jimmying

    jimmying

    Joined:
    Sep 20, 2017
    Posts:
    8
    Hey Lanre,

    I've got a possible issue where when I launch the app on device (Android for now) where, I think, it's turning on the microphone and then off again.

    So when I have my bluetooth headphones connected, when I launch the app, any current audio gets killed, and I can briefly hear static, then my headphones give a notification that a "call has ended" when there was no call. This happens every time I launch the app.

    To produce the error, I created a new project, imported the latest NatCorder and built the ReplayCam scene to my devices. I also tried it with an empty scene (just in case the mic initialization in ReplayCam was the issue).

    Happens on an up to date Pixel 2 and an old Sony Z3c on Lollipop. Latest NatCorder on Unity 2019.2.10f1.

    Any thoughts on this?
     
  11. takoy

    takoy

    Joined:
    Jul 2, 2015
    Posts:
    20
    I think I found the issue. I changed the MP4Recorder file path to a different location. Once changed it back to the original path - the error stopped.
    Is there a way to save the file at a different location instead of moving it after the video is complete?
     
  12. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Can you share the logs from Xcode? Make sure to use code tags or attach a .txt file.
     
  13. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    What version of NatCorder are you running on? ReplayCam in NatCorder 1.6.6 will always start the microphone when the scene is opened, regardless of whether you will be recording with microphone audio. This is because it makes Unity's microphone audio more stable.
     
  14. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Do not change NatCorder internals, especially the recording path. You would need to get an understanding of platform-specific file system differences.
     
  15. mindmovies

    mindmovies

    Joined:
    May 9, 2019
    Posts:
    6
    trying record camera with Post Processing. Noticed it fails to record HDR colors into texture, what should be done to do this ?
     
  16. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    How do you mean "it fails to record HDR colors into texture". CameraInput will record to an RGBA32 RenderTexture (no HDR) because native codecs don't accept HDR pixel data.
     
  17. tz

    tz

    Joined:
    Oct 13, 2007
    Posts:
    87
    What is the most efficient way to record from a RenderTexture in the most recent version of Natcorder?
    Previously, we would use the AquireFrame() interface, then do a Graphics.Blit.

    ie.
    var frame = natcorder.AcquireFrame();
    Graphics.Blit(GetRenderTarget(), frame);
    natcorder.CommitFrame(frame, time);

    Now that AquireFrame is deprecated, what approach will yield the best performance?
    We tried ReadPixels(), but it appears to be noticeably slower than the old way of doing it.
     
  18. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    AsyncGPUReadback.Request.
     
  19. tz

    tz

    Joined:
    Oct 13, 2007
    Posts:
    87
  20. jimmying

    jimmying

    Joined:
    Sep 20, 2017
    Posts:
    8
    I am using 1.6.6. I've realised that it isn't a NatCorder thing so don't worry about it.

    Just for reference though, it seems like it's something with unity 2019+. I create a new project, create a script and put "Microphone.GetPosition(null);" in a new method. I don't even have to attach the script to anything. Then when I run it on the device my bluetooth headphones indicate that a call has been ended. It doesn't happen with 2018.
     
    Lanre likes this.
  21. TerabyteTim

    TerabyteTim

    Joined:
    Oct 20, 2011
    Posts:
    111
    Does this plugin support recording audio to separate files? If so, does it support multiple simultaneous audio captures?

    I would like to do a capture where the export is 1 video file, and 3 separate audio files. For three separate tracks of audio.

    This would be so I can combine and edit them in premier separately
     
  22. mindmovies

    mindmovies

    Joined:
    May 9, 2019
    Posts:
    6
    Yes I noticed. I'm using Post Process Bloom with combination with material/shader , containing emission with intensive HDR color for nice volume effect, but video recorder do not catch this. So HDR must be converted to LDR. Could your ReadableTexture readback handle with HDR and convert to LDR ? Or other fixes are welcome.
    Im quite new to Unity so maybe there are other solutions?
     
  23. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    AsyncGPUReadback is supported on Android devices running Vulkan. You might need to enable it in Player Settings. The `CameraInput` script uses this for recording.
     
  24. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    NatCorder currently doesn't have an audio-only recorder. In the NatCorder 1.7.0 update, we have added a simple WAVRecorder for recording to waveform audio files.
     
  25. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    NatCorder 1.6.6 doesn't use ReadableTexture. Make sure you are using 1.6.6 as we use Unity-provided functions for performing the readback. You can then investigate and tweak it to get the effect you want (I have no experience with HDR rendering in Unity).
     
  26. TerabyteTim

    TerabyteTim

    Joined:
    Oct 20, 2011
    Posts:
    111
    Is there an expected release for the 1.7.0 update?
     
  27. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Planning on sometime next week.
     
  28. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    350
    Hi, interested in NatCorder as my game required recoding gameplay for about 5 min (User highlight saving so that he can save the video file for upload or personal use) I just want to check the following before I make any payments.

    1. Does it support HDRP? (2019.3f6, 7.18) I've seen that it does, but that was about a year ago, and I want to know if it supports the latest version and that it will be supported throughout the HDRP cycle.

    2. Can the saved video be played back in Unity?

    3. Can the video recording stop, pause, resume at any time without issues?

    Thanks!
     
  29. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Yes, there's nothing that prevents it from working with HDRP. All NatCorder needs to do is to render to a texture,
    Yes, using the VideoPlayer component.
    From the NatCorder perspective, there is no such thing as pausing or resuming recording. If you have a recording session active (you've created a recorder) and you don't commit frames to it, then you could say you are in a paused state. There is a constraint that for mobile platforms (and maybe on macOS and Windows too, I haven't tried), if the app gets suspended, then recording will automatically fail.
     
  30. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    350
    Lanre, I just implemented NatCorder to my game and it is working great!

    But I have one last concern,

    it looks like mp4encoder encodes the videos using H264 codec, right?

    And my game allows users to make videos of the game content and allows access to the video files (stored as mp4)

    Will I run into any license issues regarding H264?
    I read this article here
    https://www.engadget.com/2010/05/04/know-your-rights-h-264-patent-licensing-and-you/

    and it looks like the commercial use of H264 is not free.
    Could you kindly clarify?
     
  31. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    AFAIK if you aren't selling H.264-encoded video content, then you don't have to pay any royalties. See this.
     
  32. jjejj87

    jjejj87

    Joined:
    Feb 2, 2013
    Posts:
    350
    I read through the post you linked, and it looks like the users of software (my client) don't have to pay, but software developers (you and I) have to. Here is the exert.

    I am very worried about this situation and that the licensing of NatCorder doesn't cover this.

    I think you should really contact MPEG LA, and get written confirmation that natCorder and its uses are covered. Unless of course, I understood the whole thing wrong.

    But, provided that I am wrong, you might have to contact your clients (those who bough NatCorder) to make sure they are aware of this and that all this is straightened out - before it becomes a clusterfk of all ages....
     
    Last edited: Feb 1, 2020
  33. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Using an integrated encoder (AVFoundation on macOS and iOS, MediaCodec on Android, MediaFoundation on Windows) subverts the need to acquire a license. This makes the most sense, as it would not be realistic to expect every app out there that does some form of video recording to pay licensing fees to MPEG LA.
     
  34. rhedgeco

    rhedgeco

    Joined:
    Dec 2, 2013
    Posts:
    2
    Hi, I have been trying to get this to work for a short while now and seem to be running into a few bugs. I searched through this thread but couldn't seem to find out what to do. (I apologize if this has been answered millions of times.)

    I have a coroutine that should render a video from a render texture, but it freezes about one second in every time. Any help would be appreciated, thanks.

    Code (CSharp):
    1.  
    2. private IEnumerator RenderRoutine(float length)
    3. {
    4.     float fps = 60;
    5.     int audioSamples = 44100;
    6.     int channels = 2;
    7.     Camera renderCam = CreateRenderingCamera();
    8.     RenderTexture tex = renderCam.targetTexture;
    9.     MP4Recorder recorder =
    10.         new MP4Recorder(tex.width, tex.height, fps, audioSamples, channels, s => { Debug.Log(s); });
    11.     FixedIntervalClock clock = new FixedIntervalClock(fps);
    12.  
    13.     for (int frame = 0; frame < length * fps; frame++)
    14.     {
    15.         yield return new WaitForEndOfFrame();
    16.         Texture2D fTex = RenderTextureToTexture2D(tex);
    17.         recorder.CommitFrame(fTex.GetPixels32(), clock.Timestamp);
    18.         clock.Tick();
    19.         DestroyImmediate(fTex);
    20.     }
    21.  
    22.     recorder.Dispose();
    23. }
    24.  
     
  35. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    You are setting a non-zero sample rate and channel count without actually committing any audio. On Windows, this causes weird behaviour (app starts to lag, no clue why). Set those arguments to zero.

    Minor note: a fixed interval clock will auto-tick by default when Timestamp is accessed, so no need to manually tick. You can disable this in the constructor.
     
  36. rhedgeco

    rhedgeco

    Joined:
    Dec 2, 2013
    Posts:
    2
    Thanks for the reply! That helped immensely!
    Another less pressing question, Am I able to specify the output path? Or should I just move it after its created? Thanks.
     
  37. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Move it after it is created. Don't try modding the sources to change the output path.
     
  38. Dotberrys

    Dotberrys

    Joined:
    May 29, 2019
    Posts:
    7
    Hi!

    I have been using NatCorder for quite some time now. It was working fine on both my Mac, Android, and iPhone. But post the update, I'm unable to hear any sound in Android devices. I am yet to check on iPhones. I checked this forum and tried all possible things that i could do but still am unable to get the audio recording working. Can you please help with this?

    Note that the audio recording is working fine in the unity editor (mac)

    Spec:
    NacCorder - 1.6.6
    Unity - 2019.3
    Android Version - 10

    Any help would be appreciated.
    Thanks
     
  39. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Can you share the unfiltered logs from logcat in a .txt file?
     
  40. Dotberrys

    Dotberrys

    Joined:
    May 29, 2019
    Posts:
    7
    Hi,

    Thanks for replying.
    Attached the log file

    Thanks
     

    Attached Files:

    • log.txt
      File size:
      38.6 KB
      Views:
      14
  41. timbokoppers

    timbokoppers

    Joined:
    Nov 14, 2016
    Posts:
    31
    Hi Lanre,

    What is the best way to record a cropped screen? The newest iPhone have a 19.5:9 screen ratio, but I want to record a 16:9, so chop off a rectangle from the top and bottom. I've read that there was an example called 'GreyWorld' that used a custom material, but can't find it in the current NatCorder.

    Can you advice how to do this? THanks!
     
  42. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Can you share your recording code? Are you recording from a microphone AudioSource or an AudioListener?
     
  43. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    You could just create a recorder with a 16:9 aspect, and Unity will render your app for the recorder like it is being displayed on a screen with a 16:9 aspect.
     
  44. timbokoppers

    timbokoppers

    Joined:
    Nov 14, 2016
    Posts:
    31
    The outcome of doing that is that the content of the 19.5:9 ratio is crammed in the 16:9 video, and the content looks distorted. See screenshot attached. Recording and displaying in the app looks good, but when saving that video to the camera roll show the wierd result.
     

    Attached Files:

  45. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Are you rendering the app with a CameraInput. You should never get squishing with CameraInput, as Unity will always re-render to match the size of the recorder.
     
  46. timbokoppers

    timbokoppers

    Joined:
    Nov 14, 2016
    Posts:
    31
    Yes, this is the code :

    Code (CSharp):
    1. recordingClock = new RealtimeClock();
    2.             HEVCVideoRecorder = new HEVCRecorder(_videoWidth, _videoHeight, 30, AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode, OnReplay);
    3.             cameraInput = new CameraInput(HEVCVideoRecorder, recordingClock, Camera.main);
    4.            
    5.             // Start the microphone
    6.             audioDevice = AudioDevice.GetDevices()[0];
    7.             audioDevice.StartRecording(AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode, this);
    8.         }
    9.        
    10.         public void OnSampleBuffer(float[] sampleBuffer, int sampleRate, int channelCount, long timestamp)
    11.         {
    12.             HEVCVideoRecorder.CommitSamples(sampleBuffer, recordingClock.Timestamp);
    13.         }
     
  47. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    How is your UI setup? If you change the aspect of the game view in the editor, does it come out squished?
     
  48. Dotberrys

    Dotberrys

    Joined:
    May 29, 2019
    Posts:
    7
    Hi,

    I am using the Audiosource as a microphone and attached is the code that I've used.
     

    Attached Files:

  49. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,860
    Check that you have granted microphone permissions to your app.
     
  50. AAwezz

    AAwezz

    Joined:
    Aug 1, 2017
    Posts:
    3
    Hi,

    I recently updated to the newest version of NatCorder because of a null error fix.
    In the process i lost the CameraInput.OnPreRender(); and CameraInput.OnPostRender(); which was made to hide UI elements in the recording. Is there a new alternative for this?

    Thanks,
    Anders
     
unityunity