Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

NatCorder - Video Recording API

Discussion in 'Assets and Asset Store' started by Lanre, Nov 18, 2017.

  1. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Your channel count is zero, which is invalid.
     
  2. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    What I'm primarily concerned about is the hanging you said happens. Beyond that, if your app is CPU- or GPU-bound already (without recording), then you can't expect adding recording to be unaffected by the lack of processing resources. I need to see Xcode profiler stats. A good first step would be to share what your app's performance and memory behaviour looks like (CPU, memory, and frame profilers in Xcode) without any recording. Then share them when recording and we can work from there.
     
  3. frimarichard

    frimarichard

    Joined:
    Jul 24, 2017
    Posts:
    31
    Awesome, I removed the audio sample rate, which now presumably stops the audio initializer completely.

    Feature request: For 1.6.1, can we have a validation check in the MP4 constructor to detect this issue? :)

    Thanks!
     
    Lanre likes this.
  4. nikosurfing

    nikosurfing

    Joined:
    Mar 11, 2014
    Posts:
    45
    Thank you lanre, your asset is totally worth! Recording works both on editor and android!
     
    Lanre likes this.
  5. Aidan-Wolf

    Aidan-Wolf

    Joined:
    Jan 6, 2014
    Posts:
    59
    edee1337 and Lanre like this.
  6. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    And that's why the new API makes all the difference :D
     
  7. Aidan-Wolf

    Aidan-Wolf

    Joined:
    Jan 6, 2014
    Posts:
    59
    Oh! I just see it released a couple days ago.

    I will try it out after I push this next update :)
     
    Lanre likes this.
  8. rexcheung

    rexcheung

    Joined:
    Feb 2, 2017
    Posts:
    35
    Hi Lanre,

    Thanks for this plugin, it is good.

    I am trying to adopt it at ipad.
    Unity - 2018.4.3f1
    Xcode - 10
    ipad ios - 12

    I am wondering if the plugin can record audio from the scene. My output is a short video with 5s, which record background music and video captured by main camera (not webcamera).
     
  9. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    You can record from an AudioSource or AudioListener using the AudioInput helper class. See the ReplayCam example for how it is used.
     
  10. Mraoge

    Mraoge

    Joined:
    Jul 23, 2015
    Posts:
    2
    Hi Lanre,
    The new version 1.6.0 , i test it on iphonex , there happend some mistake, the screen just freeze when i start recording(Examples/ReplayCam),but i release the record button, it can show the replay and the screen not freeze anymore,
    btw,Everything works fine on Android
    please check
     
    Last edited: Jul 8, 2019
  11. MichalHuuuge

    MichalHuuuge

    Joined:
    May 8, 2019
    Posts:
    4
    We are experiencing the same issue on IPhoneXR. It sometimes works at first recording and game logics always works just fine in the background - it's just as the camera was not rendering while being captured. It only happens with the CameraInput.
     
  12. HappyShip

    HappyShip

    Joined:
    Jun 15, 2017
    Posts:
    7
    Hey, any update on this? I'm still completely lost on the issue, and I've tried just about anything I could think of...
     
  13. Aidan-Wolf

    Aidan-Wolf

    Joined:
    Jan 6, 2014
    Posts:
    59
    Hi HappyShip,

    Shader:
    Code (CSharp):
    1. Shader "Custom/Landscape Video" {
    2.     Properties {
    3.         _MainTex ("Texture", 2D) = "white" {}
    4.         _Rotation ("Rotation", float) = 0
    5.         _Scale ("Scale", float) = 1
    6.     }
    7.     SubShader {
    8.  
    9.         Tags {
    10.             "Queue"="Transparent"
    11.             "RenderType"="Transparent"
    12.             "IgnoreProjector"="True"
    13.             "PreviewType"="Plane"
    14.             "CanUseSpriteAtlas"="True"
    15.         }
    16.  
    17.         Cull Off
    18.         ZWrite Off
    19.         ZTest Always
    20.         Lighting Off
    21.         Fog { Mode off }
    22.         ZTest[unity_GUIZTestMode]
    23.         Blend SrcAlpha OneMinusSrcAlpha
    24.  
    25.         Pass {
    26.             CGPROGRAM
    27.             #pragma vertex vert
    28.             #pragma fragment frag
    29.            
    30.             #include "UnityCG.cginc"
    31.  
    32.             struct appdata {
    33.                 float4 vertex : POSITION;
    34.                 float2 uv : TEXCOORD0;
    35.             };
    36.  
    37.             struct v2f {
    38.                 float2 uv : TEXCOORD0;
    39.                 float4 vertex : SV_POSITION;
    40.             };
    41.  
    42.             uniform fixed _Rotation, _Scale;
    43.  
    44.             v2f vert (appdata v) {
    45.                 v2f o;
    46.                 o.vertex = UnityObjectToClipPos(v.vertex);
    47.                 // Rotate and mirror UV
    48.                 o.uv = v.uv - float2(0.5, 0.5);
    49.                 float s, c;
    50.                 sincos(_Rotation, s, c);
    51.                 float2x2 transform = mul(float2x2(
    52.                     float2(c, -s),
    53.                     float2(s, c)
    54.                 ), float2x2(
    55.                     float2(_Scale, 0.0),
    56.                     float2(0.0, 1.0)
    57.                 ));
    58.                 o.uv = mul(transform, o.uv) + float2(0.5, 0.5);
    59.                 return o;
    60.             }
    61.            
    62.             sampler2D _MainTex;
    63.  
    64.             fixed4 frag (v2f i) : SV_Target {
    65.                 return tex2D(_MainTex, i.uv);
    66.             }
    67.             ENDCG
    68.         }
    69.     }
    70. }
    71.  
    Code (this is based on ReplayCam.cs)
    Code (CSharp):
    1. private void Start() {
    2.             landscapeTex = new RenderTexture(videoWidth, videoHeight, 16, RenderTextureFormat.ARGB32);
    3.             landscapeCamera.targetTexture = landscapeTex;
    4. //landscapeCamera is a separate camera attached to my main camera that also has the AR background texture
    5.         }
    6.  
    7.         private void Update() {
    8.             if (landscape && videoRecorder != null) {
    9.                 var frame = videoRecorder.AcquireFrame();
    10.                 Graphics.Blit(landscapeTex, frame, landscapeMaterial);
    11.                 videoRecorder.CommitFrame(frame, recordingClock.Timestamp);
    12.             }
    13.         }
    14.  
    15. void StartRecording () {
    16. if (landscape) {
    17.                 if (Input.deviceOrientation == DeviceOrientation.LandscapeLeft) {
    18.                     landscapeMaterial.SetFloat("_Rotation", -90f * Mathf.Deg2Rad);
    19.                 } else {
    20.                     landscapeMaterial.SetFloat("_Rotation", -90f * Mathf.Deg2Rad);
    21.                 }
    22.             }
    23.  
    24. videoRecorder = new MP4Recorder(
    25.                 !landscape ? videoWidth : videoHeight,
    26.                 !landscape ? videoHeight : videoWidth,
    27.                 30,
    28.                 sampleRate,
    29.                 channelCount,
    30.                 callback
    31.             );
    32.  
    33. if (!landscape) {
    34.                 cameraInput = new CameraInput(videoRecorder, recordingClock, Camera.main);
    35.             }
    }
     
    edee1337 and Lanre like this.
  14. frimarichard

    frimarichard

    Joined:
    Jul 24, 2017
    Posts:
    31
    Hey,

    We've noticed significantly worse performance with 1.6.0 compared to 1.5.1, specifically when using a Samsung S9 device.

    I've attempted to profile it. The device uses AsyncGPUReadbackES.Request() inside CameraInput.cs, which itself takes ~1ms (although allocates 2kb), and mediaRecorder.CommitFrame() again takes only ~1ms. This suggests the problem is inside the java request.

    I've also forced the device to use the stalling method. Reading the pixels and texture data takes ~110ms.

    I haven't profiled the S9 on 1.5.1 but just visually it wasn't lagging at all, whereas on 1.6.0 it very clearly is.

    Any ideas? Thanks.
     
    serginian likes this.
  15. serginian

    serginian

    Joined:
    Nov 22, 2013
    Posts:
    1
    The same problem here! Tested on Samsung Galaxy S7, S9, S10;
     
  16. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Check that you have enabled camera permissions for the test app.
     
  17. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Huh this is weird. And it only freezes while recording?
     
  18. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    This is weird, because once you actually create the request, everything else happens on a dedicated GL thread that shares the Unity render thread context. Can you post a screenshot of the Unity CPU profiler, showing the raw hierarchy?
     
  19. HappyShip

    HappyShip

    Joined:
    Jun 15, 2017
    Posts:
    7
    Thanks for your reply, I do appreciate it.
    What I don't understand however, is how it does work for you, and doesn't work for me... As soon as the Graphics.Blit comes into play, it messes things up. I've tried many things to get it to work, and found out that if I comment out the Material parameter in the Graphics.Blit call, I do get video. Whenever it isn't commented out, my video turns out completely black.
    The material uses the Shader you sent. The shader also works on mobile, as I've tested it with a rendertexture and an image on canvas (the cameraview projected on the image did get rotated)
    I am also using a secondary camera attached to my Main camera to render the video. I've also tried to render to the RenderTexture first, like what is done in the original CameraInput script. All to no avail.

    If you, or anyone else is able to help at all, it's much appreciated.

    The code I use:
    Code (CSharp):
    1.  
    2. public class CustomCameraInput : IDisposable
    3. {
    4.     private IMediaRecorder mediaRecorder;
    5.     private IClock clock;
    6.     private Camera camera;
    7.     private int frameCount;
    8.     private DeviceOrientation orientation;
    9.     private Material rotateUV;
    10.  
    11.     RenderTexture _render;
    12.  
    13.     public int recordEveryNthFrame = 1;
    14.  
    15.     public CustomCameraInput(IMediaRecorder mediaRecorder, IClock clock, Camera camera, Material mat)
    16.     {
    17.         this.mediaRecorder = mediaRecorder;
    18.         this.clock = clock;
    19.         this.camera = camera;
    20.         rotateUV = mat;
    21.         DispatchUtility.onFrame += OnFrame;
    22.         orientation = GetOrientation();
    23.  
    24.         float rotation = 0f;
    25.         int width, height;
    26.  
    27.         switch (orientation)
    28.         {
    29.             case DeviceOrientation.LandscapeLeft:
    30.                 rotation = -90;
    31.                 width = Screen.height;
    32.                 height = Screen.width;
    33.                 break;
    34.             case DeviceOrientation.LandscapeRight:
    35.                 rotation = 90;
    36.                 width = Screen.height;
    37.                 height = Screen.width;
    38.                 break;
    39.             default:
    40.                 rotation = 0f;
    41.                 width = Screen.width;
    42.                 height = Screen.height;
    43.                 break;
    44.         }
    45.  
    46.         _render = new RenderTexture(width, height, 16, RenderTextureFormat.ARGB32);
    47.         camera.targetTexture = _render;
    48.  
    49.         rotateUV.SetFloat("_Rotation", rotation * Mathf.Deg2Rad);
    50.     }
    51.  
    52.     public void Dispose()
    53.     {
    54.         DispatchUtility.onFrame -= OnFrame;
    55.     }
    56.  
    57.     private void OnFrame()
    58.     {
    59.         // Check frame index
    60.         if (frameCount++ % recordEveryNthFrame != 0)
    61.             return;
    62.         // Acquire frame
    63.         var encoderFrame = mediaRecorder.AcquireFrame();
    64.  
    65.         //var prevTarget = camera.targetTexture;
    66.         //camera.targetTexture = _render;
    67.         //camera.Render();
    68.         //camera.targetTexture = prevTarget;
    69.  
    70.         Graphics.Blit(_render, encoderFrame, rotateUV);
    71.  
    72.         // Commit frame              
    73.         mediaRecorder.CommitFrame(encoderFrame, clock.Timestamp);
    74.     }
    75.  
    76.     public DeviceOrientation GetOrientation()
    77.     {
    78.         Vector3 grav = Input.gyro.gravity;
    79.         if (Mathf.Abs(grav.x) > Mathf.Abs(grav.y) && Mathf.Abs(grav.z) > -0.8f)
    80.         {
    81.             if (grav.x < 0)
    82.                 return DeviceOrientation.LandscapeLeft;
    83.             else
    84.                 return DeviceOrientation.LandscapeRight;
    85.         }
    86.         else return DeviceOrientation.Portrait;
    87.     }
    88. }
     
  20. jasonwaltersxyz

    jasonwaltersxyz

    Joined:
    Jul 16, 2014
    Posts:
    24
    Hi Lanre, Any chance you have an experimental branch that works for linear? Thanks!
     
    inear_sthlm likes this.
  21. MichalHuuuge

    MichalHuuuge

    Joined:
    May 8, 2019
    Posts:
    4
    Exactly, the moment recording stops everything is again in order. For 100% we have a permission for a camera, as we can see a preview before recording.
     
  22. SominHC

    SominHC

    Joined:
    Jul 20, 2018
    Posts:
    8
    Hi Lanre!
    I started implementing a Ringbuffer for capturing the last x seconds using version 1.6.0 and it (almost) works like a charm.
    The issue I'm having is that the encoder suddenly becomes very slow when I pass a certain recording length.
    I crudely checked and one frame takes about 0.012 seconds to encode (with Debug.Log() etc ). But when I encode a clip that is longer than ~2.3s the last few frames start to take pretty much exactly 1 second longer.
    I also checked if it might be connected to the size but I reduced the image size from 1080p to 800x600 and it starts to slow down at exactly the same frame (136)
    Any idea what might be wrong? I tried starting it in the main-thread but had the same issue. So its (probably) not a thread thing?

    I'm at a loss as to what I might do wrong on my end. I simply loop through all the frames and commit the byte[] array.
    The resulting video looks fine as well.

    edit: I might have found a lead: If I don't record Audio it encodes fine. I first commit all video frames and only afterwards commit the samples. Maybe I need to do it in parallel?

    edit2: seems like that did the trick.
     
    Last edited: Jul 10, 2019
  23. mtalbott

    mtalbott

    Joined:
    Dec 21, 2011
    Posts:
    125
    @Lanre,
    I can confirm this bug as well. I am getting a camera freeze on Unity 2018.4.3f1 using Vuforia 8.3.8 on an iPhone X and and iPhone SE. The weird thing is that the video records everything fine and without the freeze. I've tried toggling Multithreaded Rendering, GPU Skinning, and Graphics Jobs in the Player Settings as a long shot at fixing it with no luck.
     
  24. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Since the shader is being used for a blit, you might want to remove the transparency tags and other unnecessary things. Look for Unity's Internal/BlitCopy shader online and start with that as a base instead. Beyond that if it's still black then it's probably a Unity issue. I haven't had the best experience with lower level rendering with Unity.
     
  25. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    We support linear on macOS, Windows, and iOS. We don't support it on Android, and have no plans to add it.
     
  26. Mraoge

    Mraoge

    Joined:
    Jul 23, 2015
    Posts:
    2
    I have enabled camera permissions for the test app.
    When it start recording and game logics always works just fine in the background,but the screen just freeze .
     
  27. DanFlanaganCodes

    DanFlanaganCodes

    Joined:
    Jun 21, 2014
    Posts:
    17
    @Lanre
    I can't find any documentation regarding how to pass in a render texture's data into the pixelBuffer or I don't know what byte[] to pass in


    Code (CSharp):
    1.                 webcamTexture.GetPixels32(pixelBuffer);
    2.                 videoRecorder.CommitFrame(pixelBuffer, clock.Timestamp);
    I have tried using graphics.bilt to copy the render texture to a Texture2D, and a few other conversions, Graphics.CopyTexture, etc.. maybe I am missing something, can I pass in the render texture data? I need to use a render texture for my application. Also this would be a great thing to include in the examples section.

    Side note, your thing runs like a freaking charm, was using FFMPEG bind before and it was super slow.
     
  28. JoakimOlsson

    JoakimOlsson

    Joined:
    May 20, 2019
    Posts:
    11
    1. The app can have been installed before the storage of the device was filled up.
    2. "impossible" is a strong word to use. Devices have storage in abundance, yes, but the storage is not unlimited. If we want to built robust programs (and we do) we very much prefer to fail gracefully in the scenarios we can vs throwing a native "the app has crashed" in the face of the user.

    I have a scenario where the user is able to start recording and the program will continue to create multiple video snippets until the user stops recording. Each recording will be stopped when a certain trigger occur, then "post-processed", and finally a new recording automatically begins. This means that even if the user has a few hundred of megabytes around before the first recording, that may not be the case after a while. Especially since there may be other applications adding stuff to the storage as well.

    Other measures, such as checking the available storage space before each recording and possible disallow a recording to begin if the storage space is low, does not change the fact that we need to handle "out of storage" scenarios.

    I tried to wrap the dispose of the IMediaRecorder and ICameraInput in a try-catch block, but it seemed that nothing was caught? Other solutions available?
     
  29. HappyShip

    HappyShip

    Joined:
    Jun 15, 2017
    Posts:
    7
    If anyone else is struggling with Landscape video in a portrait locked application now, or in the future, I hope ill be able to help with this post.
    After struggling for a long time with it and with the help of @Aidan-Wolf and @Lanre it's finally working for me. I will be attaching the scripts that I've used to get it working.

    In short, what managed to work, is a combination of a separate recording camera (with AR-background renderer), and changing the dimensions of the RenderTexture that gets acquired from the MediaRecorder (if needed).
    Keep in mind you will need to check the device orientation when creating the Mp4Recorder aswell:
    Code (CSharp):
    1.  
    2. DeviceOrientation ori = GetOrientation();
    3.  
    4.         if (ori == DeviceOrientation.Portrait)
    5.             _videoRecorder = new MP4Recorder(Screen.width, Screen.height, 30, AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode, RecorderCallback);
    6.         else if (ori == DeviceOrientation.LandscapeLeft || ori == DeviceOrientation.LandscapeRight)
    7.             _videoRecorder = new MP4Recorder(Screen.height, Screen.width, 30, AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode, RecorderCallback);
    8.  
    Good luck, and thanks for the help Aidan and Lanre!
     

    Attached Files:

    jiraphatK likes this.
  30. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Can you email me a repro?
     
  31. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    You're probably on Windows. This has been reported to happen on Windows. No clue why.
     
  32. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    You have to perform a readback of the RenderTexture's data from GPU memory to system memory (a plain array). You can use AsyncGPUReadback for this, or Texture2D.ReadPixels.
     
  33. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    What I have seen others do (Everyplay and pmjo's Next Gen Recorder) is simply enforcing a minimum available storage size before allowing recording to start. We are not adding an error handler because it is another thing for the client (devs) to worry about. Unity doesn't allow assets throw exceptions. If you prevent recording from starting when there is low space, you should never have to handle an out-of-storage scenario.
     
  34. JoakimOlsson

    JoakimOlsson

    Joined:
    May 20, 2019
    Posts:
    11
    Again, I have learnt that "never" is a poor word to use when it comes to programming, especially when in regards to making assumptions about what, when and why the users will do things.

    I will implement the "pre-check" and it will simply have to suffice. The rare (but certainly not impossible) times this is not enough I believe that the users can understand why.

    Thanks for quick and informative replies.
     
  35. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    In this case, never is an exact quantifier. If you don't start recording because there isn't enough space, then you can never have an out-of-storage error during recording (because recording was never started in the first place). This is why it is commonly used.
     
    MattLovell likes this.
  36. JoakimOlsson

    JoakimOlsson

    Joined:
    May 20, 2019
    Posts:
    11
    Kind of impossible to know in advance how large the recording will be, however. We can make a reasonable estimate, but that does not change the fact that other applications may affect the available storage simultaneously during our recording.
     
  37. andrewdSuper77

    andrewdSuper77

    Joined:
    Jul 12, 2019
    Posts:
    15
    Hi, I'm having an odd issue where the audio is played back at a slowed down rate. Video looks fine. I'm using AudioSettings.outputSampleRate for the media MP4Recorder audio sample rate parameter. Is this a common or known issue?
     
  38. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Have you checked the channel count? Can you share your recording code?
     
  39. jonathaniscarroll

    jonathaniscarroll

    Joined:
    Oct 6, 2015
    Posts:
    4
    Also seem to be newly having this problem, using Unity 2018.14f1.
    Everything freezes while recording (on iPhone 6s), but successfully opens video when recording is done.
     
  40. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    I have attached an updated CameraInput script. Try it and let me know if the freezing on iOS is still present.
     

    Attached Files:

  41. GuyInception

    GuyInception

    Joined:
    Jun 12, 2019
    Posts:
    4
    Hi guys,

    Does "CameraInput" actually work? I only get empty, invalid video on windows editor and the following error on Android:


    NatCorder: Preparing MP4 video encoder with format: {color-format=2130708361, i-frame-interval=3, mime=video/avc, width=1280, bitrate=5909760, frame-rate=24, height=640}
    07-15 09:43:49.296 6176 6214 V Unity : NatCorder: Preparing MP4 audio encoder with format: {channel-mask=12, sample-rate=22, mime=audio/mp4a-latm, channel-count=2, aac-profile=2, bitrate=64000, max-input-size=8192}
    07-15 09:43:49.387 6176 6214 D Unity : Uploading Crash Report
    07-15 09:43:49.423 6176 6214 E Unity : AndroidJavaException: android.media.MediaCodec$CodecException: Error 0x80001001
    07-15 09:43:49.423 6176 6214 E Unity : android.media.MediaCodec$CodecException: Error 0x80001001
    07-15 09:43:49.423 6176 6214 E Unity : at android.media.MediaCodec.native_configure(Native Method)
    07-15 09:43:49.423 6176 6214 E Unity : at android.media.MediaCodec.configure(MediaCodec.java:1935)
    07-15 09:43:49.423 6176 6214 E Unity : at android.media.MediaCodec.configure(MediaCodec.java:1864)
    07-15 09:43:49.423 6176 6214 E Unity : at com.yusufolokoba.natcorder.MP4Recorder.<init>(MP4Recorder.java:73)
    07-15 09:43:49.423 6176 6214 E Unity : at com.unity3d.player.UnityPlayer.nativeRender(Native Method)
    07-15 09:43:49.423 6176 6214 E Unity : at com.unity3d.player.UnityPlayer.c(Unknown Source:0)
    07-15 09:43:49.423 6176 6214 E Unity : at com.unity3d.player.UnityPlayer$e$1.handleMessage(Unknown Source:88)
    07-15 09:43:49.423 6176 6214 E Unity : at android.os.Handler.dispatchMessage(Handler.java:101)
    07-15 09:43:49.423 6176 6214 E Unity : at android.os.Looper.loop(Looper.java:164)
    07-15 09:43:49.423 6176 6214 E Unity : at com.unity3d.player.UnityPlayer$e.run(Unknown Source:20)
    07-15 09:43:49.423 6176 6214 E Unity : at UnityEngine.AndroidJNISafe.CheckException () [0x00000] in <00000000000000000000000000000000>:0
    07-15 09:43:49.423 6176 6214 E Unity : at UnityEngine.AndroidJNISafe.NewObject (System.IntPtr clazz, System.IntPtr methodID, UnityEngine.jvalue[] args) [0x00000] in <00000000000000000000000000000000>:0
     
  42. MichalHuuuge

    MichalHuuuge

    Joined:
    May 8, 2019
    Posts:
    4
    Thank you for your help and the new script.

    I'm afraid sending repo won't be necessary. We can reproduce it in empty project with only NacCorder on example scene "ReplayCam". Checked on Unity 2018.3.4 and 2018.3.14 with iPhone XS, iPhone XR and iPhone 8.

    I've tried use this script but had to change one line in order to compile it with the newest version of the plugin available in the store. Line 104:


    request.CopyTo(pixelBuffer); // FROM: request.pixelBuffer.CopyTo(pixelBuffer, pixelBuffer.Length);


    it should not matter tho, as it is only executed on Android. Still - no improvement in the same tests as mentioned above (empty project, example scene, 3 devices on 2 Unity versions). Thank you again for your support and we all hope it's only a small fix and won't bother you too much. Cheers!
     
  43. andrewdSuper77

    andrewdSuper77

    Joined:
    Jul 12, 2019
    Posts:
    15
    Absolutely, here's the configuration:
    Code (CSharp):
    1.  
    2. int width = 480;
    3. int height = Mathf.FloorToInt(width * Screen.height / (float)Screen.width);
    4. int framerate = 30;
    5.  
    6. m_MediaRecorder = new MP4Recorder(width, height, framerate, AudioSettings.outputSampleRate, 1, OnVideoFinishedProcessing);
    7. m_RecordingClock = new RealtimeClock();
    8. m_VideoInput = new CameraInput(m_MediaRecorder, m_RecordingClock, m_RecordingCamera);
    9. m_AudioInput = new AudioInput(m_MediaRecorder, m_RecordingClock, m_AudioListener);
    I have the audio listener in an arbitrary location, and audio comes though my computer correctly.
     
  44. ZAUBAR

    ZAUBAR

    Joined:
    Jun 18, 2019
    Posts:
    53
    @Aidan-Wolf Sounds great! How do I need to implement it? At which point?
     
    Last edited: Jul 16, 2019
  45. ZAUBAR

    ZAUBAR

    Joined:
    Jun 18, 2019
    Posts:
    53
    @Lanre - I am migrating to 1.6 - how do I record from RenderTexture now? A documentation on this would be great, specifically how the following needs to be adapted:

    Code (CSharp):
    1. var frame = recorder.AcquireFrame();
    2.                 Graphics.Blit(renderTexture, frame);
    3.                 recorder.CommitFrame(frame, clock.Timestamp);

    When I try the following:

    Code (CSharp):
    1.     AsyncGPUReadback.Request(videoCameraRenderTexture, 0, request => {
    2.                     mediaRecorder.CommitFrame(request.GetData<byte>().ToArray(), clock.Timestamp);
    3.                 });
    ... I get this error:


    NullReferenceException: Object reference not set to an instance of an object
    RecordingController.<Update>b__16_1 (UnityEngine.Rendering.AsyncGPUReadbackRequest request):


    If I create a media recorder at start, Unity crashes.
    Can you please just add an example for renderTextures, this is such a requested use case! Best a full script of starting recording, stopping recording, etc. My whole app flow is now broken :(
     
    Last edited: Jul 15, 2019
  46. ZAUBAR

    ZAUBAR

    Joined:
    Jun 18, 2019
    Posts:
    53

    Hey @DanFlanaganCodes , did you get it to work, if yes, how?
     
  47. mtalbott

    mtalbott

    Joined:
    Dec 21, 2011
    Posts:
    125
    Unfortunately, the script update does not fix the freezing. Also, it doesn't compile on iOS as-is. I had #if def out the Android pixelBuffer stuff. Still freezing.
     
  48. Aidan-Wolf

    Aidan-Wolf

    Joined:
    Jan 6, 2014
    Posts:
    59
    @Lanre

    This would be great:
    Code (CSharp):
    1. AsyncGPUReadback.Request(videoCameraRenderTexture, 0, request => {
    2. mediaRecorder.CommitFrame(request.GetData<byte>().ToArray(), clock.Timestamp);
    3. });
    But it doesn't work on Android (Google Pixel 3a 2019)

    Code (CSharp):
    1. This GfxDevice does not support asynchronous readback
    2. 07-16 00:19:49.034 24895 24911 E Unity   : (Filename: ./Runtime/GfxDevice/GfxDevice.cpp Line: 1625)
    3. 07-16 00:19:49.039 24895 24911 E Unity   : InvalidOperationException: Cannot access the data as it is not available
    Can we get accquireFrame back?
     
  49. DanFlanaganCodes

    DanFlanaganCodes

    Joined:
    Jun 21, 2014
    Posts:
    17
    I haven't gotten it to work yet. I am going to try a few things in the next day or so. I will post updates!
     
  50. Aidan-Wolf

    Aidan-Wolf

    Joined:
    Jan 6, 2014
    Posts:
    59
    - CommitFrame is rather robust and I was able to use OpenCV to quickly convert Texture to Color[] and send it.
    - Putting recording logic in a background thread crashes the app on Android
     
    NAKAS likes this.