Search Unity

Bug Failed to add video frame to ProRes encoder

Discussion in 'Audio & Video' started by Luke_161, Nov 30, 2021.

  1. Luke_161

    Luke_161

    Joined:
    Mar 3, 2015
    Posts:
    19
    Hello,

    I'm trying to use Recorder to capture gameplay in editor, but I'm getting the following errors for every frame when trying to output to ProRes (any format), H264 however works fine. Wondering if anyone has had similar problems?

    Code (CSharp):
    1. Failed to add video frame to ProRes encoder
    2. UnityEngine.Debug:LogError (object)
    3. Unity.Media.ProResEncoderCore:AddFrameImpl (int,int,int,UnityEngine.TextureFormat,Unity.Collections.NativeArray`1<byte>,long,int,int) (at Library/PackageCache/com.unity.recorder@4.0.0-pre.2/Editor/ProRes/ProResEncoderPlugin.cs:472)
    4. Unity.Media.ProResEncoderCore:AddFrame (int,int,int,UnityEngine.TextureFormat,Unity.Collections.NativeArray`1<byte>,UnityEditor.Media.MediaTime) (at Library/PackageCache/com.unity.recorder@4.0.0-pre.2/Editor/ProRes/ProResEncoderPlugin.cs:449)
    5. Unity.Media.ProResEncoder:AddFrame (int,int,int,UnityEngine.TextureFormat,Unity.Collections.NativeArray`1<byte>,UnityEditor.Media.MediaTime) (at Library/PackageCache/com.unity.recorder@4.0.0-pre.2/Editor/ProRes/ProResEncoderPlugin.cs:657)
    6. Unity.Media.MediaEncoderManager:AddFrame (Unity.Media.MediaEncoderHandle,int,int,int,UnityEngine.TextureFormat,Unity.Collections.NativeArray`1<byte>,UnityEditor.Media.MediaTime) (at Library/PackageCache/com.unity.recorder@4.0.0-pre.2/Editor/Encoding/MediaEncoder.cs:406)
    7. UnityEditor.Recorder.BaseTextureRecorder`1<UnityEditor.Recorder.MovieRecorderSettings>:ReadbackDone (UnityEngine.Rendering.AsyncGPUReadbackRequest) (at Library/PackageCache/com.unity.recorder@4.0.0-pre.2/Editor/Sources/BaseTextureRecorder.cs:133)
    8.  
    I'm using the Apple Silicon editor on the latest MacBook Pro, recording was working fine on my old intel MacBook Pro so maybe that's the issue? I've also tried upgrading from 3.0.1 to 4.0.0-pre.2 but that made no difference.

    I get a similar error for audio as well 'Failed to add audio samples to ProRes encoder'
     
    Luke_161, Nov 30, 2021
    #1

  2. cguertin

    cguertin

    Unity Technologies

    Joined:
    Aug 20, 2019
    Posts:
    83
    Hey @Luke_161! I tried reproducing the issue on my end but wasn't able to get that error. Just to make sure I have the correct setup, could you please provide us the following information:
    1. Unity Editor version
    2. Unity Editor architecture ("Silicon" or "Intel".)
      1. You can validate this by looking at the title bar of your Unity Editor. The answer is "Intel" if the title bar ends with "<Rosetta>" like this screenshot Screen Shot 2021-11-30 at 4.50.51 PM.png . Otherwise it's "Silicon".
    3. Render Pipeline used in your project (Legacy Render Pipeline, HDRP, URP)
    4. Are you launching the recording from the Recorder Window or through Recorder clips on a Timeline?
    5. If possible, a screenshot of your recording settings would be great
    Cheers!
     
    cguertin, Nov 30, 2021
    #2

  3. Luke_161

    Luke_161

    Joined:
    Mar 3, 2015
    Posts:
    19
    Hey @cguertin
    1. Editor version: 2021.2.4f1
    2. Architecture: Silicon
    3. Render Pipeline: URP (12.1.1)
    4. I'm launching it through code (at points in the game flow) but if I hit 'start recording' manually from the recorder window I get the same result.
    I've attached a screenshot :) thanks for the quick response!
     

    Attached Files:

    Luke_161, Nov 30, 2021
    #3

  4. cguertin

    cguertin

    Unity Technologies

    Joined:
    Aug 20, 2019
    Posts:
    83
    Thanks for the extra infos! I tried a few more things on my end but somehow still can't get it to fail :/.

    I'm wondering if the Recorder might possibly be conflicting with other things in the project. Just to rule that out, could you try recording in a "fresh" URP project and validate if you also have the error there?

    I doubt they are causing the issue but can you also try adjusting the following Recorder settings just in case:
    1. Change the "Source" from "Game View" to "Targeted Camera"
    2. Change the "Codec Format" from "Apple ProRes 4444 XQ (ap4x)" to "Apple ProRes4444 (ap4h)"
     
    cguertin, Dec 1, 2021
    #4

  5. Luke_161

    Luke_161

    Joined:
    Mar 3, 2015
    Posts:
    19
    Hello, I've tried creating fresh projects with the following setups:
    • 2021.2.4f1 - 2D (URP) Core template
    • 2021.2.4f1 - 2D (Legacy) Core template
    • 2020.3.23f1 (Intel) - URP Demo scene
    I get the same error with Pro Res in all of them and the same error if I try with the Recorder settings you suggested.

    Wondering if maybe it's something to do with the setup on my machine? Are there any library dependencies / extras installs I might be missing? I'm on macOS 12.0.1.

    I did on one occasion get a hard crash when I hit 'start recording' a few seconds after hitting 'play' in the editor. I've attached that log incase it's of any use.
     

    Attached Files:

    Luke_161, Dec 2, 2021
    #5

  6. cguertin

    cguertin

    Unity Technologies

    Joined:
    Aug 20, 2019
    Posts:
    83
    So, I tried a few more things and even did a clean re-install of MacOS 12.0.1 on machine but still no luck on reproducing that error. It's very weird, I'm a bit puzzled as to what could be causing that. Your hunch about libraries is likely a good direction but I didn't do anything special on my end after re-installing the OS.

    I'm really sorry that we don't have any workaround for you at the moment. I need do some more digging on my end with my colleagues and will let you know as soon as we have more info or questions for you!
     
    cguertin, Dec 2, 2021
    #6

  7. Luke_161

    Luke_161

    Joined:
    Mar 3, 2015
    Posts:
    19
    @cguertin thanks for your efforts on this! Do let me know if you think of anything, or if there are any extra tests I could do my end. I will have a go at a clean OS install at some point, incase maybe migrating my old Mac to this new one has caused the issue.
     
    Luke_161, Dec 3, 2021
    #7

  8. cguertin

    cguertin

    Unity Technologies

    Joined:
    Aug 20, 2019
    Posts:
    83
    One more question: which CPU does your Macbook have (M1, M1 Pro, M1 Max)?

    I couldn't reproduce with my M1 but we had another user report the same issue as you when using an M1 Max.
     
    cguertin, Dec 6, 2021
    #8
    unity_28C4C8FB7C957FE2FA60 likes this.

  9. Luke_161

    Luke_161

    Joined:
    Mar 3, 2015
    Posts:
    19
    Hey, I have the M1 Max also. I'm using a MacBook Pro 16-inch 2021 model, M1 Max.
     
    Luke_161, Dec 7, 2021
    #9
    unitybru and cguertin like this.

  10. unitybru

    unitybru

    Unity Technologies

    Joined:
    Jan 28, 2020
    Posts:
    225
    An internal user has reported a similar issue with M1 Max, and has pointed us towards this discussion on the Apple forums regarding issues with M1 Max chips: https://developer.apple.com/forums/thread/694622

    We are waiting for an M1 Max laptop to arrive for our testing. We'll keep you updated, but in the meantime you should probably update your OS as Apple releases fixes.
     
    Last edited: Jan 13, 2022
    unitybru, Dec 7, 2021
    #10
    Luke_161 likes this.

  11. unity_28C4C8FB7C957FE2FA60

    unity_28C4C8FB7C957FE2FA60

    Joined:
    Oct 31, 2021
    Posts:
    1
    I had the same problem. I also use M1 MAX.
     
    unity_28C4C8FB7C957FE2FA60, Dec 17, 2021
    #11
    unitybru likes this.

  12. luispedrofonseca

    luispedrofonseca

    Joined:
    Aug 29, 2012
    Posts:
    945
    I'm also on the same boat (M1 Max).
     
    luispedrofonseca, Dec 21, 2021
    #12
    unitybru likes this.

  13. vladala

    vladala

    Unity Technologies

    Joined:
    Mar 3, 2017
    Posts:
    189
    Are you using the x86 build of the editor or are running a native Arm64 Unity Editor build ?
     
    vladala, Jan 4, 2022
    #13

  14. AntonioFroger

    AntonioFroger

    Joined:
    Jul 19, 2019
    Posts:
    5
    I'm also on the same boat (M1 Max).
     
    AntonioFroger, Jan 19, 2022
    #14

  15. AntonioFroger

    AntonioFroger

    Joined:
    Jul 19, 2019
    Posts:
    5
    Failed to add video frame to ProRes encoder
    UnityEngine.Debug:LogError (object)
    Unity.Media.ProResEncoderCore:AddFrame (int,int,int,UnityEngine.TextureFormat,Unity.Collections.NativeArray`1<byte>) (at Library/PackageCache/com.unity.recorder@3.0.3/Editor/ProRes/ProResEncoderPlugin.cs:418)
    Unity.Media.ProResEncoder:AddFrame (int,int,int,UnityEngine.TextureFormat,Unity.Collections.NativeArray`1<byte>) (at Library/PackageCache/com.unity.recorder@3.0.3/Editor/ProRes/ProResEncoderPlugin.cs:600)
    Unity.Media.MediaEncoderManager:AddFrame (Unity.Media.MediaEncoderHandle,int,int,int,UnityEngine.TextureFormat,Unity.Collections.NativeArray`1<byte>) (at Library/PackageCache/com.unity.recorder@3.0.3/Editor/Encoding/MediaEncoder.cs:396)
    UnityEditor.Recorder.BaseTextureRecorder`1<UnityEditor.Recorder.MovieRecorderSettings>:ReadbackDone (UnityEngine.Rendering.AsyncGPUReadbackRequest) (at Library/PackageCache/com.unity.recorder@3.0.3/Editor/Sources/BaseTextureRecorder.cs:118)
     
    AntonioFroger, Jan 19, 2022
    #15

  16. AntonioFroger

    AntonioFroger

    Joined:
    Jul 19, 2019
    Posts:
    5
    2020.3.26f1c1 LTS intel : Failed
    2021.2.8f1c1 apple silicon : Failed

    macOS Montery 12.1
    Apple M1 Max
     
    AntonioFroger, Jan 19, 2022
    #16

  17. vladala

    vladala

    Unity Technologies

    Joined:
    Mar 3, 2017
    Posts:
    189
    This is unfortunately an Apple bug: https://developer.apple.com/forums/thread/694622?login=true
    Hopefully they'll fix it soon.
     
    vladala, Jan 20, 2022
    #17

  18. sameng

    sameng

    Joined:
    Oct 1, 2014
    Posts:
    184
    Same problem here, still unfixed.
     
    sameng, Apr 27, 2022
    #18

  19. vladala

    vladala

    Unity Technologies

    Joined:
    Mar 3, 2017
    Posts:
    189
    Hey,

    Recorder version 4.0.0-pre.4 should support 422* codecs on M1 Max (compatible only with Unity 2022.1 and newer)
    Hopefully the next recorder version will contain the fix for 4444 codecs.

    Hope this helps.
     
    vladala, Apr 28, 2022
    #19
    ilmario and sameng like this.

  20. sameng

    sameng

    Joined:
    Oct 1, 2014
    Posts:
    184
    Thanks for the update! I'm glad to hear it's fixed in 2022.1.

    Is there any chance of a new Recorder version compatible with Unity 2021 LTS?
    Or is Recorder entirely leaving behind 2021 from now on? (The last compatible version being pre.3 ?)
     
    sameng, Apr 29, 2022
    #20

  21. vladala

    vladala

    Unity Technologies

    Joined:
    Mar 3, 2017
    Posts:
    189
    Unfortunately it is close to impossible to back port this fix for older recorder versions (compatible with 2021 LTS) since it relies on a lot of refactors we did in the latest version.

    You are stuck with either using a non-M1 Max CPU or wait for apple to fix the underlying issue in the OS. Here's a thread: https://developer.apple.com/forums/thread/694622
     
    vladala, May 4, 2022
    #21

  22. sameng

    sameng

    Joined:
    Oct 1, 2014
    Posts:
    184
    Prores encoder still broken under M1 on macOS 12.4
     
    sameng, May 18, 2022
    #22

  23. vladala

    vladala

    Unity Technologies

    Joined:
    Mar 3, 2017
    Posts:
    189
    Hey,
    Can you update to recorder 4.0.0-pre.4 ? (Unity 2022.1 and newer). Because of a refactor we did, we were able to fix the M1 Max ProRes issues.
     
    Last edited: May 25, 2022
    vladala, May 24, 2022
    #23

  24. sameng

    sameng

    Joined:
    Oct 1, 2014
    Posts:
    184
    I'm glad it seems to be fixed but unfortunately my project is totally broken under Unity 2022.1 so I cannot update.

    (due to a change in the blender importer, which destroys the scaling of all the .blend files in my project, making it unplayable)

    For now I am just using my old intel mac on 2021.3 LTS to record footage, where the prores encoder works great.
     
    sameng, May 25, 2022
    #24

  25. thesoulharmonic

    thesoulharmonic

    Joined:
    Sep 25, 2020
    Posts:
    2
    This worked for me, is there anyway to get recorder 4.0.0 for 2021.2 LTS?
     
    thesoulharmonic, Jun 14, 2022
    #25

  26. vladala

    vladala

    Unity Technologies

    Joined:
    Mar 3, 2017
    Posts:
    189
    Unfortunately no. This fix is based a very large refactor we did for this version (that contains a lot of 2022.x dependent features)
     
    vladala, Jun 14, 2022
    #26

  27. thesoulharmonic

    thesoulharmonic

    Joined:
    Sep 25, 2020
    Posts:
    2
    OK, Great work though, thank you
     
    thesoulharmonic, Jun 14, 2022
    #27

  28. Lorin_Atzberger

    Lorin_Atzberger

    Joined:
    Dec 11, 2013
    Posts:
    119
    I too am having issues with this on a M1 Pro macbook using Unity 2021.3.1f1 and recorder 3.0.3.
    Unfortunately migrating to Unity 2022 is not a solution and seems unreasonable.
     
    Lorin_Atzberger, Jul 4, 2022
    #28

  29. nitrocaphane

    nitrocaphane

    Joined:
    Mar 27, 2022
    Posts:
    5
    Omg M1 user here, it's 2023 and I still encounter the same issue. -_-
     
    nitrocaphane, May 23, 2023
    #29