Search Unity

Video Unity recorder alpha background

Discussion in 'Audio & Video' started by yanndrevon, Apr 13, 2018.

  1. yanndrevon

    yanndrevon

    Joined:
    Jul 15, 2014
    Posts:
    1
    Hi guys,
    I'm trying to record a png sequence of an animated character but i need an alpha BG behind it.
    I've made several test to get that result.
    1. I've tried deferred rendering, checked capture alpha in the recorderclip, my camera clear flags is set to solid color, but it seems that the post processing effects(which i need for the AA) prevent the alpha to be written in the png, i simply get an opaque BG with my character.
    2. I've changed to forward rendering, i've set the AA in the quality settings to 8xMS, made sure the camera rendering path is set to forward, no post processing, same camera flags as above and checked capture alpha. This way is working i finally get a transparent BG behind my character BUT the MSAA doesn't seems to be applied. I'm not sure why but when i change my collection method in my recorderclip to "screen" the MSAA seems to be applied to the png, but i have an opaque BG.:confused:
    So to you what is the best way to record a png sequence with a transparent BG ?
    How would you do that but still have a fine AA on the character ?
    I'm simply looking for the best way to achieve that with the best quality !

    Could you help me with that ?;)

    My unity version: 2017.3.1.f.1

    Cheers

    Yann
     
    newguy123 likes this.
  2. Thr3dee

    Thr3dee

    Joined:
    Aug 29, 2017
    Posts:
    5
    I've been rendering twice - Once to get alpha (no Post Processing and Camera Clear Flag to solid) and then exporting again to get the scene as it was originally. [Alpha Matte the Beauty essentially]
    Aliasing is there for the Matte pass but I just "super-sample" and it works for my uses - Better than faking Greenscreen and Keying lol...

    Maybe try this plugin? "Offline Render".
    Let us know if it works for what you're trying to do.
     
    angelsm85 likes this.
  3. InfiniteDesign8

    InfiniteDesign8

    Joined:
    Aug 22, 2017
    Posts:
    38
    1-make a camera
    2-check mark transparency
    3-Export
     
  4. Ramzesmeister

    Ramzesmeister

    Joined:
    May 2, 2020
    Posts:
    1
    Hi!
    Check this video
     
  5. NStamal

    NStamal

    Joined:
    Oct 20, 2012
    Posts:
    1
    But my exported sequence somehow detecting black shadows as alpha! Not working properly. Anybody facing this issue?
     
    xiangshushu likes this.
  6. tansteven96249464

    tansteven96249464

    Joined:
    Mar 9, 2021
    Posts:
    3
    I am facing this issue after I change my pipeline to Universal RP...But don't know how to fix this..Anyone fix this before??
     
  7. xiangshushu

    xiangshushu

    Joined:
    Mar 17, 2015
    Posts:
    2
    use URP, I can not get transparent backgroud,neither.
     
  8. ErdTod

    ErdTod

    Joined:
    Jan 7, 2018
    Posts:
    11
    Can't get alpha in mah exports too :(
     
  9. C4ro

    C4ro

    Joined:
    Oct 5, 2014
    Posts:
    9
    I can't get this to work either :( fiddled with a lot of settings. Some other post mentioning something about post processing messing it up, but I need PP on the exported images. I tried turning it off anyway on the camera settings, and it didn't help.

    Confirmed it works in standard renderer, it's only in URP it doesn't work.
     
  10. warrenLD

    warrenLD

    Joined:
    Jul 2, 2019
    Posts:
    9
    I was able to render using a transparent background using URP by setting the capture Source in the Recorder to "Texture Sampling" and in my camera component settings setting the background to a solid color with a transparent alpha Unity_261_1816_February04_2022.png Unity_262_1816_February04_2022.png
     
  11. rob11

    rob11

    Joined:
    Mar 7, 2017
    Posts:
    59
    Struggled with this for a while on URP (2021.2.11), here's what I had to do to make it work for me with WebM transparent video capture or transparent PNG image sequences :
    • Downgrade Unity Recorder to 2.5.7
    • Do as @warrenLD suggested above, except instead of using Texture Sampling, use TargetedCamera
    Hope this helps !
     
  12. laessnb

    laessnb

    Joined:
    Jun 10, 2014
    Posts:
    101
    THANK YOU!

    Just wanted to bump this because it was the only thing that worked for me. 2019.4.31f1, built-in forward. I had to set the background color to black with 0 alpha (rather than a color with 0 alpha) because the color was showing up in partly transparent portions.

    Recorder 2.5.5 and 3.x fails to save the alpha, in PNGs and in WebM.
     
  13. Thall33

    Thall33

    Joined:
    Sep 18, 2013
    Posts:
    134
    Works great, but doesn't take in post fx like bloom is there isn't an opaque surface behind it...
     
  14. OtakuD

    OtakuD

    Joined:
    Apr 24, 2014
    Posts:
    49
    I noticed when trying to troubleshoot this, PNGs do not work if post processing is on at all, downgraded from 2021.3 to 2021.2.19f and it works but cannot get WEBMs to work tho.
     
  15. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    makes no difference if you switch on post or not, the alpha still has non existant AA
     
  16. Thall33

    Thall33

    Joined:
    Sep 18, 2013
    Posts:
    134
    I'm not having aliasing problems TAA with frame accumulation between 32 and 128 gives me extremely clean shots.
     
  17. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    So what are you saying? You dont have the alpha problem anymore if using those settings for accumulation?

    So your post processing works great and you get nice alpha against a blank backplate with high accumulation settings?
     
  18. Thall33

    Thall33

    Joined:
    Sep 18, 2013
    Posts:
    134
    No, looking at your problem, you have major degradation in the how alpha edges is computed, giving it really aggressive aliasing. - I had that problem previous to this one, I solved it by rendering out higher resolution shots with higher accumulation sampling using targeted camera in the recorder. - Assuming I understand your problem correctly.

    My problem is that recorder will not consider bloom post effects into any of render passes available in HDRP - it's now my strong belief that this is a result of design. So not a bug, but also not a feature either.
     
  19. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    Yeah I'm rendering 4K then going back down to 1080. But that frankly is rediculous, because what if my final output should be 4K? I cant render 8K and take it down to 4K to fix alpha AA!

    Might not be by design, but more of a design oversight. I mean, how's virtual production suppose to work then if you cant have bloom over CG element edges that should seemlesly gel with live backplates (or 3rd party video backplates)
     
  20. Thall33

    Thall33

    Joined:
    Sep 18, 2013
    Posts:
    134
    I just want some transparency from Unity on this.
     
    Retrokat likes this.
  21. Thall33

    Thall33

    Joined:
    Sep 18, 2013
    Posts:
    134
    Have you had any luck with Deckard or Offline renderer?
     
  22. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    I have a Deckard license and its nice, although it is too slow for my taste in my initial experiments. Olly is very knowlegeable. I played around with it for a short time, too short to make an absolute conclusion about it. I may try it again in the future.

    I havent tried Offline Render, however I know the devs from our background from DCC and 3rd party renderers. Between the 2 of them, I'd say Deckard is probably the better option. However, offline render has some of the typical render elements (AOV) it can output for you very easily.

    Neither seems to get updated often, however both's devs are quick to responses to queries (if you follow the correct non-official channels LOL). For Deckard, its seems Olly is more responsive if you contact him via non Unity channels (aparantly), and Offline Render is quick to respond via social media (although in my case that may just be because I know them for a long time already before I even started using Unity).

    I also tried a certain version 5 of another realtime engine which shall not be named (lol), and I absolutely LOVE the fact that you get automatic reflections with their new system, without sticking in reflection probes, nevermind the their nice new GI system. The system in general feels more responsive and runs smoother than verion 4.x. However Ive had some other hurdles that I could not solve yet, unsure if lack of my knowledge or bugs...

    All that being said, I'm sticking with Unity's own Recorder for now and sometimes throwing in some Raytracing also. The new APV (adaptive probe volume) that is experimental in 2022.2 seems VERY Promising for GI and the next iteration will apparantly have a bunch of fixes. Just wish they can come up with an automatic reflection thing like that other engine...
     
  23. uwdlg

    uwdlg

    Joined:
    Jan 16, 2017
    Posts:
    150
    That used to work for me, now on a new project the only way I got it working again was to set the camera up to render to a RenderTexture with an alpha channel in addition to downgrading the package and using TargetedCamera. For everything else I tried the alpha channel was always dropped for some reason.
     
  24. TranSystems

    TranSystems

    Joined:
    Jan 10, 2018
    Posts:
    11
    This is getting super aggravating because none of this seems to be working and on any other software rendering with alpha channel is no problem at all... but I want to do it in Unity! With Recorder 2.5.7 (Verified) it isn't even an option. With Recorder 3.0.3, nothing I can change in settings seems to help. Using a Targeted & Tagged Camera with all the geometry on a culling mask layer. PNG doesn't save an alpha channel at all, and EXR saves a alpha that is solid white. Using HDRP in Unity 2020.3.24 - what am I missing?
     
  25. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    set your camera background to black and dont use fog
    then it will save with alpha.

    But be warned, the alpha will likely have serious AA artefacts/issues, or should I say total lack of AA, as per here:
    https://forum.unity.com/threads/recorder-case-1423485-premultiplied-alpha-bug-png-and-exr.1195888/
     
  26. goals936

    goals936

    Joined:
    Nov 2, 2018
    Posts:
    1
    It's not worked for me even I checked Include Alpha & changed BG type to Solid and made alpha is 0 , here is my setting in Renderer View and Camera, I'm using Unity Recorder 2.5.7

    upload_2022-9-2_3-9-47.png

    upload_2022-9-2_3-10-26.png

    upload_2022-9-2_3-12-55.png
     
  27. uwdlg

    uwdlg

    Joined:
    Jan 16, 2017
    Posts:
    150
    Have you tried using a RenderTexture with alpha channel as the camera's Output Texture? As I said before this additional step was the only way I could get proper alpha to work on new projects.
     
  28. ggreer

    ggreer

    Joined:
    May 22, 2019
    Posts:
    5
    When I do this I'm just getting a totally transparent image - when I slot in a render tex I get "No cameras rendering" in my gameview - do I need a secondary camera?
     
  29. uwdlg

    uwdlg

    Joined:
    Jan 16, 2017
    Posts:
    150
    I don't think you need a second camera as long as the recorder is set to capture from a "Targeted Camera" that is picked up correctly (it's also not a problem then if no camera renders to the screen). These are the settings that work for me with Unity Recorder version 2.5.7 in Unity 2021.3.1f1 (top right are the camera settings, that game object has the tag "ItemCamera" referenced in the Recorder settings on the left):

    Note that the RenderTexture resolution doesn't seem to matter, with those settings I get images with 2160x1260 pixels as set in the Recorder settings, not the 2048x2048 from the RenderTexture.
     
  30. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    HEY GUYS!!!! Did you look at the code in Unity Recorder? I found that RGB24 ONLY was configured in imageRecorder.CS!!! That completely foooked my copy of UnityRecorder in U3D 2017. It's probably fixed for later versions.

    I changed the RGB reference to RGBA32. Now transparency works fine on default settings.
    Camera background solidcolor Alpha channel must be zero.

    You should study the codes for your issues though. it's a really simple program, it takes a render texture, and does a tex2D to PNG on it. For the render pipeline, it can be fine-tuned through the imageRecorder.cs code. Please keep me updated for your findings I would be very interested to know about finer details like bloom and smoke become fudged up by the tex2d to PNG process. I would consider implementing a green screen / chroma key and running my own pixel loop on the PNG encoder so that "Green + any other colors is converted to "transparent + colors" ... you can also select blue/red, as long as you code your own chroma exporter, chances are that you can work around the buggy encoder used by the default code. Any ideas on that for bloom and render path would be cool. How can I detect smoke levels of transparency and Alpha levels from textures in between the background and the camera? Why wouldn't unity use the Z buffer to turn alpha levels on smoke textures and bloom to alpha levels in the tex2D from the texture renderer? best ask the shader graphics card forum pro's that. they could have some advice.


    For that reason, every element of the Tex2D to PNG encoding process can be accessed... I believe that renderTexture has to have some alpha setting in mediarecorder.cs like:

    Code (CSharp):
    1.             var cbRenderTextureInput = m_Inputs[0] as CBRenderTextureInput;
    2.  
    3.             bool includeAlphaFromTexture = cbRenderTextureInput != null && cbRenderTextureInput.cbSettings.m_AllowTransparency;
    4.             if (includeAlphaFromTexture && m_Settings.m_OutputFormat == MediaRecorderOutputFormat.MP4)
    5.             {
    6.                 Debug.LogWarning("Mp4 format does not support alpha.");
    7.                 includeAlphaFromTexture = false;
    8.             }
    and the line which was wrong from imageRecorder.cs was this:
    Code (CSharp):
    1. #if UNITY_2017_3_OR_NEWER
    2.             if (m_Inputs[0] is ScreenCaptureInput)
    3.             {
    4.                 tex = ((ScreenCaptureInput)m_Inputs[0]).image;
    5.                 if (m_Settings.m_OutputFormat == PNGRecordeOutputFormat.EXR)
    6.                 {
    7.                     var textx = new Texture2D(tex.width, tex.height, TextureFormat.RGBAFloat, false);
    8. ABOVE LINE WAS WRONG ^^^^^^^^
    9.                     textx.SetPixels(tex.GetPixels());
    10.                     tex = textx;
    11.                 }
    12.                 else if (m_Settings.m_OutputFormat == PNGRecordeOutputFormat.PNG)
    13.                 {
    14.                     var textx = new Texture2D(tex.width, tex.height, TextureFormat.RGBA32, false); <<<<<<<<<<<<<<WAS RGB24
    15.                     textx.SetPixels(tex.GetPixels());
    16.                     tex = textx;
    17.                 }
    18.             }
    19.             else
    20. #endif
    21.             {
    22.                 var input = (BaseRenderTextureInput)m_Inputs[0];
    23.                 var width = input.outputRT.width;
    24.                 var height = input.outputRT.height;
    25.                 tex = new Texture2D(width, height, m_Settings.m_OutputFormat != PNGRecordeOutputFormat.EXR ? TextureFormat.RGBA32 : TextureFormat.RGBAFloat, false);
    26.                 var backupActive = RenderTexture.active;
    27.                 RenderTexture.active = input.outputRT;
    28.                 tex.ReadPixels(new Rect(0, 0, width, height), 0, 0, false);
    29.                 tex.Apply();
    30.                 RenderTexture.active = backupActive;
    31.             }
    32.  
    33.             byte[] bytes;
    34.             string ext;
    35.             switch (m_Settings.m_OutputFormat)
    36.             {
    37.                 case PNGRecordeOutputFormat.PNG:
    38.                     bytes = tex.EncodeToPNG();
    39.                     ext = "png";
    40.                     break;
    This is the image i had first attempt after changing the line, smoke and fire works fine, my version of unity recorder is from 2017 it's so old it's called "generic frame recorder" here is my copy.
     
    Last edited: Nov 13, 2022
    newguy123 likes this.
  31. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    I dunno about offline render and deckard, if you want good editing and screen capture, the best capture on windows is OBS.... Shotcut is a free high quality video editor, you can drag the first image of the transparent sequence, click "image sequence" on it, and it will run as a movie with up to 20 other tracks in foreground and background and a choice of 40 video effects like resize crop move mosaic color grading, and audio filters and so forth. very stable, although adobe tools are more cool if you have them.
     
  32. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    Yeah OBS is great. However for our purpose with Unity its useless as we do offline rendering with Unity and raytricing/pathtracing and generally that does not run in realtime 60 FPS, therefore offline rendering
     
  33. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    That's very hi-tech I'd be interested to see your work if it's on the web.
     
  34. Kuroco965

    Kuroco965

    Joined:
    Oct 26, 2021
    Posts:
    1
    I am just facing this Issue.
    How fast does the Unity team usually fix issues? (I'm a Unity newbie)

    The fixes shared above were too complicated and unavailable for me.
    It would be possible to get away with chromakey, but not ideal.

    Now you guys are talking about OBS, is there a well-known practice to output transparent video using OBS?
     
  35. Doc_Martins

    Doc_Martins

    Joined:
    Jun 1, 2020
    Posts:
    14
    Hi all, this topic became relevant for me lately. My goal was to have a few moving objects from a Unity scene recorded as a video, with everything else (e.g. background/environment) showing up as transparent in the recording; so that this can be used in compositing with real-world footage (a mixed-reality mockup).

    Below is my current setup for capturing videos with transparency in Unity, in a URP project.
    • Create a new camera distinct from the main camera
    • Set the camera's Environment > Background Type to a solid color, with zero alpha
    • Add a Tag of your choice to the camera, e.g. "Transparency Camera"
    • In the Recorder, add a new Movie Recorder
    • Set Capture > Source to "Texture Sampling", Camera to "Tagged Camera" and Tag to the tag you chose above (e.g. "Transparency Camera").
    • Set Format > Media File Format to a format that supports transparency (e.g. ProRes QuickTime), you can see an "Include Alpha" checkbox below if that's the case
    • Check Format > include Alpha
    Optional steps (from this YouTube Video)
    • Set the camera's Rendering > Priority to -1 so that it does not supplant the Main Camera in Game View
    • Reassign the objects you want to capture to an "Opaque Capture" layer, and set the camera's Rendering > Culling Mask to that layer only (so that it only renders these objects)
    Below is the result playback in MPV with checkerboard background indicating transparent areas.

    Recording With Alpha.jpg
     
  36. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    Which unity recorder and unity versions did you run?
     
  37. Lepisto3D

    Lepisto3D

    Joined:
    Oct 6, 2019
    Posts:
    27
    Hi, I've solved the issue by just turning off HDR and Post-processing on my camera. I'm on 2020.3.42f1.

    My setup is to use a camera with a tag, and only use culling mask on a layer with exactly those objects I want to render.
    Select Solid Color with 0 alpha on the camera.
    Select PNG and Include alpha in the recorder.

    Under the renderer file, I have opaque layer mask on Nothing and Transparent layer mask on my same layer as the objects.

    Voila.

    If I recall correctly, the issue here is that Post-processing in URP can not render an alpha pass.
     
    Doc_Martins and (deleted member) like this.
  38. mitaywalle

    mitaywalle

    Joined:
    Jul 1, 2013
    Posts:
    247
    Hello everyone.

    I' ve successfuly made screenshots capture in URP, with RenderTexture.
    I've didn't tested video recording
     
  39. Hazzel

    Hazzel

    Joined:
    Feb 26, 2022
    Posts:
    14
    i tried everything in this thread with Uniy 2021.3.11f1 and the release version of the recorder nothing worked!

    i couldnt find how to get Unity Recorder 2.5.7, can anyone help me ?
     
  40. Hazzel

    Hazzel

    Joined:
    Feb 26, 2022
    Posts:
    14
    i think urp is/was the problem, with the built-in render pipeline i could at least get image seuqence to work, video not.
    this video helped me, according to the comments hdrp could work too
     
    ggreer likes this.
  41. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    I got it to work in the end by changing the code, I checked all the formats and saw that it wasn't RGBA at some stage, so i changed the format of the png write or whatever and it worked.
     
    Hazzel likes this.
  42. Hazzel

    Hazzel

    Joined:
    Feb 26, 2022
    Posts:
    14
    i saw your post about that and tried to find where the code is located for the recoder but i couldnt find it ! images are working now anyways, would like to get video to work
     
  43. Doc_Martins

    Doc_Martins

    Joined:
    Jun 1, 2020
    Posts:
    14
    I've managed to render transparent video using URP.
    When selecting the codec for the video (ProRes QuickTime, 4444XQ), I do not see the "include alpha" checkbox when on PC, but I do on macOS, so this was an issue for me.

    Another issue I've has was, as Lepisto3D mentioned:
    Having post-processing enabled on the recording camera effectively removes the transparency. This seems to be an issue with the post-processing shader (it does not return a useful alpha value), see: https://forum.unity.com/threads/transparent-rendertexture-with-postprocessing.1265873/
     
  44. Doc_Martins

    Doc_Martins

    Joined:
    Jun 1, 2020
    Posts:
    14
    Sorry for the late reply, hope it helps nonetheless. I am currently using

    Unity 2021.3.13f1
    URP 12.1.7
    Recorder 3.0.0
    on macOS 13.3
     
  45. Renatusdev

    Renatusdev

    Joined:
    Dec 9, 2018
    Posts:
    35
    To downgrade to @2.5.7

    you need to in package window
    (top left) add package by name

    name: com.unity.recorder
    version: 2.5.7
     
    hypnoslave likes this.
  46. Lorrak

    Lorrak

    Joined:
    Sep 17, 2019
    Posts:
    50
    so one may think taking transparent image will be easy... lol
    does it work in urp 12? 2021.3f?

    tried disabling enabling PP, changing alpha etc..

    thanks
     
    Last edited: Oct 17, 2023
  47. Mori_X

    Mori_X

    Joined:
    Mar 17, 2023
    Posts:
    25
    any trick to get alpha rendered for built-in?
     
  48. dimitris_baud

    dimitris_baud

    Joined:
    Jan 15, 2018
    Posts:
    30
    Recorder 2.5.7 doesn't seem to work with Unity 2022 and latest URP.

    Anyone managed to record with transparency with the latest versions of Unity/URP?
     
    Lorrak likes this.