Video Unity recorder alpha background

Hi guys,
I'm trying to record a png sequence of an animated character but i need an alpha BG behind it.
I've made several test to get that result.

1. I've tried deferred rendering, checked capture alpha in the recorderclip, my camera clear flags is set to solid color, but it seems that the post processing effects(which i need for the AA) prevent the alpha to be written in the png, i simply get an opaque BG with my character.
2. I've changed to forward rendering, i've set the AA in the quality settings to 8xMS, made sure the camera rendering path is set to forward, no post processing, same camera flags as above and checked capture alpha. This way is working i finally get a transparent BG behind my character BUT the MSAA doesn't seems to be applied. I'm not sure why but when i change my collection method in my recorderclip to "screen" the MSAA seems to be applied to the png, but i have an opaque BG.

So to you what is the best way to record a png sequence with a transparent BG ?
How would you do that but still have a fine AA on the character ?
I'm simply looking for the best way to achieve that with the best quality !

Could you help me with that ?

My unity version: 2017.3.1.f.1

Cheers

Yann

 

I've been rendering twice - Once to get alpha (no Post Processing and Camera Clear Flag to solid) and then exporting again to get the scene as it was originally. [Alpha Matte the Beauty essentially]
Aliasing is there for the Matte pass but I just "super-sample" and it works for my uses - Better than faking Greenscreen and Keying lol...

Maybe try this plugin? "Offline Render".
Let us know if it works for what you're trying to do.

 

1-make a camera
2-check mark transparency
3-Export

 

Hi!
Check this video

 

But my exported sequence somehow detecting black shadows as alpha! Not working properly. Anybody facing this issue?

 

I am facing this issue after I change my pipeline to Universal RP...But don't know how to fix this..Anyone fix this before??

 

use URP, I can not get transparent backgroud,neither.

 

Can't get alpha in mah exports too

 

I can't get this to work either fiddled with a lot of settings. Some other post mentioning something about post processing messing it up, but I need PP on the exported images. I tried turning it off anyway on the camera settings, and it didn't help.

Confirmed it works in standard renderer, it's only in URP it doesn't work.

 

I was able to render using a transparent background using URP by setting the capture Source in the Recorder to "Texture Sampling" and in my camera component settings setting the background to a solid color with a transparent alpha

 

Struggled with this for a while on URP (2021.2.11), here's what I had to do to make it work for me with WebM transparent video capture or transparent PNG image sequences :

• Downgrade Unity Recorder to 2.5.7
• Do as @warrenLD suggested above, except instead of using Texture Sampling, use TargetedCamera

Hope this helps !

 

THANK YOU!

Just wanted to bump this because it was the only thing that worked for me. 2019.4.31f1, built-in forward. I had to set the background color to black with 0 alpha (rather than a color with 0 alpha) because the color was showing up in partly transparent portions.

Recorder 2.5.5 and 3.x fails to save the alpha, in PNGs and in WebM.

 

Works great, but doesn't take in post fx like bloom is there isn't an opaque surface behind it...

 

I noticed when trying to troubleshoot this, PNGs do not work if post processing is on at all, downgraded from 2021.3 to 2021.2.19f and it works but cannot get WEBMs to work tho.

 

makes no difference if you switch on post or not, the alpha still has non existant AA

 

I'm not having aliasing problems TAA with frame accumulation between 32 and 128 gives me extremely clean shots.

 

So what are you saying? You dont have the alpha problem anymore if using those settings for accumulation?

So your post processing works great and you get nice alpha against a blank backplate with high accumulation settings?

 

No, looking at your problem, you have major degradation in the how alpha edges is computed, giving it really aggressive aliasing. - I had that problem previous to this one, I solved it by rendering out higher resolution shots with higher accumulation sampling using targeted camera in the recorder. - Assuming I understand your problem correctly.

My problem is that recorder will not consider bloom post effects into any of render passes available in HDRP - it's now my strong belief that this is a result of design. So not a bug, but also not a feature either.

 

Yeah I'm rendering 4K then going back down to 1080. But that frankly is rediculous, because what if my final output should be 4K? I cant render 8K and take it down to 4K to fix alpha AA!

Might not be by design, but more of a design oversight. I mean, how's virtual production suppose to work then if you cant have bloom over CG element edges that should seemlesly gel with live backplates (or 3rd party video backplates)

 

I just want some transparency from Unity on this.

 

Have you had any luck with Deckard or Offline renderer?

 

I have a Deckard license and its nice, although it is too slow for my taste in my initial experiments. Olly is very knowlegeable. I played around with it for a short time, too short to make an absolute conclusion about it. I may try it again in the future.

I havent tried Offline Render, however I know the devs from our background from DCC and 3rd party renderers. Between the 2 of them, I'd say Deckard is probably the better option. However, offline render has some of the typical render elements (AOV) it can output for you very easily.

Neither seems to get updated often, however both's devs are quick to responses to queries (if you follow the correct non-official channels LOL). For Deckard, its seems Olly is more responsive if you contact him via non Unity channels (aparantly), and Offline Render is quick to respond via social media (although in my case that may just be because I know them for a long time already before I even started using Unity).

I also tried a certain version 5 of another realtime engine which shall not be named (lol), and I absolutely LOVE the fact that you get automatic reflections with their new system, without sticking in reflection probes, nevermind the their nice new GI system. The system in general feels more responsive and runs smoother than verion 4.x. However Ive had some other hurdles that I could not solve yet, unsure if lack of my knowledge or bugs...

All that being said, I'm sticking with Unity's own Recorder for now and sometimes throwing in some Raytracing also. The new APV (adaptive probe volume) that is experimental in 2022.2 seems VERY Promising for GI and the next iteration will apparantly have a bunch of fixes. Just wish they can come up with an automatic reflection thing like that other engine...

 

That used to work for me, now on a new project the only way I got it working again was to set the camera up to render to a RenderTexture with an alpha channel in addition to downgrading the package and using TargetedCamera. For everything else I tried the alpha channel was always dropped for some reason.

 

This is getting super aggravating because none of this seems to be working and on any other software rendering with alpha channel is no problem at all... but I want to do it in Unity! With Recorder 2.5.7 (Verified) it isn't even an option. With Recorder 3.0.3, nothing I can change in settings seems to help. Using a Targeted & Tagged Camera with all the geometry on a culling mask layer. PNG doesn't save an alpha channel at all, and EXR saves a alpha that is solid white. Using HDRP in Unity 2020.3.24 - what am I missing?

 

set your camera background to black and dont use fog
then it will save with alpha.

But be warned, the alpha will likely have serious AA artefacts/issues, or should I say total lack of AA, as per here:
https://forum.unity.com/threads/recorder-case-1423485-premultiplied-alpha-bug-png-and-exr.1195888/

 

It's not worked for me even I checked Include Alpha & changed BG type to Solid and made alpha is 0 , here is my setting in Renderer View and Camera, I'm using Unity Recorder 2.5.7

 

Have you tried using a RenderTexture with alpha channel as the camera's Output Texture? As I said before this additional step was the only way I could get proper alpha to work on new projects.

 

When I do this I'm just getting a totally transparent image - when I slot in a render tex I get "No cameras rendering" in my gameview - do I need a secondary camera?

 

I don't think you need a second camera as long as the recorder is set to capture from a "Targeted Camera" that is picked up correctly (it's also not a problem then if no camera renders to the screen). These are the settings that work for me with Unity Recorder version 2.5.7 in Unity 2021.3.1f1 (top right are the camera settings, that game object has the tag "ItemCamera" referenced in the Recorder settings on the left):

Note that the RenderTexture resolution doesn't seem to matter, with those settings I get images with 2160x1260 pixels as set in the Recorder settings, not the 2048x2048 from the RenderTexture.

 

HEY GUYS!!!! Did you look at the code in Unity Recorder? I found that RGB24 ONLY was configured in imageRecorder.CS!!! That completely foooked my copy of UnityRecorder in U3D 2017. It's probably fixed for later versions.

I changed the RGB reference to RGBA32. Now transparency works fine on default settings.
Camera background solidcolor Alpha channel must be zero.

You should study the codes for your issues though. it's a really simple program, it takes a render texture, and does a tex2D to PNG on it. For the render pipeline, it can be fine-tuned through the imageRecorder.cs code. Please keep me updated for your findings I would be very interested to know about finer details like bloom and smoke become fudged up by the tex2d to PNG process. I would consider implementing a green screen / chroma key and running my own pixel loop on the PNG encoder so that "Green + any other colors is converted to "transparent + colors" ... you can also select blue/red, as long as you code your own chroma exporter, chances are that you can work around the buggy encoder used by the default code. Any ideas on that for bloom and render path would be cool. How can I detect smoke levels of transparency and Alpha levels from textures in between the background and the camera? Why wouldn't unity use the Z buffer to turn alpha levels on smoke textures and bloom to alpha levels in the tex2D from the texture renderer? best ask the shader graphics card forum pro's that. they could have some advice.


For that reason, every element of the Tex2D to PNG encoding process can be accessed... I believe that renderTexture has to have some alpha setting in mediarecorder.cs like:

Code (CSharp):

1.             var cbRenderTextureInput = m_Inputs[0] as CBRenderTextureInput;
2.  
3.             bool includeAlphaFromTexture = cbRenderTextureInput != null && cbRenderTextureInput.cbSettings.m_AllowTransparency;
4.             if (includeAlphaFromTexture && m_Settings.m_OutputFormat == MediaRecorderOutputFormat.MP4)
5.             {
6.                 Debug.LogWarning("Mp4 format does not support alpha.");
7.                 includeAlphaFromTexture = false;
8.             }

and the line which was wrong from imageRecorder.cs was this:

Code (CSharp):

1. #if UNITY_2017_3_OR_NEWER
2.             if (m_Inputs[0] is ScreenCaptureInput)
3.             {
4.                 tex = ((ScreenCaptureInput)m_Inputs[0]).image;
5.                 if (m_Settings.m_OutputFormat == PNGRecordeOutputFormat.EXR)
6.                 {
7.                     var textx = new Texture2D(tex.width, tex.height, TextureFormat.RGBAFloat, false);
8. ABOVE LINE WAS WRONG ^^^^^^^^
9.                     textx.SetPixels(tex.GetPixels());
10.                     tex = textx;
11.                 }
12.                 else if (m_Settings.m_OutputFormat == PNGRecordeOutputFormat.PNG)
13.                 {
14.                     var textx = new Texture2D(tex.width, tex.height, TextureFormat.RGBA32, false); <<<<<<<<<<<<<<WAS RGB24
15.                     textx.SetPixels(tex.GetPixels());
16.                     tex = textx;
17.                 }
18.             }
19.             else
20. #endif
21.             {
22.                 var input = (BaseRenderTextureInput)m_Inputs[0];
23.                 var width = input.outputRT.width;
24.                 var height = input.outputRT.height;
25.                 tex = new Texture2D(width, height, m_Settings.m_OutputFormat != PNGRecordeOutputFormat.EXR ? TextureFormat.RGBA32 : TextureFormat.RGBAFloat, false);
26.                 var backupActive = RenderTexture.active;
27.                 RenderTexture.active = input.outputRT;
28.                 tex.ReadPixels(new Rect(0, 0, width, height), 0, 0, false);
29.                 tex.Apply();
30.                 RenderTexture.active = backupActive;
31.             }
32.  
33.             byte[] bytes;
34.             string ext;
35.             switch (m_Settings.m_OutputFormat)
36.             {
37.                 case PNGRecordeOutputFormat.PNG:
38.                     bytes = tex.EncodeToPNG();
39.                     ext = "png";
40.                     break;

This is the image i had first attempt after changing the line, smoke and fire works fine, my version of unity recorder is from 2017 it's so old it's called "generic frame recorder" here is my copy.

 

I dunno about offline render and deckard, if you want good editing and screen capture, the best capture on windows is OBS.... Shotcut is a free high quality video editor, you can drag the first image of the transparent sequence, click "image sequence" on it, and it will run as a movie with up to 20 other tracks in foreground and background and a choice of 40 video effects like resize crop move mosaic color grading, and audio filters and so forth. very stable, although adobe tools are more cool if you have them.

 

Yeah OBS is great. However for our purpose with Unity its useless as we do offline rendering with Unity and raytricing/pathtracing and generally that does not run in realtime 60 FPS, therefore offline rendering

 

That's very hi-tech I'd be interested to see your work if it's on the web.

 

I am just facing this Issue.
How fast does the Unity team usually fix issues? (I'm a Unity newbie)

The fixes shared above were too complicated and unavailable for me.
It would be possible to get away with chromakey, but not ideal.

Now you guys are talking about OBS, is there a well-known practice to output transparent video using OBS?

 

Hi all, this topic became relevant for me lately. My goal was to have a few moving objects from a Unity scene recorded as a video, with everything else (e.g. background/environment) showing up as transparent in the recording; so that this can be used in compositing with real-world footage (a mixed-reality mockup).

Below is my current setup for capturing videos with transparency in Unity, in a URP project.

• Create a new camera distinct from the main camera
• Set the camera's Environment > Background Type to a solid color, with zero alpha
• Add a Tag of your choice to the camera, e.g. "Transparency Camera"
• In the Recorder, add a new Movie Recorder
• Set Capture > Source to "Texture Sampling", Camera to "Tagged Camera" and Tag to the tag you chose above (e.g. "Transparency Camera").
• Set Format > Media File Format to a format that supports transparency (e.g. ProRes QuickTime), you can see an "Include Alpha" checkbox below if that's the case
• Check Format > include Alpha

Optional steps (from this YouTube Video)

• Set the camera's Rendering > Priority to -1 so that it does not supplant the Main Camera in Game View
• Reassign the objects you want to capture to an "Opaque Capture" layer, and set the camera's Rendering > Culling Mask to that layer only (so that it only renders these objects)

Below is the result playback in MPV with checkerboard background indicating transparent areas.

 

Which unity recorder and unity versions did you run?

 

Hi, I've solved the issue by just turning off HDR and Post-processing on my camera. I'm on 2020.3.42f1.

My setup is to use a camera with a tag, and only use culling mask on a layer with exactly those objects I want to render.
Select Solid Color with 0 alpha on the camera.
Select PNG and Include alpha in the recorder.

Under the renderer file, I have opaque layer mask on Nothing and Transparent layer mask on my same layer as the objects.

Voila.

If I recall correctly, the issue here is that Post-processing in URP can not render an alpha pass.

 

Hello everyone.

I' ve successfuly made screenshots capture in URP, with RenderTexture.
I've didn't tested video recording

 

i tried everything in this thread with Uniy 2021.3.11f1 and the release version of the recorder nothing worked!

i couldnt find how to get Unity Recorder 2.5.7, can anyone help me ?

 

i think urp is/was the problem, with the built-in render pipeline i could at least get image seuqence to work, video not.
this video helped me, according to the comments hdrp could work too

 

I got it to work in the end by changing the code, I checked all the formats and saw that it wasn't RGBA at some stage, so i changed the format of the png write or whatever and it worked.

 

i saw your post about that and tried to find where the code is located for the recoder but i couldnt find it ! images are working now anyways, would like to get video to work

 

I've managed to render transparent video using URP.
When selecting the codec for the video (ProRes QuickTime, 4444XQ), I do not see the "include alpha" checkbox when on PC, but I do on macOS, so this was an issue for me.

Another issue I've has was, as Lepisto3D mentioned:

Having post-processing enabled on the recording camera effectively removes the transparency. This seems to be an issue with the post-processing shader (it does not return a useful alpha value), see: https://forum.unity.com/threads/transparent-rendertexture-with-postprocessing.1265873/

 

Sorry for the late reply, hope it helps nonetheless. I am currently using

Unity 2021.3.13f1
URP 12.1.7
Recorder 3.0.0
on macOS 13.3

 

To downgrade to @2.5.7

you need to in package window
(top left) add package by name

name: com.unity.recorder
version: 2.5.7

 

so one may think taking transparent image will be easy... lol
does it work in urp 12? 2021.3f?

tried disabling enabling PP, changing alpha etc..

thanks

 

any trick to get alpha rendered for built-in?

 

Recorder 2.5.7 doesn't seem to work with Unity 2022 and latest URP.

Anyone managed to record with transparency with the latest versions of Unity/URP?