Question Extracting G-buffers for URP deferred shader

I'm interested in using Unity's Barracuda Machine learning platform, and one of the first steps to my project is extracting G-buffers to textures (for passing to a machine learning algorithm).

I was wondering what the best way to do this with URP would be.

 

Perhaps it would be better to simplify the question. How do I write G-buffers to texture in URP or HDRP? I'd like to do this for every frame.

 

Others may chime in here, but I'll take a stab.

IANAGP (I am not a graphics programmer), but I'd say you could probably create a deferred pass with multiple render targets to unpack any data you want. This will likely be helpful.

I also came across this old post and sample code.

 

Thank you so much for your response amirebrahmini!
It looks like every single one of the old posts you linked is for the legacy render pipeline. The new SRP based rendering pipelines presumably have slightly different interfaces for extracting these data. I am going to start much simpler and see if I can figure out how to extract render buffers ala this blog post: https://alexanderameye.github.io/notes/edge-detection-outlines/

Edit: If you have examples of G-buffer extraction using SRPs in URP or HDRP, that would be extremely useful!

 

The more I dive into doing this in URP the more difficult this is seeming.
So far I can vaguely ascertain that
com.unity.render-piplines.universal/shaders/utils/StencilDeferred.shader
pulls the GBuffers, extracts BRDFData and Surface data from these buffers then caclulates the final color.

I don't know anything about programming shaders, but as far as I can tell, one way to do this is to make a custom shader for each Buffer, figure out the correct operation for each buffer, then output that to a texture within the shader?????

The following (admittedly crappy) attempt at doing this results in glitchy output.

Code (CSharp):

1.  
2. Shader "CustomDeferred/GBuffer0"
3. {
4.     SubShader
5.     {
6.         Tags
7.         {
8.             "RenderType"="Opaque" "RenderPipeline" = "UniversalPipeline" "LightMode" = "UniversalGBuffer" "UniversalMaterialType" = "Lit"
9.         }
10.  
11.         HLSLINCLUDE
12.         #pragma vertex vert
13.         #pragma fragment frag
14.  
15.         #include "Packages/com.unity.render-pipelines.universal/ShaderLibrary/Core.hlsl"
16.         #include "Packages/com.unity.render-pipelines.universal/Shaders/Utils/Deferred.hlsl"
17.         #include "Packages/com.unity.render-pipelines.universal/ShaderLibrary/Shadows.hlsl"
18.  
19.         struct Attributes
20.         {
21.             float4 positionOS : POSITION;
22.             float2 uv : TEXCOORD0;
23.         };
24.  
25.         struct Varyings
26.         {
27.  
28.             float4 positionHCS : SV_POSITION;
29.             float2 uv : TEXCOORD0;
30.         };
31.      
32.         #define GBUFFER0 0
33.  
34.  
35.         FRAMEBUFFER_INPUT_HALF(GBUFFER0);
36.  
37.  
38.         Varyings vert(Attributes IN)
39.         {
40.             Varyings OUT;
41.             OUT.positionHCS = TransformObjectToHClip(IN.positionOS.xyz);
42.             return OUT;
43.         }
44.         ENDHLSL
45.  
46.         Pass
47.         {
48.             Name "GBUFFER EXTRACTION"
49.  
50.             HLSLPROGRAM
51.             half4 frag(Varyings IN) : SV_TARGET
52.             {
53.                 half4 gbuffer0 = LOAD_FRAMEBUFFER_INPUT(GBUFFER0, IN.positionHCS.xyz);
54.  
55.  
56.                 return gbuffer0;
57.             }
58.             ENDHLSL
59.         }
60.     }  
61. }
62. }

Applying this shader to a material results in some seriously glitchy behavior. Omitting the tag
"LightMode" = "UniversalGBuffer", also seemed to result in a completely black render. Anyone with better shader programming knowledge want to chime in?

 

Bump - I just needed to do the same thing and I'm seeing that it's not yet possible???

In the URP source, I see that GBufferPass.cs has a OnCameraCleanup which seems to be ran right after gbuffer stuff is handled, which releases the rendertextures for the gbuffers? What if I want to access them?? I'm trying to access them to both read and then write to (for screen-space reflections) - but it seems like it's just not possible?

It's so close! Can anyone at Unity chime in?

 

Bump. It's stuff like this that will make tools like Barracuda actually more useful, imo

 

Bump, is the an easy way to have access to _CameraGBufferTexture2, like in previous Standard Pipeline ?

This is a nightmare so far, i have spent a day to do the simplest of thing with zero success

 

Have you tried with BuiltinRenderTextureType.GBuffer2?

 

Hi, thanks . this worked and without do anything else

 

So if anyone stumbles through this like I did and still can't make it work, try this:

Code (CSharp):

1.             CommandBuffer cmd = new CommandBuffer();
2.             cmd.name = "TESTING GBuffer TESTING";
3.             var gBuff = Shader.PropertyToID("GetBuff0");
4.  
5.             RenderTargetIdentifier rti = new RenderTargetIdentifier("_GBuffer1");
6.             cmd.GetTemporaryRT(gBuff, Screen.width, Screen.height, 0, FilterMode.Bilinear, RenderTextureFormat.ARGB32);
7.             cmd.Blit(rti, gBuff);
8.             cmd.SetGlobalTexture(ShaderIDs.CommandBuffer0, gBuff);
9.             cmd.ReleaseTemporaryRT(gBuff);
10.             context.ExecuteCommandBuffer(cmd);
11.             CommandBufferPool.Release(cmd);

For some reason

            cmd.Blit(BuiltinRenderTextureType.GBuffer0, gBuff); 

DOES NOT WORK. I had to go into the Deferred pass .cs, find these identifiers and do that by hand.

 

For me this didn't work as they have renamed _GBuffer1 etc. to include the screen resolution and buffer format etc.. Personally, I ended up having to use reflection to unpack the internal GbufferAttachments array from inside URP... I imagine this is potentially problematic though.. Praying that unity give us an official way to access the Gbuffers, like in the built in pipeline...

 

Wow this almost comical except it makes me cry.

 

In URP DeferredLights.cs:

Code (CSharp):

1. internal static readonly string[] k_GBufferNames = new string[]
2. {
3.     "_GBuffer0",
4.     "_GBuffer1",
5.     "_GBuffer2",
6.     "_GBuffer3",
7.     "_GBuffer4",
8.     "_GBuffer5",
9.     "_GBuffer6"
10. };

Doing `Shader.GetGlobalTexture("_GBuffer0");` worked for me with ScriptableRendererFeature

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.Rendering;
3. using UnityEngine.Rendering.Universal;
4.  
5. public class Gbuffer : ScriptableRendererFeature
6. {
7.     public ComputeShader computeShader;
8.     public RenderTexture rt;
9.  
10.     private CommandBuffer _cb;
11.  
12.     private GbufferPass _gbufferPass;
13.  
14.     public Gbuffer(ComputeShader computeShader)
15.     {
16.         this.computeShader = computeShader;
17.     }
18.  
19.     public override void Create()
20.     {
21.         _cb = new CommandBuffer();
22.         _cb.name = "CopyGbuffer";
23.  
24.         rt = new RenderTexture(Screen.width, Screen.height, 0, RenderTextureFormat.ARGB32, RenderTextureReadWrite.Linear);
25.         rt.enableRandomWrite = true;
26.         rt.useMipMap = false;
27.         rt.Create();
28.  
29.         _gbufferPass = new GbufferPass(computeShader, rt);
30.     }
31.  
32.     public override void AddRenderPasses(ScriptableRenderer renderer, ref RenderingData renderingData)
33.     {
34.         renderer.EnqueuePass(_gbufferPass);
35.     }
36.  
37.     protected override void Dispose(bool disposing)
38.     {
39.         DisposeUtil.Dispose(ref rt);
40.     }
41.  
42.     class GbufferPass : ScriptableRenderPass
43.     {
44.         private RenderTexture _rt;
45.         private ComputeShader _computeShader;
46.  
47.         public GbufferPass(ComputeShader computeShader, RenderTexture rt)
48.         {
49.             _computeShader = computeShader;
50.             _rt = rt;
51.             renderPassEvent = RenderPassEvent.AfterRenderingGbuffer;
52.         }
53.  
54.         public override void OnCameraSetup(CommandBuffer cb, ref RenderingData renderingData)
55.         {
56.             base.OnCameraSetup(cb, ref renderingData);
57.  
58.             cb.SetComputeTextureParam(_computeShader, 0, "_Gbuffer0", Shader.GetGlobalTexture("_GBuffer0"));
59.             cb.SetComputeTextureParam(_computeShader, 0, "_RT", _rt);
60.             cb.DispatchCompute(_computeShader, 0, Mathf.CeilToInt(_rt.width / 8f), Mathf.CeilToInt(_rt.height / 8f), 1);
61.         }
62.  
63.         public override void Execute(ScriptableRenderContext context, ref RenderingData renderingData)
64.         {
65.         }
66.     }
67. }
68.  

 

Can we do a gbuffer rendering in Forward plus ?

 

Yes, you just need to create a RendererList and set the shader tag ID to "UniversalGBuffer".

URP will strip all the deferred rendering variants if there's no deferred renderer in your project, so make sure you add one or modify the shader stripping in some way.

 

Thanks a lot , will try it asap

 

Thanks, will be very good to have

Also another question that is not yet clear

In the example of RenderList with Graph (Link), where is the target texture for each of the renderer list results defined ?

For example i want to render the Gbuffer group using "new ShaderTagId("UniversalGBuffer")" in the below textures

ConfigureTarget(new RTHandle[] {
albedoHandle,
specularHandle,
worldPosHandle,
depthNormalHandle,
});

Thanks

 

Also is this available in 6000.0.b13 already or will be in next version ?

Thanks

Another note that come to mind, the Draw single camera of URP seems to have issues in the Graph case, i had to revert to the BiRP pipeline on the fly to perform a replacement shader operation and back to URP for the rest of processing, while in non Graph URP i could do it with that function in a 3D texture with render objects feature in pure URP mode.