Direct3D12 native plugin render to screen

I'm working on a Direct3D12-based native rendering plugin but I can't find a way to render directly to the screen. The rendering plugin example unfortunately omits this particular bit (https://bitbucket.org/Unity-Technologies/graphicsdemos/src/77f014c12161e5c25d902e2c5697dd0c45ce3e35/NativeRenderingPlugin/PluginSource/source/RenderAPI_D3D12.cpp#lines-162).
Is there any way to get an RTV and DSV for the current render/depth target?

 

To clarify: by "rendering directly to the screen" I actually meant rendering to the active camera buffer, regardless of whether that is a render texture or not (I'd rather not force users of the plugin to always use a render texture). Unfortunately this precludes the GetNativeTexturePtr solution.

To my knowledge I have to provide my own ID3D12GraphicsCommandList to use with ExecuteCommandList in the plugin API. This means I cannot have RTV/DSV set up for me because a freshly created command list has no resources bound (https://docs.microsoft.com/en-us/wi...nf-d3d12-id3d12graphicscommandlist-clearstate).

Hope this helps in finding a solution.

 

Hi @JDeConinck_PresenZ,

The current d3d12 native plugin interface only provides access to our internal ID3D12CommandQueue. What you really need is access to our internal ID3D12CommandList. Alternatively, if you could execute a command list of type bundle, then you could inherit the resources and state bound to the engine's internal pipeline state object and command list. This is similar to how our Metal and Vulkan native plugin implementations work. Unfortunately, our d3d12 native plugin implementation does not provide access to our internal command list and the interface's ExecuteCommandList() method does not support executing command lists of type bundle. If you try, you will get GPU validation issues. This is because internally Unity issues a direct call to its ID3D12CommandQueue::ExecuteCommandLists() which does not support bundled command lists.

Due to these constraints, there are some less performant options you could try.

You may be able to exploit the Display class to gain access to the actively bound render surface. Please take a look at the following API
https://docs.unity3d.com/ScriptReference/Display-colorBuffer.html

Display.main.colorBuffer will provide access to the active bound color render surface. Likewise, Display.main.depthBuffer will provide access to the active depth render surface.
These render surfaces are unity specific objects that contain unity specific data describing the render surface in addition to the native GPU resource backing the surface (i.e. ID3D12Resource*).

We do not have a C# side API that will allow you to extract the ID3D12Resource* from C# script. However, we do have a native rendering interface method that will allow you to do that in C++. It is defined like so
ID3D12Resource* IUnityGraphicsD3D12v5::TextureFromRenderBuffer(UnityRenderBuffer * rb);

Therefore if you feed the TextureFromRenderBuffer() method a UnityRenderBuffer, which is what Display.main.colorBuffer represents, then you can gain access to the underlying Unity owned ID3D12Resource* backing the actively bound render texture.

If you choose to use this approach, you will need to create your own d3d12 pipeline state object, command allocator, and command list. Your command list will need to bind the color render buffer via a call to OMSetRenderTarget() on your d3d12 command list.

When filling out a description for your ID3D12PipelineState object, you may struggle to define the D3D12_GRAPHICS_PIPELINE_STATE_DESC::RTVFormats and D3D12_GRAPHICS_PIPELINE_STATE_DESC:: DSVFormat DXGI_FORMATs as we don't currently provide those to you in C# or C++. Therefore my recommendation is to take a RenderDoc capture of the scene without executing your native rendering plugin. From there you can examine Unity's internal PSO and the various settings we use based on your player, graphics, quality, and camera settings.

Once you are done recording into your d3d12 command list, you will then be able to use the native plugin's interface method, ExecuteCommandList(), to execute the command list on Unity's internal d3d12 command queue.

Another approach you could try is blitting your render into Unity's actively bound render texture. For this approach, you would again need to create your own d3d12 pipeline state object, command allocator, and command list. You would also need to create your own color and depth surfaces and direct your command list to render into them. Next, you would need an API to inform your C# script driving the native rendering plugin as to what the pointer address is to your native d3d12 shader resource view pointing to your color surface. Once you have done that, you can pass the SRV pointer into a method like Texture2D.CreateExternalTexture().
https://docs.unity3d.com/ScriptReference/Texture2D.CreateExternalTexture.html

Unity will then create a Texture2D that wraps your native rendering plugin's resource without creating any internal d3d12 resources within Unity. Once you have done that, you can then invoke a Graphics.Blit() call in your camera's OnPostImage() or OnRenderTexture() methods.

Obviously blitting will incur a performance cost but I believe it may be the only viable workaround if the Display approach doesn't work for you.

I hope that helps!

 

Thanks @BrandonFogerty,

The Display solution sounds like something that would work.
For now I have shifted my approach to a hybrid D3D11/D3D12 one with shared resources so it will be a while before I get around to trying it. But once I'm back on the full D3D12 path I'll post an update here with how it went.

 

Hi @BrandonFogerty,

This is the only thread I could spot on this issue - I'm currently trying to pass the active colour render buffer down to D3D12 native code as you suggest in your above post (as there doesn't seem to be another way to grab the current RT?) but just seem to get crashes in 'TextureFromRenderBuffer()' or invalid pointers returned.

I'm using Unity 2019.4.1f1 and passing down 'Display.main.colorBuffer.GetNativeRenderBufferPtr()' - I tried in D3D11 and it works as expected (from render thread):

Code (CSharp):

1.  
2. IUnityGraphicsD3D11* d3d11(s_UnityInterfaces->Get<IUnityGraphicsD3D11>());
3. if (d3d11)
4. {
5.    UnityRenderBuffer unityRenderBuffer(static_cast<UnityRenderBuffer>(renderTarget));
6.    ID3D11Resource* d3d11Resource(d3d11->TextureFromRenderBuffer(unityRenderBuffer));
7.    if (d3d11Resource)
8.    {
9.        ID3D11Texture2D* d3d11Texture(static_cast<ID3D11Texture2D*>(d3d11Resource));
10.        D3D11_TEXTURE2D_DESC desc;
11.        d3d11Texture->GetDesc(&desc);
12.        int x = 0;
13.    }
14. }
15.  

..I get a sensible 'D3D11_TEXTURE2D_DESC' fed back. But in D3D12:

Code (CSharp):

1.  
2. IUnityGraphicsD3D12v5* d3d12(s_UnityInterfaces->Get<IUnityGraphicsD3D12v5>());
3. if (d3d12)
4. {
5.    UnityRenderBuffer unityRenderBuffer(static_cast<UnityRenderBuffer>(renderTarget));
6.    ID3D12Resource* d3d12Resource(d3d12->TextureFromRenderBuffer(&unityRenderBuffer));
7.    if (d3d12Resource)
8.    {
9.        D3D12_RESOURCE_DESC desc(d3d12Resource->GetDesc());
10.        int x = 0;
11.    }
12. }
13.  

..I get access violations on 'TextureFromRenderBuffer()' or crashes in 'd3d12Resource->GetDesc()' as the pointer isn't valid - I also tried in Unity 2020.1.0b13 but results seem the same.

One difference I spotted is the D3D12 'TextureFromRenderBuffer()' function takes a pointer to 'UnityRenderBuffer' (which is already a typedef'd pointer) whereas on D3D11 (and other platforms) it's passed by value.

Are you aware of any issues here whilst I continue digging?

Cheers,
Richard

 

@RichRH, did you ever find a solution? In the most recent releases, the function signature has been fixed, but it seems TextureFromRenderBuffer still always returns null. I may be missing a magical incantation, I cannot find a single example that uses it.

 

My mistake. I was testing on 2020.2.1f1, which still has the incorrect signature and seems to be broken even if you manually correct it. 2021.1.0 Beta 1 seems to work. (not that it is stable enough for us to use)

 

Hi @cphinton,

I haven't had chance to look at this for a while so not sure of the current interface state, but our workaround at the time was to manually blit the screen to a render texture (using a command buffer) and pass that to native code.

Cheers,
Richard