Question Enabling Motion Vector Pass For DrawProcedural?

I'm doing some rendering of some procedural objects in HDRP and finding that I cannot get them to generate motion vectors during rendering.

I've verified that my shader is working by forcing motion vector values - it works on mesh objects, but not procedural.

Is there a way to turn this on easily or it is something deeper?

Thanks

 

Im sorry, but what does it have to do with having access to the mesh? I dont think thats what the question was about. We are handling the motion vectors calculation in our shaders ourselves, thats not the problem. The problem is that even if you have a motion vector pass with correct motion vectors information, its simply ignored, we now have to use custom passes and command buffers to force rendering motion vectors, which bypasses instancing/batching/culling etc. In DrawProcedural there is an option to choose shadow casting mode, which works correctly. There should be an option like that to choose motion vectors mode similar to shadow casting mode.

 

The shader in question is a custom shader written from scratch. It has a valid custom motion vectors pass as well. Again, the problem is not in the shader code, or how we handle the motion vectors data, but in the fact that that pass is completely ignored by the pipeline. I have a shadow caster pass in there as well - that one is not ignored. But the motion vectors pass is ignored. (Our project is on 2019.4.3 / HDRP 7.2.1 btw, maybe it was fixed in later versions?) I assume it cause it works similar to regular objects, and only renders motion vectors if the object's transform has changed, or it has its motion vectors mode set to Force No Motion (neither of which we can affect with DrawProcedural).

 

Yup.

Yup.

Opaque. Not working in editor, didnt test at runtime because we need it to work in editor anyway.

 

Hi Seb,

Thanks for your reply.

It looks like DrawProcedural is not queuing the MotionVector Pass. Looking in FrameDebug I only see 1 ObjectMotionVector pass for a test mesh object that has the same material as the procedural object. So perhaps there is a per-object tag that is not getting set to enable the pass.

For testing, in the Lit material, I am forcing the outMotionVector like this in the motion vector Frag, which works for the aforementioned mesh object but not the procedural one.
outMotionVector = float4(0.0, 1.0, 0.0, 0.0);

I also explicitly set this but it still seems that it is missing the pass.
MaterialOpaque.SetShaderPassEnabled("MotionVectors", true);

I'm using Unity 2020.1b, not using Shader Graph.

 

Hi, I have the same problem. I'm also trying to get he MotionVectors Pass working with DrawProcedural. Maybe it's not supported?

 

Looks like Motion Vectors haven't been support since years! Why does Unity not give a better support for Graphics.Functions they are so powerful and useful for Procedural Objects!
https://github.com/keijiro/KvantSprayMV
This shows that the problem already exists 4 years ago!

 

i can confirm that using drawmeshinstanced or drawmeshinstancedindirect both do not write motion vectors either but just get ignored by the pipeline.

in our case we do vertex animations (grass) so it is relatively easy to calculate the motion vectors. the shader of course has a proper and properly named pass but still: the pipeline simply ignores it.
i was able once to make them render by editing the hdrp renderpipeline and changing somehow:

Code (CSharp):

1. var rendererList = RendererList.Create(CreateOpaqueRendererListDesc(cullResults, hdCamera.camera, HDShaderPassNames.s_MotionVectorsName, PerObjectData.MotionVectors, stateBlock: stateBlock));

i guess i left out the "PerObjectData.MotionVectors" or so. i can't remember exactly...

hacking hdrp of course is not nice.
and vfx graph manages to write out motionvectors although it might do something similar to drawmeshinstanced or drawmeshinstancedindirect.
would be great if we somehow could add a flag to drawmeshinstanced or drawmeshinstancedindirect so it registers properly to the pipeline.

 

interesting. which custom pass entry point gives you access to the motion vector buffer? or how do you set it as render target?

 

Yeah, we had to modify the pipeline to expose additional injection points

 

Appreciate the update. Thanks kindly!

 

great news, thanks!

 

oh, that sounds evil

 

@SebLagarde: Great news, thank you!

 

Btw.: I also found a way to get it working but it needs some additional functions in the HDRP pipeline and also a Custom Pass, which renders the Mesh to the MotionVectors-Texture... so very hacky and dirty. I would recommend to wait until the Unity team fixed it.

 

If anyone is interested, this is how my custom pass looks like:

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.Rendering.HighDefinition;
3. using UnityEngine.Rendering;
4. using System.Reflection;
5. using System.Collections.Generic;
6.  
7. class WriteMV : CustomPass
8. {
9.     [SerializeField] Mesh _mesh;
10.     [SerializeField] Shader _motionVectorShader;
11.     [SerializeField] Transform _transform;
12.  
13.     Matrix4x4 _previousModelMatrix;
14.     Material _motionVectorMaterial;
15.  
16.     static HDRenderPipeline _hdrp;
17.     static MethodInfo _mGetMotionVectors;
18.     static MethodInfo _mGetDepthStencilBuffer;
19.     static PropertyInfo _sharedRTManager;
20.     static Dictionary<HDCamera, RenderTargetIdentifier[]> _MotionVectorsPassBuffersRTI_List;
21.     static Dictionary<HDCamera, RTHandle> _DepthStencilBuffer_List;
22.  
23.     protected override void Setup(ScriptableRenderContext renderContext, CommandBuffer cmd)
24.     {
25.         if (_motionVectorShader != null) _motionVectorMaterial = new Material(_motionVectorShader);
26.         if (_transform != null) _previousModelMatrix = _transform.localToWorldMatrix;
27.         if (_hdrp == null) _hdrp = (RenderPipelineManager.currentPipeline as HDRenderPipeline);
28.         if (_sharedRTManager == null) _sharedRTManager = _hdrp.GetType().GetProperty("sharedRTManager", BindingFlags.Instance | BindingFlags.NonPublic);
29.         if (_mGetMotionVectors == null) _mGetMotionVectors = _sharedRTManager.PropertyType.GetMethod("GetMotionVectorsPassBuffersRTI");
30.         if (_mGetDepthStencilBuffer == null) _mGetDepthStencilBuffer = _sharedRTManager.PropertyType.GetMethod("GetDepthStencilBuffer");
31.         if (_MotionVectorsPassBuffersRTI_List == null) _MotionVectorsPassBuffersRTI_List = new Dictionary<HDCamera, RenderTargetIdentifier[]>();
32.         if (_DepthStencilBuffer_List == null) _DepthStencilBuffer_List = new Dictionary<HDCamera, RTHandle>();
33.     }
34.  
35.     public static RenderTargetIdentifier[] GetMotionVectorsPassBuffersRTI(HDCamera hdCamera)
36.     {
37.         return (RenderTargetIdentifier[])_mGetMotionVectors.Invoke(_sharedRTManager.GetValue(_hdrp), new object[] { hdCamera.frameSettings });
38.         //return hdrp.sharedRTManager.GetMotionVectorsPassBuffersRTI(hdCamera.frameSettings);
39.     }
40.  
41.     public static RTHandle GetDepthStencilBuffer(HDCamera hdCamera)
42.     {
43.         return (RTHandle)_mGetDepthStencilBuffer.Invoke(_sharedRTManager.GetValue(_hdrp), new object[] { hdCamera.frameSettings.IsEnabled(FrameSettingsField.MSAA) });
44.         //return hdrp.sharedRTManager.GetDepthStencilBuffer(hdCamera.frameSettings.IsEnabled(FrameSettingsField.MSAA));
45.     }
46.  
47.     protected override void Execute(ScriptableRenderContext renderContext, CommandBuffer cmd, HDCamera hdCamera, CullingResults cullingResult)
48.     {
49.         // Executed every frame for all the camera inside the pass volume
50.         if (_motionVectorMaterial == null || _transform == null || _mesh == null)
51.             return;
52.  
53.         _motionVectorMaterial.SetMatrix("_PreviousM", _previousModelMatrix);
54.         _motionVectorMaterial.SetMatrix("_PreviousVP", CameraMatrixProvider.GetPreviousVPMatrix(hdCamera.camera));
55.         _motionVectorMaterial.SetMatrix("_NonJitteredVP", CameraMatrixProvider.GetVPMatrix(hdCamera.camera));
56.  
57.         //cmd.SetRenderTarget(BuiltinRenderTextureType.MotionVectors, BuiltinRenderTextureType.CameraTarget);
58.         hdCamera.camera.depthTextureMode |= DepthTextureMode.MotionVectors | DepthTextureMode.Depth;
59.  
60.         if (!_MotionVectorsPassBuffersRTI_List.ContainsKey(hdCamera)) _MotionVectorsPassBuffersRTI_List.Add(hdCamera, GetMotionVectorsPassBuffersRTI(hdCamera));
61.         if (!_DepthStencilBuffer_List.ContainsKey(hdCamera)) _DepthStencilBuffer_List.Add(hdCamera, GetDepthStencilBuffer(hdCamera));
62.  
63.         //CoreUtils.SetRenderTarget(cmd, HDUtils.GetMotionVectorsPassBuffersRTI(hdCamera), HDUtils.GetDepthStencilBuffer(hdCamera));
64.         CoreUtils.SetRenderTarget(cmd, _MotionVectorsPassBuffersRTI_List[hdCamera], _DepthStencilBuffer_List[hdCamera]);
65.  
66.         cmd.DrawMesh(_mesh, _transform.localToWorldMatrix, _motionVectorMaterial, 0, 0); //<- replace cmd.DrawMesh with your cmd.DrawProcedural
67.         //cmd.DrawProcedural(...);
68.  
69.         _previousModelMatrix = _transform.localToWorldMatrix;
70.     }
71.  
72.     protected override void Cleanup()
73.     {
74.         if (_MotionVectorsPassBuffersRTI_List != null) _MotionVectorsPassBuffersRTI_List.Clear();
75.         if (_DepthStencilBuffer_List != null) _DepthStencilBuffer_List.Clear();
76.         CoreUtils.Destroy(_motionVectorMaterial);
77.     }
78. }

The rest of the code you can find here:
https://github.com/keijiro/DrawMeshWithMotionVectors
You need to fix the shader to be correctly working for HDRP however you can already see results with the existing one.

Cheers,
Chris

 

please also consider "fixing" this for drawmeshinstanced and drawmeshinstancedindirect

 

Ok, but you are working on a "fix" for newer versions like 2020.1 ?
I basically fixed the issue with my code snippet, but a solution from your side would be much better, hope this will work soon. Thanks!

 

Wanted to check when this will be resolved. We'd prefer to avoid a custom pass and have it in the core version.

 

Hi, just wanted to check if this fix will be available in 2021.x soon?
I had a quick look at the changelogs for 2021.1 and 2021.2a and it seems like it hasn't happened yet
(been using DrawProceduralIndrect a lot, it would be great to have the extra controls exposed)

 

I think I have the same Problem using the Alembic Importer (2.1.3) with the Import Points option which uses the AlembicPointsRenderer in HDRP (11 and 12).

 

Any updates on this? I'd appreciate a nudge in this thread once the api is available in the 2021.2 alpha version

 

https://github.com/Unity-Technologies/Graphics/pull/4190

Looks like this branch is related to this.

BTW. We are using DrawInstancedIndirect for foliage with vertex animation. Does anybody know if "Motion Vector For Vertex Animation" material option works with DrawInstancedIndirect?

Thanks.

 

Still not working as of 2021.2.0b16.

The motion vector pass still does not seem to be enabled/available.
Using DrawProcedural / MyMaterial.SetShaderPassEnabled("MotionVectors", true);

 

hi jarkko,
thanks for the heads up.

first question
my system uses DrawMeshInstancedIndirect do draw a vast amount of grass. why does it need a new api to support motion vectors? why couldn't it just be added to already existing apis?
i saw that you were involved in this: https://github.com/Unity-Technologies/Graphics/pull/4190 so what went wrong?

second question
having a first glance at RenderPrimitives and RenderParams i only see support for
instanceCount: The number of instances to render. i use compute to do per instance frustum culling and obviously get back the resulting instance count in a bufferWithArgs: The GPU buffer containing the arguments for how many instances of this mesh to draw.
how should this be implemented using the new api?
asyn readback? delay?

best,
lars

 

@JarkkoUnity
Hi,

I can be wrong but in the first question Lars is probably asking about rendering custom meshes with standard DrawMeshInstancedIndirect but with motion vectors enabled.

There is also another thread with discussion about motion vectors for DrawMeshInstanced and DrawMeshInstancedIndirect but there is no answer from developers there. Also the new terrain details/trees system use one of this function (probalby the Indirect variant) so it's affected too.

https://forum.unity.com/threads/sev...sh-trees-do-not-write-motion-vectors.1182244/

Do you know if it's possible to have motion vectors with DrawMeshInstanced and/or DrawMeshInstancedIndirect somehow?

Thanks!

 

Good to know! I was worried that this is some abandoned bug or that motion vectors won't be supported.

So I assume that if you will introduce/add support for any new functions in the future (like ray-tracing for example) they will be added to Graphics.Render functions and not Graphics.Draw functions.

 

hi jarkko,

thanks for the fast answer. any advice how to add the needed code to an HDRP shader graph?
best,
lars

 

Can this be used now in 2021.2 to render instanced meshes with motion vectors?

Terrain detail mesh & trees needs to use this, currently it does not and there's no motion vectors on all the grass/trees.

 

As explained before, the problem with the old APIs is that they don't expose control over the motion vector mode used for that draw command, using the default value "camera" which causes the motion vectors shader pass to be ignored.

That pull request is just adding shader support for motion vectors calculation for instanced meshes (reading the previous object matrix from the instanced properties), it does not solve the API problem (which is not an HDRP limitation).

The new APIs offer full replacements of the old ones and allow you to specify the motion vector mode to be used, so they are required for the above pull request to be of any use.

 

that is fine testing RenderMeshIndirect i just run into the issue that the "InstanceID" node in shader graph always returns 0... any help on this would be great.
i managed to fix the shadow caster pass simply commenting all the #if unityanyinstancing and manually pushing the instance id to my custom vertex function but this is just a pain!

 

hey jarkko, after doing some test with HDRP and shader graph it looks like none of the instancing keywords are set?
so the meshes get renderd but all on top of each other as instanceID just always is 0.

 

@JarkkoUnity
using something like this in a custom function in shader garph and HDRP 12.1 always return white.
so no instancing is enabled by the engine?
thus instanceID will always be 0?

Code (CSharp):

1. albedo = 1;
2. #ifdef UNITY_PROCEDURAL_INSTANCING_ENABLED
3. albedo = float3(1,0,0);
4. #endif
5. #if UNITY_ANY_INSTANCING_ENABLED
6. albedo = float3(0,1,0);
7. #endif

 

Thanks very much for letting us know about the new functionality. This and AddPrecomputedVelocity have been a huge help which saves me writing a custom pass and more time for coffee.

 

@JarkkoUnity
Reporting a similar issue to @larsbertram1 , I have tried moving my code over from DrawMeshInstancedIndirect to RenderMeshIndirect in order to take advantage of the motion vectors but it does not seem to be working for me. It's hard to see what I could be doing wrong from the limited documentation available. This is using URP in 2021.2.
Do you have an example scene that you could share that shows a working implementation of these new Render functions?

 

it works for me know, using hlsl or shadergraph injecting a setp function.

 

@JarkkoUnity

I'm trying to replace DrawMeshInstancedIndirect with RenderMeshIndirect in HDRP but I can't make it work properly with motion vectors mode set to 'Object'.

I'm using custom cginc in the shader subgraph and set unity_ObjectToWorld and unity_WorldToObject manually.

It generally works for RenderMeshIndirect. Meshes are rendered with correct RenderParams settings (like shadows, reflection probes mode etc.) but I have a problem with motion vectors.

When I set motion vectors mode to 'Object' in the RenderParams and the rendered mesh is not at 0, 0 ,0 position it jitters visually (when using DLSS or TAA).

While using motion vectors full-screen debug mode to see what is happening it's clearly visible that the mesh writes motion vectors every frame and all of them are pointing in the same direction (direction from 0, 0, 0 to this mesh).

The cube on the attached screenshot is placed at -0.5, 0, 0. I'ts not moving and tehre is no vertex animation. Camera is static too.

Any help will be much appreciated.

@larsbertram1

Have you managed to make it work with object motion vectors and shader graph shader?

 

yes. you have to set unity_MatrixPreviousM and unity_MatrixPreviousMI as well. otherwise motion vectors will be calculated based on the distance to the origin

 

@larsbertram1

Thanks for the fast reply!

I'm getting 'undeclared identifier 'unity_MatrixPreviousIM'' error.

Anyway it looks like it works with setting only the 'unity_MatrixPreviousM'. I'm not sure.

I've tested object motion and vertex animation too.
Motion vectors looks correct in the debug mode when moving the mesh position (I'm setting 'unity_MatrixPreviousM' to the position from the previous frame). Same for testing vertex animation.

I'm on Unity 2021.2.15 by the way. No idea why I can't set unity_MatrixPreviousIM and why it (probably) works without it. Maybe it's not required anymore?