Search Unity

How to Translate Instanced Meshes Relative to Transform?

Discussion in 'Shaders' started by amasinton, Jul 25, 2019.

  1. amasinton

    amasinton

    Joined:
    Aug 12, 2006
    Posts:
    131
    I'm working on an effect which relies on DrawMeshInstancedIndirect. I'm adapting the first shader (surface shader) which the scripting documentation provides as an example: here.

    What I would like is to position the instanced meshes relative to the Transform of a GameObject in-scene, and be able to have those mesh positions move with the Transform when the GameObject moves.

    I suspect this can be achieved by transforming the Vector3 array of instanced positions in the shader by the Transform of a specified GameObject.

    I just have now clue how to do this.

    Any suggestions?
     
  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    7,319
    If you want to do it from script, then you would do this:
    https://docs.unity3d.com/ScriptReference/Transform.TransformPoint.html

    If you want to do it in the shader, you would have to pass the local to world matrix for that game object to the shader, either by setting it on the material used, or by setting it as a global shader property, or on the material property block.
    https://docs.unity3d.com/ScriptReference/Transform-localToWorldMatrix.html
    https://docs.unity3d.com/ScriptReference/Material.SetMatrix.html
    https://docs.unity3d.com/ScriptReference/Shader.SetGlobalMatrix.html
    https://docs.unity3d.com/ScriptReference/MaterialPropertyBlock.SetMatrix.html
     
  3. amasinton

    amasinton

    Joined:
    Aug 12, 2006
    Posts:
    131
    Thank you so much - and for such a quick reply.

    Erm ... I'm a little stuck, tho...

    Two things:
    1. I don't know how to declare and initialize a matrix in the shader itself - I assume it's a float 4x4. When I attempt to declare it in the shader's property block I get a TVAL error no matter what I do:

    Code (CSharp):
    1. _TransformMatrix("TransformMatrix", float4x4) = (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
    or

    Code (CSharp):
    1. _TransformMatrix("TransformMatrix", float4x4)
    When I attempt to declare it in the 'Setup' function of the shader like this:

    Code (CSharp):
    1. void setup()
    2.     {
    3. #ifdef UNITY_PROCEDURAL_INSTANCING_ENABLED
    4.      
    5.         float4x4 _TransformMatrix = (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
    6.      
    7. #endif
    8.     }
    I only seem to be setting it back to zero rather than using the values I'm passing in from my transform's local to world matrix.

    2. I guess I don't know what to do with this matrix should I ever succeed in passing it into the shader. I'm using the example shader in the Unity docs for DrawMeshInstancedIndirect (as linked I'm my original post, above). The position of each instanced mesh is determined by an array of float3. This array is in world space. (Just to recap:) I would like to transform this array into the object's transform space so that the instanced meshes will move when the transform moves.
    So, I thought I could just multiply the transform's local to world matrix by each position float3 from the array of positions in the shader and that would be that. But I really don't understand what is going on in the sample shader in the 'setup' function at this point:

    Code (CSharp):
    1. void setup()
    2.     {
    3. #ifdef UNITY_PROCEDURAL_INSTANCING_ENABLED
    4.         float4 data = positionBuffer[unity_InstanceID];
    5.  
    6.         unity_ObjectToWorld._11_21_31_41 = float4(data.w, 0, 0, 0);
    7.         unity_ObjectToWorld._12_22_32_42 = float4(0, data.w, 0, 0);
    8.         unity_ObjectToWorld._13_23_33_43 = float4(0, 0, data.w, 0);
    9.         unity_ObjectToWorld._14_24_34_44 = float4(dataTrans.xyz, 1);
    10.         unity_WorldToObject = unity_ObjectToWorld;
    11.         unity_WorldToObject._14_24_34 *= -1;
    12.         unity_WorldToObject._11_22_33 = 1.0f / unity_WorldToObject._11_22_33;
    13.      
    14. #endif
    15.     }
    So, I don't know where to do the multiply operation in the shader.

    Any advice?

    Thank you!
     
  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    7,319
    See the documentation I linked to above, it has examples for the first few questions:
    https://docs.unity3d.com/ScriptReference/Material.SetMatrix.html

    One big confusion you're having is there's no need to put the matrix in the shader's properties. A shader's properties mainly exist for serialization and display in the inspector, and matrix properties aren't supported. So just skip that. All you need to do is define it in the shader code itself, inside the CGPROGRAM block, but outside a function. Again, see the above documentation.

    Sounds like you need to read up on how to use transform matrices. Unfortunately I don't know of any good tutorials on this. Most tutorials on matrix math tend to go deep into the math itself, which is great if that's what you want to learn, but is more confusing than it needs to be for people starting out who just want to know what they're for and how they're used.

    No, that's not actually what you want. You want to transform the array of positions into world space positions relative to the object's transform. The positions already are in "object space". I'll explain.

    By default those positions you pass to the instanced objects are "in world space". What makes them in world space? Nothing, really, just they're treated as if they're world space positions by virtue of those positions being used when defining the matrices used to transform the mesh vertices into world space. In your case you want to treat the positions as being in the space of another game object. To do that you need to transform those positions from that object's space into world space and use those new positions to build the object to world matrices. Transforming vectors from one space to another is what a transform matrix exists for.

    So, for your use case you'll want to do something like this:
    Code (csharp):
    1. // in the shader code, outside a function
    2. float4x4 _TransformMatrix;
    3.  
    4. // in the setup function
    5. float4 data = positionBuffer[unity_InstanceID];
    6.  
    7. // apply transform matrix to position data
    8. // You need to transform in a float4 with a w of 1 otherwise it'll be transformed as a direction
    9. // instead of a position. Read up on matrix math if you want to understand why.
    10. data.xyz = mul(_TransformMatrix, float4(data.xyz, 1)).xyz;
    11.  
    12. // then the rest of the code as is
    An additional note. The object to world transform matrices the example code generates has no rotation, only position and scale. This means that your instanced objects will follow the transform you pass in, but stay aligned to the world. If you want the objects to rotate and scale with the object, that'll require handling things in the shader code a little different.
     
  5. amasinton

    amasinton

    Joined:
    Aug 12, 2006
    Posts:
    131
    Thank you for your gentle explanation, your concrete examples, and the theory you've shared. I've learned a lot from your posts throughout this forum over the past few years!

    It's starting to make a lot more sense to me. I was getting very close, but I didn't understand that properties could be declared outside the Properties block. That's very good to know. Also, I'm educating myself about transformation matrices in general - fascinating stuff.

    It makes so much more sense that I want to transform the positions array into world space relative to the object's transform. Yes, that's exactly it, of course!

    So, I've declared a float4x4 in the shader as a property called '_TransformMatrix' as per above (not the best name for clarity's sake now, I see that). And in my C# script I've set it using MaterialPropertyBlock.SetMatrix - using the transform's local to world space matrix. Using MaterialPropertyBlock.GetMatrix, I can confirm in the C# script that I've successfully set my '_TransformMatrix' property. So far so good. I set the matrix on Update.

    But this:

    Code (CSharp):
    1. data.xyz = mul(_TransformMatrix, float4(data.xyz, 1)).xyz;
    ... seems to have no effect. The instanced objects still render relative to world 0,0,0. I've tried sending the Renderer's local to world matrix rather than the Transform, just to check (and also sending the world to local matrices, too) and nothing seems to make any difference. It's like this multiplication step is being ignored.

    For now, when I transform the position array in the C# script using TransformPoint (as you suggested earlier) it works okay, but this may start to chug once I'm updating tens or hundreds of thousands of points.

    Any thoughts, again, on why that multiply doesn't seem to have any effect?
     
  6. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    7,319
    I just noticed that’s “dataTrans” instead of “data”. Not sure where that variable name came from, but that should probably be “data”.

    Otherwise, post your entire shader as is, and at least some of your script code.
     
  7. amasinton

    amasinton

    Joined:
    Aug 12, 2006
    Posts:
    131
    That's certainly odd. It's not actually in the shader in my project - must be a copy-paste typo on my part. Sadly. It would be great if this was the problem...

    Okay here's the script (based very largely on this example by noisecrime except that I've modified it to use a List of position arrays so I can change 'models' when pressing up or down arrows):
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4.  
    5. [System.Serializable]
    6. public class PositionGroup
    7. {
    8.     public Vector3[] positionSet;
    9. }
    10.  
    11. public class MyInstancedController : MonoBehaviour
    12. {
    13.     public List<PositionGroup> positionVectors = new List<PositionGroup>();
    14.     public float baseSize = 0.04f;
    15.     public float randSizeVariation = 0.01f;
    16.     public Mesh instanceMesh;
    17.     public Material instanceMaterial;
    18.  
    19.     private MaterialPropertyBlock block;
    20.     public Matrix4x4 myTransformMatrix;
    21.  
    22.     public int instanceCount = 0;
    23.     public int cachedInstanceCount = -1;
    24.  
    25.     private ComputeBuffer positionBuffer;
    26.     private ComputeBuffer argsBuffer;
    27.  
    28.     private int counter = 0;
    29.  
    30.     private uint[] args = new uint[5] { 0, 0, 0, 0, 0 };
    31.  
    32.     void Start()
    33.     {
    34.         myTransformMatrix = transform.localToWorldMatrix;
    35.         block = new MaterialPropertyBlock();
    36.         block.SetMatrix("_TransformMatrix", myTransformMatrix);
    37.  
    38.         instanceCount = positionVectors[counter].positionSet.Length;
    39.         argsBuffer = new ComputeBuffer(1, args.Length * sizeof(uint), ComputeBufferType.IndirectArguments);
    40.         UpdateBuffers();
    41.     }
    42.  
    43.     public void UpdateBuffers()
    44.     {
    45.         instanceCount = positionVectors[counter].positionSet.Length;
    46.  
    47.         if (instanceCount < 1) instanceCount = 1;
    48.  
    49.         // Positions
    50.         if (positionBuffer != null) positionBuffer.Release();
    51.  
    52.         positionBuffer = new ComputeBuffer(instanceCount, 16);
    53.  
    54.         Vector4[] positions = new Vector4[instanceCount];
    55.  
    56.         Vector3[] tempPositions = new Vector3[instanceCount];
    57.         tempPositions = positionVectors[counter].positionSet;
    58.  
    59.         for (int i = 0; i < instanceCount; i++)
    60.         {
    61.             float randSize = baseSize + Random.Range(-randSizeVariation, randSizeVariation);
    62.             positions[i] = new Vector4(tempPositions[i].x, tempPositions[i].y, tempPositions[i].z, randSize);
    63.         }
    64.  
    65.         positionBuffer.SetData(positions);
    66.  
    67.         instanceMaterial.SetBuffer("positionBuffer", positionBuffer);
    68.  
    69.         // indirect args
    70.         uint numIndices = (instanceMesh != null) ? (uint)instanceMesh.GetIndexCount(0) : 0;
    71.         args[0] = numIndices;
    72.         args[1] = (uint)instanceCount;
    73.         argsBuffer.SetData(args);
    74.  
    75.         cachedInstanceCount = instanceCount;
    76.  
    77.     }
    78.  
    79.     void Update()
    80.     {
    81.         // Update starting position buffer
    82.         if (cachedInstanceCount != instanceCount) UpdateBuffers();
    83.  
    84.         myTransformMatrix = transform.localToWorldMatrix;
    85.         block.SetMatrix("_TransformMatrix", myTransformMatrix);
    86.         shaderMatrix = block.GetMatrix("_TransformMatrix");
    87.  
    88.         if (Input.GetKeyDown(KeyCode.UpArrow))
    89.         {
    90.             counter++;
    91.             if (counter == positionVectors.Count)
    92.             {
    93.                 counter = 0;
    94.             }
    95.             UpdateBuffers();
    96.         }
    97.  
    98.         if (Input.GetKeyDown(KeyCode.DownArrow))
    99.         {
    100.             counter--;
    101.             if (counter <= 0)
    102.             {
    103.                 counter = positionVectors.Count - 1;
    104.             }
    105.             UpdateBuffers();
    106.         }
    107.  
    108.         // Render
    109.         //  instanceMaterial.SetBuffer("positionBuffer", positionBuffer);
    110.         Graphics.DrawMeshInstancedIndirect(instanceMesh, 0, instanceMaterial, new Bounds(Vector3.zero, new Vector3(100.0f, 100.0f, 100.0f)), argsBuffer);
    111.     }
    112.  
    113.     void OnDisable()
    114.     {
    115.         if (positionBuffer != null) positionBuffer.Release();
    116.         positionBuffer = null;
    117.  
    118.         if (argsBuffer != null) argsBuffer.Release();
    119.         argsBuffer = null;
    120.     }
    121. }
    122.  
    And here's the shader - based almost entirely on the sample shader in the Unity docs for DrawMeshInstancedIndirect (as linked in my original post here):
    Code (CSharp):
    1. Shader "Instanced/MyInstancedTest"
    2. {
    3.     Properties{
    4.         [HDR]_EmissiveColor("Emissive Color (RGB)", Color) = (1, 1, 1, 1)
    5.     }
    6.         SubShader{
    7.         Tags{ "RenderType" = "Opaque" }
    8.         LOD 200
    9.  
    10.         CGPROGRAM
    11.         // Physically based Standard lighting model
    12. #pragma surface surf Standard addshadow
    13. #pragma multi_compile_instancing
    14. #pragma instancing_options procedural:setup
    15.  
    16.         sampler2D _MainTex;
    17.      
    18.         float4x4 _TransformMatrix;
    19.  
    20.     struct Input {
    21.         float2 uv_MainTex;
    22.     };
    23.  
    24. #ifdef UNITY_PROCEDURAL_INSTANCING_ENABLED
    25.     StructuredBuffer<float4> positionBuffer;
    26. #endif
    27.  
    28.     void setup()
    29.     {
    30. #ifdef UNITY_PROCEDURAL_INSTANCING_ENABLED
    31.         float4 data = positionBuffer[unity_InstanceID];
    32.              
    33.         data.xyz = mul(_TransformMatrix, float4(data.xyz, 1)).xyz;
    34.  
    35.         unity_ObjectToWorld._11_21_31_41 = float4(data.w, 0, 0, 0);
    36.         unity_ObjectToWorld._12_22_32_42 = float4(0, data.w, 0, 0);
    37.         unity_ObjectToWorld._13_23_33_43 = float4(0, 0, data.w, 0);
    38.         unity_ObjectToWorld._14_24_34_44 = float4(data.xyz, 1);
    39.      
    40.         unity_WorldToObject = unity_ObjectToWorld;
    41.         unity_WorldToObject._14_24_34 *= -1;
    42.         unity_WorldToObject._11_22_33 = 1.0f / unity_WorldToObject._11_22_33;
    43.      
    44. #endif
    45.     }
    46.  
    47.     half3 _EmissiveColor;
    48.  
    49.     void surf(Input IN, inout SurfaceOutputStandard o)
    50.     {
    51.         o.Emission = _EmissiveColor.rgb;
    52.     }
    53.     ENDCG
    54.     }
    55.         FallBack "Diffuse"
    56. }
    57.  
    Soooo - seems like all of the data is getting from the script to the shader, but I'm not certain why the multiply line doesn't seem to be actually multiplying. I'm certain I'm doing something obviously wrong, but I can't see it.

    Thanks for taking a look!
     
  8. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    7,319
    No, the data is never making it to the shader. You create a material property block and set the matrix on it, but never actually use the material property block, so that data just sits in the material property block unused.

    Look at the documentation for DrawMeshInstancedIndirect again for how to use that material property block.
     
  9. amasinton

    amasinton

    Joined:
    Aug 12, 2006
    Posts:
    131
    That was it! Of course that was it. It says right in the documentation (for DrawMesh, but NOT in DrawMeshInstancedIndirect):
    "modifying material properties between calls to this function won't make the meshes pick up them. If you want to draw series of meshes with the same material, but slightly different properties (e.g. change color of each mesh), use MaterialPropertyBlock parameter."​

    Thank you! I've learned so much over the past couple of days.