Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Question How would you access skinned mesh pre-skinned vertex positions in shaders, these days?

Discussion in 'Graphics Dev Blitz Day 2023 - Q&A' started by Yoraiz0r, May 25, 2023.

  1. Yoraiz0r

    Yoraiz0r

    Joined:
    Apr 26, 2015
    Posts:
    64
    Currently in Unity, when you use a skinned mesh renderer, the vertex positions you get in the shaders, are the already-skinned ones.

    For certain visual effects it is desirable to have the pre-skinned vertex position. Consider things like making a character's hand glow and have the glow extend towards the arm, but not to their chest if they hold their hand to said chest.

    Other effects could include remaining wet up to a waistline after walking in water and moving out, but when crouching not have the effect immediately 'teleport' to the arms and torso due to the skinned positions suddenly being lower.

    I know that we can write a unity custom asset post processor and include this information in an extra UV layer (as each one can contain up to a vector4), but is there a more intuitive / easier to work with way?

    I'd also love to use this information with VFX Graph!

    Thanks!
     
    Yoraiz0r, May 25, 2023
    #1

  2. arkano22

    arkano22

    Joined:
    Sep 20, 2012
    Posts:
    1,719
    When skinning takes place in the CPU, vertices reach the vertex shader already transformed, there's no way around that that I can think of.

    There's a chance that asking Unity to perform GPU skinning and then using a custom shader would allow you to access pre-transform vertex positions. However, I'm unsure how the GPU skinning is actually performed, there's a chance it is done using compute shaders and in that case, vertices would still arrive at the vertex shader after being transformed.

    The surefire way to do this is to pass the bone transforms and the vertex weights to the shader and perform your own skinning in the vertex shader :).
     
    arkano22, May 25, 2023
    #2

  3. Jebtor

    Jebtor

    Unity Technologies

    Joined:
    Apr 18, 2018
    Posts:
    111
    The vertex positions you receive in the vertex shader are indeed the deformed vertices after skinning and blend shapes are applied. This buffer can be retrieved with SkinnedMeshRenderer.GetVertexBuffer. Similarly, you can retrieve the graphics buffer for the shared mesh through SkinnedMeshRenderer.sharedMesh.GetVertexBuffer - this will contain the undeformed vertices. You could bind this buffer to the material, then use the vertex ID to retrieve the undeformed vertex.

    In the case of VFX, the functionality to sample the original mesh exists but this is not exposed. For now to achieve this you will have to take the approach described above. Please submit a new idea with the request here. The team will receive the request and consider options for exposing this functionality.
     
    Jebtor, May 25, 2023
    #3
    Yoraiz0r likes this.

  4. Yoraiz0r

    Yoraiz0r

    Joined:
    Apr 26, 2015
    Posts:
    64
    Thank you for the response! I hadn't considered using SetVertexBuffer!

    I don't believe Shader Graph has any way to retrieve buffer information though, is that correct? if so, would a custom function node be able to do so? what would the code for that look like please?

    The mesh would also have to be set to readwrite in addition to the runtime code necessary to set the buffer on the material, is that correct? an interesting tradeoff between import-time setting a uv channel with it being available even in read only mode, or that... hmm
     
    Yoraiz0r, May 25, 2023
    #4

  5. Pema-Malling

    Pema-Malling

    Unity Technologies

    Joined:
    Jul 3, 2020
    Posts:
    257
    Another option could be to bake the unskinned position into another mesh channel, like vertex colors for example. Data would be easily accessible and you get interpolation for free if you need. Don't believe we have any built in way to do that, though, so it would involve some custom scripting.
     
    Pema-Malling, May 25, 2023
    #5
    Yoraiz0r and icauroboros like this.

  6. icauroboros

    icauroboros

    Joined:
    Apr 30, 2021
    Posts:
    103
    What if we want it after Blend Shapes but before skinning ?
     
    icauroboros, May 25, 2023
    #6

  7. Jebtor

    Jebtor

    Unity Technologies

    Joined:
    Apr 18, 2018
    Posts:
    111
    I will ask around if we have any samples to retrieve data from a buffer in ShaderGraph. Likely, this will involve a somewhat complicated setup.

    GetVertexBuffer should not require read/write to be enabled as long as the Mesh.vertexBufferTarget is not changed.

    As both you and Pema have mentioned an alternative would be to pack the positions and/or masks in other mesh channels such as UVs and vertex colors.
     
    Jebtor, May 25, 2023
    #7
    Yoraiz0r and Pema-Malling like this.

  8. Jebtor

    Jebtor

    Unity Technologies

    Joined:
    Apr 18, 2018
    Posts:
    111
    I believe this is not possible at the moment. Is the use case still the same as the original post?
     
    Jebtor, May 25, 2023
    #8
    Yoraiz0r likes this.

  9. icauroboros

    icauroboros

    Joined:
    Apr 30, 2021
    Posts:
    103
    icauroboros, May 25, 2023
    #9
    Jebtor likes this.