Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. The 2023.1 beta is now available for testing. To find out what's new, have a look at our 2023.1 feature highlights.
    Dismiss Notice

Feedback Does Raytracing support instancing in HDRP 12.1?

Discussion in 'HDRP Ray Tracing' started by newguy123, Nov 8, 2021.

  1. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156
    Hi Guys

    I does raytracing support instancing?

    I'm asking as I can't figure out why instanced prefabs do not work with raytraced shadows or raytraced reflections. It also doesn not seem to work AT ALL with pathtracing.

    Even if I apply regular HDRP/Lit shaders to the plants and refresh everything, it still does not work.

    However, if I directly drag the prefabs into the scene and manually duplicate them, then everything is fine.

    This makes me believe there is some kinf of issue with either instances, or Vegetation Studio Pro
     
  2. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    536
    Hey, If by instancing you mean using this function at runtime, it is indeed working and supposed to. (i.e you're supposed to see your instanced object in reflections and they should have raytraced shadows.. etc)

    Maybe your setup have something more to it.
    If you can't what it is, please make a report with a simple repro, drop the case number here, and we'll look into it.
    Thanks !
     
  3. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156
    Hi @chap-unity sorry for the late reply. Here is a reply I received from Lennart from Vegetation Studio Pro:

    "Apparently Unity has not made any API to add meshes rendered with the instancing API to the raytracing structure"

    Is that statement correct? If not, could you please point me to the correct information?

    Apparently that is the reason why I can't get my spawned objects to work with Raytracing. (Objects spawned with Vegetation Studio Pro)
     
  4. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156
    Some further info:

    upload_2022-1-19_10-30-23.png

    Raytracing does not suppor the 1st one in that list. But what about the other 2?

    From the sounds of it, DrawMesh uses CPU, but DrawMeshInstanced uses GPU........
     
  5. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    Hi!

    Graphics.Draw* functions don't make sense in ray tracing since ray tracing only works using a RayTracingAccelerationStructure and the source geometry in GPU memory. You can't "draw" into an acceleration structure. If a geometry is not in the acceleration structure then rays can't hit it. There's a plan to add similar functions to RayTracingAccelerationStructure where we could add instances of a Mesh using a RayTracingAccelerationStructure.AddInstances function. For example a very possible function signature could be:

    RayTracingAccelerationStructure.AddInstances(Mesh mesh, int submeshIndex, Material material, Matrix4x4[] matrices, int count, MaterialPropertyBlock properties);

    This function should be a cheap alternative to create many many ray tracing instances of the same Mesh into the acceleration structure where only the transformation matrix is different between instances. The shader setup is the same for all instances and will happen only once (in DXR, there will be only one hit group setup for all instances).
     
    TerraUnity and newguy123 like this.
  6. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156

    Thanks @INedelcu
    Do you know a rough ETA of when such a feature could become available?

    Since its also mostly for foilage, as I understand the wind animation is mostly handled by the shaders, at the same time, could we get something that works with the instancing to also support the shader wind animations?

    Currently this is a major hurdle that prevent us from using DXR/Pathtracing for exterior Film sets
     
  7. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    I'll be working on that function in the near future. I don't have a general solution for where you change vertex positions individually in a vertex shader (ie wind animation). The requirement for building an acceleration is that the source vertex buffer to be in main GPU memory. If you have a vertex shader for that you use in rasterization where you animate vertices individually then that data is kept in GPU registers, then the acceleration structure build process has no idea about that. I'll think about it but I don't see a cheap solution.
     
  8. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156
    Yeah since its for film and offline rendering with DXR, we're not overly worried about realtime performance, but of course if it ends up being so bad that frames take longer than traditional DCC rendering, then there's no point of course...
     
  9. m0nsky

    m0nsky

    Joined:
    Dec 9, 2015
    Posts:
    206
    This sounds great! I'm looking foward to it. (my main project is a racing game with a lot of instances of the same wheels, cones, fences, guard rails, etc)
     
    newguy123 likes this.
  10. G-Mika

    G-Mika

    Joined:
    Jul 17, 2012
    Posts:
    99
    I really need this function! I'm using DrawMeshInstancedIndirect but I cannot add the instanced meshes to the ray tracing acceleration structure because it requires non-existant Mesh Renderers.

    I'm thinking about creating a "hidden" gameobject with a renderer for each instances in a layer not visible to the camera but I really don't like this workaround.
     
    Last edited: Mar 12, 2022
    newguy123 likes this.
  11. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    I started working on this today!
     
  12. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156
    Hi @INedelcu
    This is the thread I meant to have posted in.
    How is work coming along for this?
     
  13. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    Hi @newguy123

    I figured out that we first need the equivalent of Graphics.RenderMesh in ray tracing.

    There will be a new function in 2023.1 RayTracingAccelerationStructure.AddInstance which will use Mesh instead of MeshRenderer. This will add some nice flexibility regarding what can be added to an acceleration structure.
     
    Zarbuz and newguy123 like this.
  14. Cascho01

    Cascho01

    Joined:
    Mar 19, 2010
    Posts:
    1,347
    Also figured out that prefabs do not cast raytraced shadows for now.
    Please "fix" that soon, thanks!
     
  15. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156
    We also desperately need foilage to be able to blow in the wind, while using raytracing/pathtracing, and make LOOOOOADS of duplicates of foilage all over our scenes, using non terrain imported meshes
     
    keeponshading and TerraUnity like this.
  16. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    536
    Hey, I was worried about this one and just tried to instantiate a few prefabs with multiple lights (dir and punctals) and it seems to work as expected on latest version.
    Meaning, instantiated prefabs at runtime are indeed added automatically to the ray tracing acceleration structure and casts ray traced shadows.

    Do you care to explain a bit more about your setup ? Or have a repro project by any chance ?
    Thanks.
     
  17. Cascho01

    Cascho01

    Joined:
    Mar 19, 2010
    Posts:
    1,347
    I copied some torusknots in 3dsmax as instances and imported the fbx in Unity.
    All knots share the same original mesh, so 3dsmax instances work fine in Unity.
    But the knots do not cast raytraced shadows.

    I did not instantiate anything at runtime.
     
  18. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    536
    I'd be happy to help if you can manage to give a repro and/or report a bug !
    Thanks :)
     
  19. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    13
    any updates on this topic? running into a similar problem...
     
  20. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    I finished RayTracingAccelerationStructure.AddInstances (the equivalent of Graphics.RenderMeshInstanced). It will be available in one of the future 2023.1 alpha versions.
     

    Attached Files:

    Bzuco, valarus, Rewaken and 6 others like this.
  21. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156
    Awesome!!!
    Looking forward to the nice things this brings
     
  22. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156
    I take it that wont make it into the changelog. So how would I know this is in whichever alpha version. Do I look at pulls in github or how does it work? Just trying to sound smart, no idea how to look for a "pull" in github.....
     
  23. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    It will make it into the usual release notes (e.g. https://unity3d.com/unity/alpha/2023.1.0a4) I'm not sure which code do you want to pull. You'll have to install whichever Unity alpha version the function will make into.
     
    newguy123 likes this.
  24. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    13
    @INedelcu could you share working project with 2023.1 on how to instantiate meshes with raytracing, like the example image you posted above? much appreciated
     
    Last edited: Aug 26, 2022
    newguy123 likes this.
  25. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    @jffonseca @valarus I don't have the project anymore. I'll try to redo it once RayTracingAccelerationStructure.AddInstances is available in a public Unity version.
     
    m0nsky and newguy123 like this.
  26. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    13
    Hi @INedelcu thanks for your answer. Any idea when? i cant real find any information or samples online. I really would like to play with this.

    Best,
    jf
     
  27. MichaelHVedeler

    MichaelHVedeler

    Joined:
    Feb 2, 2021
    Posts:
    2
    Hey @INedelcu!

    I just downloaded the newest unity version (2023.1), where i can call the AddInstances as you spoke above in this thread - using the MeshInstanceConfig method. However i cant really get it to work. I would love to to get a sample project showing how this would be set up.

    Hope to hear back soon,
    Michael V
     
    jffonseca likes this.
  28. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    13
  29. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    Hi! RayTracingAccelerationStructure.AddInstances wasn't merged in any Unity version. I don't know when this will be available.
     
  30. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    13
    Hi @INedelcu,

    wasnt it added here? maybe its not fully implemented, you probably know better than me, just checking.

    Best,
    j

    upload_2022-10-18_20-19-31.png
     
  31. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    That AddInstance will add only one Mesh instance to the RTAS and is the equivalent of Graphics.RenderMesh. The function signatures are quite similar. The new AddInstances will be the equivalent of Graphics.RenderMeshInstanced and you’ll have to pass an array of matrices, one matrix for each instance. The PR is in review now.
     
    matthewhxq, TerraUnity and m0nsky like this.
  32. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134

    Attached Files:

    • 2.png
      2.png
      File size:
      1.4 MB
      Views:
      23
  33. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,156
    Excuse my ignorance please @INedelcu, but how would I apply this to a simple use case where I have a shrub fbx that I want to instance 10 000 times over a plain, or a tree fbx that I want to instance 50 000 times over same plain?
     
  34. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    @newguy123 you can check my example. Just replace the sphere with your shrub.
     
    newguy123 likes this.
  35. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,131
    @INedelcu This is amazing, thanks for the efforts and sample projects!
     
    INedelcu likes this.
  36. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    13
    INedelcu likes this.
  37. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    13
    Hi @INedelcu looking at the HDRP example, is there a way to disable the path tracing, so still have the instances raytraced but in HDRP?

    If i disable the path tracing all the instance objects disapear, just the floor remains. Do i need to set any flag on the script?

    Cheers,
    j
     
    Last edited: Dec 1, 2022
  38. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    13
    Hi @INedelcu do you have any hints for me on this, much apreciated.

    Cheers,
    j
     
  39. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    jffonseca likes this.
  40. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    752
    @INedelcu any chance there be a RayTracingAccelerationStructure.AddInstancesIndirect() or a Graphic.RenderMeshInstancedIndirect()? As expected, Graphics.DrawMeshInstancedIndirect does not work with HDRP ray-tracing (SSR/SSGI/Shadows) and since the things I need to draw is defined on the GPU I can't feed an array of matrices to RayTracingAccelerationStructure.AddInstances().

    EDIT: I presumed Graphics.RenderMesh() was working with HDRP Ray Tracing, but it does not. The MeshRenderer exposes Ray Tracing settings that are not contained in the RenderParams passed to the method.
     
    Last edited: Jan 19, 2023
  41. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    Hi @cecarlsen! Graphics.RenderX will not work in ray tracing. Anything ray tracing related needs to work using an acceleration structure and not a Camera as in the case of Graphics.RenderX.
    RayTracingAccelerationStructure.AddInstancesIndirect might be possible in the future but is a bit tricky. For example when the acceleration structure buffer (TLAS) is allocated we need to specify the amount of ray tracing instances.
     
  42. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    752
    VFXGraph now supports instanced particles with ray-tracing (within certain limits). How does that work? Do they download the particle count to allocate ray tracing instances?
     
  43. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    No. VFX particles are ray traced using intersections shader where each particle has an associated AABB.
    The acceleration structure (BLAS) associated with a VFX effect is built based on a GraphicsBuffer of AABBs and the size of this GraphicsBuffer is constant and is set to maximum amount of particles (128 in the example attached). For particles that are not alive yet or not active, all values in the AABB are set to NaN and this way DXR knows that the AABB is not active so it's not included in the build. The DXR specs about this is here: https://microsoft.github.io/DirectX-Specs/d3d/Raytracing.html#inactive-primitives-and-instances
     

    Attached Files:

    cecarlsen and m0nsky like this.
  44. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    134
    m0nsky likes this.
  45. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    752
    Thanks, this is what I needed to know.

    I see that the overload of RayTracingAccelerationStructure.AddInstance() that takes a AABB GraphicsBuffer calls AddInstance_Procedural() internally. Wouldn't it be more clear to expose the method as RayTracingAccelerationStructure.AddInstanceProcedural()?