Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only. On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live. Read our full announcement for more information and let us know if you have any questions.

Feedback Does Raytracing support instancing in HDRP 12.1?

Discussion in 'HDRP Ray Tracing' started by newguy123, Nov 8, 2021.

  1. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,281
    Hi Guys

    I does raytracing support instancing?

    I'm asking as I can't figure out why instanced prefabs do not work with raytraced shadows or raytraced reflections. It also doesn not seem to work AT ALL with pathtracing.

    Even if I apply regular HDRP/Lit shaders to the plants and refresh everything, it still does not work.

    However, if I directly drag the prefabs into the scene and manually duplicate them, then everything is fine.

    This makes me believe there is some kinf of issue with either instances, or Vegetation Studio Pro
     
  2. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    808
    Hey, If by instancing you mean using this function at runtime, it is indeed working and supposed to. (i.e you're supposed to see your instanced object in reflections and they should have raytraced shadows.. etc)

    Maybe your setup have something more to it.
    If you can't what it is, please make a report with a simple repro, drop the case number here, and we'll look into it.
    Thanks !
     
  3. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,281
    Hi @chap-unity sorry for the late reply. Here is a reply I received from Lennart from Vegetation Studio Pro:

    "Apparently Unity has not made any API to add meshes rendered with the instancing API to the raytracing structure"

    Is that statement correct? If not, could you please point me to the correct information?

    Apparently that is the reason why I can't get my spawned objects to work with Raytracing. (Objects spawned with Vegetation Studio Pro)
     
  4. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,281
    Some further info:

    upload_2022-1-19_10-30-23.png

    Raytracing does not suppor the 1st one in that list. But what about the other 2?

    From the sounds of it, DrawMesh uses CPU, but DrawMeshInstanced uses GPU........
     
  5. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    Hi!

    Graphics.Draw* functions don't make sense in ray tracing since ray tracing only works using a RayTracingAccelerationStructure and the source geometry in GPU memory. You can't "draw" into an acceleration structure. If a geometry is not in the acceleration structure then rays can't hit it. There's a plan to add similar functions to RayTracingAccelerationStructure where we could add instances of a Mesh using a RayTracingAccelerationStructure.AddInstances function. For example a very possible function signature could be:

    RayTracingAccelerationStructure.AddInstances(Mesh mesh, int submeshIndex, Material material, Matrix4x4[] matrices, int count, MaterialPropertyBlock properties);

    This function should be a cheap alternative to create many many ray tracing instances of the same Mesh into the acceleration structure where only the transformation matrix is different between instances. The shader setup is the same for all instances and will happen only once (in DXR, there will be only one hit group setup for all instances).
     
    TerraUnity and newguy123 like this.
  6. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,281

    Thanks @INedelcu
    Do you know a rough ETA of when such a feature could become available?

    Since its also mostly for foilage, as I understand the wind animation is mostly handled by the shaders, at the same time, could we get something that works with the instancing to also support the shader wind animations?

    Currently this is a major hurdle that prevent us from using DXR/Pathtracing for exterior Film sets
     
  7. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    I'll be working on that function in the near future. I don't have a general solution for where you change vertex positions individually in a vertex shader (ie wind animation). The requirement for building an acceleration is that the source vertex buffer to be in main GPU memory. If you have a vertex shader for that you use in rasterization where you animate vertices individually then that data is kept in GPU registers, then the acceleration structure build process has no idea about that. I'll think about it but I don't see a cheap solution.
     
  8. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,281
    Yeah since its for film and offline rendering with DXR, we're not overly worried about realtime performance, but of course if it ends up being so bad that frames take longer than traditional DCC rendering, then there's no point of course...
     
  9. m0nsky

    m0nsky

    Joined:
    Dec 9, 2015
    Posts:
    263
    This sounds great! I'm looking foward to it. (my main project is a racing game with a lot of instances of the same wheels, cones, fences, guard rails, etc)
     
    newguy123 likes this.
  10. G-Mika

    G-Mika

    Joined:
    Jul 17, 2012
    Posts:
    99
    I really need this function! I'm using DrawMeshInstancedIndirect but I cannot add the instanced meshes to the ray tracing acceleration structure because it requires non-existant Mesh Renderers.

    I'm thinking about creating a "hidden" gameobject with a renderer for each instances in a layer not visible to the camera but I really don't like this workaround.
     
    Last edited: Mar 12, 2022
    newguy123 likes this.
  11. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    I started working on this today!
     
  12. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,281
    Hi @INedelcu
    This is the thread I meant to have posted in.
    How is work coming along for this?
     
  13. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    Hi @newguy123

    I figured out that we first need the equivalent of Graphics.RenderMesh in ray tracing.

    There will be a new function in 2023.1 RayTracingAccelerationStructure.AddInstance which will use Mesh instead of MeshRenderer. This will add some nice flexibility regarding what can be added to an acceleration structure.
     
    Zarbuz and newguy123 like this.
  14. Cascho01

    Cascho01

    Joined:
    Mar 19, 2010
    Posts:
    1,347
    Also figured out that prefabs do not cast raytraced shadows for now.
    Please "fix" that soon, thanks!
     
  15. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,281
    We also desperately need foilage to be able to blow in the wind, while using raytracing/pathtracing, and make LOOOOOADS of duplicates of foilage all over our scenes, using non terrain imported meshes
     
    keeponshading and TerraUnity like this.
  16. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    808
    Hey, I was worried about this one and just tried to instantiate a few prefabs with multiple lights (dir and punctals) and it seems to work as expected on latest version.
    Meaning, instantiated prefabs at runtime are indeed added automatically to the ray tracing acceleration structure and casts ray traced shadows.

    Do you care to explain a bit more about your setup ? Or have a repro project by any chance ?
    Thanks.
     
  17. Cascho01

    Cascho01

    Joined:
    Mar 19, 2010
    Posts:
    1,347
    I copied some torusknots in 3dsmax as instances and imported the fbx in Unity.
    All knots share the same original mesh, so 3dsmax instances work fine in Unity.
    But the knots do not cast raytraced shadows.

    I did not instantiate anything at runtime.
     
  18. chap-unity

    chap-unity

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    808
    I'd be happy to help if you can manage to give a repro and/or report a bug !
    Thanks :)
     
  19. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    15
    any updates on this topic? running into a similar problem...
     
  20. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    I finished RayTracingAccelerationStructure.AddInstances (the equivalent of Graphics.RenderMeshInstanced). It will be available in one of the future 2023.1 alpha versions.
     

    Attached Files:

    Bzuco, valarnur, Rewaken and 6 others like this.
  21. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,281
    Awesome!!!
    Looking forward to the nice things this brings
     
  22. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,281
    I take it that wont make it into the changelog. So how would I know this is in whichever alpha version. Do I look at pulls in github or how does it work? Just trying to sound smart, no idea how to look for a "pull" in github.....
     
  23. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    It will make it into the usual release notes (e.g. https://unity3d.com/unity/alpha/2023.1.0a4) I'm not sure which code do you want to pull. You'll have to install whichever Unity alpha version the function will make into.
     
    newguy123 likes this.
  24. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    15
    @INedelcu could you share working project with 2023.1 on how to instantiate meshes with raytracing, like the example image you posted above? much appreciated
     
    Last edited: Aug 26, 2022
    newguy123 likes this.
  25. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    @jffonseca @valarus I don't have the project anymore. I'll try to redo it once RayTracingAccelerationStructure.AddInstances is available in a public Unity version.
     
    m0nsky and newguy123 like this.
  26. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    15
    Hi @INedelcu thanks for your answer. Any idea when? i cant real find any information or samples online. I really would like to play with this.

    Best,
    jf
     
  27. MichaelHVedeler

    MichaelHVedeler

    Joined:
    Feb 2, 2021
    Posts:
    2
    Hey @INedelcu!

    I just downloaded the newest unity version (2023.1), where i can call the AddInstances as you spoke above in this thread - using the MeshInstanceConfig method. However i cant really get it to work. I would love to to get a sample project showing how this would be set up.

    Hope to hear back soon,
    Michael V
     
    jffonseca likes this.
  28. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    15
  29. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    Hi! RayTracingAccelerationStructure.AddInstances wasn't merged in any Unity version. I don't know when this will be available.
     
  30. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    15
    Hi @INedelcu,

    wasnt it added here? maybe its not fully implemented, you probably know better than me, just checking.

    Best,
    j

    upload_2022-10-18_20-19-31.png
     
  31. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    That AddInstance will add only one Mesh instance to the RTAS and is the equivalent of Graphics.RenderMesh. The function signatures are quite similar. The new AddInstances will be the equivalent of Graphics.RenderMeshInstanced and you’ll have to pass an array of matrices, one matrix for each instance. The PR is in review now.
     
    matthewhxq, TerraUnity and m0nsky like this.
  32. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190

    Attached Files:

    • 2.png
      2.png
      File size:
      1.4 MB
      Views:
      132
    Lionious, Bzuco, TerraUnity and 5 others like this.
  33. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,281
    Excuse my ignorance please @INedelcu, but how would I apply this to a simple use case where I have a shrub fbx that I want to instance 10 000 times over a plain, or a tree fbx that I want to instance 50 000 times over same plain?
     
  34. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    @newguy123 you can check my example. Just replace the sphere with your shrub.
     
    newguy123 likes this.
  35. TerraUnity

    TerraUnity

    Joined:
    Aug 3, 2012
    Posts:
    1,287
    @INedelcu This is amazing, thanks for the efforts and sample projects!
     
    INedelcu likes this.
  36. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    15
    INedelcu likes this.
  37. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    15
    Hi @INedelcu looking at the HDRP example, is there a way to disable the path tracing, so still have the instances raytraced but in HDRP?

    If i disable the path tracing all the instance objects disapear, just the floor remains. Do i need to set any flag on the script?

    Cheers,
    j
     
    Last edited: Dec 1, 2022
  38. jffonseca

    jffonseca

    Joined:
    Mar 4, 2020
    Posts:
    15
    Hi @INedelcu do you have any hints for me on this, much apreciated.

    Cheers,
    j
     
  39. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    jffonseca likes this.
  40. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    874
    @INedelcu any chance there be a RayTracingAccelerationStructure.AddInstancesIndirect() or a Graphic.RenderMeshInstancedIndirect()? As expected, Graphics.DrawMeshInstancedIndirect does not work with HDRP ray-tracing (SSR/SSGI/Shadows) and since the things I need to draw is defined on the GPU I can't feed an array of matrices to RayTracingAccelerationStructure.AddInstances().

    EDIT: I presumed Graphics.RenderMesh() was working with HDRP Ray Tracing, but it does not. The MeshRenderer exposes Ray Tracing settings that are not contained in the RenderParams passed to the method.
     
    Last edited: Jan 19, 2023
  41. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    Hi @cecarlsen! Graphics.RenderX will not work in ray tracing. Anything ray tracing related needs to work using an acceleration structure and not a Camera as in the case of Graphics.RenderX.
    RayTracingAccelerationStructure.AddInstancesIndirect might be possible in the future but is a bit tricky. For example when the acceleration structure buffer (TLAS) is allocated we need to specify the amount of ray tracing instances.
     
  42. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    874
    VFXGraph now supports instanced particles with ray-tracing (within certain limits). How does that work? Do they download the particle count to allocate ray tracing instances?
     
  43. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    No. VFX particles are ray traced using intersections shader where each particle has an associated AABB.
    The acceleration structure (BLAS) associated with a VFX effect is built based on a GraphicsBuffer of AABBs and the size of this GraphicsBuffer is constant and is set to maximum amount of particles (128 in the example attached). For particles that are not alive yet or not active, all values in the AABB are set to NaN and this way DXR knows that the AABB is not active so it's not included in the build. The DXR specs about this is here: https://microsoft.github.io/DirectX-Specs/d3d/Raytracing.html#inactive-primitives-and-instances
     

    Attached Files:

    cecarlsen and m0nsky like this.
  44. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    Noisecrime, Lex4art and m0nsky like this.
  45. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    874
    Thanks, this is what I needed to know.

    I see that the overload of RayTracingAccelerationStructure.AddInstance() that takes a AABB GraphicsBuffer calls AddInstance_Procedural() internally. Wouldn't it be more clear to expose the method as RayTracingAccelerationStructure.AddInstanceProcedural()?
     
  46. imKoi

    imKoi

    Joined:
    Nov 23, 2016
    Posts:
    3
    @nedelcupaulpetru Hello, what is the easiest solution right now to render ray traced meshes? I would render different meshes one by one, like with Graphics.RenderMesh api
     
  47. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    Lex4art likes this.
  48. INedelcu

    INedelcu

    Unity Technologies

    Joined:
    Jul 14, 2015
    Posts:
    190
    The other simpler method for adding a single Mesh instance to an acceleration structure is the RayTracingAccelerationStructure.AddInstance method that takes a RayTracingMeshInstanceConfig as parameter + instance transform. It's available starting with Unity 2023.1 a3.
     
    Lex4art and Lionious like this.
  49. kisai3

    kisai3

    Joined:
    Sep 10, 2017
    Posts:
    2

    pic1.jpg pic2.jpg pic3.jpg
    This HDRP sample works correctly in path tracing, but the spheres do not seem to be rendered in ray tracing.
    How can I get it to work with ray tracing?
    I tried unchecking path tracing and checking the various checkboxes for ray tracing and pressing the play button, and I could see the spheres rendered as specular images in specular reflections, occluders in ambient occlusion, and indirect light casters, but the spheres itself and the direct shadows are not rendered.
    I would appreciate it if you could tell me how to make this HDRP sample work with ray tracing instead of path tracing.
     
  50. m0nsky

    m0nsky

    Joined:
    Dec 9, 2015
    Posts:
    263
    The "problem" here is that it does actually work with ray tracing (the instances are present in the acceleration structure) but not with rasterization. :D