MRT approach to picking as an alternative to ray casting

Hi,

I'm working on a system that in some ways resembles a CAD tool. There is a 3D environment where you can place objects, align and connect them, change properties, etc.

Currently we are using a ray cast approach to selecting/picking objects, but we do have a few types of objects where the geometry is not fully known on the CPU side: wires, terrain, primitive types like cones and capsules, etc. All these meshes are shaped in geometry and vertex shaders.

In a similar system we have developed using a different technology, we have use an approach that renders IDs and coordinates into render buffers. Then we read the values in the buffer index under the mouse every frame. The good thing about this is that it always works and it is easy to support new types of objects, as long as you have control of the shaders. It supports picking through wire frames and camera clip planes out of the box too.

I have done my best and looked for information about how this can be set up in Unity. The best I have found is a few examples of rendering to multiple textures, but I have not found any examples of a GPU side picking system like the one we consider.

Does anyone know of good examples of similar efforts and documentation that we can leverage? Are there better ways of doing this that we should explore instead?

Thanks.

 

possibly some references here
https://matthewyan.com/2015/10/25/gpu-based-object-picking-without-physics/

maybe can do something with
https://docs.unity3d.com/2021.2/Documentation/ScriptReference/Camera.RenderRequestMode.ObjectId.html

or, older ways, custom shader to render with id
http://answers.unity.com/answers/1421403/view.html

adding some alternative ideas:
- could also select with manual ray intersection calculations (to lines, or triangles, or other shapes)
- depth ray (can get position, not object id's) https://github.com/staggartcreations/Graphics-Raycast
- raycast without colliders https://forum.unity.com/threads/a-solution-for-accurate-raycasting-without-mesh-colliders.134554/

 

Thank you, @mgear! I will go through the references you provided.