Question Can I use Articulations withouth physics being applied to them?

Hello everyone!

I have a specific use case I want to implement, and I don't know how to go about it.

TLDR;
I have a simulation of a robotic arm in unity created as an Articulation Body chain. I want to create an invisible copy of that robotic arm for testing potential collisions - same articulation chain, same colliders (but only as triggers), no mesh renderers. I would like to efficiently manipulate the joints (snapping them into certain angles) and ask for collisions outside the Update loop in a motion planning algorithm.

Long version
I am implementing a simulation of a robotic arm, and currently, I am mainly experimenting with the Pick-and-Place tutorial. So I have the Niryo One robot in the scene imported from a URDF. I would like to try implementing an online motion planning algorithm, and before moving the actual robotic arm, I would like to check various joint configurations and detect if there would not be any collisions with the robotic arm.

And this was my thought process on how I wanted to approach this:

• Make a copy of the robot (mainly to have a copy of the original articulation chain).
• Set all colliders to triggers so they don't collide with the original robot's colliders and explode.
• Remove mesh renderers, or set a transparent material for debugging.
• In some coroutines outside the update loop, repeat as many times as possible for a given time interval:
  • Snap joints to some configuration (no physics applied).
  • Ask if any of the colliders are in collision with something.

Is there any flaw in my plan, and is something like this even doable with articulations? My main concern is if I can somehow disable physics for the articulation movement and snap its joint rotations around. The useless physics calculations would probably make this very ineffective, and I need to iterate over many joint configurations. Or would the trigger colliders kill the performance? If articulation isn't the right way, what might be better?

How would you approach this?
Many thanks!

 

So, it seems this is not doable.

I was able to figure out the instant repositioning of the robotic arm, though I doubt some redundant physics calculations aren't computed in the background.

The problem is in the colliders. With MeshColliders, there is no way to check if it collides with something on demand. The only option was to add a small script with OnTriggerEnter and OnTriggerExit methods on every game object of the articulation that has some colliders and update the number of collisions across the whole robotic arm to check if there are any or not. Which sounds super ineffective but still manageable, I guess. The problem is that the colliders will update only in FixedUpdate, and therefore I won't be able to do extensive state-space searches in a short time. I would be limited to one configuration state per FixedUpdate cycle. Bummer...

 

I hope I understand what you want to do: You want to investigate potential arm positions to ensure clearance, without actually doing anything with the physics of the arm, then, when you have some positions which you consider acceptable, you will power the arm into those positions? I guess a simplistic approach would be to calculate your potential joint positions / orientations and then sphere cast along that virtual arm location. If you get a hit you will know where the potential clash is and you can avoid it. You can repeatedly sphere cast in one FixedUpdate to explore potential positions before selecting a target position to power your arm to.

 

Hi and thank you for your insight! You understand correctly, but to be a bit more clear about my intentions: I would like to run Monte Carlo simulations during the physical movement of the robotic arm. I would discretize the range of motion for all the joints and perform Monte Carlo search for the next step (target configuration of the joints). While the robotic arm is physically moving from the previous state to the next (in the normal Update/FixedUpdate), I want the Monte Carlo simulation (most likely on a different thread) to search for the following joint configuration, so the arm movement can continue after it reaches the current target. Therefore, in the simulation itself, I would rapidly change the potential joint positions and check for collisions.

I didn't think of sphere casting, which sounds like a viable option to explore. Can I run sphere casts on demand without the need to do them inside the FixedUpdate, for example, on a different thread? I feel like I wouldn't get away with running the Monte Carlo simulation inside the FixedUpdate.

I was also thinking about the possibility of approximating the robotic arm (and probably other objects around it) with some custom sphere/box colliders and implementing my own simplified on-demand collision detection. The colliders would move with the transforms, and whenever asking for a collision in the simulation step, I would run an instant collision detection check between the custom colliders. Does that sound too crazy?

And with both of these options, my main concern is performance. For the Monte Carlo search to have some meaningful outcome, I reckon I need to be checking thousands (if not more) of joint positions while the robotic arm is moving to its target configuration.

EDIT: I have read some other discussions on the forums and found out that using Physics (or anything else from Unity's API) outside the main thread is a big no-no because it is not thread-safe. So I'm back to square one.

 

Shape and ray casting happens on demand (it's basically querying the current physics state, not changing it).

Collider collision events are handled by the PhysX physics cycle (https://docs.unity3d.com/Manual/ExecutionOrder.html), the event fires on collisions and will not fire again until the next physics cycle.

I have no experience of multi-threading in Unity, but if your scene is static (except for the robot arm, which you control) then being thread-safe does not seem such a problem: You know when you start and finish an arm move, so you can control that aspect of the execution sequence, and you are only reading data from the scene, not changing it.

If your scene is dynamic then I guess you are into DOTS if you want to multi-thread

There are two possible approaches:
1. Your scene is static and you are OK to experiment, then you might like to look at this: https://forum.unity.com/threads/multi-threaded-usage-of-unity-api.348072/#post-2253818
2. You want an official solution, then move to DOTS, e.g. here is someone raycasting in multi-thread:
https://forum.unity.com/threads/raycast-jobs-and-multi-threading.1082381/ but as the reply says, ask on the DOTS physics forum. I can't emphasis enough how little I know about DOTS, so ask there https://forum.unity.com/forums/dots-physics.422/.

Obviously I don't know the details of your use case, but the Monte Carlo method seems like an approach of last resort, it might be worth thinking about whether there are more efficient ways to determine the arm movement.

 

Hey! I've done some experiments with multithreading today, and the problem is that Unity API is (in most cases) checking if you are calling it from the main thread. Otherwise, it throws an exception in runtime when you try to use it from a child thread. So, for example, you cannot access

transform.position

in a child thread, let alone modify it. A way around this would be to make a

Vector3

with the position, work with it in the child thread, and then queue up a function using this

Vector3

to modify the

transfrotm.position

. You'll then pick it up from the queue and execute it in the next

Update()

call (from the main thread).

So, unfortunately, I would not be able to call

Physics.SphereCast

from the child thread. And I expect I also wouldn't be able to manipulate an articulation. Unless I'd implement my own representation and manipulation of the articulation, but then still there's the sphere cast problem.

I really like the concept of DOTS, but from my experience (over 1.5 years ago) it doesn't feel that it is a fit for such a project like this. Maybe a lot has changed since then, and it would be doable to use it only for the simulation part. But I feel like it would be a huge risk to invest time into it as this is my school research project and I'm running out of time. Though, I might spend at least a few hours experimenting with DOTS to be sure.

The whole idea of the project was to use the Monte Carlo Tree Search as an online motion planning approach for a robotic arm and compare it to some offline approaches. But it's beginning to look like I should start thinking of changing my research topic.

 

I think DOTS could be an answer. You can have a separate DOTS based representation of the arm (just the colliders) where positions are updated from the articulation bodies, then perform whatever spatial queries you need on the DOTS version, with all the performance benefits (parallelism, fast math, vectorization...).