Rotating to new Up Vector based on Raycast Hit's normal, not getting consistent results.

I'm trying to develop the best method for letting a character walk continuously along the faces of a mesh, no matter the orientation of surfaces or any obstructions.

Currently i'm using raycasting and if it hits a surface that has a different normal than the surface it is currently on, it'll rotate towards that normal.

This works well about half the time, and the other half it gives a strange rotation. I'm looking for help with keeping a natural looking rotation so the rigidbody forward vector kind of stays the same.

I shared the Unity project here: https://github.com/GregFinger/Unity-WorldWalker
Main things to look at are the "rayWalk" scene and the "rayWalk" script attached to the "player" gameobject.

============

The raycasting code. The important part here is the endRotation variable. That's the one that determines the rotation the rigid body should end up on when transitioning to a new surface.

Code (CSharp):

1. if (Physics.Raycast(transform.position, -transform.up, out hit, Mathf.Infinity) && startRotate==false)
2. {
3.     hitDistance = hit.distance;
4.     if (hit.normal != upVector)
5.     {
6.         prevUpVector = upVector;
7.         print("down");
8.         upVector = hit.normal;
9.         endRotation = Quaternion.FromToRotation(Vector3.up, hit.normal);
10.  
11.         startTime = Time.time;
12.         startRotate = true;
13.     }
14. }

The rotation code. I'm also not so sure the little snippet for determining the shortest rotation (ie. turn -90° rather than 270°) is doing it correctly.

Code (CSharp):

1. if (startRotate)
2. {
3.     float playback = (Time.time - startTime) / transitionDuration;
4.  
5.     // makes sure you do the shortest rotation
6.     if (Quaternion.Dot(transform.rotation, endRotation) < 0f)
7.     {
8.         endRotation = endRotation* Quaternion.AngleAxis(180f, Vector3.up);
9.     }
10.  
11.     if (playback< 1f)
12.     {
13.         transform.rotation = Quaternion.Lerp(transform.rotation, endRotation, playback);
14.     }
15.     else
16.     {
17.         startRotate = false;
18.     }
19. }

=============

\/ This is an example of a good transition rotation to a new surface \/

\/ This is an example of a bad transition rotation to a new surface \/

I am looking for help so all the transition rotations look like the first video.

=============

The one other thing that I can't figure out is how to allow turning while the rigidbody is also transitioning to a new surface. The rotation code seems to block the code to spin the rigidbody. You can notice this in the "bad rotation" video. As the rigidbody, i'm holding the key for rotation, but it only begins when the transition to the new surface finishes. The code for the rotation is pretty simple. I tried using addTorque instead because then it can spin while transitioning, but the torque gets built up and makes physics unpredictable.

Code (CSharp):

1. if (Input.GetKey("a"))
2. {
3.     transform.RotateAround(transform.position, transform.up, -spinSpeed* Time.deltaTime);
4. }
5.  
6. if (Input.GetKey("d"))
7. {
8.     transform.RotateAround(transform.position, transform.up, spinSpeed* Time.deltaTime);
9. }

=============

Extras/Notes:
In the Unity project there's some extra methods that don't work as well, but the NavMesh ones have potential.

1. Raycast using trigger colliders for steep transitions: This one works fairly well and eliminates the need for raycasting forward and backward. However the setup is a bit more intense. To see how this works, activate the trigger colliders that are childs to the "Tower" gameobject, uncomment the OnTriggerEnter part of the rayWalk script and comment out the forward/backward raycasting.
2. Rebuild NavMesh every time the player transitions to a new surface: For this one, instead of rotating the player, the world itself would rotate under the player and realign so the surface normal under the player has the same up vector as the player. After rotation it would rebuild the NavMesh. I tried doing this but can't figure out the proper code for rotating the world. Right now, it just rotates very spastically. Check out the "navWalk - Rotate World" scene.
3. Break the "world" into chunks and then auto generate NavMesh links: In the end, I think this is the way to go. It does require some preparation with the world in 3D modeling software, since it needs the world to be broken up into separate meshes so none of them have angles that are too steep for the NavAgent. But then each chunk would have it's own local up vector, allowing it to have a NavMesh be built on it. I have it already set up and auto-generating the NavMeshes on runtime, however i don't know how to make it solve generating the NavMesh Links. Check out the scene: "navWalk - Link Generation".

There's various issues happening within all this so any help with them is much appreciated.

Cheers!

 

I can't dig through all of this right now but I can offer a couple of tips:

- Try using Slerp instead of Lerp to maybe solve the rotation issue where it takes "the long way".
- Use Rotate with just the Y and Space.Self instead of RotateAround for turning left and right to prevent changing your position.
- In the end you'll probably want to build a target rotation, including your turning, and feed that to Rigidbody.MoveRotation() rather than hard setting the transform's rotation to ensure the rest of your physics plays nice.

 

Changing it over to Slerp didn't seem to work.

Code (CSharp):

1. if (playback< 1f)
2. {
3.     transform.rotation = Quaternion.Slerp(transform.rotation, endRotation, playback);
4. }

======

I changed it to this, and it still gets blocked by the rotation code (the second code snippet i posted above):

Code (CSharp):

1. if (Input.GetKey("a"))
2. {
3.     //transform.RotateAround(transform.position, transform.up, -spinSpeed * Time.deltaTime);
4.     transform.Rotate(Vector3.up* -spinSpeed* Time.deltaTime, Space.Self);
5. }

=====

Changed the rotation code to and it works, but of course still retains the some of the same bad rotations as before:

Code (CSharp):

1. if (playback< 1f)
2. {
3.     //transform.rotation = Quaternion.Slerp(transform.rotation, endRotation, playback);
4.     rb.MoveRotation(Quaternion.Slerp (transform.rotation, endRotation, playback));
5. }

and i also changed the key-press spin code to MoveRotation, but it still seems to get blocked (i even added a print within the GetKey if statement, and it does continuously print, but something is overriding that Transform.Rotate/RigidBody.MoveRotation):

Code (CSharp):

1.  
2. if (Input.GetKey("a"))
3. {
4.     //transform.RotateAround(transform.position, transform.up, -spinSpeed * Time.deltaTime);
5.     //transform.Rotate(Vector3.up* -spinSpeed* Time.deltaTime, Space.Self);
6.     rb.MoveRotation(rb.rotation * Quaternion.Euler(new Vector3(0, -spinSpeed, 0) * Time.deltaTime));
7. }
8.  

Is there a way to add both rotations together so they can happen simultaneously?

 

Instead of using

Code (CSharp):

1. endRotation = Quaternion.FromToRotation(Vector3.up, hit.normal);

Use:

Code (CSharp):

1. endRotation = transform.rotation * Quaternion.FromToRotation(transform.up, hit.normal);

Then you shouldn't even need the Dot check to find the shortest rotational path

 

didn't work. rigidbody ended up on its back like this: