Question How to maintain an object's upward direction when it's forward vector changes?

I have a target for an animation rigging IK system that i'm trying to simultaneously align it's forward vector with a surface normal while also having its upward vector follow the direction the mouse is moving.

this works on a flat mesh fine, but once the normal of the plane changes, the upwards vector is effected. you can see this in the video here about 12 seconds in:

what i'm trying to do is maintain the direction of the upward vector when the forward vector changes, so that hand isn't awkwardly flipping directions everytime the hand moves to a surface with a different normal.

here is the code i'm currently using, CalculateLookDirection() gets the direction of the mouse:

Code (CSharp):

1. target.transform.rotation = Quaternion.LookRotation(-normalRay.direction, CalculateLookDirection());

In case anyone was curious here's the code for the mouse rotation:

Code (CSharp):

1.    private Vector3 CalculateLookDirection() {
2.         Vector3 mousePos = Mouse.current.position.ReadValue();
3.         mousePos.z = mainCamera.farClipPlane * 5f;
4.  
5.         var cam_rot = mainCamera.ScreenPointToRay(mousePos).direction.normalized;
6.         var ray = new Ray(target.transform.position, cam_rot);
7.         return ray.direction;
8.     }

I really have no clue what to do here, I've tried doing the rotations separately, I've tried multiplying them, no matter what when the forward vector of the target changes it always rotates in a way that disregards the current upwards direction

 

Hey, ran into a similar problem recently. This answer on Stack Exchange is relevant and might help you solve this:

https://gamedev.stackexchange.com/q...always-face-the-target-position/199843#199843