Laggy cinemachine movement when using a custom script inheriting componentbase

Hi,
I followed how the CinemachineMixingCamera was done, and wrote this script.
What I need is that, the camera should add all of the children's transform together and update that to the brain.

My scenario when I am using this:
1.one person will controlling a customly created camera, working very similar to POV, so in that case what I mean is that this guy has total control over how the camera move, and where it should look, by using external contorllers such as XBOX controller or vive controllers.
2.In the scene, I will have a cinemachine dolly cart and have a virtual camera following it.
3.Both Vcam of 1 and 2, will be placed under the Vcam that this script 'MixerVCam' is attached to.
4.The player is supposed to be using his/her own camera like a cameraOperator, sitting on a dollied Jib. Because the player can move the camera, based on the dolly track's current moving location, and the player also has control over the look. So I am imitating how on a real large set, you have a camera operator sitting on a Jib/Crane, that is moving along a track.

My issue:
1.When I do the step 4, the Brain Camera Component becomes very laggy, resulting in stuttering motion.
2.Also, if I move the number 1 Vcam(controller by player) to the opposite direction of where the number 2 VCam (dolly cart Vcam) is moving toward, the brain camera trembles drastically and won't move to where it is supposed to move to.
3.I would like my number 1 Vcam to be influenced exactly by where the number 2 Vcam is.(imitating a camera op sitting on a moving crane) However, it is in its own position. So Baiscally, it should be where the BrainCamera is, in this use scenario.
4.When I drag the number 1 Vcam under the MixerVCam Vcam, it teleports to a random place?

public class MixerVCam : CinemachineVirtualCameraBase
{
/// <summary>The maximum number of tracked cameras. If you want to add
/// more cameras, do it here in the source code, and be sure to add the
/// extra member variables and to make the appropriate changes in
/// GetWeight() and SetWeight().
/// The inspector will figure itself out based on this value.</summary>
public const int MaxCameras = 8;
private Vector3 _finalPos;
private Quaternion _finalRot;
public Vector3 finalPos
{
get
{
return _finalPos;
}
private set
{
_finalPos = value;
}
}
public Quaternion finalRot
{
get
{
return _finalRot;
}
private set
{
_finalRot = value;
}
}

/// <summary>Blended camera state</summary>
private CameraState m_State = CameraState.Default;

/// <summary>The blended CameraState</summary>
public override CameraState State { get { return m_State; } }

/// <summary>Not used</summary>
override public Transform LookAt { get; set; }

/// <summary>Not used</summary>
override public Transform Follow { get; set; }

/// <summary>Remove a Pipeline stage hook callback.
/// Make sure it is removed from all the children.</summary>
/// <param name="d">The delegate to remove.</param>
//public override void RemovePostPipelineStageHook(OnPostPipelineStageDelegate d)
//{
// base.RemovePostPipelineStageHook(d);
// ValidateListOfChildren();
// foreach (var vcam in m_ChildCameras)
// vcam.RemovePostPipelineStageHook(d);
//}

/// <summary>Makes sure the internal child cache is up to date</summary>
protected override void OnEnable()
{
base.OnEnable();
InvalidateListOfChildren();
}

/// <summary>Makes sure the internal child cache is up to date</summary>
public void OnTransformChildrenChanged()
{
InvalidateListOfChildren();
}

private CinemachineVirtualCameraBase[] m_ChildCameras;
private Dictionary<CinemachineVirtualCameraBase, int> m_indexMap;


/// <summary>Get the cached list of child cameras.
/// These are just the immediate children in the hierarchy.
/// Note: only the first entries of this list participate in the
/// final blend, up to MaxCameras</summary>
public CinemachineVirtualCameraBase[] ChildCameras
{
get { ValidateListOfChildren(); return m_ChildCameras; }
}

/// <summary>Invalidate the cached list of child cameras.</summary>
protected void InvalidateListOfChildren()
{
m_ChildCameras = null;
m_indexMap = null;
}

/// <summary>Rebuild the cached list of child cameras.</summary>
protected void ValidateListOfChildren()
{
if (m_ChildCameras != null)
return;

m_indexMap = new Dictionary<CinemachineVirtualCameraBase, int>();
List<CinemachineVirtualCameraBase> list = new List<CinemachineVirtualCameraBase>();
CinemachineVirtualCameraBase[] kids
= GetComponentsInChildren<CinemachineVirtualCameraBase>(true);
foreach (CinemachineVirtualCameraBase k in kids)
{
if (k.transform.parent == transform)
{
int index = list.Count;
list.Add(k);
if (index < MaxCameras)
m_indexMap.Add(k, index);
}
}
m_ChildCameras = list.ToArray();
}

/// <summary>Called by CinemachineCore at designated update time
/// so the vcam can position itself and track its targets. This implementation
/// computes and caches the weighted blend of the tracked cameras.</summary>
/// <param name="worldUp">Default world Up, set by the CinemachineBrain</param>
/// <param name="deltaTime">Delta time for time-based effects (ignore if less than 0)</param>
public override void InternalUpdateCameraState(Vector3 worldUp, float deltaTime)
{
//UnityEngine.Profiling.Profiler.BeginSample("CinemachineMixingCamera.UpdateCameraState");
CinemachineVirtualCameraBase[] children = ChildCameras;

//haixu: always reset to zero and use children cams' locals.
finalPos = Vector3.zero;
finalRot = Quaternion.identity;
for (int i = 0; i < MaxCameras && i < children.Length; ++i)
{
CinemachineVirtualCameraBase vcam = children;
if (vcam.isActiveAndEnabled)
{
//finalPos += transform.InverseTransformPoint(vcam.State.FinalPosition);
finalPos += vcam.State.FinalPosition;
finalRot *= vcam.State.FinalOrientation;
}
}

m_State.RawPosition = finalPos;
m_State.RawOrientation = finalRot;
//UnityEngine.Profiling.Profiler.EndSample();
}
}
}


Thanks so much!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

 

Hi again,

Thanks for the tip!

Yes your understanding is correct. So, I have 3 classes altogether to immitate this effect. The script I mainly have trouble is attached below, this is the one that adds the outcome of each of its child virtual camera.

Code (CSharp):

1.  
2. namespace Cinemachine
3. {
4.     [ExecuteInEditMode, DisallowMultipleComponent]
5.     [AddComponentMenu("Cinemachine/BrainslumSumVCam")]
6.     public class SumVCam : CinemachineVirtualCameraBase
7.     {
8.         /// <summary>The maximum number of tracked cameras.  If you want to add
9.         /// more cameras, do it here in the source code, and be sure to add the
10.         /// extra member variables and to make the appropriate changes in
11.         /// GetWeight() and SetWeight().
12.         /// The inspector will figure itself out based on this value.</summary>
13.         public const int MaxCameras = 8;
14.         private Vector3 _finalPos;
15.         private Quaternion _finalRot;
16.         public Vector3 finalPos
17.         {
18.             get
19.             {
20.                 return _finalPos;
21.             }
22.             private set
23.             {
24.                 _finalPos = value;
25.             }
26.         }
27.         public Quaternion finalRot
28.         {
29.             get
30.             {
31.                 return _finalRot;
32.             }
33.             private set
34.             {
35.                 _finalRot = value;
36.             }
37.         }
38.  
39.         /// <summary>Blended camera state</summary>
40.         private CameraState m_State = CameraState.Default;
41.  
42.         /// <summary>The blended CameraState</summary>
43.         public override CameraState State { get { return m_State; } }
44.  
45.         /// <summary>Not used</summary>
46.         override public Transform LookAt { get; set; }
47.  
48.         /// <summary>Not used</summary>
49.         override public Transform Follow { get; set; }
50.  
51.         /// <summary>Remove a Pipeline stage hook callback.
52.         /// Make sure it is removed from all the children.</summary>
53.         /// <param name="d">The delegate to remove.</param>
54.         //public override void RemovePostPipelineStageHook(OnPostPipelineStageDelegate d)
55.         //{
56.         //    base.RemovePostPipelineStageHook(d);
57.         //    ValidateListOfChildren();
58.         //    foreach (var vcam in m_ChildCameras)
59.         //        vcam.RemovePostPipelineStageHook(d);
60.         //}
61.  
62.         /// <summary>Makes sure the internal child cache is up to date</summary>
63.         protected override void OnEnable()
64.         {
65.             base.OnEnable();
66.             InvalidateListOfChildren();
67.         }
68.  
69.         /// <summary>Makes sure the internal child cache is up to date</summary>
70.         public void OnTransformChildrenChanged()
71.         {
72.             InvalidateListOfChildren();
73.         }
74.  
75.         private CinemachineVirtualCameraBase[] m_ChildCameras;
76.         private Dictionary<CinemachineVirtualCameraBase, int> m_indexMap;
77.  
78.  
79.         /// <summary>Get the cached list of child cameras.
80.         /// These are just the immediate children in the hierarchy.
81.         /// Note: only the first entries of this list participate in the
82.         /// final blend, up to MaxCameras</summary>
83.         public CinemachineVirtualCameraBase[] ChildCameras
84.         {
85.             get { ValidateListOfChildren(); return m_ChildCameras; }
86.         }      
87.  
88.         /// <summary>Invalidate the cached list of child cameras.</summary>
89.         protected void InvalidateListOfChildren()
90.         {
91.             m_ChildCameras = null;
92.             m_indexMap = null;        
93.         }
94.  
95.         /// <summary>Rebuild the cached list of child cameras.</summary>
96.         protected void ValidateListOfChildren()
97.         {
98.             if (m_ChildCameras != null)
99.                 return;
100.  
101.             m_indexMap = new Dictionary<CinemachineVirtualCameraBase, int>();
102.             List<CinemachineVirtualCameraBase> list = new List<CinemachineVirtualCameraBase>();
103.             CinemachineVirtualCameraBase[] kids
104.                 = GetComponentsInChildren<CinemachineVirtualCameraBase>(true);
105.             foreach (CinemachineVirtualCameraBase k in kids)
106.             {
107.                 if (k.transform.parent == transform)
108.                 {
109.                     int index = list.Count;
110.                     list.Add(k);
111.                     if (index < MaxCameras)
112.                         m_indexMap.Add(k, index);
113.                 }
114.             }
115.             m_ChildCameras = list.ToArray();
116.         }
117.  
118.         /// <summary>Called by CinemachineCore at designated update time
119.         /// so the vcam can position itself and track its targets.  This implementation
120.         /// computes and caches the weighted blend of the tracked cameras.</summary>
121.         /// <param name="worldUp">Default world Up, set by the CinemachineBrain</param>
122.         /// <param name="deltaTime">Delta time for time-based effects (ignore if less than 0)</param>
123.         public override void InternalUpdateCameraState(Vector3 worldUp, float deltaTime)
124.         {
125.             //UnityEngine.Profiling.Profiler.BeginSample("CinemachineMixingCamera.UpdateCameraState");
126.             CinemachineVirtualCameraBase[] children = ChildCameras;
127.  
128.             //haixu: always reset to zero and use children cams' locals.
129.             finalPos = Vector3.zero;
130.             finalRot = Quaternion.identity;
131.             for (int i = 0; i < MaxCameras && i < children.Length; ++i)
132.             {
133.                 CinemachineVirtualCameraBase vcam = children[i];
134.                 if (vcam.isActiveAndEnabled)
135.                 {                
136.                     //finalPos += transform.InverseTransformPoint(vcam.State.FinalPosition);
137.                     finalPos += vcam.State.FinalPosition;
138.                     finalRot *= vcam.State.FinalOrientation;
139.                 }
140.             }
141.  
142.             m_State.RawPosition = finalPos;
143.             m_State.RawOrientation = finalRot;
144.             //UnityEngine.Profiling.Profiler.EndSample();
145.             InvokePostPipelineStageCallback(
146.           this, CinemachineCore.Stage.Finalize, ref m_State, deltaTime);
147.         }
148.     }
149. }
150.  

The other two scripts:
1. VR_VirtualCameraBody:
This one calculates how far Vive controller has moved, and applies that distance to Virtual Camera Body Stage.

When the user pulls the trigger, the camera will move according to how far the Vive controller has moved in real space and then, times a public float distanceMultiplier. This way the player can move fast or slow, depending on the shot, all according to the Vive controller.

However, you are not confined by the size of the tracked area. Essentially, you can pull trigger and move, when you reach the tracked boundary, release the trigger so the user can go back to the center of the room and pull the trigger to offset animate the camera again.

2. VR_VirtualCameraHead
This one works exactly as VR_VirtualCameraBody, but it offsets the rotation. So when you pull the trigger, how much Vive controller has rotated will apply to virtual camera.

3. How I use them with SumCamera:
I set up a virtual camera shot roughly, maybe using an autodolly with a fixed speed, and then stitch together a few dolly shots.(I am not fine tuning stuff, especially 'look at', thats very time consuming for me usually because I came directly from a DP background in filmmaking I am kinda picky about composition every frame)

So instead of relying on keyframing to achieve a fine-tuned body and lookat, I throw all the dolly show I roughly created under the SumCamera. And THEN I throw in another VCam with VR_VirtualCameraBodyand VR_VirtualCameraHead.

Now, I could just watch how Cinemachine is procedurally creating the shot, and do the correction on the fly with my hand controlling a VR_VirtuaCameraHead and VR_VirtualCameraBody.


Why I am having problem:
1.the SumCamera somehow becomes very laggy, even if I have just my own VR_VirtualCameraBody and a dolly VCam on it.
2.If I move my vive controller to the opposite direction of where dolly VCam is moving toward, it will jiggle(so the CameraBrain does not want to ). I am expecting to use it smoothly, like the dolly is moving forward at a fixed speed so I want to pull back a little bit with my VR_VirtualCameraBody, but again, it jitters.

 

Code (CSharp):

1. using UnityEngine;
2. using Opsive.ThirdPersonController.Input;
3. using Brainslum;
4.  
5. namespace Cinemachine
6. {
7.     [AddComponentMenu("")]
8.     [SaveDuringPlay]
9.     public class VRCameraBody : CinemachineComponentBase, INeedPlayerInput, INeedVR
10.     {
11.         #region Vars
12.         [SerializeField] private string rightControllerTrigger = "RightControllerTriggerAxis";
13.         [SerializeField] public float moveSpeedMultiplier = 100;
14.         public int moveDamping = 8; // the dampig for lerp
15.  
16.         #region INeedVR
17.         private VRDeviceManager vrDeviceManager;
18.         //VR Transofrms
19.         private Transform rightControllerTransform;
20.         private Vector3 moveTargetstartRot; //unused
21.         private Vector3 startControllerRot; //unused
22.  
23.         public GameObject playAreaGO;
24.         public GameObject controllerLeftGO;
25.         public GameObject controllerRightGO;
26.  
27.         public bool GetVRDevices()
28.         {
29.             if (controllerRightGO != null)
30.             {
31.                 Debug.Log("HAIXU:VRCameraBody.GetVRDevices() success");
32.                 return true;
33.             }
34.  
35.             else if (controllerRightGO == null)
36.             {
37.                 if (vrDeviceManager != null)
38.                 {
39.                     playAreaGO = vrDeviceManager.playAreaGO; //playAreaGO.GetComponent<SteamVR_PlayArea>();
40.                     controllerLeftGO = vrDeviceManager.controllerLeftGO;
41.                     controllerRightGO = vrDeviceManager.controllerRightGO;
42.                     GetVRTKTransforms();
43.                 }
44.                 return false;
45.             }
46.             return false;
47.         }
48.  
49.         private void GetVRTKTransforms()
50.         {
51.             rightControllerTransform = controllerRightGO.transform.parent;
52.         }
53.         #endregion
54.  
55.         #region INeedPlayerInput
56.         private RewiredInput rewiredInput;
57.         public void GetPlayerInput()
58.         {
59.             if (rewiredInput == null)
60.             {
61.                 if (Singleton_PlayerInput.Instance != null)
62.                 {
63.                     rewiredInput = Singleton_PlayerInput.Instance.rewiredInput;
64.                 }
65.             }
66.         }
67.         #endregion
68.  
69.         #region MOVE LOGICS
70.         //bool
71.         private bool hasPinSet;
72.         //Controller
73.         private Vector3 controllerStartPos;
74.         private Vector3 curFrameControllerPos;
75.         //targets
76.         private Vector3 targetStartWorldPos;
77.         private Vector3 targetDstWorldPos;
78.         private Vector3 distanceFromPin; // curlocalpos - moveTargetStartPos
79.         private Vector3 lerpStartWorldPos;
80.         #endregion
81.  
82.         #endregion
83.  
84.         #region override cinemachine
85.         public override CinemachineCore.Stage Stage
86.         {
87.             get
88.             {
89.                 return CinemachineCore.Stage.Body;
90.             }
91.         }
92.  
93.         public override bool IsValid
94.         {
95.             get
96.             {
97.                 return enabled;
98.             }
99.         }
100.         #endregion
101.  
102.  
103.         private void Start()
104.         {
105.             if (Singleton_PlayerInput.Instance != null)
106.             {
107.                 vrDeviceManager = Singleton_PlayerInput.Instance.gameObject.GetComponent<VRDeviceManager>();
108.             }
109.         }
110.  
111.         private void OnEnable()
112.         {
113.             GetPlayerInput();
114.             GetVRDevices();
115.         }
116.  
117.         public override void MutateCameraState(ref CameraState curState, float deltaTime)
118.         {
119.             if (rewiredInput != null)
120.             {
121.                 if (GetVRDevices())
122.                 {
123.                     if (rewiredInput.GetAxis(rightControllerTrigger) > 0.1f)
124.                     {
125.                         Debug.Log("VRCameraBody: " + rewiredInput.GetAxis(rightControllerTrigger));
126.                         if (hasPinSet == false)
127.                         {
128.                             SetControllerStartPos();//run once only.
129.                             SetTargetStartPos();
130.                             SetLerpStartPos();
131.                         }
132.  
133.                         CalculateDistanceDiff();
134.                         Move(ref curState);
135.                     }
136.                     else if (rewiredInput.GetAxis(rightControllerTrigger) < 0.1f)
137.                     {
138.                         hasPinSet = false;
139.                         //otherwise the rawposition will be reset to curState's default position.
140.                         curState.RawPosition = lerpStartWorldPos;
141.                     }
142.                 }
143.             }
144.  
145.             else if (rewiredInput == null)
146.             {
147.                 GetPlayerInput();
148.             }
149.             //UpdateCameraForwardDirection();
150.         }
151.  
152.         private void Move(ref CameraState curState)
153.         {
154.             //do not change targetStartWorldPos after SetTargetStartPos();
155.             targetDstWorldPos = targetStartWorldPos + distanceFromPin * moveSpeedMultiplier;// * CameraOperate.globalMoveMulti;
156.             //do not lerp rawposition directly or it will be jerky.
157.             lerpStartWorldPos = Vector3.Lerp(lerpStartWorldPos, targetDstWorldPos, Time.deltaTime * moveDamping);
158.             curState.RawPosition = lerpStartWorldPos;
159.         }
160.  
161.         //lerp and targetstartpos must be seperate.
162.         private void SetTargetStartPos()
163.         {
164.             targetStartWorldPos = VirtualCamera.State.FinalPosition;
165.             //targetStartWorldPos = lerpStartPos;
166.         }
167.         private void SetLerpStartPos()
168.         {
169.             lerpStartWorldPos = VirtualCamera.State.FinalPosition;
170.             // TargetstartRot = vCam.transform.localEulerAngles;
171.         }
172.  
173.         //to calculate how much we have moved.
174.         private void SetControllerStartPos()
175.         {
176.             controllerStartPos = rightControllerTransform.position;
177.             hasPinSet = true;
178.         }
179.         public void CalculateDistanceDiff()
180.         {
181.             curFrameControllerPos = rightControllerTransform.position;
182.             distanceFromPin = curFrameControllerPos - controllerStartPos;
183.         }
184.     }
185. }

Here is the VR_VirtualCameraBody.. The VR_VirtualCameraHead is just the same, only doing rotation with Slerp

 

This is really crucial to my workflow... Please help because I just have no single clue why this wouldn't work the way I expected.

Here is a picture example of how I use it with, i.g., a dolly cart vcam.(both the link and attachment)
https://drive.google.com/open?id=1a8HCDwSM1ZgQsGVYfqxAwpFVVZWaBFQ1

UnlockCameraRotation and RTrigger in the inspector are inputs to start applying the offset of the Virtual Camera

THANKSS SO MUCH this could save me from deadlines.

 

Hey!
Thanks for the help and I got the position to work!

However, I want to check whether the vcam is the current live vcam, when I am setting the position.
This way I can set up every virtual camera with this extension. And just watch it cuts itself and I will always be tweaking the active one.

However the following implementation did not work in order to check which one is active? I guess the base class alays return false?
I also tried if (vcam as ICinemachineCamera == cmBrain.ActiveVirtualCamera). It just bugged out with null reference???

by the way I'm getting calling my GetCinemachineBrain() in in Awake(), or OnEnbale(). Both did not work.

private void GetCinemachineBrain()
{
if (cmBrain == null)
{
cmBrain = CinemachineCore.Instance.GetActiveBrain(0);
}
}

Could you please point me

Code (CSharp):

1.         /// <summary>
2.         /// This callback will be called after the virtual camera has implemented
3.         /// each stage in the pipeline.  This method may modify the referenced state.
4.         /// If deltaTime less than 0, reset all state info and perform no damping.
5.         /// </summary>
6.         protected override void PostPipelineStageCallback(
7.             CinemachineVirtualCameraBase vcam,
8.             CinemachineCore.Stage stage, ref CameraState state, float deltaTime)
9.         {
10. // if (vcam as ICinemachineCamera == cmBrain.ActiveVirtualCamera) will bug out!!!!!!!!!!!!!!!!!!!
11.             if (vcam.IsLiveChild(vcam))
12.             {
13.                 if (stage == CinemachineCore.Stage.Body)
14.                 {
15.                     ApplyOffsetCorrection(vcam, stage, ref state, deltaTime);
16.                 }
17.             }
18.         }

 

Also, I have not tested it with the collider extension yet.

How do I make sure that my correction happens before the collider does its work?

Thanks!

 

Hey Greg I tried everything but I cant get state.OrientationCorrection working... I am not so famaliar with quaternions so could you please point to me which direction to go? Please diregard the variable
private bool LerpBackWhenRelease;
I havent got that far to get it working yet.

I get the Quaternion To Matrix conversion failed because input Quaternion is invalid in editor mode.

And the script is setting the Vcam to look at quaternion.identity at the start and it just stuck there.
I got it working once, but it seems like the X and Z axis are totally inversed, meaning if I drag them they move the opposite direction. But this code corrupts the project and I lost the code that was working the opposite way... Sometimes it freezes when I hit the play mode button and it gets stuck all day. I could only force close it and re open Unity.


Could you please point to me:
1. why I am getting the editor bug
2. How I should get it working again?

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using Opsive.ThirdPersonController.Input;
5. using Brainslum;
6. using System;
7.  
8. namespace Cinemachine
9. {
10.     [AddComponentMenu("")]
11.     [SaveDuringPlay]
12.     public class CMExtension_RotationOffset : CinemachineExtension
13.     {
14.         //drag one here if you do not want use VR Device automatically
15.         public GameObject objectToFollow;
16.         //Automatically Get VR Devices?
17.         public EAutoGetVRDevice eAutoGetVRDevice = EAutoGetVRDevice.Vive_RightHand;
18.  
19.         [SerializeField] private bool SkipInput;
20.         [SerializeField] private string startOffsetAction = "StartOffsetPosition"; //Action used
21.         [SerializeField] public float moveDistanceMultiplier = 1; //move speed
22.         public int lerpSpeedDamping = 8; // the damping for lerp
23.         [SerializeField] private bool LerpBackWhenRelease;
24.  
25.         //Optional Dependencies, if we are using Input, or want to use VR Device to track
26.         private PlayerInput playerInput;
27.         private VRDeviceManager vrDeviceManager;
28.  
29.         #region Check & Get PlayerInput
30.         private bool CheckPlayerInput()
31.         {
32.             if (playerInput != null)
33.             {
34.                 return true;
35.             }
36.             else
37.             {
38.                 GetPlayerInput();
39.                 return false;
40.             }
41.         }
42.         private void GetPlayerInput()
43.         {
44.             if (playerInput == null)
45.             {
46.                 if (Singleton_PlayerInput.Instance != null)
47.                 {
48.                     playerInput = Singleton_PlayerInput.Instance.rewiredInput;
49.                 }
50.             }
51.         }
52.         #endregion
53.  
54.         #region Check & Get Follow Obejct. Automatically Get VR Controller by accessing the VRDeviceManager Singleton.
55.         private bool CheckFollowObjectValide()
56.         {
57.             if (objectToFollow != null)
58.             {
59.                 return true;
60.             }
61.             else
62.             {
63.                 GetFollowObject();
64.                 return false;
65.             }
66.         }
67.  
68.         private void GetFollowObject()
69.         {
70.             if (eAutoGetVRDevice == EAutoGetVRDevice.DontUseVRDevice)
71.             {
72.                 Debug.Log("You have forgotten to drag a Follow Object instance if DontFollowVR");
73.             }
74.             else if (eAutoGetVRDevice == EAutoGetVRDevice.Vive_LeftHand)
75.             {
76.                 if (CheckVRDeviceManager() && vrDeviceManager.controllerLeftGO != null)
77.                 {
78.                     objectToFollow = vrDeviceManager.controllerLeftGO;
79.                 }
80.             }
81.             else if (eAutoGetVRDevice == EAutoGetVRDevice.Vive_RightHand)
82.             {
83.                 if (CheckVRDeviceManager() && vrDeviceManager.controllerRightGO != null)
84.                 {
85.                     objectToFollow = vrDeviceManager.controllerRightGO;
86.                 }
87.             }
88.         }
89.         #endregion
90.  
91.         #region Check & Get VRDeviceManager Singleton
92.         private bool CheckVRDeviceManager()
93.         {
94.             if (vrDeviceManager != null)
95.             {
96.                 return true;
97.             }
98.             else
99.             {
100.                 GetVRDeviceManager();
101.                 return false;
102.             }
103.         }
104.         //if this is null, a new one will be automatically created.
105.         private void GetVRDeviceManager()
106.         {
107.             if (Singleton_PlayerInput.Instance != null)
108.             {
109.                 vrDeviceManager = Singleton_PlayerInput.Instance.gameObject.GetComponent<VRDeviceManager>();
110.             }
111.         }
112.         #endregion
113.  
114.         #region Movement
115.         //Final Position Calculation Outcome
116.         private Quaternion finalRotationCorrection;
117.  
118.         #region Distance Offset  
119.         //bool
120.         private bool controllerInitialOffsetCalculated;
121.         private bool firstResultCalculated;
122.         //Controller
123.         private Quaternion startControllerRot;
124.         private Quaternion curFrameControllerRot;
125.         //targets
126.         private Quaternion distanceTraveled; // curlocalpos - moveTargetStartPos
127.         private Quaternion lerpingTravelDistance;
128.         private Quaternion moveTargetStartRot;
129.         //Back
130.         private Quaternion lastWorldPosition;
131.         private Quaternion lerpBackRot;
132.  
133.         //to calculate how much we have moved.
134.         private void SetStartPositions()
135.         {
136.             startControllerRot = objectToFollow.transform.rotation;
137.             moveTargetStartRot = VirtualCamera.transform.rotation;
138.         }
139.         private void SetLerpStartPos()
140.         {
141.             //In order to start from the object's place at the first frame.
142.             //lerpPos = VirtualCamera.transform.position;
143.             //lerpingTravelDistance = Quaternion.identity;
144.             Debug.Log("SetLerpStartPos:" + lerpingTravelDistance);
145.         }
146.         public void CalculateObjectDistanceTraveled()
147.         {
148.             curFrameControllerRot = objectToFollow.transform.rotation;
149.             //startControllerPos -> curFrameControllerPos
150.             distanceTraveled = curFrameControllerRot * Quaternion.Inverse(startControllerRot);
151.         }
152.         //To slowly generate the move position, because sometimes the multiplier is too drastic.
153.         private void LerpDistanceTraveled()
154.         {
155.             //do not lerp rawposition directly or it will be jerky.
156.             lerpingTravelDistance = Quaternion.Slerp(lerpingTravelDistance, distanceTraveled, Time.deltaTime * lerpSpeedDamping);
157.             Debug.Log("startControllerRot" + startControllerRot);
158.             Debug.Log("curFrameControllerRot" + curFrameControllerRot);
159.             Debug.Log("distanceTraveled" + distanceTraveled);
160.             Debug.Log("lerpingTravelDistance" + lerpingTravelDistance);
161.             Debug.Log("lastWorldPosition" + lastWorldPosition);
162.         }
163.         private void CalculateFinalResult()
164.         {
165.             //if pressed before, remember the last results and don't start from zero
166.             if (firstResultCalculated == true)
167.             {
168.                 finalRotationCorrection = lerpingTravelDistance * lastWorldPosition;
169.             }
170.             else
171.             {
172.                 finalRotationCorrection = lerpingTravelDistance;
173.             }
174.         }
175.  
176.         private void DistanceCorrection()
177.         {
178.             if (SkipInput || playerInput.GetAxis(startOffsetAction) > 0.2f)
179.             {
180.                 Debug.Log("CMExtension_PositionOffset: " + playerInput.GetAxis(startOffsetAction));
181.                 if (controllerInitialOffsetCalculated == false)
182.                 {
183.                     SetStartPositions();//run once only
184.                     SetLerpStartPos();
185.                     controllerInitialOffsetCalculated = true;
186.                 }
187.                 CalculateObjectDistanceTraveled();
188.                 LerpDistanceTraveled();
189.  
190.                 CalculateFinalResult();
191.             }
192.             else if (!SkipInput && playerInput.GetAxis(startOffsetAction) < 0.2f)
193.             {
194.                 //dont touch finalcorrection here, because that + virtualcamera.transform.position = Brain position.
195.                 if (LerpBackWhenRelease)
196.                 {
197.                     LerpBackToRawPosition();
198.                 }
199.                 controllerInitialOffsetCalculated = false;
200.                 firstResultCalculated = true;
201.                 lastWorldPosition = finalRotationCorrection;
202.             }
203.         }
204.  
205.         private void LerpBackToRawPosition()
206.         {
207.             //lerpPos = Vector3.Lerp(lerpPos, Vector3.zero, Time.deltaTime * moveDamping * 0.001f);
208.             lerpBackRot = lerpingTravelDistance;
209.             lerpBackRot = Quaternion.Slerp(lerpBackRot, Quaternion.identity, Time.deltaTime * lerpSpeedDamping);
210.             finalRotationCorrection = lerpBackRot;
211.         }
212.         #endregion
213.         #endregion
214.  
215.         public void ApplyOffsetCorrection(CinemachineVirtualCameraBase vcam, CinemachineCore.Stage stage, ref CameraState state, float deltaTime)
216.         {
217.                 if (CheckPlayerInput())
218.                 {
219.                     if (CheckFollowObjectValide())
220.                     {
221.                         DistanceCorrection();
222.                     }
223.                 }
224.            
225.             state.OrientationCorrection = finalRotationCorrection;
226.         }
227.  
228.         /// <summary>
229.         /// This callback will be called after the virtual camera has implemented
230.         /// each stage in the pipeline.  This method may modify the referenced state.
231.         /// If deltaTime less than 0, reset all state info and perform no damping.
232.         /// </summary>
233.         protected override void PostPipelineStageCallback(
234.             CinemachineVirtualCameraBase vcam,
235.             CinemachineCore.Stage stage, ref CameraState state, float deltaTime)
236.         {
237.             //if (vcam as ICinemachineCamera == cmBrain.ActiveVirtualCamera)  if (vcam.IsLiveChild(vcam))
238.             if (CinemachineCore.Instance.IsLive(vcam))
239.             {
240.                 ApplyOffsetCorrection(vcam, stage, ref state, deltaTime);
241.             }
242.         }
243.     }
244. }

 

what I am trying to do in a nutshell:
Use current frame target.transform.rotation to minus a marked at the start of tracking.curFrameControllerRot * Quaternion.Inverse(startControllerRot);

And the difference will be the orientation correction, which will be combined with RawOrientation in CameraState

Thanks Greg!