Free Look camera jittering between Middle and Bottom Rig

I am seeing some werid jittering when moving the free look camera between the bottom and middle rigs.
I have tried changing a lot of parameters and from what i can tell its related to the Tracked Object Offset.
If these numbers are different the blending that is happening does not seem stable.

It seems it may also happen some on the middle to upper but not as bad. Changing the Aim mode from Composer to Hard Look At makes it go away. But I can no longer use the Tracked Object Offset.

I am driving the Axis values manually and its not a jitter from that input.
I have tried various settings on the composer to no avail.

It seems its something with the middle rig. Leaving the top and bottom on composer and setting the middle to Hard Look At also makes the issue go away. But again I loose my ability to adjust the look at offset.

I want the hard look at functionality but with the ability to have an offset without having to drive another transform.

 

Let me see if i can put something together.

 

What is the best way to send you?

 

? How do i send it?