Question How to specify the Composer's rotation with an extension?

I hope I'm describing this alright. I have an extension on my Virtualcamera that rotates the camera toward walls in order to avoid clipping (which is done by using the Framing Composer for body, so rotating the camera towards the wall will also move it away from the wall), which otherwise uses Composer to rotate. It disables the Composer while this is happening, as the Composer always overrides my adjustments to RawOrientation.

My problem is that once the extension finishes rotating the camera away from the wall, and I then re-enable the composer, it will snap the camera back inside the wall where it would go if the extension wasn't there. It does this even if I max out damping and increase the dead zone and soft zone sizes so that the Composer would normally do nothing.

Here is the extension script:

 

Sure thing.

In the video my solution works exactly how I want it to up until the camera rotates far away enough from the wall that the spherecast no longer hits. There I reenable Composer because thats what I want for when there's no terrain. It is the snapping back that is the problem.
Additionally, in the video I have damping maxed at 20 for Composer as well as both zones maxed, so Composer should normally be doing nothing at all yet its still snapping back.

 

Thinking about it, I could probably make a work around by disabling Composer entirely and coding what it does in the extension, as I use fairly little of the total Composer functionality.