AR Camera moves out of parent collider at room scale

Hi

I am using AR foundation for my mobile AR app. I have AR session origin with AR camera as a child. I am allowing users to move in two ways, character controller using on screen joystick button and room scale physically.

The character controller is attached to AR session origin and it moves the child camera. However, the child camera moves as the user moves physically. This leads to camera moving outside the character controller collider. Hence now I can move through walls, not climb stairs etc.
What is the best way to keep the AR session origin and AR camera transform in sync.
I believe this would be the same issue with VR headset as well with locomotion.

Has anyone solved this problem for AR?

Many Thanks

 

Code (CSharp):

1.    Vector3 place = new Vector3(aRCamera.transform.localPosition.x, controller.center.y, aRCamera.transform.localPosition.z);
2.             controller.center = place;