AR camera with trail renderer way points issue

I have this industrial plant model that is over a Vuforia Image Target. The green fluid in the pipe is a sphere with a trail renderer following some way points. It works just how I want it to.
The image behind it is a diagram of the plant in 2D form and is a child of the camera because we want it to rotate with the camera and maintain that distance. I wanted to show the same movement on the diagram, also using a sphere, trail renderer and waypoints but the trail renderer just draws all over the place when the AR camera is shakey.
So when the image target moves around, the trail renderer doesnt move erratically, it is basically keeping its local position, but I cant get the diagram to behave in the same way. Hope this is a clear explanation, any thoughts on how I can fix this or another approach to get the same effect?