ARCore/ARKit drifting scene(please help)

Hello community,

Right to the point... I'm working on a project for a client. It is both for Android and iOS, so it is simultaneously using ARCore and ARKit(Unity 2018.3.12f1).

This project has already been made and I'm reworking it for a different purpose.

The AR code had already been made and it worked well in the previous app in the sense of anchors and tracking(I mostly had to focus on UI and to rework the augmented image detection flow(flat surface->image->shoot from image).

Now the problem that I have is with the exact same manipulation/anchoring/tracking/parenting code which the scene from the previous project works and doesn't drift, my scene drifts.

Everything points out that there is something wrong with the scene itself.


This is how the new scene looks like in the hierarchy.
It contains rigs(a whole lot of levels going down), animations, particle effects, colliders, a model with a texture for shadow).

And this is the previous scene:


This one contains only models and a model with a texture for shadows nothing else.

They are both using exactly the same settings for everything and the only parenting that happens is to the anchor and to the root(scene1_root, Mongolian_Ger_v11), all the manipulations also happen via the root.

The Mongolian scene doesn't drift at all while the new scene moves almost as if it is attached to the camera.
The original scene doesn't drift in either ARCore or ARKit, while the new scene drifts in both.
Any ideas are very welcome as I'm completely stuck.

Thanks in advance!