Question Overlay gameobjects onto previously recorded video and maintain perspective

This is what I’m trying to accomplish:

I have a live video shot in first person shooter perspective. The video has the FPS player look around and engage with virtual objects that aren’t there - they will be added later.

I need to create the objects in Unity, and for the perspective to look correct between the Unity camera and the real life video FPS perspective.

What is the easiest way of doing this?

For example, in the real life YouTube video the player is shooting a nerf gun. Created in some 3D software are the robots.

What I have tried: In Unity, I created a robot animation of it getting hit and falling down on a green screen and recorded it to an mp4 video. I can then take the Unity clip into Adobe Premiere and overlay it onto the live video, then use the color key effect to remove the green screen. Sure, it overlays onto the live video just fine but the problem with this, the camera used in the live video and the Unity scene camera with the robot clip - the perspective / camera orientation is not synced.

What is the easiest way to sync a camera used in a live action video with a Unity scene camera? Is it just a matter of time spent tweaking and animating the Unity camera so the robot perspective appears correct with the live camera? Should I create markers in the real world around the live action scene that Unity can use to determine camera or object placement within the Unity scene - Something like tall signs that have big QR codes so Unity can translate that to XYZ coordinates in the scene?