problem with moving camera for prerendered background

i've been testing for a while to create a workflow to make provper prerendered playable backgrounds.

creating the bg from scratch
matching maya camera with unity
creating diferent layers (front bg mid....)
collision etc etc

and here is the final result

the problem comes when i try to give a litte bit of movement to the bg, the perspective efect ruins everyting



the problem is that nothing matches, layers collisions.... basically the scene breaks and make no sense
even wit a small movement of the camera

I would like to achive something like this, I think its easy co make it with an ortographic camera.. but i'll lose all the perspective when the character walk to the horizon.

one of the ideas I had was to move all the layers and collisions with the camera(character),

I would like to know if anyone can figure out how to do it

 

i've found that.... and think thats the solution at least to avoid having missmatching with the front layers when paning the camera.

but the problem now is how to apply the zdepth from the render to the unity scene
_____

Solution 1: Z-buffering for pre-rendered backgrounds
Ideally, you would want to exploit this mechanism for your pre-rendered backgrounds as well. Technically, you would do this by rendering two passes of your background: the actual color dataand a depth map.

1. The color pass would be a simple color image. It should obviously be rendered with a camera identical to the one used in your game.
2. The depth map is a grayscale image that represents the distance of pixels to the camera. White pixels are very close to the camera's near plane, whereas black pixels are very far away from the camera (close to the far plane). Again, this map should be rendered with a camera identical to the one used in your game. Pay special attention to the near and far plane settings here! (You can do a Google search for Depth Map to see an example of these maps)

What you want to do in the draw phase of your game loop, is render the background first. This means, you'd write the color image to the screen. Additionally, you want to copy the depth map to the depth buffer, so that it is filled with the same depth data, as if it were truly rendered as 3D geometry. Note that after these steps, the screen and depth buffer will be in exactly the same state, as if you has truly drawn the state in 3D. You however can render as much detail as you want, as the color and depth data are pre-rendered, thus they don't have any real-time constraints. After that you can simply draw all of your objects on top. Objects that appear behind segments of your pre-rendered background will be occluded correctly if depth buffering is enabled.

Now, this is where you might run into a problem. I'm not very familiar with Unity myself, but I do not know if it is possible to write to the depth buffer directly. If it is not, you could try the following.