Translating X,Y mouse coordinates from one cameras viewport to another one?

Hi,

hopefully I can explain my problem correct so it makes sense...here is what I have: an off-screen scene is captured by an off-screen camera and the result is projected to a rendering texture - visible to the player. It's like a monitor that shows the action, but just occupies a part of the visible screen. Similar to a monitor or TV, placed on a desk. You, as the player, watches the monitor.

And this is what I want: if I - or the player - clicks or "touches" onto the monitor - these x,y coordinates should be translated to the off-screen cameras viewport and an ray should be cast from this x,y position into the off-screen scene. So the x,y coordinates in the one cameras viewport need to be translated to the other cameras viewport. Resulting in a ray from that starts from the main cameras center point (viewport coord 0.5, 0.5), then runs through the point clicked or touched INTO the on the monitor projected scene - hitting any objects given there.

Does this makes any sense?

I though I capture the mouseclick with ScreenToViewportPoint from the main camera and then use this coordinates in the second camera with ViewportPointToRay. However, as the two cameras setup, position and viewport recs are differ I'm getting not the desired result. And now I've run out of ideas.

Any help is appreciated! Thanks!

-Axel

 

Have you tried scaling the mouse positions?

mainPosX / mainScreenWidth = tvPosX / tvScreenWidth, solving for tvPosX

mainPoxY / mainScreenHeight = tvPosY / tvScreenHeight, solving for tvPosY

 

As the TV is a 3D object in the main cameras world space it got no width or height in pixel values as the mouse coordinates. But yes, basically I'm looking for these pseudo-mouse coordinates - in pixel values - on the TV to cast a ray into the second cameras world space.That's the problem...

 

I'm really missing something here.. How can a face on a 3D object, in this case a TV screen, not have a width or height? If you UV wrap that thing, it must have relative pixel dimensions, no?

 

Right what you need to do is:
Shoot a ray from the main camera. If the ray intersects your 3D Monitor object (specifically the faces showing your render texture) you can then access the UV coordinated from the RaycastHit class.

You then transorm these UVs into normalised viewport coords for your second camera (eg may need to flip the Y or something along those lines). This should be straightforward since you will probably have mapped a rect using full uv range.

From these coordinates, you can shoot another ray using Camera.ViewportPointToRay but using your second camera.

Make sure you only shoot the second ray if your first ray hit the actual polys which are mapped with your render texture. Maybe put that obj in a seperate layer...

I'm on my phone right now so can't really write any pseudo code for you, but the above should be enough to get you started.

 

Thanks Paul - checking the UV coordinates was the missing piece. It's working now! I've just a problem with getting inaccurate Y coordinates now, for some reason the UV Y-coordinates are not exact, but the X coordinates are pixel correct. Strange. However, the main problem is solved. Thanks again...to booth of you!

-Axel