ARFoundation - How to get camera orientation?

When calling TryGetLatestImage() I get the camera image in whatever orientation the device might consider its default. What is the recommended way to query that orientation?
For example if I start the app in Portrait orientation the data I get from TryGetLatestImage() might be in Landscape Left or Landscape Right. Where is that information exposed?

 

TryGetLatestImage exposes the raw, unrotated camera data. It will always be in the same orientation (landscape right, I believe). The purpose of this API is to allow for additional CPU-based image processing, such as with OpenCV or other computer vision library. These libraries usually have a means to rotate images, or accept images in various orientations, so we there is no built-in functionality to rotate the image.

For what are you using the camera image, and why do you need to rotate it?

 

We have an existing tensorflow segmentation model that was trained on portrait oriented images. I would like to feed the AR Foundation image to it but for that I first need to know whether any rotation needs to be applied or not.

 

I guess what I am looking for is the proper way to retrieve this information on iOS & Android:
https://developers.google.com/ar/re...ameraImage_1a124654de0a837fb0643820e392757f60

 

You can use Screen.orientation to determine which orientation you're in. ARCore's TransformCoordinates are informed by this value, so they should be giving you the same information.

 

I would think Screen.orientation only solves half the problem. It tells me how the screen is oriented. It doesn't tell me how the device's camera is oriented/mounted with regards to the screen.

 

It should always be the same. Are you finding it different on different devices?

 

I don't think the Android hardware guidelines require manufacturers to mount the camera in a particular direction. Here's an older example of the Nexus 5 vs 5x :
https://www.captechconsulting.com/blogs/android-camera-orientation-made-simple

 

A previous version of Google's computer vision sample for the ARCore Unity SDK obtained the physical camera's orientation. Here is how they did it: https://github.com/google-ar/arcore...sion/Scripts/ComputerVisionController.cs#L248

Luckily, it's all in C#, so you should just be able to copy and paste

_GetCameraImageToDisplayRotation

for your own needs. I'll also see about getting this into AR Foundation proper.

 

Great. Thank you for your help Tim!

 

For anyone finding this thread, I put some code using Screen.orientation on this GitHub issue

 

Incredible how nothing has been implemented yet in the ARFoundation example. It seems to me a very fundamental thing for the full functioning of the example itself. Could I ask you to fully coexist the part of code to rotate the preview correctly? There are some parts that I can't understand. Thank you