ARFoundation : I need RenderTexture to Texture2D every single frame

Hi ~!

I need RenderTexture to Texture2D every single frame.

How to get camera texture in ARFoundation?

Not RenderTexture, only I need Texture or Texture2D.

ReadPixel and Apply is to heavy for every single frame.

 

Hi ~!

Thank you for answer and it works ~!

I have one more question

I tested it from iPhone XS MAX and I use Portrait mode.

However when I get single frame image, it's resolution is W:1920, H:1440.

I think it's Orientation.Landscape mode image.

I wanna get Portrait mode image from Camera image.

Is it have any func to make a image like what I look on screen on Camera image API?

 

Thank you for answer.

I use the image for Vision Project and the project support only Portrait mode.

Vision Project works analysis the image and detect some target and show the what is it and tracking it on the image.

Now it works on ARKit Plugin( use OnRenderImage and RenderTexture to Texture2D.ReadPixel ). It's heavy cost >.<

I'll use ARFoundation, so I try to use CameraImage.

Do you have a plan to support camera image rotation or orientation output?

 

thank you for answer ~!

I solved it to use opencv.

I have other question.

== ARFoundation CameraImage Code ==
CameraImage image;
if (!ARSubsystemManager.cameraSubsystem.TryGetLatestImage(out image))
================================

== output ==
image.width : 1920
image.height : 1440

I tested on iPhoneX ( 1125, 2436 )

why image output size is like that?

I can see extra side image on output but I can't see on display.

Can I setup output image size?


I think, I can cut the image to use CameraImageConversionParams.inputRect.

Can you show how can I get a part of image from output image ?

 

Thank you for answer

I uploaded screenshot.

It's iPhoneX screenshot.

"Screen : 1125 - 2436" is device resolution.
"TexScreen : 1280 - 720" is output image resolution.

void Start()
{
ARSubsystemManager.cameraSubsystem.SetCurrentConfiguration(new CameraConfiguration(new Vector2Int(1280, 720), 60));
}

var conversionParams = new CameraImageConversionParams
{
// Get the entire image
inputRect = new RectInt(0, 0, image.width, image.height),

// Downsample by 2
outputDimensions = new Vector2Int(image.width, image.height),

// Choose RGBA format
outputFormat = TextureFormat.RGB24,

// Flip across the vertical axis (mirror image)
//transformation = CameraImageTransformation.MirrorY
};

I used this code in OnCameraFrameReceived.

I marked on image, what I asked "I can see extra side image on output but I can't see on display."

I tested iPhoneX, iPhoneXS Max, iPhone7 and iPhone8S.

iPhone7 no problem exactly same on display and image output.

However, iPhoneX and iPhoneSX Max have problem just I uploaded file.


thank you for help me.

 

Thank you for answer.

I didn't resize Camera Image like the aspect ratio on my screenshot because it's just test now.

I just want to see output result.

Maybe I didn't explain well, so I uploaded CameraImage output and device screen.

I marked red rect on CameraImage output. it's not see on device screen, but CameraImage have it.

When I make a screenshot from RenderTexture, it's just look like device screen.

So is it possible to hide some parts of camera image on screen for differences in the aspect ratio of different devices?

I use cameraSubsystem.Configurations() to get lowest supported resolution and use it, thank you for recommend.

Do you have a plan to support more low resolution like 640 x 360 ? for mobile?

 

Thank you for answer ~!

I have other question about the same project for iOS, AOS.

I made Native Plugin and I send Image data to Native every single frame.

iOS not bad for perfomance but AOS is to much slow.

Android Device : Galaxy S8

fixed width ratio image size :
iOS : 720P
AOS : 360P

iOS : 10-30 FPS
AOS : less than 5 FPS

I tried 720P but Android send a too slow error.
So now I try 360P.

Does AOS Native Call very slow than iOS Native Call ?

Do you have any tip for send data to native to fast ?

thank you.