How can I preprocess mlagents camera image or use different source for the camera?

Hi, I would like to perform GAUGAN on segmentation image in unity.
I was hoping to take segmentation image from unity, then perform a GAN method in it.
I can get image segmentation using https://bitbucket.org/Unity-Technologies/ml-imagesynthesis/src/master/

How can I use use this image as input for mlagents model?

 

hi i asked many similar questions here with no results, but I believe u can explore ISensor. I made mine work by editing the source code on python's receiving end

 

hey, thanks for the reply, I could somehow get the image from camera.
but I am struggling with camera interpolation.
I made a segmentation map from camera image, but since it is resized to 84x84, all of the segmentation got jumbled. Do you know how to prevent this?

 

Can I ask how you did that? I'm looking to use a live camera feed from my local network.

 

May I know how did you obtain the custom visual observation in python? Thank you! @TiranianHoward @MrOCW