Discussion Unity main camera feed to python

I am working on an autonomous system. I need to get live video feed from unity camera and send it to python for deep learning process and then get it into unity again. I find some codes taking snaoshot from camera and save it to hard disk. But couldn't send it. Can anyone help me out?
Thanks in advance.

 

Just because you are using Unity doesn't mean rest of the world stops existing. Your operating system with it's APIs still exist, .NET standard library still exists (with some limitations but it's mostly there). Microsoft documentation for them still exists. If you want to cobble some contraption of multiple technologies, there is a high chance you will have to look at the underlying layer bellow the technologies that you are trying to mix.

Windows has like a dozen different IPC mechanisms available https://learn.microsoft.com/en-us/windows/win32/ipc/interprocess-communications . Choose whatever fits your performance and latency requirements, then read .NET and python documentations for how to use it.

If you didn't have to afterwards communicate back to unity I would have suggested to ignore the Unity half completely. And use whatever interface your OS and python provides for scarping display output from other programs. That way it doesn't matter if the program generating images is written in Unity or any other game engine. But if you need bidirectional communication might as well use the same thing both ways. Although if the amount of information which needs to be sent back to unity is small like movement inputs, you might still choose the display scraping approach.

If by autonomous system you are thinking about robotics related stuff, you might want to look into Unity ROS integration. That will give you tools for exchanging higher level standardized messages not only for camera picture but also other properties which might be relevant for robotics applications. Standardized ROS messages makes it easier to switch between multiple different simulation environments (different simulation software) and real robots with almost no changes in your main software. Might not be worth getting ROS involved just for the sake sending images if you aren't already using or planning to use rest of ROS functionality.