Unity Ml_Agent : Object detection in real-time

Unity Ml_Agent : Object detection in real-time

Hello, I would like to ask you a question regarding the ML-Agent project that is underway these days. I am currently studying self-driving drones at graduate school. More specifically, we are conducting research to describe the ability of Unity to identify targets in real time as drones launch into target areas after takeoff.
Currently, I put Raycast on drone agent that identifies objects recognized by sensors through tags, but that is not what I wanted.
Real AI artificial intelligence is not information identified by contact with the Raycast sensor, but I think I should actually detect the target I want among the various objects seen on the Unity Agent's camera, so I'm trying to solve it, but I haven't found the right answer yet.

I'd like to know about the technology that identifies objects from the drone's first-person perspective in real time as they play the game, and the announcement is displayed around the bounding-box around the target.
To realize it, I have tried various methods for packages such as barracks, OpenCV, OpenVINO, and perception that are currently available in Unity, but I have not solved them.
Currently, the simulation implemented is using vector-observation via raycast (Picture 1), and my research goal is to target-detection using vision-sensor of agent camera.

What I am curious about is
1) whether the above explanation is actually possible?
2) If possible, I would like to know if there are any examples of such cases or git hubs.We sincerely hope to find Insight through your advice. Thank you.


Picture 1 is an example of a scene that I want to implement. This is a picture of me randomly putting bounding-box and announcement through Photoshop in the simulation scene I created.
Picture 2 shows an example of a bounding-box found on Unity's official blog. https://unity.com/kr/how-to/ai-machine-learning-explained

 

Looks like you want to run an image classifier prior to agent observations. This can be done using Barracuda within Unity
https://github.com/Unity-Technologi...ndex.md#multi-object-static-image-recognition