Question Need help understanding ML model inference I/O for object detection/YOLO

Hi there. I have spent the last week deep diving object detection machine learning. I have an understanding of how to train models with YOLO (considering using v4 tiny) but my issue is with what comes after doing that successfully.

Once you have your trained model, my understanding is you convert it into ONNX and run inference on it in Barracuda. But in this week of learning, I have yet to find somewhere that explains how to do that or how to gain an understanding of what is required. My main questions are:

• How do you make your ONNX file compatible with Barracuda?/How do you gain the understanding of what is required in order to make it compatible?
  
  
• How do you figure out (and interpret) what input is needed for inference in Unity once you have your trained model?
  
  
• How do you carry out the inference?
  
  
• How do you interpret then use the output of the inference?

 

I have messed around with Baraccudda. So I'll attempt to answer your questions as no-one else has:

1. How to make ONNX compatible? Well, mostly you can't since Baraccuda hasn't implemented every function so you'll just have to try it and see if it works. Or maybe wait for Baracuda version 4.0. I haven't tried YOLO yet so I can't tell you if it works.
2. When you click on the onnx file in your assets folder, in the inspector it shows you what format the input should be in.
3. To do inference you pass an input float array (for example) representing your image and get back a float array representing the output.
4. Depending on the model it should tell you what the output means. For example the output might just be an array of probabilities, then you find the maximum value and that tells you what your input image is.

Hope that helps.

P.S. I Googled to see if there are any YOLO implementations in onnx and there might be here:
https://docs.openvino.ai/latest/omz_models_model_yolo_v3_onnx.html

The smallest YOLO onnx is about 500MB so you'll need a lot of graphics RAM to run it. And it probably won't be very fast.