Showcase What the right way to infer onnx on an image in barracuda

Hello ,

I cant find a proper example on using Barracuda that will do a preprocessing on the image to use later for inference (yolo).I came across the following code but it doesn't seem to work. Im getting a shape mismatch but despite that error, is the below code the right way to infer an image using onnx.
void Start()
{
m_RuntimeModel = ModelLoader.Load(modelAsset);
var worker = WorkerFactory.CreateWorker(m_RuntimeModel, WorkerFactory.Device.GPU);
Texture2D t = (Texture2D)AssetDatabase.LoadAssetAtPath("Assets/download.jpg", typeof(Texture2D));

Tensor tensor = new Tensor(t,3);
ITensorData tensorData = tensor.data;

TensorShape shape = tensor.shape;


var downdata = tensorData.Download(shape);

Tensor input = new Tensor(1, 168, 299, 3, downdata);

var output = worker.Execute(input);
}

If my understanding is correct ,a tensor is created by using the texture and specifying the number of channels ,then it's data is accessed and downloaded which it seems like getting all the pixels data in one go .

Also I have another inquiry ,is it possivler to use yolo5 or yolo6 ? Im able to load properly in using
ModelLoader.Load ,does this mean it will work ?

Thanks
Ayad

 

Any help ?

 

Not sure if this will be helpful but after some research I was successful at loading a image as a tensor and retrieving results from a worker.

The onnx model I used was from the Keras cats vs. dogs image classification example



Basically my worker code looks like this:

Code (CSharp):

1. using System;
2. using System.Linq;
3. using Unity.Barracuda;
4. using UnityEngine;
5.  
6. public class BarracudaWorker : MonoBehaviour
7. {
8.     public Texture2D t;
9.  
10.     public NNModel modelAsset;
11.  
12.     void Start()
13.     {
14.         Model m_RuntimeModel = ModelLoader.Load(modelAsset);
15.  
16.         var worker = WorkerFactory.CreateWorker(WorkerFactory.Type.ComputePrecompiled, m_RuntimeModel);
17.  
18.         var input = new Tensor(1, 180, 180, 3);
19.  
20.         for (var y = 0; y < 180; y++)
21.         {
22.             for (var x = 0; x < 180; x++)
23.             {
24.                 var tx = x * t.width  / 180;
25.                 var ty = y * t.height / 180;
26.  
27.                 /*
28.                 input[0, 179 - y, x, 0] = t.GetPixel(tx, ty).r; //for values from 0 - 1
29.                 input[0, 179 - y, x, 1] = t.GetPixel(tx, ty).g;
30.                 input[0, 179 - y, x, 2] = t.GetPixel(tx, ty).b;
31.                 */
32.  
33.                 input[0, 179 - y, x, 0] = t.GetPixel(tx, ty).r * 255; //for values from 0 - 255
34.                 input[0, 179 - y, x, 1] = t.GetPixel(tx, ty).g * 255;
35.                 input[0, 179 - y, x, 2] = t.GetPixel(tx, ty).b * 255;
36.             }
37.         }
38.  
39.         worker.Execute(input);
40.  
41.         Tensor output = worker.PeekOutput("dense_1");
42.  
43.         float[] outputBuffer = output.ToReadOnlyArray();
44.  
45.         print(outputBuffer[0]); //print the result as a percentage
46.  
47.         output.Dispose();
48.     }
49. }
50.  

The nets were trained on values between 0 - 255 so I had to multiply each pixel by 255, but most nets are trained on values between 0 - 1

For reference here's the Unity MnistBarracuda example.