Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Question I don't know how to approach image recognition using Barracuda.

Discussion in 'ML-Agents' started by chealin, Mar 5, 2024.

  1. chealin

    chealin

    Joined:
    Sep 10, 2017
    Posts:
    81
    hello

    I use Barracuda

    I want to create something that adds texture to the nails when they are shown on the webcam.
    I even created Onnx with the dataset.
    (After learning with data Yolo, change to onnx
    After learning with Yolo, it comes out well when verified using predict.py)


    However, when I input a texture using Unity, I am unable to make something appear in the shape of the nail.
    What should I do?

    [My Unity Code]
    Code (CSharp):
    1. using System;
    2. using System.Linq;
    3. using Unity.Barracuda;
    4. using UnityEngine;
    5. using System.Collections;
    6. using System.Collections.Generic;
    7.  
    8.  
    9.  
    10. public class MyScript : MonoBehaviour
    11. {
    12.  
    13.     public Texture2D inputTexture;
    14.  
    15.     public NNModel modelAsset;
    16.  
    17.     Dictionary<string, Tensor> InputDic = new Dictionary<string, Tensor>();
    18.  
    19.     //public RenderTexture renderTexture;
    20.  
    21.     public Texture2D outputT;
    22.  
    23.     public List<float> pointList;
    24.  
    25.     void Start()
    26.     {
    27.         Model model = ModelLoader.Load(modelAsset);
    28.         var worker = WorkerFactory.CreateWorker(WorkerFactory.Type.ComputePrecompiled, model);
    29.  
    30.         //Excute
    31.         InputDic = new Dictionary<string, Tensor>();
    32.         Tensor t_Image = new Tensor(inputTexture, 3);
    33.         InputDic.Add("images", t_Image);
    34.  
    35.  
    36.         worker.Execute(InputDic);
    37.  
    38.  
    39.         var outputTensor = worker.PeekOutput("output0");
    40.  
    41.  
    42.         // Get the data from the output tensor
    43.         float[] outputData = outputTensor.ToReadOnlyArray();
    44.  
    45.         // Create a new texture for the nail mask
    46.         Texture2D nailMaskTexture = new Texture2D(inputTexture.width, inputTexture.height);
    47.  
    48.         // Iterate over each pixel in the output tensor
    49.         for (int y = 0; y < inputTexture.height; y++)
    50.         {
    51.             for (int x = 0; x < inputTexture.width; x++)
    52.             {
    53.                 // Assuming the output tensor is 1-dimensional
    54.                 int index = y * inputTexture.width + x;
    55.  
    56.                 // Assuming output value represents the nail area (0 or 1)
    57.                 float nailValue = outputData[index];
    58.  
    59.                 // Set the corresponding pixel in the nail mask texture
    60.                 Color nailMaskColor = (nailValue > 0.5f) ? Color.white : Color.black;
    61.                 nailMaskTexture.SetPixel(x, y, nailMaskColor);
    62.             }
    63.         }
    64.         nailMaskTexture.Apply();
    65.         outputT = nailMaskTexture;
    66.     }
    67.  
    68. }
    69.  

    upload_2024-3-5_16-55-37.png

    upload_2024-3-5_17-7-8.png
    [My Onnx File]

    upload_2024-3-5_16-59-25.png upload_2024-3-5_16-59-25.png
     
    Last edited: Mar 5, 2024