Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Convert Texture2d to opencvsharp to Mat (and vice versa)

Discussion in 'Image Effects' started by Totoro205, Apr 3, 2018.

  1. Totoro205

    Totoro205

    Joined:
    Dec 12, 2017
    Posts:
    18
    I'm using opencvsharp with Unity to detect edges from a webcam, the process is like this:

    1. converting the current frame to OpenCVSharp Mat object

    2. processing the image (using the canny edge detector)

    3. converting the OpenCVSharp Mat of canny image to Texture2D
    I have tried to follow the conversion between OpenCVSharp image format (Mat) toUnity3D image holder (Texture2D) from here but the unity editor keeps crashing every time. This is my first project using Opencv (and opencvsharp) and any help would be much appreciated.

    For reference here is the code for the convertion that I used:

    TextureToMat
    Code (CSharp):
    1. void TextureToMat()
    2. {
    3.    // Color32 array : r, g, b, a
    4.    Color32[] c = _webcamTexture.GetPixels32();
    5.  
    6.    // Parallel for loop
    7.    // convert Color32 object to Vec3b object
    8.    // Vec3b is the representation of pixel for Mat
    9.    Parallel.For(0, imHeight, i =>
    10.    {
    11.        for (var j = 0; j < imWidth; j++)
    12.        {
    13.            var col = c[j + i * imWidth];
    14.            var vec3 = new Vec3b
    15.            {
    16.                Item0 = col.b,
    17.                Item1 = col.g,
    18.                Item2 = col.r
    19.            };
    20.            // set pixel to an array
    21.            videoSourceImageData[j + i * imWidth] = vec3;
    22.        }
    23.    });
    24.    // assign the Vec3b array to Mat
    25.    videoSourceImage.SetArray(0, 0, videoSourceImageData);
    26. }
    MatToTexture

    Code (CSharp):
    1. void MatToTexture()
    2. {
    3.    // cannyImageData is byte array, because canny image is grayscale
    4.    cannyImage.GetArray(0, 0, cannyImageData);
    5.    // create Color32 array that can be assigned to Texture2D directly
    6.    Color32[] c = new Color32[imHeight * imWidth];
    7.  
    8.    // parallel for loop
    9.    Parallel.For(0, imHeight, i =>
    10.    {
    11.        for (var j = 0; j < imWidth; j++)
    12.        {
    13.            byte vec = cannyImageData[j + i * imWidth];
    14.            var color32 = new Color32
    15.            {
    16.                r = vec,
    17.                g = vec,
    18.                b = vec,
    19.                a = 0
    20.            };
    21.            c[j + i * imWidth] = color32;
    22.        }
    23.    });
    24.  
    25.    processedTexture.SetPixels32(c);
    26.    // to update the texture, OpenGL manner
    27.    processedTexture.Apply();
    28. }
     
  2. muki0000

    muki0000

    Joined:
    Jun 13, 2020
    Posts:
    1
    Hi were you able to solve your problem? I am doing something similar. I would really appreciate it if you suggested anything.
     
  3. LT23Live

    LT23Live

    Joined:
    Jul 8, 2014
    Posts:
    83
    I could use this too. Open CV has a built in tool, but it crashes unity UWP builds