Search Unity

Updating thread from C++ uchar*

Discussion in 'Scripting' started by thunderwiring, Sep 21, 2021.

  1. thunderwiring

    thunderwiring

    Joined:
    Aug 28, 2021
    Posts:
    5
    In my Unity project (for Android) im reading a frame from the device camera, passing it to a C++ plugin (uses opencv) for processing and in another scene the processed image from the c++ plugin is rendered on a material texture to be shown on a sphere gameobject. i don't see that the texture is changing even though i verified all functions in the c# and c++ are called in the correct order.

    In the scene there is a sphere with a material that uses a skybox/cubemap shader

    My code is heavily based on this example by unity for reading texture pointer and updating it.:
    https://github.com/Unity-Technologi...aster/PluginSource/source/RenderingPlugin.cpp

    my project setup is this:

    C++ plugin with exposed functions:
    1. read a pointer of the texture that would be modified
    Code (CSharp):
    1. extern "C" void UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API
    2. SetTextureFromUnity(void *textureHandle, int w, int h) {
    3.     // update global variables:
    4.     g_TextureHandle = textureHandle;
    5.     g_TextureWidth = w;
    6.     g_TextureHeight = h;
    7. }
    2. read camera frame
    Code (CSharp):
    1. static void ReadTexturePixels() {
    2.     void *textureHandle = g_TextureHandle;
    3.     int width = g_TextureWidth;
    4.     int height = g_TextureHeight;
    5.     if (!textureHandle) {
    6.         LOGE("ModifyTexturePixels", "ModifyTexturePixels - textureHandle is null");
    7.         return;
    8.     }
    9.     std::thread processingThread(
    10.             [&]() {
    11. /** For now this call just saves the frame in a class member in cv::Mat variable to be retrieved later*/
    12.                 cameraReader.readImageFrame(
    13.                         (uchar*)textureHandle, 4 * height * width, 4, height, width);
    14.             }
    15.     );
    16.     processingThread.join();
    17. }
    3. apply image to the texture from step 1.
    Code (CSharp):
    1. extern "C" void UNITY_INTERFACE_API SetVrMapView() {
    2.     void *textureHandle = g_TextureHandle;
    3.     int width = g_TextureWidth;
    4.     int height = g_TextureHeight;
    5.     if (!textureHandle) {
    6.         LOGE("SetVrMapView", "SetVrMapView - textureHandle is null");
    7.         return;
    8.     }
    9.    uchar * textureDataPtr = cameraReader.getPanoramaImage();/* returns frame.data, where frame is cv::Mat*/
    10.     s_CurrentAPI->EndModifyTexture(textureHandle, width, height, textureDataPtr);
    11. }
    and EndModifyTexture:
    Code (CSharp):
    1. void RenderAPI_OpenGLCoreES::EndModifyTexture(
    2.         void* textureHandle, int textureWidth, int textureHeight,
    3.         void* dataPtr) {
    4.     GLuint gltex = (GLuint)(size_t)(textureHandle);
    5.     // Update texture data, and free the memory buffer
    6.     glBindTexture(GL_TEXTURE_2D, gltex);
    7.     glTexSubImage2D(
    8.             GL_TEXTURE_2D, 0, 0, 0,
    9.             textureWidth, textureHeight, GL_RGBA, GL_UNSIGNED_BYTE, dataPtr);
    10. }

    C# script:
    Code (CSharp):
    1.  
    2. public class VrViewer : MonoBehaviour
    3. {
    4.     Texture writableTexture;
    5.     [SerializeField] private Material screenImageFromPlugin;
    6.  
    7.     void Start()
    8.     {
    9.         Debug.Log("VrViewer  Start function");
    10.         CreateWritableTexture(
    11.             1080,
    12.             2340);
    13.         SetVrMapView(); // from C++
    14.     }
    15.  
    16.     private void CreateWritableTexture(int width, int height)
    17.     {
    18.         Debug.Log("CreateWritableTexture " + width + " : " + height);
    19.         writableTexture = new Texture2D(width, height, TextureFormat.RGBA32, false);
    20.         SetTextureFromUnity(writableTexture.GetNativeTexturePtr(), writableTexture.width, writableTexture.height); // c++ function
    21.     }
    22.  
    23. }