Search Unity

Bug Low performance when using WebCam

Discussion in 'Web' started by De-Panther, Oct 15, 2022.

  1. De-Panther

    De-Panther

    Joined:
    Dec 27, 2009
    Posts:
    589
    On a laptop with RTX 2070 Max Q, The process of grabbing the webcam image takes 10ms.
    _JS_WebCamVideo_GrabFrame
    upload_2022-10-16_1-17-22.png

    I couldn't find a way to set WebCamTexture readable to false.

    Code (JavaScript):
    1.   function _JS_WebCamVideo_GrabFrame(deviceId, buffer, destWidth, destHeight) {
    2.     if (!MediaDevices[deviceId].video) {
    3.       console.error("WebCam not initialized.");
    4.       return
    5.     }
    6.     var context = webcam.canvas.getContext("2d");
    7.     if (context) {
    8.       canvas.width = destWidth;
    9.       canvas.height = destHeight;
    10.       context.drawImage(MediaDevices[deviceId].video, 0, 0, MediaDevices[deviceId].video.videoWidth, MediaDevices[deviceId].video.videoHeight, 0, 0, destWidth, destHeight);
    11.       var imageData = context.getImageData(0, 0, destWidth, destHeight);
    12.       writeArrayToMemory(imageData.data, buffer)
    13.     } else {
    14.       console.log("2d Context is null")
    15.     }
    16.   }
    Looking at the code, I think that if the WebCamTexture readable could be set to false, using an external canvas 2D, getting image data, and writing it to memory, could have been redundant, as drawing the image on a texture on the main context(similar to how VideoPlayer.targetTexture is implemented in WebGL) would be much faster.
     
  2. CodeSmile

    CodeSmile

    Joined:
    Apr 10, 2014
    Posts:
    6,005
    Is it possible to reuse the context? This may or may not carry a significant overhead.

    Also this seems to be polling the camera. Does it support pushing (events)? Since most webcams only deliver 30 fps you may be getting images twice when webgl runs at 60 fps.
     
  3. De-Panther

    De-Panther

    Joined:
    Dec 27, 2009
    Posts:
    589
    The performance tab show that getcontext costs almost nothing (I guess the browser cache it).

    I guess they can optimize it by polling only once in x frames, based on the camera refresh rate.