Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Access the RGB camera (or Locatable Camera) at 30fps on HoloLens

Discussion in 'VR' started by bnuernberger, Aug 2, 2016.

  1. bnuernberger

    bnuernberger

    Joined:
    Jul 29, 2016
    Posts:
    2
    Does anyone know if Unity is planning to wrap the RGB (or LocatableCamera) in C# code that is accessible at 30fps? I've tried using PhotoCapture.TakePhotoAsync and WebCamTexture, but both of them drop the framerate to below 30fps. The most I've gotten is 20fps with WebCamTexture with an 896x504 image. I know it's possible to get the RGB image at 30fps, because I've successfully ran https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CameraGetPreviewFrame on the HoloLens. However, in that code example, they're sending the image to a windows UI element called CaptureElement and I'm not sure if we can access the image through that at 30fps.

    I know Unity has a VideoCapture class but that only saves the video to a file and doesn't let me access the raw pixel information for each frame.

    Is Unity planning to write a wrapper to access the RGB video frames at 30fps?
     
  2. BrandonFogerty

    BrandonFogerty

    Joined:
    Jan 29, 2016
    Posts:
    83
    Hi @bnuernberger

    Anytime Mixed Reality Capture is used on the HoloLens, the fps will drop to 30 fps as the price of doing business. However I was unaware that it dropped below 30 fps. Do you have a repro project that I could evaluate? Either way I will look into this further. Thanks for bringing this to my attention!
     
  3. bnuernberger

    bnuernberger

    Joined:
    Jul 29, 2016
    Posts:
    2
    Hi @BrandonFogerty I'm not doing Mixed Reality Capture, but just regular RGB video capture. And I don't want to simply save the video frames to disk (e.g., with the VideoCapture class) but I want to access the raw pixels (e.g., to use it in a texture). Currently, I don't have a clean project to send you, but if I am able to get one, I'll send it to you ASAP.

    Do you know if the frame rate for non mixed reality capture (i.e., only RGB camera capture) should make the fps drop to 30? Or is this only for mixed reality capture?