Search Unity

Control over Hololens camera exposure

Discussion in 'Windows Mixed Reality' started by rcyr_cimmi, Nov 15, 2016.

  1. rcyr_cimmi

    rcyr_cimmi

    Joined:
    May 17, 2013
    Posts:
    7
    We would like to combine our markers tracking system (computer vision) with Hololens. We did some preliminary tests for image and video quality using files from Device Portal. These are compressed therefore quality is affected but our main issue is camera exposure. Frames are (very) blurred when moving head such that tracking is very poor or plainly impossible. Is there a way to control Hololens camera properties such as exposure? I don't see any such control in Device Portal. Maybe through coding?
     
  2. Unity_Wesley

    Unity_Wesley

    Unity Technologies

    Joined:
    Sep 17, 2015
    Posts:
    557
    Hello,

    The device portal is a good way to grab video's however it does impact frame rate while taking a video. I would suggest exploring the Video and Photo Capture API's that you can use for scripting in your unity project. They might provide the answer you are looking for.

    Link to Documentation:
    https://docs.unity3d.com/550/Documentation/ScriptReference/VR.WSA.WebCam.VideoCapture.html

    Let us know if you have questions.

    Thank you,
    Wesley
    QA Team
     
  3. BrandonFogerty

    BrandonFogerty

    Unity Technologies

    Joined:
    Jan 29, 2016
    Posts:
    83
    Hi @rcyr_cimmi,

    We do not currently provide an API that will allow you to adjust the camera's exposure. We do however provide the following API "GetUnsafePointerToVideoDeviceController"
    https://docs.unity3d.com/550/Docume....GetUnsafePointerToVideoDeviceController.html

    This method will return a pointer to the native IVideoDeviceController object. If you write a custom plugin, you can use this pointer to adjust the exposure yourself. You can find more information about the IVideoDeviceController object here.
    https://msdn.microsoft.com/en-us/li...dows.media.devices.videodevicecontroller.aspx
     
  4. gsabiniok

    gsabiniok

    Joined:
    Sep 12, 2017
    Posts:
    1
    Hi BrandonFogerty,

    could you please help us with idea how to get the object from the pointer? What we try is:


    Code (CSharp):
    1.  
    2.         VideoDeviceController vdc = Marshal.PtrToStructure<VideoDeviceController>(m_PhotoCaptureObj.GetUnsafePointerToVideoDeviceController());
    3.         Debug.Log("Exposure" + vdc.Exposure);
    But we get MissingMethodException because VideoDeviceController is not support a default constructor.

    Any hint will be a help! :)
     
  5. rcyr_cimmi

    rcyr_cimmi

    Joined:
    May 17, 2013
    Posts:
    7
    I originally started this post it has been a while now. You can basically call anything on this page: VideoDeviceController class for UWP

    Code (CSharp):
    1. using System;
    2. using System.Runtime.InteropServices;
    3. using UnityEngine;
    4.  
    5. #if NETFX_CORE
    6. using Windows.Media.Devices;
    7. #endif
    8.  
    9. public sealed class VideoDeviceControllerWrapper : IDisposable
    10. {
    11.     private static readonly string PLATFORM_NOT_SUPPORTED_ERROR_MESSAGE = string.Format("[{0}] API is restricted to Universal Windows Platform", typeof(VideoDeviceControllerWrapper));
    12.  
    13. #if NETFX_CORE
    14.     private VideoDeviceController m_VideoDeviceController = null;
    15. #endif
    16.  
    17.     public VideoDeviceControllerWrapper(IntPtr unknown)
    18.     {
    19. #if !NETFX_CORE
    20.         throw new PlatformNotSupportedException(PLATFORM_NOT_SUPPORTED_ERROR_MESSAGE);
    21. #else
    22.         m_VideoDeviceController = (VideoDeviceController)Marshal.GetObjectForIUnknown(unknown);
    23. #endif
    24.     }
    25.  
    26.     ~VideoDeviceControllerWrapper()
    27.     {
    28.         Dispose();
    29.     }
    30.  
    31.     public void SetExposure(int exposure)
    32.     {
    33. #if !NETFX_CORE
    34.         throw new PlatformNotSupportedException(PLATFORM_NOT_SUPPORTED_ERROR_MESSAGE);
    35. #else
    36.         if (!m_VideoDeviceController.Exposure.TrySetAuto(false))
    37.         {
    38.             Debug.LogErrorFormat("[{0}] HoloLens locatable camera has failed to set auto exposure off", typeof(VideoDeviceControllerWrapper));
    39.         }
    40.        
    41.         if (!m_VideoDeviceController.Exposure.TrySetValue((double)exposure))
    42.         {
    43.             Debug.LogErrorFormat("[{0}] HoloLens locatable camera has failed to set exposure to {1} as requested", typeof(VideoDeviceControllerWrapper), exposure);
    44.         }
    45. #endif
    46.     }
    47.  
    48.     public void Dispose()
    49.     {
    50. #if NETFX_CORE
    51.         if (m_VideoDeviceController != null)
    52.         {
    53.             Marshal.ReleaseComObject(m_VideoDeviceController);
    54.             m_VideoDeviceController = null;
    55.         }
    56. #endif
    57.     }
    58. }
    And you use VideoDeviceControllerWrapper like this:

    Code (CSharp):
    1. IntPtr unknown = m_PhotoCapture.GetUnsafePointerToVideoDeviceController(); // pointer to IUnknown COM interface
    2. using (VideoDeviceControllerWrapper wrapper = new VideoDeviceControllerWrapper(unknown))
    3. {
    4.     int exposure; // TODO set proper exposure value
    5.     wrapper.SetExposure(exposure);
    6. }
     
  6. alexxjaz

    alexxjaz

    Joined:
    Sep 18, 2018
    Posts:
    29
    Hey, it has been a while since this answer was posted, but how do you get the IntPtr?
    I get "NullReferenceException: Object reference not set to an instance of an object" in the console when you do
    "IntPtr unknown = m_PhotoCapture.GetUnsafePointerToVideoDeviceController();"

    This is my class, the VideoDeviceControllerWrapper.cs is exactly the same:

    Code (CSharp):
    1. using System;
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using UnityEngine;
    5. using UnityEngine.XR.WSA.WebCam;
    6.  
    7. public class WindowsVideoDevice : MonoBehaviour
    8. {
    9.     PhotoCapture m_PhotoCapture;
    10.     int exposure = 1;
    11.  
    12.     // Start is called before the first frame update
    13.     void Start()
    14.     {
    15.         Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
    16.  
    17.         // Create a PhotoCapture object
    18.         PhotoCapture.CreateAsync(false, delegate (PhotoCapture captureObject) {
    19.             m_PhotoCapture = captureObject;
    20.             CameraParameters cameraParameters = new CameraParameters();
    21.             cameraParameters.hologramOpacity = 0.0f;
    22.             cameraParameters.cameraResolutionWidth = cameraResolution.width;
    23.             cameraParameters.cameraResolutionHeight = cameraResolution.height;
    24.             cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;
    25.  
    26.             // Activate the camera
    27.             m_PhotoCapture.StartPhotoModeAsync(cameraParameters, delegate (PhotoCapture.PhotoCaptureResult result) {
    28.             });
    29.         });
    30.         TestExposure();
    31.     }
    32.  
    33.     void TestExposure()
    34.     {
    35.         IntPtr unknown = m_PhotoCapture.GetUnsafePointerToVideoDeviceController(); // pointer to IUnknown COM interface
    36.         using (VideoDeviceControllerWrapper wrapper = new VideoDeviceControllerWrapper(unknown))
    37.         {
    38.             wrapper.SetExposure(exposure);
    39.         }
    40.     }
    41. }
     
  7. rcyr_cimmi

    rcyr_cimmi

    Joined:
    May 17, 2013
    Posts:
    7
    In the PhotoCapture#StartPhotoModeAsync's callback, is the result a success? Have you enabled the Webcam device capability?