Search Unity

Using the controller to interact with Unity UI

Discussion in 'Daydream' started by SubhanUrRehman, Dec 2, 2016.

  1. SubhanUrRehman

    SubhanUrRehman

    Joined:
    Jan 24, 2016
    Posts:
    6
    Hey,

    I created a project earlier for Cardboard and have been trying to port it to Daydream and use the controller for input instead of the "gaze" functionality of the cardboard.

    Previously, I had the GazeInputModule on the EventSystem and that allowed the reticle to be used as a sort of cursor to select objects. This worked on both 3D objects and the UI elements.

    I have tried to replicate this functionality to where the controller is aiming, and have it working with 3D objects, but its not working with the UI elements.

    Here is the code I have tested:

    RaycastHit hitInfo;
    Vector3 rayDirection = GvrController.Orientation * Vector3.forward;
    Debug.DrawRay(transform.position, rayDirection * 100000, Color.green);

    PointerEventData cursor = new PointerEventData(EventSystem.current);
    cursor.position = rayDirection;
    List<RaycastResult> objectsHit = new List<RaycastResult>();
    EventSystem.current.RaycastAll(cursor, objectsHit);

    This doesn't have any effect on the UI elements. Is there a method similar to the cardboard implementation that I had? That allowed me to have a single code base and set events for the elements in the Unity editor.

    If not, whats the best way to proceed?
     
  2. bradweiers

    bradweiers

    Unity Technologies

    Joined:
    Nov 3, 2015
    Posts:
    59
    The problem is most likely that you are still raycasting from the HMD UI camera. You will need a separate UI camera on the controller.

    So in Awake() in your custom input module you'll need to create a new UI camera and make sure that all of the canvases in the scene use this new camera as the UI camera.

    Have a look at this for an example:
    https://github.com/wacki/Unity-VRIn...nputModule/Scripts/LaserPointerInputModule.cs

    Let me know if you have any trouble.
     
  3. SubhanUrRehman

    SubhanUrRehman

    Joined:
    Jan 24, 2016
    Posts:
    6
    Thanks for the help. I have implemented the following code, and the hover events enter the objects but don't exit. Am I missing something?


    Code (CSharp):
    1.     public static DaydreamControllerInputModule instance { get { return _instance; } }
    2.     private static DaydreamControllerInputModule _instance = null;
    3.  
    4.     private Camera UICamera;
    5.  
    6.     protected override void Awake() {
    7.         base.Awake();
    8.         if (_instance != null) {
    9.             Debug.LogWarning("Trying to instantiate multiple.");
    10.             DestroyImmediate(this.gameObject);
    11.         }
    12.         _instance = this;
    13.     }
    14.  
    15.     protected override void Start() {
    16.         base.Start();
    17.  
    18.         // Create a new camera that will be used for raycasts
    19.         UICamera = new GameObject("UI Camera").AddComponent<Camera>();
    20.         UICamera.clearFlags = CameraClearFlags.Nothing;
    21.         UICamera.cullingMask = 0;
    22.         UICamera.fieldOfView = 5;
    23.         UICamera.nearClipPlane = 0.01f;
    24.  
    25.         // Find canvases in the scene and assign our custom UICamera to them
    26.         Canvas[] canvases = Resources.FindObjectsOfTypeAll<Canvas>();
    27.         foreach (Canvas canvas in canvases) {
    28.             canvas.worldCamera = UICamera;
    29.         }
    30.     }
    31.  
    32.     private PointerEventData eventData;
    33.     private GameObject currentPoint;
    34.  
    35.     public override void Process() {
    36.         UpdateCameraPosition();
    37.  
    38.         eventData = new PointerEventData(eventSystem);
    39.  
    40.         eventData.Reset();
    41.         eventData.delta = Vector2.zero;
    42.         eventData.position = new Vector2(UICamera.pixelWidth * 0.5f, UICamera.pixelHeight * 0.5f);
    43.      
    44.         eventSystem.RaycastAll(eventData, m_RaycastResultCache);
    45.         eventData.pointerCurrentRaycast = FindFirstRaycast(m_RaycastResultCache);
    46.         m_RaycastResultCache.Clear();
    47.      
    48.         // Send control enter and exit events
    49.         var hitControl = eventData.pointerCurrentRaycast.gameObject;
    50.         currentPoint = hitControl;
    51.  
    52.         // Handle enter and exit events on the GUI controlls that are hit
    53.         base.HandlePointerExitAndEnter(eventData, currentPoint);
    54.     }
    55.  
    56.     protected void UpdateCameraPosition() {
    57.         UICamera.transform.rotation = GvrController.Orientation;
    58.     }
     
  4. bradweiers

    bradweiers

    Unity Technologies

    Joined:
    Nov 3, 2015
    Posts:
    59
    That problem would probably have something to do with other parts of your implementation. Can't see how anything here would cause that.
     
  5. SubhanUrRehman

    SubhanUrRehman

    Joined:
    Jan 24, 2016
    Posts:
    6
    Every other method of input works fine (including the gaze functionality if I enable it). The only thing I can think of is that I'm missing something crucial with the implementation of this input module. Is there some sort of "guideline" or something to help make sure I haven't missed something obvious?
     
  6. bradweiers

    bradweiers

    Unity Technologies

    Joined:
    Nov 3, 2015
    Posts:
    59
    You can send me your input module and I can have a look.
     
  7. SubhanUrRehman

    SubhanUrRehman

    Joined:
    Jan 24, 2016
    Posts:
    6
    I've attached my best attempt. Thanks
     

    Attached Files:

  8. bradweiers

    bradweiers

    Unity Technologies

    Joined:
    Nov 3, 2015
    Posts:
    59
    OK found your problem. You weren't saving the event data between frames. In Process() look under the commented section "NEW CODE" to see the change I made to get it to work.

    Let me know how that goes for you.

    Code (CSharp):
    1. using UnityEngine;
    2. using UnityEngine.EventSystems;
    3.  
    4. public class DaydreamControllerInputModule : BaseInputModule
    5. {
    6.     public static DaydreamControllerInputModule instance { get { return _instance; } }
    7.     private static DaydreamControllerInputModule _instance = null;
    8.  
    9.     private Camera UICamera;
    10.     private PointerEventData eventData;
    11.  
    12.     protected override void Awake() {
    13.         base.Awake();
    14.  
    15.         if (_instance != null) {
    16.             Debug.LogWarning("Trying to instantiate multiple.");
    17.             DestroyImmediate(this.gameObject);
    18.         }
    19.  
    20.         _instance = this;
    21.     }
    22.  
    23.     protected override void Start() {
    24.         base.Start();
    25.  
    26.         // Create a new camera that will be used for raycasts
    27.         UICamera = new GameObject("UI Camera").AddComponent<Camera>();
    28.         UICamera.clearFlags = CameraClearFlags.Nothing;
    29.         UICamera.cullingMask = 0;
    30.         UICamera.fieldOfView = 5;
    31.         UICamera.nearClipPlane = 0.01f;
    32.  
    33.         // Find canvases in the scene and assign our custom UICamera to them
    34.         Canvas[] canvases = Resources.FindObjectsOfTypeAll<Canvas>();
    35.         foreach (Canvas canvas in canvases) {
    36.             canvas.worldCamera = UICamera;
    37.         }
    38.     }
    39.  
    40.     public override void Process()
    41.     {
    42.         UpdateCameraPosition();
    43.  
    44.         // NEW CODE: Only create a new instance of event data if there isn't an existing instance
    45.         if (eventData == null)
    46.             eventData = new PointerEventData(eventSystem);
    47.         else
    48.             eventData.Reset();
    49.  
    50.         eventData.delta = Vector2.zero;
    51.         eventData.position = new Vector2(UICamera.pixelWidth * 0.5f, UICamera.pixelHeight * 0.5f);
    52.  
    53.         eventSystem.RaycastAll(eventData, m_RaycastResultCache);
    54.         eventData.pointerCurrentRaycast = FindFirstRaycast(m_RaycastResultCache);
    55.         m_RaycastResultCache.Clear();
    56.  
    57.         // Send control enter and exit events to our controller
    58.         var hitControl = eventData.pointerCurrentRaycast.gameObject;
    59.         var currentPoint = hitControl;
    60.  
    61.         // Handle enter and exit events on the GUI controlls that are hit
    62.         base.HandlePointerExitAndEnter(eventData, currentPoint);
    63.  
    64.         if (base.eventSystem.currentSelectedGameObject != null)
    65.         {
    66.             ExecuteEvents.Execute(eventSystem.currentSelectedGameObject, GetBaseEventData(), ExecuteEvents.updateSelectedHandler);
    67.         }
    68.     }
    69.  
    70.     protected void UpdateCameraPosition()
    71.     {
    72. //        UICamera.transform.position = controller.transform.position;
    73.         UICamera.transform.rotation = GvrController.Orientation;
    74.     }
    75.    
    76.     private Vector3 GetControllerPosition()
    77.     {
    78.         return GvrController.Orientation * Vector3.forward;
    79.     }  
    80. }
     
  9. SubhanUrRehman

    SubhanUrRehman

    Joined:
    Jan 24, 2016
    Posts:
    6
    Oh thanks so much! It seems so obvious now. It works great, I'll implement the other functionality now :)
     
  10. UmairEm

    UmairEm

    Joined:
    Nov 7, 2015
    Posts:
    31
    @SubhanUrRehman Are you doing something different than provided demos?
    Here is how i adapted controller input for World space UI:
    1. Add GVRPointerInputModule on EventSystem object.
    2. Add GVRPointerGraphicsRaycster to the canvas object(s).
     
    fmielke likes this.
  11. SubhanUrRehman

    SubhanUrRehman

    Joined:
    Jan 24, 2016
    Posts:
    6
    As far as I know these were included in GVR13, but not in the earlier versions (which I was using when I make the scripts). So the desired functionality was the same but it was not bundled at the time. Now I've implemented a mixture of my script and the GVRPointer
     
  12. fmielke

    fmielke

    Joined:
    Oct 17, 2016
    Posts:
    35
    @UmairEm Okay, for buttons it works, but for e.g. sliders not. It seems that the controller does not know the exact position of the slider. No matter where I click the slider, the handle is always in the middle.
     
  13. UmairEm

    UmairEm

    Joined:
    Nov 7, 2015
    Posts:
    31
    I have created a thread on this issue already. Please keep an eye on that.
     
  14. ireth_86

    ireth_86

    Joined:
    May 18, 2016
    Posts:
    23
    Hi r3dthrawn,
    I'm using the same code that you linked (LaserPointerInputModule) in a HTC Vive app.
    But I'm trying to do something slightly different: I would like to get the position of the hit on the world canvas, since I'd like to be able to (for example) drag it around.
    This seems to be difficult, since the pointerCurrentRaycast.screenPosition always returns the (UICamera.pixelWidth * 0.5f, UICamera.pixelHeight * 0.5f) values, and worldPosition always returns Vector3.zero.

    Am I missing something obvious here? Maybe I need a different approach of raycasting?
    (for the sake of completion, I'm using Unity 5.4 and SteamVR 1.2)

    Thanks in advance!
     
unityunity