Feedback Implement a mouse drag composite !

I was looking for a 'mouse drag' action,

So ... it was bit of a pain but apparently I have something that works !

So we can all understand:

• Action Type should be Value ?
• Control Type should be Vector2 ?
• but what about Interactions, Processors, should they be left off ?

One of the problem that I have currently is Canceled is being emitted when I'm not moving anymore while mouse button is still being held.

Do Unity team has plans for this ?

Else, can someone at Unity help getting this proper ?

Thanks

Code (CSharp):

1. using System.Diagnostics.CodeAnalysis;
2. using JetBrains.Annotations;
3. using UnityEngine;
4. using UnityEngine.InputSystem;
5. using UnityEngine.InputSystem.Layouts;
6. #if UNITY_EDITOR
7. using UnityEditor;
8.  
9. #endif
10.  
11. namespace z
12. {
13. #if UNITY_EDITOR
14.     [InitializeOnLoad]
15. #endif
16.     public class MouseDragComposite : InputBindingComposite<Vector2>
17.     {
18.         static MouseDragComposite()
19.         {
20.             InputSystem.RegisterBindingComposite<MouseDragComposite>();
21.         }
22.  
23.         [RuntimeInitializeOnLoadMethod]
24.         [SuppressMessage("Code Quality", "IDE0051:Remove unused private members", Justification = "<Pending>")]
25.         private static void Init()
26.         {
27.         }
28.  
29.         public override Vector2 ReadValue(ref InputBindingCompositeContext context)
30.         {
31.             var b = context.ReadValueAsButton(Button);
32.             var x = context.ReadValue<float>(Axis1);
33.             var y = context.ReadValue<float>(Axis2);
34.             var v = new Vector2(x, y);
35.  
36.             return b && v.magnitude > 0.0f ? v : default;
37.         }
38.  
39.         public override float EvaluateMagnitude(ref InputBindingCompositeContext context)
40.         {
41.             return ReadValue(ref context).magnitude;
42.         }
43.  
44.         #region Fields
45.  
46.         [InputControl(layout = "Button")]
47.         [UsedImplicitly]
48.         public int Button;
49.  
50.         [InputControl(layout = "Axis")]
51.         [UsedImplicitly]
52.         public int Axis1;
53.  
54.         [InputControl(layout = "Axis")]
55.         [UsedImplicitly]
56.         public int Axis2;
57.  
58.         #endregion
59.     }
60. }

EDIT

I have been able to get proper interaction:

• started
• processed over and over
• canceled

Code (CSharp):

1. using System;
2. using UnityEngine;
3. using UnityEngine.InputSystem;
4. using UnityEngine.InputSystem.Controls;
5. #if UNITY_EDITOR
6. using UnityEditor;
7.  
8. #endif
9.  
10. namespace z
11. {
12.     /// <summary>
13.     ///     Mouse drag interaction.
14.     /// </summary>
15. #if UNITY_EDITOR
16.     [InitializeOnLoad]
17. #endif
18.     public class MouseDragInteraction : IInputInteraction
19.     {
20.         static MouseDragInteraction()
21.         {
22.             InputSystem.RegisterInteraction<MouseDragInteraction>();
23.         }
24.  
25.         public void Reset()
26.         {
27.         }
28.  
29.         public void Process(ref InputInteractionContext context)
30.         {
31.             if (context.timerHasExpired)
32.             {
33.                 context.Performed();
34.                 return;
35.             }
36.  
37.             var phase = context.phase;
38.  
39.             switch (phase)
40.             {
41.                 case InputActionPhase.Disabled:
42.                     break;
43.                 case InputActionPhase.Waiting:
44.                     if (context.ControlIsActuated())
45.                     {
46.                         context.Started();
47.                         context.SetTimeout(float.PositiveInfinity);
48.                     }
49.  
50.                     break;
51.                 case InputActionPhase.Started:
52.                     context.PerformedAndStayPerformed();
53.                     break;
54.                 case InputActionPhase.Performed:
55.                     if (context.ControlIsActuated())
56.                     {
57.                         context.PerformedAndStayPerformed();
58.                     }
59.                     else if (!((ButtonControl) context.action.controls[0]).isPressed)
60.                     {
61.                         context.Canceled();
62.                     }
63.  
64.                     break;
65.                 case InputActionPhase.Canceled:
66.                     break;
67.                 default:
68.                     throw new ArgumentOutOfRangeException(nameof(phase), phase, null);
69.             }
70.         }
71.  
72.         [RuntimeInitializeOnLoadMethod]
73.         private static void Init()
74.         {
75.         }
76.     }
77. }

So no more funny state when debugging the input.

 

I have the same issue, mouse drag is a pain to implement, there is no way to define that easily, and once you succeed using UPDATE function to simulate it like you did with the old system, then you have to decide how to distinguish drag vs mouse click.

I can see the beauty and evolution of the new system, but the basic functionality are missing.
Like mouse drag, distinction between a mouse drag and a click on the same button
Keyboard event that are not repeated in callback, so that force you to use update to have a constant drag with keyboard.
Touchpad how are considered as mouse and not as an independent device like touch
Cancel function that is not always call.

The new system show potential, but at least for what I need, I ended up doing everything in update using polling, and almost defining no action, even wondering why I should...

Sure that device connect/disconnect, multi player are nice, but if once connected you can't do basic stuff, what does it server

 

I've edited first post and added the second part of the code I just wrote, the interaction. It works as expected: now there's only single started and cancelled phases.

This is how it should be configured:

Action

Action Type = Pass Through
Control Type = Vector2​

Interactions

Mouse Drag​

Add Mouse Drag composite

Then use it, in my case I was looking for implementing a panning function so I fetched delta, so far it's working well !

But we still would like an official answer

 

I'm hoping something like this can be built in, too; Essentially, I want to be able to bind an axis that only fires updates for my Action whilst a modifier is being held down (along with start and end events).

(It shouldn't be too mouse-specific, since I'm actually hoping to use this with XR buttons and tracked position axes instead!)

 

I too think this is a core feature that should come with the package. I just wanted to rotate an object by horizontal mouse movement only during mouse drag. Previously i tracked all of it manually and worked out the phases from mouse click and delta myself, which is fine... but making this stuff easier is kinda the point of actions no?

 

Gesture support is indeed a missing piece. It is one of the highest priority items for after 1.0. What's there ATM is indeed ill-equipped to do this gracefully. There's ways to route compound pointer input into a higher level recognizes in the way that the Touch Samples do but the input system adds little there other than mapping input sources.

What this will look like exactly is too early to tell. My *thoughts* (it's really not more than that ATM) are that I'd like to see two things happen:

1. Interactions being extended to where the "recognizer" part splits off and is its own thing. Meaning that instead of an interaction just sitting as its own object *on* bindings, there's a recognizer that can feed interactions *in* from outside. Including interactions that have been recognized externally (e.g. by the OS).
2. A set of MonoBehaviours that wrap this nicely such that a reasonable set of "default" interactions can be directly correlated with GameObjects and made contextual on them.

 

What i was hoping for as I'm trying to learn this new system is:

1. configure 1-n conditions that when satisfied will trigger the event to be fired.
2. configure what data gets sent with the event.

So with my example earlier:

Mouse left pressed to start Player/Dragging action. (Started phase)

* Action includes mouse Position.​

While held Action fires continually (Active phase)

* Action includes mouse Delta.​

Mouse left released (Finished phase)

* Action includes mouse Position.​

I was able to get the phases sorted using the script provided earlier in this thread for the custom interaction.
But i couldn't figure out from the Actions Editor how to have a trigger condition on the action that is different than the data to be delivered. What i came up with was using a custom composite. Is there a way to do this is directly in the Editor?

Code (CSharp):

1. using Unity.Mathematics;
2. using UnityEditor;
3. using UnityEngine;
4. using UnityEngine.InputSystem;
5. using UnityEngine.InputSystem.Utilities;
6. using UnityEngine.InputSystem.Layouts;
7. using UnityEngine.Scripting;
8.  
9. public struct MouseDragCompositeResult
10. {
11.     public float2 Position;
12.     public float2 Delta;
13.  
14.     public override string ToString() => $"{{ Position: {Position}, Delta:{Delta} }}";
15. }
16.  
17. [Preserve]
18. #if UNITY_EDITOR
19. [InitializeOnLoad]
20. #endif
21. [DisplayStringFormat("{Modifier}+{Delta}+{Position}")]
22. public class MouseDragComposite : InputBindingComposite<MouseDragCompositeResult>
23. {
24.     [RuntimeInitializeOnLoadMethod]
25.     static void Init() { }
26.  
27.     static MouseDragComposite()
28.     {
29.         InputSystem.RegisterBindingComposite<MouseDragComposite>();
30.     }
31.  
32.     [InputControl(layout = "Button")]
33.     public int Modifier;
34.  
35.     [InputControl(layout = "Vector2")]
36.     public int Delta;
37.  
38.     [InputControl(layout = "Vector2")]
39.     public int Position;
40.  
41.     private Vector2MagnitudeComparer _comparer = new Vector2MagnitudeComparer();
42.  
43.     public override MouseDragCompositeResult ReadValue(ref InputBindingCompositeContext context)
44.     {
45.         if (context.ReadValueAsButton(Modifier))
46.         {
47.             var delta = context.ReadValue<Vector2, Vector2MagnitudeComparer>(Delta, _comparer);
48.             var position = context.ReadValue<Vector2, Vector2MagnitudeComparer>(Position, _comparer);
49.  
50.             return new MouseDragCompositeResult
51.             {
52.                 Delta = delta,
53.                 Position = position,
54.             };
55.         }
56.         return default;
57.     }
58.  
59.     public override float EvaluateMagnitude(ref InputBindingCompositeContext context)
60.     {
61.         var value = ReadValue(ref context);
62.         return math.length(value.Delta);
63.     }
64. }

 

How did you do it with the delta?

 

Did you add this to the script the new input system made or created as a different script then add to player?

 

Hi! Thanks for this solution!
It works pretty well, but I'm seeing 2 issues.
When using only the MouseDragComposition( without the interaction ), the action is still being executed sending a 0 Delta when mouse button is not being pressed.
If I add the MouseDragInteraction then the action is not executed if I not press the mouse button, but when I release it, the action is no executed with a 0 Delta, thus my local values remain with previous values, which are never 0.

 

Following the examples above I was able to make a functioning mouse drag input. It works fine when launched in a local scene from the editor but in a compiled build it complains that the binding composite has not been registered. Am I missing something?

InvalidOperationException: No binding composite with name 'MouseDrag' has been registered
at UnityEngine.InputSystem.InputBindingResolver.InstantiateBindingComposite (System.String nameAndParameters) [0x00038] in <7cd406f3e9c44161a27643afba86ee8e>:0
at UnityEngine.InputSystem.InputBindingResolver.AddActionMap (UnityEngine.InputSystem.InputActionMap map) [0x00336] in <7cd406f3e9c44161a27643afba86ee8e>:0
UnityEngine.DebugLogHandler:Internal_LogException(Exception, Object)
UnityEngine.DebugLogHandler:LogException(Exception, Object)
UnityEngine.Logger:LogException(Exception, Object)
UnityEngine.Debug:LogException(Exception)
UnityEngine.InputSystem.InputBindingResolver:AddActionMap(InputActionMap)
UnityEngine.InputSystem.InputActionMap:ResolveBindings()
UnityEngine.InputSystem.InputActionMap:ResolveBindingsIfNecessary()
UnityEngine.InputSystem.InputActionMap:Enable()
UnityEngine.InputSystem.PlayerInput:set_currentActionMap(InputActionMap)
UnityEngine.InputSystem.PlayerInput:SwitchCurrentActionMap(String)
UnityEngine.InputSystem.PlayerInput:ActivateInput()
UnityEngine.InputSystem.PlayerInput:OnEnable()

 

So I figured this out and just wanted to share.

By switching all of the use cases of

Code (CSharp):

1. [RuntimeInitializeOnLoadMethod]

to

Code (CSharp):

1. [RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.BeforeSceneLoad)]

resolved the problem. Might be something specific to my setup that required this.

 

I'm a tad confused, what is the purpose of the "MouseDragInteraction" Class by aybe?

Isn't drag only useful for UI (such as inventory screen), so since the new Input system doesn't support the IDropHandler/IBeginDraghandler etc. events.

wouldn't you need to use your UI canva's graphic raycaster and simply get the current mouse position from the new input system and then use the objects hit to do the drag/drop interaction in code?

Or am I misunderstanding something?

 

When setting up like this, I get all events firing repeatedly even when not holding down the mouse. Is this not correctly setup?



Edit:
Ah, you needed both the composite and the interaction! Thank you for sharing the script

 

Works like a charm, thanks so much. This absolutely needs to be part of the input system by default.

 

Extremely valuable post! Any news from Unity on when this functionality will be part of the InputSystem by default ? @Rene-Damm

@aybe 's solution works like a charm indeed, if all the properties are set correctly. Here's the setup that works for me:

1.Create the Action using the appropriate Action Type and Control Type, and add a Mouse Drag Interaction (should appear in the list if you have added @aybe 's second script in your project)


2. Create a Moude Drag Composite for that Action (should appear in the list if you have added @aybe 's first script in your project)


3. Add the Mouse Drag Interaction to the Composite as well (didn't work without this in my case)


4. Set the button of your choice (in my case Left Button) as the Binding for Button


5. Set the X & Y axes of your choice (either

<Mouse>/position/x

&

<Mouse>/position/y

or

<Mouse>/delta/x

&

<Mouse>/delta/y

Finally you should be able to read the Vector2 axis whenever a drag is happening, using the method of your choice. Here I am using a callback subscribed to the Drag event of the Player Input Component:

Code (CSharp):

1.  public void Drag(InputAction.CallbackContext ctx)
2.     {
3.         Debug.Log(ctx.ReadValue<Vector2>());
4.     }

 

Hey @aybe & @GeorgeAdamon thanks a bunch for this gentlemen, it really shed some light onto my poor InputSystem internals understanding!
@aybe's custom composite and processor work grear except one thing - i am having an issue where the RadValue<Vector2> only returns whole positive numbers (still 2 values, thats correct), no floats and no negative values at all.
It happens when i use both pointer x/y position and delta.
I have used @aybe's code as is, without any modifications and I believe I am folowing the setup you suggested.
Did you guys also experience this?

EDIT
It turned out it was because I had the action defined for general pointer to handle mouse and touch input simultaneously (button bound to ponter press, axes bound to pointer xy delta). When I sseparated the action into two, one for mouse and one for touch, it started to behave.
Slick! You guys rock Merry Xmass and happy new year!

 

@aybe and @GeorgeAdamon, thank you so much for the great explanation! And it works great However I'd like to tweak what you did to allow me to get the mouse drag start position (so I can get the direction from start to current pos and return that value), but I can't seem to find how to do that.

Any clue ? Binding composite being stateless isn't helping...

 

@tarahugger this is great, any idea how you'd adjust it to prevent events from firing entirely if `delta` is zero? Is this even something possible?

 

i cant get this to work with other controls bound to the mouse capture has anyone got this going with multiple other control options.

 

This works great. Thanks. One issue i have is that the i can only get it to trigger once the mouse starts moving. I would like the context.started to be available once the button is pressed regardless if i move the mouse or not. Any ideas?

Code (CSharp):

1.         if (context.started)
2.         {
3.  
4.             Debug.Log("Test: Start");
5.         }
6.         if (context.performed)
7.         {
8.  
9.             Debug.Log("Test: Performing");
10.         }
11.         if (context.canceled)
12.         {
13.  
14.             Debug.Log("Test: End");
15.         }

 

You can assign event like this. It works for me!

Code (CSharp):

1.         inputControls.Player.Drag.performed += MouseDrag;
2.         inputControls.Player.Drag.canceled += MouseDragExit;
3.  
4.     public void MouseDragExit(InputAction.CallbackContext ctx)
5.     {
6.         Debug.Log("Drag canceled");
7.     }
8.