Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Bug Windows mouse Raw Input mouse handling broken in Rewired

Discussion in '2021.2 Beta' started by guavaman, Sep 22, 2021.

  1. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    A change made in this beta branch has broken Rewired's native mouse input handling in Windows. Raw Input mouse messages sent by Windows to Rewired's messaging window are now intermittently missed.

    When the user moves the mouse and clicks or releases a mouse button simultaneously, the button down/up events may be missed causing both button press misses and stuck forever-on buttons. I have logged this at the lowest level possible when the messages are delivered to Rewired's message window.

    I cannot explain this as Rewired consumes input directly from the Windows Raw Input API, registering with Windows for HID events for devices and does not rely on Unity for any of this information. This beta version of Unity definitely causes Rewired to miss mouse events.

    Tested in Unity 2021.2.0b12.
    This cannot be reproduced in 2021.1.21f1.
     
    Last edited: Sep 22, 2021
  2. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    OOF, that's not good :(.

    The only thing I can think of that could have caused this is this fix: https://issuetracker.unity3d.com/is...ing-mouse-with-high-polling-rate-in-play-mode

    We found that when using very high input polling rate mice (tested with 8000 Hz), the framerate would drop dramatically, to the point where processing mouse events was taking longer than receiving them. The bottleneck was between PeekMessageW() and GetRawInputData() Windows APIs - it would acquire a mutex inside the kernel every time you tried to retrieve a raw input message, and with 8000 events per second the contention was just too high. I could reproduce this issue even in an empty win32 app that did practically nothing (no Unity involved).

    The fix was to use the GetRawInputBuffer() API instead, which allows processing all queued raw input events at once without pumping them through the Windows message loop. Using that API removes these events from Windows event queue so they no longer arrive through the WndProc.

    That fix landed to Unity 2021.2.0b9. Could you try using 2021.2.0b8 and 2021.2.0b9 to see if it started happening in b9?

    How do you implement raw input? Do you spawn your own invisible window that you register for raw input notifications (like DirectInput)? Or are you piggybacking off Unity's WndProc?
     
    TJHeuvel-net, LeonhardP and Prodigga like this.
  3. Prodigga

    Prodigga

    Joined:
    Apr 13, 2011
    Posts:
    1,123
  4. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    Thanks for the reply and the helpful information. I also received many reports of the high refresh-rate mouse issue, bought an 8000 Hz mouse, discovered the issue happened without Rewired in the mix, and filed a bug report with Unity. It's very interesting to know the underlying cause is the Raw Input API. That means I would also need to change Rewired to use the buffered method to avoid this, which means forwarding Raw Input messages to Unity is no longer an option (see below).

    I have verified the issues does not occur in 2021.2.0b8 and does occur in 2021.2.0b9

    That is definitely the cause.

    1. Rewired creates a hidden message window.
    2. Rewired registers with Raw Input to receive mouse, keyboard, and HID controller events. This prevents Unity from receiving those events because only one Window may receive events per application.
    3. Rewired processes the mouse event, then forwards it to Unity's WncProc so Unity can use the event also so Rewired doesn't break Unity's mouse input. Keyboard and HID device events are not forwarded because Unity does not use them (in the legacy input system). (And the forwarding trick fails in the new input system because of threading.)
    4. Rewired processes the first message, then forwards it to Unity, at which point GetRawInputBuffer is called, consuming all the remaining Raw Input events for the mouse for that frame.

    Because Unity is consuming all the Raw Input messages from the queue, Rewired doesn't receive anything after the first event, and therefore misses many events.

    The only solution I can see would be to change Rewired to no longer forward the messages to Unity and just break Unity mouse support when using Rewired.
     
    Last edited: Sep 23, 2021
  5. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    Would it help if we exposed a way to forward the raw input events to Unity without going through SendMessage/PostMessage? That way you could consume all the events with GetRawInputBuffer yourself, and then forward them all to Unity without going through Windows message queue. I'm thinking of something like this:

    Code (csharp):
    1. namespace UnityEngine.Windows
    2. {
    3.     class Input
    4.     {
    5.         static void ForwardRawInput(NativeSlice<uint> rawInputHeaderIndices, NativeSlice<uint> rawInputDataIndices, NativeSlice<byte> rawInputData);
    6.     }
    7. }
    Then you could build the data on C++ side and send it to managed land in one call:

    Code (csharp):
    1.  
    2. typedef void (*DispatchRawInputToUnityFunc)(const void* rawInputHeaderIndicesPtr, const void* rawInputDataIndicesPtr, uint32_t rawInputEventCount, const void* rawInputData, uint32_t rawInputDataSize);
    3. static DispatchRawInputToUnityFunc DispatchRawInputToUnity;
    4. HWND g_RewiredHwnd;
    5.  
    6. template <bool DataIsFromGetRawInputBuffer>
    7. static void AddRawInputEvent(RAWINPUT* rawInput, std::vector<uint32_t>& rawInputHeaderIndices, std::vector<uint32_t>& rawInputDataIndices, std::vector<uint8_t>& rawInputData)
    8. {
    9.    auto oldSize = static_cast<uint32_t>(rawInputData.size());
    10.    rawInputHeaderIndices.push_back(oldSize);
    11.  
    12.    auto dataOffset = offsetof(RAWINPUT, data);
    13.    if (DataIsFromGetRawInputBuffer)
    14.    {
    15. #if defined(_M_IX86) || defined(_M_ARM)
    16.        static bool isWow64 = IsWow64();
    17.        if (isWow64)
    18.        {
    19.            // RAWINPUTHEADER is 16 bytes on 32-bit, but 24 bytes on 64-bit. Since this data is memcpy-ed from the kernel,
    20.            // we need to adjust the data pointer by 8 bytes if we're in a 32-bit process but 64-bit kernel.
    21.            // We only need to do this with data coming from GetRawInputBuffer as GetRawInputData fixes up the single
    22.            // event it returns.
    23.            dataOffset += 8;
    24.        }
    25. #endif
    26.    }
    27.  
    28.    rawInputDataIndices.push_back(oldSize + dataOffset);
    29.  
    30.    rawInputData.resize(oldSize + rawInput->header.dwSize);
    31.    memcpy(&rawInputData[oldSize], rawInput, rawInput->header.dwSize);
    32. }
    33.  
    34. static bool IsWow64()
    35. {
    36.    BOOL isWow64 = FALSE;
    37.    if (IsWow64Process(GetCurrentProcess(), &isWow64))
    38.        return isWow64 == TRUE;
    39.  
    40.    return false;
    41. }
    42.  
    43. void ProcessRawInputMessages(HRAWINPUT rawInputHandle)
    44. {
    45.    std::vector<uint32_t> rawInputHeaderIndices;
    46.    std::vector<uint32_t> rawInputDataIndices;
    47.    std::vector<uint8_t> rawInputData;
    48.  
    49.    uint8_t rawInputBuffer[8 * 1024];
    50.    uint32_t result = GetRawInputData(rawInputHandle, RID_INPUT, rawInputBuffer, sizeof(rawInputBuffer), sizeof(RAWINPUTHEADER));
    51.    if (result == (static_cast<uint32_t>(-1))
    52.    {
    53.        LogError(GetLastError());
    54.        return;
    55.    }
    56.  
    57.    AddRawInputEvent<false>(reinterpret_cast<RAWINPUT*>(rawInputBuffer), rawInputHeaderIndices, rawInputDataIndices, rawInputData);
    58.  
    59.    while (true)
    60.    {
    61.        auto rawInputCount = GetRawInputBuffer(rawInputBuffer, sizeof(rawInputBuffer), sizeof(RAWINPUTHEADER));
    62.        if (rawInputCount == 0)
    63.            break;
    64.  
    65.        if (rawInputCount == static_cast<uint32_t>(-1))
    66.        {
    67.            LogError(GetLastError());
    68.            break;
    69.        }
    70.      
    71.        rawInput = reinterpret_cast<RAWINPUT*>(rawInputBuffer);
    72.        for (uint32_t i = 0; i < rawInputCount; i++)
    73.        {
    74.            AddRawInputEvent<true>(rawInput, rawInputHeaderIndices, rawInputDataIndices, rawInputData);
    75.            rawInput = NEXTRAWINPUTBLOCK(rawInput);
    76.        }
    77.    }
    78.  
    79.    DispatchRawInputToUnity(rawInputHeaderIndices.data(), rawInputDataIndices.data(), static_cast<uint32_t>(rawInputDataIndices.size()), rawInputData.data(), static_cast<uint32_t>(rawInputData.size()));
    80. }
    81.  
    82. LRESULT CALLBACK RewiredMainWndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
    83. {
    84.    if (message == WM_INPUT)
    85.        ProcessRawInputMessages(reinterpret_cast<HRAWINPUT>(lParam));
    86. }
    87.  
    88. extern "C" __declspec(dllexport) void SetDispatchRawInputToUnityCallback(DispatchRawInputToUnityFunc callback)
    89. {
    90.    DispatchRawInputToUnity = callback;
    91. }
    92.  
    93. extern "C" __declspec(dllexport) void PumpInputMessages()
    94. {
    95.    MSG msg;
    96.    while (PeekMessage(&msg, g_RewiredHwnd, 0U, 0U, PM_REMOVE))
    97.    {
    98.        TranslateMessage(&msg);
    99.        DispatchMessage(&msg);
    100.    }
    101. }
    Then on C# side:

    Code (csharp):
    1. struct CustomTimeUpdate
    2. {
    3.    [DllImport("Rewired_Core.dll")]
    4.    static extern void PumpInputMessages();
    5.  
    6.    public static PlayerLoopSystem.UpdateFunction s_EngineTimeUpdate;
    7.  
    8.    public static void OnTimeUpdate()
    9.    {
    10.        s_EngineTimeUpdate?.Invoke();
    11.        PumpInputMessages();
    12.    }
    13. }
    14.  
    15. delegate void DispatchRawInputToUnityDelegate(IntPtr rawInputHeaderIndicesPtr, IntPtr rawInputDataIndicesPtr, int rawInputEventCount, IntPtr rawInputData, uint rawInputDataSize);
    16. static DispatchRawInputToUnityDelegate dispatchRawInputToUnityDelegate = DispatchRawInputToUnity;
    17.  
    18. [DllImport("Rewired_Core.dll")]
    19. static extern void SetDispatchRawInputToUnityCallback(DispatchRawInputToUnityDelegate d);
    20.  
    21. void Awake()
    22. {
    23.     SetDispatchRawInputToUnityCallback(dispatchRawInputToUnityDelegate);
    24.  
    25.    // Insert our PumpInputMessage() right after PlayerTimeUpdate to minimize input latency
    26.    var playerLoop = PlayerLoop.GetDefaultPlayerLoop();
    27.    ReplaceTimeUpdate(ref playerLoop, 0);
    28.    PlayerLoop.SetPlayerLoop(playerLoop);
    29. }
    30.  
    31. private void ReplaceTimeUpdate(ref PlayerLoopSystem currentSystem, int depth)
    32. {
    33.    if (currentSystem.type == typeof(Initialization.PlayerUpdateTime))
    34.    {
    35.        var functionPtr = Marshal.ReadIntPtr(currentSystem.updateFunction);
    36.        if (functionPtr != IntPtr.Zero)
    37.            CustomTimeUpdate.s_EngineTimeUpdate = Marshal.GetDelegateForFunctionPointer<PlayerLoopSystem.UpdateFunction>(functionPtr);
    38.  
    39.        currentSystem.type = typeof(CustomTimeUpdate);
    40.        currentSystem.updateFunction = IntPtr.Zero;
    41.        currentSystem.updateDelegate = CustomTimeUpdate.OnTimeUpdate;
    42.    }
    43.  
    44.    if (currentSystem.subSystemList != null)
    45.    {
    46.        int subSystemCount = currentSystem.subSystemList.Length;
    47.        for (int i = 0; i < subSystemCount; i++)
    48.        {
    49.            ReplaceTimeUpdate(ref currentSystem.subSystemList[i], depth + 1);
    50.        }
    51.    }
    52. }
    53.  
    54. [MonoPInvokeCallback]
    55. static void DispatchRawInputToUnity(IntPtr rawInputHeaderIndicesPtr, IntPtr rawInputDataIndicesPtr, int rawInputEventCount, IntPtr rawInputDataPtr, uint rawInputDataSize)
    56. {
    57.     var rawInputHeaderIndices = NativeSliceUnsafeUtility.ConvertExistingDataToNativeSlice<uint>(rawInputHeaderIndicesPtr, rawInputEventCount);
    58.     var rawInputDataIndices = NativeSliceUnsafeUtility.ConvertExistingDataToNativeSlice<uint>(rawInputDataIndicesPtr, rawInputEventCount);
    59.     var rawInputData = NativeSliceUnsafeUtility.ConvertExistingDataToNativeSlice<byte>(rawInputDataPtr, rawInputDataSize);
    60.     UnityEngine.Windows.Input.ForwardRawInput(rawInputHeaderIndices, rawInputDataIndices, rawInputData),
    61. }
    Apologies if there are bugs in that code - I didn't test it (or even compile it), it's meant to be just an example of what it'd look like.
     
  6. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    Thanks for the code example! There are a number of features in there I wasn't aware of like PlayerLoopSystem.

    As a test, I implemented GetRawInputBuffer in Rewired and it works as I expected it would. Rewired receives the Raw Input event before Unity, empties the buffer, then sends the event to Unity which results in the reverse problem of Unity's mouse input only receiving the first event and missing all the rest. Even though Unity's mouse input is now broken, Rewired's now works again. I did notice in my testing that Unity's legacy input system's mouse button events still work however, leading me to conclude Unity is using some other means for that information like WM_LBUTTONDOWN.

    Rewired does not actually have its own message pump at present. It creates a message window and registers to receive the Raw Input messages for devices to that HWND. Unity's message pump dispatches the messages, Rewired's WndProc receives them, then Rewired forwards the message to Unity by calling its WndProc manually using CallWindowProc.

    Not that it's important, but Rewired's Raw Input implementation is fully C# managed code using P/Invoke.

    If you think exposing a way to forward the Raw Input messages back to Unity is feasible, I would be open to using it. Otherwise, the user has two options:
    1. Use Rewired's native mouse handling knowing Unity's mouse handling will no longer work.
    2. Disable Rewired's native mouse handling and use UnityEngine.Input as the source of mouse input in Rewired. This is the only method that will allow both Rewired's and Unity's mouse input to function.
    Having the ability to forward Raw Input events to Unity could also have an additional side benefit. In my early testing of the new input system, I was unable to forward messages back to Unity successfully I believe because input is now handled a different thread (?). This now applies to keyboard as well in the new system. (Unsure about other HID devices.) Similarly, I was also unable to implement Raw Input on a separate thread in Rewired for this same reason as Raw Input messages can only be processed by the thread to which they were delivered, so no cross-thread forwarding to Unity is possible. If Unity had a thread-safe buffer which could receive these events, handling input on different threads between Rewired and Unity while allowing both to function would be possible. Alternately, having the ability to get a callback from the new input system's input thread would allow me to create a message window on that thread so I could process input and forward it back to Unity (would also need WndProc for the input thread's message window), although with the GetRawInputBuffer issue, this is kind of moot. Just thinking out loud.
     
    Last edited: Sep 27, 2021
  7. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    Old input system uses WM_MOUSE* and WM_*BUTTON* events for button presses and to determine absolute mouse position, and only uses raw input for measuring mouse delta (to make it not be accelerated, etc).

    Oh, that makes sense. That means you're already set up to be called at the right time in the player loop.

    I thought I remembered seeing a native DLL in there somewhere, so I assumed you did raw input in it :).

    Yup, I created this bug to track it: https://issuetracker.unity3d.com/product/unity/issues/guid/1368835/. It's on my top priority list to address next.

    Unity receives raw input from Windows on the main thread. I believe we only use dedicated thread for HID devices (not 100% sure on that, would have to check).
     
    guavaman likes this.
  8. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    That makes sense.

    Yes, that was added a few years ago because IL2CPP didn't support a specific IL code used by SharpDX's Direct Input implementation so it stopped working. I added the native DLL at that time because I could no longer use SharpDX for that. Direct Input support is optional.

    Thanks!

    As of my last testing of the new input system (quite a long time ago), HID devices were not using Raw Input because registering them to Rewired's window didn't stop them from working.
     
  9. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
  10. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    Yeah, that's what I figured. I had to implement this API specifically for Xbox One controllers a few years ago due to a bug in an early update of Windows 10 that caused Xbox One controllers to crash windows when unplugged if they had been registered for Raw Input events. Good to know this is still being used in the new input system and I don't have to worry about forwarding events for those. Thanks!
     
  11. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    Alright, the raw input forwarding API landed to 2021.2.0b16. Attached is the API documentation preview. Hopefully it allows you to use buffered raw input reads yourself and then forward them to Unity too :). Lastly, this should allow you to forward both mouse and keyboard events to Unity as it will also work with the new input system too.
     

    Attached Files:

    guavaman likes this.
  12. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    Awesome! I like the use of IntPtr in the new API. When 2021.2.0b16 is available I'll test it out. That's great news it will work with the new input system as well. Until now, I've always advised users disable it, but now it should be possible to have it running alongside Rewired on Windows.
     
  13. Necka_

    Necka_

    Joined:
    Jan 22, 2018
    Posts:
    488
    I'm one of those people who reported this problem within Rewired and was made aware of this thread. I didn't expect much but wow. Thank you so much to both of you, I don't get all the technical details but it seem that Rewired will get updated later on based on the fix and API made available by Unity

    Awesome :)
     
    guavaman likes this.
  14. ZhavShaw

    ZhavShaw

    Joined:
    Aug 12, 2013
    Posts:
    168
    Is this fixed in b16?
     
  15. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    I have not made an update to Rewired to change how Raw Input works in Rewired to use the new API, so at the present time, no, it's not fixed.
     
    ZhavShaw likes this.
  16. ZhavShaw

    ZhavShaw

    Joined:
    Aug 12, 2013
    Posts:
    168
    Ahh, I downgraded according to your tested version.
     
  17. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    I downloaded 0b16 and began implementing the Raw Input forwarding, but I'm seeing it was implemented using unsafe pointers. Because Rewired_Core.dll is a DLL and I only release one version of the DLL for each major release of Unity, I handle branching dependent on Unity preprocessor defines like this using a script included in the project that implements an interface which is defined in the DLL so it make function calls using the interface. The problem is, the script included in the project must make the call to UnityEngine.Windows.Input.ForwardRawInput which is unsafe. This requires Unity be set to compile allowing unsafe code. Based on my testing, this is not the default setting, which means when Rewired is installed, it will almost certainly have a compiler error on start.

    The only workaround I can think of would be to create another DLL that includes nothing but the function call to UnityEngine.Windows.Input.ForwardRawInput, then call into that from my script to avoid the compiler errors. However, also this introduces the problem that I will now have to maintain a separate release of Rewired for Unity 2021.2. I am already currently maintaining 7 branches for the 7 major versions of Unity Rewired supports. I have made it a point to avoid having to maintain separate projects for minor Unity releases.

    It would be helpful if there was another overload of the function that used IntPtr instead of native pointers.
     
  18. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    Thanks for trying it out.

    Do you have an .asmdef for your script? You could create one and allow unsafe code inside the .asmdef. That way you wouldn't need "allow unsafe code" to be turned on for the whole project.

    I can add this. Let me know if you run into any more issues with the API.

    Apologies about the unsafe pointers... I debated whether to use those or IntPtrs. I decided on the pointers because they had more type information so it would be harder to use them incorrectly. I didn't even think about unsafe code not being on by default.
     
  19. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    I don't currently use ASMDEF files in Rewired due to a lot of issues they cause with installing integration addon-packs into the Rewired folder that need to reference content in Assembly-CSharp.dll (classes or functions defined in the supported asset scripts and therefore cannot be referenced from a separate assembly).

    It looks to me like I could use an ASMDEF w/ unsafe enabled as a workaround by including just a single static class containing a wrapper function for ForwardRawInput that takes IntPtrs. Because the assembly is compiled by Unity, I can use the #if UNITY_2021_2_OR_NEWER to exclude this call for Unity 2021.1 without having to maintain separate 2021.1 and 2021.2 branches of Rewired. This wouldn't be possible using a normal DLL. Thanks for the suggestion.

    Makes sense. It would also be acceptable to pass uint arrays for the header and data indices and an IntPtr for the data.

    Okay, thanks!
     
    Last edited: Oct 17, 2021
  20. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    @Tautvydas-Zilys

    One question:

    Is it okay to call ForwardRawInput multiple times per frame? I see your implementation expands the buffers if too many messages come in. I would like to avoid that, so I'm forwarding the events whenever the buffer gets full. I've tested with an event buffer size of 1 just to force it to be called multiple times, and it seems to work, but I just wanted to make sure this is okay.
     
    Last edited: Oct 17, 2021
  21. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    You can call it as many times as you want. And from any thread - it's thread safe. Internally it just copies the events into a buffer which gets expanded if needed.
     
    guavaman likes this.
  22. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    I just wanted to confirm whether or not you plan to add a safe overload or not for the Input.ForwardRawInput function. If so, I'll wait on posting an update. If not, I'll publish the one with the ASMDEF wrapper library.
     
  23. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    You should go ahead and publish the one with ASMDEF wrapper library. I will add it, however, I don't have an ETA for it. It could take a couple weeks (running into some issues landing the initial changes. You'll be able to remove the asmdef once it's added, but I'd hate for Rewired to be broken on Windows just because of this little thing for the time being.
     
  24. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    Actually, I'll just wait on it because removing an ASMDEF from users projects after the fact will be annoying (using editor code to find and delete it). It's fine if Rewired is broken while 2021.2 is in beta. Unless you mean the full release will be out before the safe version will land, in which case I'll do the ASMDEF and deal with it.
     
  25. longroadhwy

    longroadhwy

    Joined:
    May 4, 2014
    Posts:
    1,551
    That sounds like a good idea to wait. Did Unity answer your question about time fame when the new function will be in?
     
  26. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    @Tautvydas-Zilys Could you tell me what the minimum byte size for Raw Input data Unity is using? I'm asking because I'm getting support issues because Unity is logging this when the data in event is too small arrives, with Rewired in the stack trace:

    Invalid raw input data size: 2 bytes
    Invalid raw input data size: 0 bytes

    This was reported using the iBuffalo SNES gamepad.

    If I know what minimum data size Unity is expecting, I can filter out these events. Either that or the logging should be disabled.
     
  27. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    It's sizeof(RAWINPUTHEADER). Events smaller than that don't make much sense, as you can't even figure out what kind of event that is. With an event size of 2 bytes, technically even reading header.dwSize is undefined since it's going out of bounds... I'm really surprised to see events of that size being sent from that device.
     
  28. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    That's what I expected. Thanks. So I don't see how this could ever be thrown. Rewired always writes the RAWINPUTHEADER + the data size as reported by the header. It would never be writing fewer bytes to the buffer than sizeof(RAWINPUTHEADER), unless somehow the data size reported is a negative value... RAWINPUTHEADER.dwSize is a DWORD (unsigned), but SharpDX, the library I originally based my Raw Input implementation on, marshals this as a signed int. I'm going to have to look deeper into this, but it's going to be a challenge to reproduce.
     
  29. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    If it helps at all, this is the code that does the validation:

    Code (csharp):
    1. .....(UInt32* rawInputHeaderIndices, UInt32* rawInputDataIndices, UInt32 indicesSize, UInt8* rawInputData, UInt32 rawInputDataSize)
    2. {
    3.     ....
    4.  
    5.     for (UInt32 i = 0; i < indicesSize; i++)
    6.     {
    7.         // These are errors in the API usage, but we need to check them in order not to crash
    8.         RawInputIndex index = { rawInputHeaderIndices[i], rawInputDataIndices[i] };
    9.         if (index.headerOffset + sizeof(RAWINPUTHEADER) > rawInputDataSize)
    10.         {
    11.             ErrorStringMsg("Out of bounds error in raw input header index. Total data size: %u bytes, header size: %u bytes, index value: %u", rawInputDataSize, sizeof(RAWINPUTHEADER), index.headerOffset);
    12.             continue;
    13.         }
    14.  
    15.         auto header = reinterpret_cast<RAWINPUTHEADER*>(rawInputData + index.headerOffset);
    16.         if (header->dwSize < sizeof(RAWINPUTHEADER))
    17.         {
    18.             ErrorStringMsg("Invalid raw input data size: %u bytes", header->dwSize);
    19.             continue;
    20.         }
    21.  
    22.         if (index.headerOffset + header->dwSize > rawInputDataSize)
    23.         {
    24.             ErrorStringMsg("Out of bounds error in raw input data index. Total data size: %u bytes, data size: %u, index value: %u", rawInputDataSize, header->dwSize, index.headerOffset);
    25.             continue;
    26.         }
    27.  
    28.         ....
     
  30. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    Thanks for the code!

    After testing all scenarios, it turns out, it's another problem with the whole "32-bit application on 64-bit Windows" data format issue. The NEXTRAWINPUTBLOCK implementation I used to get the next RAWINPUT structure in the array does not work correctly in a 32-bit build on 64-bit Windows. Reworking that, it works now without any errors logged by Unity.

    Thanks for your help.
     
  31. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    Yeah, it's tricky. The sample code in the documentation shows how to do it correctly. It's important to recognize that the data is incorrect only GetRawInputBuffer, but it's fine in GetRawInputData.
     
  32. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    I was looking at your code but noticed you didn't use NEXTRAWINPUTBLOCK which calls RAWINPUT_ALIGN. I did, but the problem is it works differently for 32 and 64 bit builds, however as you've noted, when working with the data from GetRawInputBuffer on a 64-bit OS regardless of the build type, it needs to be treated as 64-bit. It looks to me like this wouldn't even work correctly in C code using the actual macro defined in winuser.h... RAWINPUT_ALIGN is defined as:

    Code (csharp):
    1. #ifdef _WIN64
    2. #define RAWINPUT_ALIGN(x) (((x) + sizeof(QWORD) - 1) & ~(sizeof(QWORD) - 1))
    3. #else // _WIN64
    4. #define RAWINPUT_ALIGN(x) (((x) + sizeof(DWORD) - 1) & ~(sizeof(DWORD) - 1))
    5. #endif // _WIN64
    I was already correctly handling the header length discrepancy when processing the data and then sending the data to Unity, but traversing the array was failing, causing the 3rd and subsequent events to be aligned incorrectly. (1st event is fine because it comes from GetRawInputData, 2nd event is fine because it's in array[0], but 3rd+ was misaligned, but did not result in any noticeable issues or errors on testing with a mouse, Xbox One controller, or Dual Shock 4.)
     
    Last edited: Nov 16, 2021
  33. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    By the way, the "safe" API with IntPtr overloads landed to 2021.2.4f1. Sorry it took so long, I ran into unexpected issues trying to land it.
     
    LeonhardP likes this.
  34. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    Okay, thanks for implementing it and thanks for letting me know!
     
  35. angelonit

    angelonit

    Joined:
    Mar 5, 2013
    Posts:
    40
    Just checking in to say that this (moving the mouse at the same time as clicking makes the click down or up not register) is happening in my project consistently on 2021.2.15f1 as of today (2022-03-18) without any specially high-polling mouse (500hz)
     
  36. angelonit

    angelonit

    Joined:
    Mar 5, 2013
    Posts:
    40
    Just came back to report that I went to the Package Manager and hit "Import" on the already Downloaded Rewired asset and accepted the changes. I don't know how but it fixed itself (I had updated my unity version recently which probably broke it)
     
  37. guavaman

    guavaman

    Joined:
    Nov 20, 2009
    Posts:
    5,605
    That would mean you were using a version of Rewired prior to 1.1.41.0. Pressing Import overwrote the older version with whatever version you had downloaded.