PointerDown trigger LAG on screen edges issue

Hello everyone,

Having an issue with significant delays (0.5 -> 1 sec) for event trigger PointerDown for a UI Image (width 300, height 100) anchored on the right bottom of the screen.
The image should act as a boost button, while the user is keeping it pressed the boost is active, so I've added on PointerDown / Up event triggers that toggle a bool value which is later checked in every update call.

How it manifests :
- If I touch and hold on the UI Image than the PointerDown trigger is delayed (0.5 - > 1sec)
- If I touch and quickly release than the PointerDown followed by PointerUp are triggered upon finger release (PointerUp is never delayed). This is why a button click would feel normal for example.
- If there is at least one other touch on the screen (holding another finger anywhere else) than the PointerDown trigger for the UI Image is triggered instantly when I touch the image.

Where it manifests :
- Reproduced on Iphone 8+ and Iphone X (both running latest IOS 13.3.1). Tried disabling 3d touch (read some forums) but didn't help.
- No issues on Ipad Pro or older Iphone 6.
- Works fine in Unity Editor or other Android devices.

How to reproduce:
- Create a new project and add a UI Image (button like dimensions, width 300, height 100), achor it to the bottom right corner. Add PointerDown & PointerUp events and log something on the screen for each event.
- It also works by only adding a UI Button with same dimensions and positioning. Tap and hold on it and the button pressed color has the same delay before it appears.

Tools Used :
Unity 2019.3.5f1, Xcode 11.3.1

Any feedback or pointers are much appreciated.
Thanks