Scrollview can't scroll with buttons that have Event Triggers

Does anyone know a way around this? I'm using an event trigger to reset the visual state of the button after the cursor leaves it (not part of the default functionality, Unity leaves the button in highlighted state on pointerexit)

Even empty event triggers disable drag scrolling of the view port while on buttons.

 

Seems you can only add this by code:

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections;
4. using UnityEngine.EventSystems;
5.  
6. public class Onpointerexit : MonoBehaviour, IPointerExitHandler {
7.  
8.     // Use this for initialization
9.     void Start () {
10.    
11.     }
12.    
13.     // Update is called once per frame
14.     void Update () {
15.    
16.     }
17.  
18.     public void OnPointerExit(PointerEventData dt) {
19.         print("test");
20.     }
21. }

 

Yeah I think the Event Trigger basically adds all event handler interfaces so is catching the ones that the scroll view needs.