2D hexagon tilemap made out of prefabs, mouse events

I have a hexagon 2d tilemap made out of prefab tiles, they have onMouseDown on them and I want to pan camera by dragging/touch and moving, but the prefabs are also getting clicked now.

For mouse I can use right button to pan the camera, but not on touch devices. I tried using only two finger touch to pan the camera, which is working but again the touch triggers the onMouseDown (if the first of the touches is on a prefab). If I set Input.simulateMouseWithTouches to false then I can't of course trigger the onMouseDown on the prefabs.

How do I solve this, that I can pan/scroll the camera around but at the same time not trigger the event on the prefab tiles? (But be able to actually touch/click on the prefabs, when not scrolling/panning the camera)

CameraControl

Code (CSharp):

1.     void Update()
2.     {
3.         if (Input.touchSupported && Application.platform != RuntimePlatform.WebGLPlayer)
4.         {
5.             this.HandleTouch();
6.         }
7.         else
8.         {
9.             this.HandleMouse();
10.         }
11.     }
12.  
13.     private void HandleTouch()
14.     {
15.         var touches = Input.touches.ToList();
16.  
17.         if (touches.Count == 2)
18.         {
19.             var center = (touches[1].position + touches[0].position) / 2;
20.  
21.             if (touches.Any(t => t.phase == TouchPhase.Began))
22.             {
23.                 this.touchStart = Camera.main.ScreenToWorldPoint(center);
24.             }
25.             else
26.             {
27.                 this.PanCamera(center);
28.             }
29.         }
30.     }
31.  
32.     private void HandleMouse()
33.     {
34.         if (Input.GetMouseButtonDown(1))
35.         {
36.             this.touchStart = Camera.main.ScreenToWorldPoint(Input.mousePosition);
37.         }
38.         else if (Input.GetMouseButton(1))
39.         {
40.             this.PanCamera(Input.mousePosition);
41.         }
42.     }
43.  
44.     private void PanCamera(Vector3 position)
45.     {
46.         var direction = touchStart - Camera.main.ScreenToWorldPoint(position);
47.         Camera.main.transform.position += direction;
48.     }

Prefabs tiles (one of them)

Code (CSharp):

1. public class PlainTile : MonoBehaviour, IGameTile
2. {
3.     public MapManager MapManager;
4.     public Sprite[] PossibleStartingSprites;
5.     public SpriteRenderer SpriteRenderer;
6.  
7.     public int UpgradePrice;
8.     public int Income { get; set; }
9.     public int Level { get; set; }
10.  
11.     void Start()
12.     {
13.         this.Level = 1;
14.         this.MapManager = GameObject.Find("MapManager").GetComponent<MapManager>();
15.         this.SpriteRenderer.sprite = this.PossibleStartingSprites[Random.Range(0, this.PossibleStartingSprites.Length)];
16.     }
17.  
18.     public void UpgradeTile()
19.     {
20.         this.MapManager.UpgradeTile(this);
21.     }
22.  
23.     public void OnMouseDown()
24.     {
25.         this.UpgradeTile();
26.     }
27. }