OnMouseOver/Down not working with rigidbody, but works without it

I have a simple code to test, which checks for mouse clicks on a 2d object. It Debug.Log-s if the object was clicked. The object has a box collider 2d and rigidbody 2d attached. The problem is that the code doesn't log anything if rigidbody exists. If i delete it in inspector, the code starts logging fine. What may be broken here?