Avoid gameObject bugging between two colliders

Hello,

It's about a 2d game.
I have a matrix of 20x20 made of prefabs. Each prefab has a box collider.
I also have a gameObject, which is dragged by the mouse. So it follows the mouse directly. This is made by Rigidbody2D.velocity.
The problem is, when I move the mouse into my matrix, the gameObject collides with the blocks, but if I move the mouse a little bit, it gets dragged into the blocks. I think probably between two of the blocks.

So how can I avoid this? How can I make, that the gameObject (which has a PolygonCollider attached to) does absolutely not get dragged between the blocks? These blocks are exactly next to each other, there is no space between.