"Weird" Physics2D behaviour when using rigidbody.addforce

I am working on a simple block breaker game following a tutorial.

I decided to turn away from the tutorial a little bit and use unity 2d physics not only to move the ball around but also use addforce to move the paddle around, rather than making it kinematic and controlling it's position directly.

However the moment i use addforce in the paddle script, the ball decides to stick to the side walls of the scene, the side walls are empties with a static 2d box collider. Because it sticks to the side it will no longer bounce off but instead scroll up and down next to it.
The weirdest part is that this behaviour only occurs when the paddle is close to the wall (it does not have to be touching.)

The moment i comment out the script that is part of the paddle, the ball behaves normally.

Does anyone have any idea why having an addforce in the paddle script cause the ball to behave like this?
(the paddle behaves as expected so i am sure the linked rigidbody in the script is that of the paddle and not of the ball.