Ball going partially through and sticking to flippers in a pinball game

Hello, I'm making a pinball style game, but I have problem with the ball going partially through the moving flippers and "sticking" to them because of this. Reducing the detection time didn't fully solve the problem.

I haven't used raycasts for the sake of simplicity, I have a ball which is always turning on its axis, and to my knowledge it would require a lot of raycasts in a lot of directions just to detect collisions. But, if someone can come up with a simple solution to my problem, I'd be more than grateful.

Thanks in advance for any answers!