Bug Ugly shadow banding / stripes because of scale (?)

I noticed some strange stripes on a texture which should otherwise be plain, see red arrows. They seem to appear from the plain wall with the blue arrow:



After some analyzation I figured the problem was with the shadows. And probably because my entire scene is a pinball table with an e. g. 2 meters x 1 meters size. Is there a way to avoid this shadow problem other than resizing the scene to something that's not the real measure?

I set the shadow values to something more aggressive and you can clearly see the problem while i increase the shadow distance: