Question Particle system noise causes particles to slowly drift away

I'm trying to add random jitter on otherwise immobile infinite lifetime particles, so that they oscillate back and forth in place. In order to do that I tried activating the Noise module, and it appears to do the trick at first, but if you leave it for a while you will notice that through those oscillations the particles slowly drift away from their initial position (which is undesirable).

Particle system:

• Duration 0.05
• Start Delay 0
• Start Lifetime Infinity
• Start size 0.02

Emission:

• Rate over Time 0
• Rate over Distance 0
• Busts 0, 200, Infinite, 0.010, 1.0

Shape:

• Shape box
• Emit from shell

Noise

• Strength 0.1
• Frequency 1
• Scroll Speed 4
• Damping off
• Octaves 1
• Quality Low
• Position Amount -2, 2 (random between two constants)

After 186.37 seconds I can clearly see the particles drifted away from their initial placement.

How can I get rid of this drift? I want particles moving back and forth returning to the same spot