Struggling with Shadergraph UV distortion shader

I'm trying to create a really simple UV distortion effect via a random noise texture. My hope is that in sampling the shader, any values < 0.5 send the pixel a proportional (from 0.5) amount to the left, and any values > 0.5 send the pixel a proportionate amount to the right. Or, in other words, (pixelColor - 0.5) would be the offset for each pixel. Let's call that 0.5 value the "Target".

What I'm finding in practice is that a Target 0.5 doesn't work well in most cases, and ends up causing most of the pixels to slide off a uniform distance in one direction. For example, here's an undistorted image:



And here's the distortion amount increased, with a Target of 0.5:




Note that most of the pixels went down and to the right, while getting distorted along the way. If, however, I change my Target from 0.5 to 0.125, it looks much better:



However, if I pick a different noise texture, then a Target of 0.125 no longer works, and I have to manually trial and error again to find a good value for Target. Maybe it's due to the average/min/max colors of the image?

Is there a completely different approach I'm missing for offsetting UV via noise?

Here's the very simple thing I'm doing to get the UV texture. (Note, I plan to separate out the U and V channels once I understand my my Target value seems off.)

 

why the distortion value are connected to Offset input? Normally it should e connected to UV input.
So basically it's DistortionUV = UV+DistortionValue

 

I think the issue is probably just that my distortion texture wasn't very balanced. The brightest pixel was not pure white. If I brighten the image, or increase the contrast, I get a more stable distortion when subtracting 0.5.

So, giving this more though, I think I just need to know the color range of my distortion texture. So I'm probably all set. Thanks for the suggestions.