Camera stuttering when camera smoothing is being applied and upscale render texture is enabled

So, I have this set up where the upscale render texture of the pixel perfect camera is enabled and I have a damping of 1 both in x and y axis in the virtual camera of the scene. The problem is that if the camera has any damping applied, it starts to stutter. Note that the stutter ONLY occurs when there's damping. Does anyone know what can be the cause of the problem?