[SOLVED] Move 2d object without setting the velocity directly

You won't stop collisions from working if you manipulate velocity directly. You can use Rigidbody2D.AddForce and set the ForceMode2D to Impulse which modifies the velocity instantly (ignoring mass). There's no special magic here though; it's simply an addition i.e. it just adds your XY force to the velocity so it is still modifying the velocity instantly. It's no different than you doing body.velocity = body.velocity + force. What you should avoid is simply overwriting the velocity because, as you hinted, doing so overwrites whatever force the physics system may apply during a collision.

That said, if you're using continuous collision detection, you won't ever be overlapping another collider meaning that you're not stopping the physics system from trying to solve the overlap when you overwrite the velocity. If continuous is used, you can still modify the velocity directly. For instance, you're contacting a wall and you set the velocity to move the object into the wall. When the solver runs, it won't simply move the object into the wall; the collision will stop that and the velocity will be updated appropriately. If however you were overlapped on the wall because you are using discrete collision detection then the physics system will apply a velocity to move you away from the wall but if you overwrite that velocity to move into the wall then you'll either keep it overlapped or maybe pass through the wall; certainly the physics system will fight you all the way possibly producing a jittering object whilst its overlapped.

Also, when you say 'drag effect', you're simply referring to the fact that you don't want to decelerate or accelerate but if you want to change direction instantly then you'll need to modify the velocity directly (or use an impulse force).