How to avoid stretched normals on modifying mesh?

I'm trying to create a terrain editor in-game and I'm using procedurally generated meshes as the terrain.
For 'editing' the terrain, I'm doing simple modifications to the mesh such as Pushing the vertices down/up.
But changing the vertices results in an artifact like this:

As you can see, there seems to be stretching of some sorts.
The mesh does not have any texture or normal map applied to it. I'm using a simple shader made in Shader Graph which uses vertex colors and the normals of the mesh are set through code using

mesh.normals

It seems like a UV problem, as I push the vertices of a face down the surrounding faces get stretched.

But I'm not using UVs at all.I'm using Vertex Colors and Vertex Normals.
What can I do to avoid stretching like this?

 

That's a shadow map artifact. Turn off shadows on your light and you will see