Resolved Texture compression makes certain textures way too shiny

I'm building for android and I'm using ASTC texture compression, but some textures get this weird shine

It's even worse from far away

I tried removing the normal map all together and it didn't fix it.
Messing with smoothness and the metallic didn't help either.
What's causing this?

 

So the shininess persists even if the material only has the color/albedo texture set?

I remember having problems with this before -- but I cannot remember if it was due to a busted normal map or due to a different map...

 

Seems like the problem isn't with the material and instead with the ground (which is a terrain FYI) because without the material it's still shiny.

 

Changing the inspector to debug mode by clicking the 3 dots in the top right and then changing the smoothness value of the material to a really high number fixed the problem.

 

I LOVE YOU SO MUCH THANK YOU
In my case however I couldn't find the smoothness value setting, but changing the Material Template from "Default-Terrain-Standard" to "Default-Terrain-Diffuse" fixed it for me