You call destroy not on the prefab itself, but on the instance of the prefab in the scene. When you call Instantiate it returns a reference to the instance of the prefab in the scene, you destroy that.