permanently deleting elements in List<> on inspector

Hey guys.
I have a spawn script with a custom editor button attached to it.
When i press the button it spawns a game object. Now what i want is when it is deleted out of play mode i want to remove it for the list<> as well.

So far i have
spawnLocations.RemoveAll(list_item => list_item == null);
but leaving playmode will put these back into the list.

Cant find much on permanently deleting the elements.
Thanks.