Strange Bug: Custom Network Client continues to receive Ping after game is shut down in Play mode?

I've been working on an experimental networking system that allows me to connect to a custom server. However, I've just encountered the strangest bug:

Shutting down the game does not shut down the socket, and it persists after.

The main way I'm able to test and tell this is that the server sends a Ping-pong keep alive, and the game responds back. After I kill the editor play, I can still see the console output growing.
It's almost acting like [ExecuteInEditMode] is active, but I'm 100% sure it does not exist in either my class or the game controller.

I have added in a number of checks to prevent this from happening, however I still feel like I should post this to see if anyone else has had it occur to them.