Resolved Play Button in Editor not doing anything

Hello, I am running the following configuration:
Unity 2021.3.11f1
Python 3.9.13
Version information:
ml-agents: 0.30.0,
ml-agents-envs: 0.30.0,
Communicator API: 1.5.0,
PyTorch: 1.7.1+cu110
[INFO] Listening on port 5004. Start training by pressing the Play button in the Unity Editor.
[INFO] Connected to Unity environment with package version 2.3.0-exp.3 and communication version 1.5.0

The above info lines show my ML Agent is successfully running in the python virtual environment. When I hit the play button in my Editor I get the following message:

Registered Communicator in Agent.
UnityEngine.Debug:Log (object)
Unity.MLAgents.Agent:Awake () (at Library/PackageCache/com.unity.ml-agents@2.3.0-exp.3/Runtime/Agent.cs:373)

I assume this means my agent has connected to the virtual environment. Last I used this component a few years ago, the agent's configuration would output in the virtual environment console. Now in this new project the console just listens, then times out. I am not sure where I went wrong, as I tried many configuration from Python 3.7 to 3.10, from ml-agents 0.26.0 - 0.30.0, each mlagents-learn command works in the virtual environment, but all of it is yielding the same result. I even tried making a simple scene with a barebones implementation of the Agent component and still get the same result. I put a breakpoint in the CollectObservations but it is never hit. So it looks like my virtual environment is correct and there is an initial connection because the agent registers itself. I see a beginepisode call but nothing after that even in a barebones TestAgent sitting in the scene.

I am pretty much going through these documents step by step, for each of the configurations I've tried:
About ML-Agents package (com.unity.ml-agents) | ML Agents | 2.2.1-exp.1 (unity3d.com)

Has anyone had this issue before? Let me know if you need more information. This is a pretty frustrating situation, seeing as how last time I used it years ago it seemed straightforward and worked with no issues.

 

Have you checked if anything else has port 5004 open on your computer?