Discussion After 1 million steps, my ai just bops around. Am I training it wrong?

My ai primarily uses the camera sensor to see AgentViewable layer objects. The camera's target display is set to 2, and the main camera for the game is set to 1, I was unsure if this is why the ai is having such a hard time learning that it can pick up guns and shoot because of the camera display setting. I feel the ai should have been quicker to get to guns and pick them up, aim. Most of my ai just bop around the map until the time runs out. I give heavy rewards for being armed, shooting, and winning, but my ai is mostly content on draw which is a small reward, and plays rubbish. My other ai for a learning project only took a few thousand steps to learn how to get to the goal. This ai doesn't seem to be learning much besides that going out of bounds is a heavy penelty.
I have a observation stack of 5 for camera, and I have stacked vector for everything else. I run with self play enabled for teams.
I am on 1,100,000 steps. Relative of 21 hours of training.