Hey everyone, I'm trying to make a simple Car Driver Agent that follows a track but I'm having trouble getting it to work. The main issue seems to be that the Agent never tries all possible actions, it always ends up repeating the same actions over and over again so the reward never improves. I have two Discrete Action vectors, each with 3 possible values. Accelerate, DontMove, BrakeReverse TurnLeft, DontTurn, TurnRight From what I understand the way machine learning works is by first testing out all actions at random and then seeing which of those random Actions results in a good reward. So I would expect the agent to try all combinations of those Actions in order to find one that seemed to work. But what I'm getting is the Agent mainly just does a single action which is different every time I run the game. Here's the simple track scenario Here's one run where it tried Reversing to the Left 70 thousand times but didn't try Accelerating even once. In this one it just tried Accelerating forward which gets the agent to the first bend but it never tried Accelerate and TurnRight. I've tried using the default config file and I've tried using this one as well as changing tons of parameters. Code (CSharp): behaviors: CarDriver: trainer_type: ppo hyperparameters: batch_size: 256 buffer_size: 10240 learning_rate: 1.0e-5 beta: 5.0e-4 epsilon: 0.2 lambd: 0.99 num_epoch: 3 learning_rate_schedule: linear network_settings: normalize: false hidden_units: 512 num_layers: 3 reward_signals: extrinsic: gamma: 0.99 strength: 1.0 max_steps: 500000 time_horizon: 2048 summary_freq: 5000 I've set the reward based on distance traveled along the track. And the Observations that I'm using are - Current Position - Next Checkpoint Position - Raycast Distance to Wall At Angle 0 - Raycast Distance to Wall At Angle +45 - Raycast Distance to Wall At Angle -45 So my issue is I don't even know what is wrong, I've tried messing around with pretty much every single parameter and I cannot get the Agent to try doing all actions in order to find the ones that work. Thanks!