[Solved] Reloading scene with agents

Hi, I am using trained agents in one of my scenes in game. When game is over player can reload the entire scene and start from beginning. The problem is that after reloading scene via SceneManager.LoadScene() my agents doesn't receive any information from brain. Method OnActionReceived is never invoked. That happens only after reloading scene. First time everything is ok. How to deal with it?

 

Hi, I found the solution. Before reloading the scene i call Academy.Instance.Dispose(). Academy class is singleton so it is initialized only once. After reloading the scene method EnvironmentStep() was not executed because of
AcademyFixedUpdateStepper GameObject was not in the scene (it is added to scene on initialization of Academy)

 

Sorry for the delayed response. We had another report of a similar issue; I think you diagnosis is correct that the AcademyFixedUpdateStepper GameObject gets removed when the scene is loaded. We have a fix that will go in the next release, but for now, the simplest workaround is probably to disable automatic stepping on the Academy (

Academy.Instance.AutomaticSteppingEnabled = false

) and call

Academy.Instance.EnvironmentStep()

somewhere else in your code.

 

hi @celion_unity
I am also facing same issue of scene reloading and getting error while training. I added

Academy.Instance.AutomaticSteppingEnabled = false

in my epsiode.begin method and then called

Academy.Instance.EnvironmentStep()

method at the end of my

OnActionReceived()

function i.e. it gets called on every iteration.
But when I press play button Unity hangs for a moment and then simply crashes. My unity version is 2019.3.12f1.
If I comment these two lines, training goes as desired till episode end and after that error comes out like these

 

Hi @mshajeehm,
Sorry, that's not enough information to go on. It looks like the error is in from your code - the NullReferenceException is coming from "snakeAgent.OnEpisodeBegin".

The original problem regarding the AcademyFixedUpdateStepper GameObject was fixed in the latest release (which came out last week), so you might want to try upgrading to that.

If you have more details on the error, please copy and paste the text, not a screenshot, and definitely not a PDF.

 

I am using version 1.0.0 *(released last week).

My snakeAgent.cs Episode begin method looks like this

Code (CSharp):

1. public override void OnEpisodeBegin()
2.     {
3.         //Academy.Instance.AutomaticSteppingEnabled = false;
4.         if (!snake.instance.isSnakeAlive())
5.         {
6.             SceneManager.LoadScene("scene-1"); //if snake is not alive then reload the scene
7.             Debug.Log("Loaded Scene Again");
8.         }
9.         Debug.Log("Loaded Episode Again");
10.        
11.         (width, height) = gameHandler.instance.levelGrid.getGridDimensions();
12.     }

The OnActionReceivedMethod is

Code (CSharp):

1.  
2.     public override void OnActionReceived(float[] vectorAction)
3.     {
4.         snake.instance.handleInputByAgent(vectorAction);
5.         snake.instance.handleGridMovement();
6.  
7.         if (snake.instance.foodEaten)
8.         {
9.             SetReward(1.0f);
10.         }
11.         if (!snake.instance.isSnakeAlive())
12.         {
13.             EndEpisode();
14.             Debug.Log("Episode Ended due to dead snake");
15.         }
16.         //Academy.Instance.EnvironmentStep();

The training goes well on first episode and error comes after episode end when new episode is to be started.
The complete AssetFiles can be accessed from this link.
From debug.log analysis, I discovered that after first episode ends, following actions take place

1. Scene is reloaded (so snake becomes alive) and all other classes must be reinitialized
2. rest goes fine in EpisodeBegin method and no error comes
3. Episode ends again. (I dont know why does episode end again here since snake is not dead)
4. Episode begins again
5. This loop goes for 4-5 iterations i.e. episode loaded, episode ended, episode loaded.....
6. After these iterations, the error comes up in episodeBegin method?

Why doesn't the error come on first episode begin? and why does episode end even if snake is not dead? Is it that load scene hasn't properly loaded scene?
My debug.log output is given below in 4 pictures numbered 1 to 4 since output is too long to fit in one picture (gameHandler is the first script to run when scene is loaded so that is why we see this in logs)