Bug Side Channel only works when env.reset() is called twice in a row

I have found an issue when adding a callback to OnEnvironmentReset. Currently, I have a custom side channel that passes relevant strings between Unity and Python. I have defined a custom environment reset function in Unity called Reset. If I add the line

Code (CSharp):

1. Academy.Instance.OnEnvironmentReset += Reset

then the side channel breaks. In particular, there is no call to

Code (Python):

1. on_message_received

in Python. In fact, this happens even if the function Reset is simply an empty function.

At first, I tried adding

Code (CSharp):

1. SideChannelManager.RegisterSideChannel(sideChannel);

to the Reset function, then it seemed to work for a single episode. But then I noticed the environment was resetting itself twice at the beginning of the first episode and thus tried to register the side channel twice. Then I realized this is because, in Python, I was calling UnityEnvironment.reset immediately after defining the environment as well as at the beginning of each episode. Basically my python code looked like this:

Code (Python):

1.  
2. env = UnityEnvironment(side_channels=[sideChannel])
3. env.reset()
4.  
5. for episode in range(2):
6.     env.reset()
7.     # insert code here
8.     env.step()

I then removed the call to UnityEnvironment.reset immediately after defining the UnityEnvironment (line 3 in the code snippet). Then for some reason, the side channel stopped working again. So for some reason, resetting twice in a row does not break the side channel, but resetting once does... I am not sure what to do. Any suggestions?

 

I'm using Python 3.8.6 and ML Agents release 7.

Thank you!

 

Sorry, let me explain it more clearly:

• The side channel breaks if reset is not called twice in python and there is no custom Reset function defined (defined meaning I append the Reset function to Academy.Instance.OnEnvironmentReset).
  
• The side channel also breaks if reset is called twice in python and there is a custom Reset function defined.

Hopefully that is more clear. Thanks!

 

@ervteng_unity Any progress?

 

I tried to reproduce the issue with the latest release and this is what I did :
I added a script in C# :

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using Unity.MLAgents.SideChannels;
5. using Unity.MLAgents;
6.  
7. public class TestingSideChannels : MonoBehaviour
8. {
9.  
10.     FloatPropertiesChannel m_floatProps;
11.  
12.     // Start is called before the first frame update
13.     void Awake()
14.     {
15.         Academy.Instance.OnEnvironmentReset += OnReset; // Also tried without this line
16.         m_floatProps = new FloatPropertiesChannel();
17.         SideChannelManager.RegisterSideChannel(m_floatProps);
18.         m_floatProps.Set("awake method", 0);
19.     }
20.  
21.     void OnReset()
22.     {
23.         Debug.Log("Custom Reset Method called");
24.     }
25.  
26.     // Update is called once per frame
27.     void Update()
28.     {
29.         m_floatProps.Set("update method", 1);
30.     }
31. }
32.  

And I added it to a game object in the 3DBall scene.

On Python, I used this script

Code (CSharp):

1. from mlagents_envs.environment import UnityEnvironment
2. from mlagents_envs.side_channel.float_properties_channel import FloatPropertiesChannel
3.  
4.  
5.  
6. sideChannel = FloatPropertiesChannel()
7.  
8. env = UnityEnvironment(side_channels=[sideChannel])
9.  
10. env.reset() ########################### Tried with this line commented as well.
11. for episode in range(2):
12.     env.reset()
13.     # insert code here
14.     env.step()
15.  
16.  

I also added something at line 24 in on_message_received inside the file
"ml-agents-envs/mlagents_envs/side_channel/float_properties_channel.py"
to see what Python was receiving.

I used Release 10 to test.

My environment behaved as expected, On Python I saw

Code (CSharp):

1.  
2. received message :  awake method 0.0
3. received message :  update method 1.0
4. received message :  update method 1.0
5.  

And in the Unity Console, I saw "Custom Reset Method called" twice.

I could not reproduce the error you are describing. Can you modify the code I sent until the error appears and send the modified code back to me? Thank you.