Resolved Creating Dependency Between Discrete Actions in Unity ML-Agents

I need to train the ML Agents to create a dependency between discrete actions. Consider this example..

Code (CSharp):

1. int sum = da0 + 5;
2. int da1 = actions.DiscreteActions[1];
3. int da2 = actions.DiscreteActions[2];
4. int da3 = actions.DiscreteActions[3];
5.  
6. int t = da1 + da2 + da3;
7. if (t != sum) {
8.     Reward(-1);
9.     return;
10. }

Even after training with 1000 agents for more than a few hours, I am not able to train the agents to create this dependency.
I am also observing all these 4 values (sum, da1, da2, da3) along with other values in my game.

Yaml config

Code (JavaScript):

1. behaviors:
2.   AO:
3.     trainer_type: ppo
4.     hyperparameters:
5.       batch_size: 512
6.       buffer_size: 20480
7.       learning_rate: 0.0003
8.       beta: 0.005
9.       epsilon: 0.2
10.       lambd: 0.95
11.       num_epoch: 3
12.       learning_rate_schedule: linear
13.     network_settings:
14.       normalize: true
15.       hidden_units: 256
16.       num_layers: 3
17.       vis_encode_type: simple
18.     reward_signals:
19.       extrinsic:
20.         gamma: 0.995
21.         strength: 1.0
22.     keep_checkpoints: 5
23.     max_steps: 30000000
24.     time_horizon: 1000
25.     summary_freq: 15000

 

i made a quick test based on your example with 4 branches with 5 possible actions each and the agent took about 700k steps to solve it.

as a side note - training time in ML agents doesn't mean much, it's more useful to state how many steps you have trained for as real world time varies due to different PC specs and time scales etc.