Search Unity

Question How to make agent remember to take action a before action b?

Discussion in 'ML-Agents' started by phantom9009, Dec 7, 2023.

  1. phantom9009


    Dec 2, 2023
    I am using unity ml agents, I have an environment where I am teaching agent to first reach for money before reaching for health. I am using negative reward if agent goes to health before getting money. And postive when agent performs as expected. I am using curiosity module. 5 hidden layers. Not using memory. Continuous space. But the agent is not learning to do as expected. It just collects money and chills out, doesn't care about the negative reward. These are exact details of what i am doing:

    I will explain what I have tried. What I want to achieve is: the agent to maintain its health about 500 (max1000) , so when his health is less than 500 it begins to recv a negative reward of -0.02*(500-health), initial health given is 1000 which reduces by 0.1 every 20 times per frame.

    Money is given 100 initially (1 money= 1 health) , I want the agent to get money first and then go to recv health. Everytime agent recvs money a reward of 5 is given. I am keeping track of agent's action, if the last action it did was to collect money and then later on it goes for health it will recv a reward of 500 (quite high but required) but when it goes to health without money a penalty of -10 is given.

    After all this, agent is just going to health with money only 10 times in 5lakh steps.

    I tried to use curiosity, and removed all the rewards except 500 one. It performed the same.

    I am using continuous space, movement in x and z direction (output of NN). Input of NN is agents position, velocity, health, money and last action.

    I have tried using LSTM but same result. I am training with 5 layers. Should I go for discrete action space? Like go left , right, forward, back?