Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

LSTM usage logic

Discussion in 'ML-Agents' started by AndrewGri, Apr 15, 2020.

  1. AndrewGri


    Jan 30, 2020
    use_recurrent: true
    sequence_length: 64
    memory_size: 256
    • use_recurrent is a flag that notifies the trainer that you want to use a Recurrent Neural Network.
    • sequence_length defines how long the sequences of experiences must be while training. In order to use a LSTM, training requires a sequence of experiences instead of single experiences.
    • memory_size corresponds to the size of the memory the agent must keep. Note that if this number is too small, the agent will not be able to remember a lot of things. If this number is too large, the neural network will take longer to train.
    How all those parameters are connected to vector of observations?
    Does "sequence_length" exactly mean that 64 x observation vectors will be used?

    What is the best practice to calculate "memory_size" based on the size of observation vector?
  2. zachduer


    Nov 7, 2016
    I had exactly these same questions and came to the forum looking for an answer, so I'm sad not to see them here. Does anyone know the answer to AndrewGri's questions?
  3. MarkTension


    Aug 17, 2019
    As I understand it, sequence_length means indeed that 64 observations are used during training to learn the temporal relationships.

    I haven't found any definite ways to calculate the memory_size (or any model architectural choices). This is generally quite difficult. Good practice would be to start by using something equal to another network with similar input, like one of the ml-agents examples. In the first place see if adding recurrence makes a positive difference to your agent's performance first by establishing a baseline.

    Then start experimenting:
    Reduce memory size by half, and see if this decreases (or even increases, because faster to train) your agent's performance. If performance decreased, you can maybe double your initial baseline, and see how that affects performance. If this also doesn't increase performance you know you're in the right ballpark, and can optionally tune further in between what worked best.
    Last edited: Jun 4, 2020
  4. sheldonelias


    Nov 6, 2017
    Is there any further progress and results to the posts provided here? I am working on an LSTM for a Unity agent. But I need to decide to create the cells inside or outside the Unity framework.