Why should batch size be >= recurrence's sequence length? Why not horizon?

Hi!

as of version 0.15.1, when using recurrence and I set batch size smaller than sequence length I get an error:

mlagents.trainers.exception.UnityTrainerException: batch_size must be greater than or equal to sequence_length when use_recurrent is True.

I don't understand why batch size should play a role in this.

As I understand it, each gradient update consists of batch size experiences with each a horizon number of steps.
It sounds more logical that not batch size but horizon should be same or greater than sequence length, so that each batch sample contains at least one recurrent sequence. Or is this related to a totally different issue?

Thank you!

 

Hi @MarkTension,
According to our research folks, we want to process at least one sequence per batch. So if the batch size is smaller than the sequence, we throw the exception.

 

Aah I thought a batch includes multiple horizons: (each batch sample being one sequence of length horizon). But from this I get that a batch really is just a sequence of experiences within one horizon. that clears things up for me

 

AFAIK:

You can fit multiple sequences within a single batch and learn from that, but since batch is the chunk size of state-action pairs used for back-prop, the sequence length must fit inside it at least once (otherwise we would inherently be trimming the sequence arbitrarily). This error is doing you a favor by indicating that you were trying to learn from a sequence that your agent's memory is not large enough to hold.

My thinking is that time-horizon and sequence length should be equal to batch size (or some fraction of it) so that there's no missed information in a given batch, and no bleed over into the beginning or end of new batches.