ML-Agents Memory - always use?

Given it’s possible to enable Long Short-Term Memory for agents, is there ever any instance where I wouldn’t want that activated? Assuming it doesn’t notably affect performance. I can’t offhand think of any situation where it wouldn’t at least be useful for an agent to remember the last 2-3 seconds of gameplay, if not more. Should I always have this ability active?

Thanks!

 

It's more input to sift through, and so if it doesn't have any bearing on current actions (such as tic-tac-toe), that means you have to train the network to de-emphasize its importance on more items in the latent space... That is: The system has to learn to ignore that which it doesn't need (more RAM, more CPU, potentially longer training times). Depending on your problem, that may (or may not) make much of a difference.