Question Increase in memory_size decreasing cumulative reward

I am experimenting with LSTM in my project and I found an odd behaviour. By keeping sequence_length constant ( = 16), increasing memory_size is leading to worse results during training.

Is it possible that since memory size increased, the agent is storing more information that is actually not helping with training? Is this an expected behaviour as we increase memory_size?

In the Cumulative Reward image below:

Red curve: Sequence Length = 16; Memory Size = 16
Yellow curve: Sequence Length = 16; Memory Size = 32
Blue curve: Sequence Length = 16; Memory Size = 64

Many thanks!

 

Yeah pretty much. your graphs is actually the method which I use myself to figure out how much memory.
I start with a small value for memory. Then keep increasing it until it starts hampering learning. Then settle on a value slightly smaller than the point where it starts hampering learning.

Umm also one of us is color blind. There is no yellow line although one might be red.
The line that learns almost fully by 40M I assume is 16
The line that jumps upwards at 40 M and then learns more till about 80M I assume is 32
The third line I assume is 64

If the above is the correct graph lines vs memory size then in this case I would go with 32 for the memory size.

 

Thank you for your reply! And I apologise for the colours I picked. I'll be more careful next time. Ideally tensorboard could give the option of "dashed" or "cross" symbols for graphs, so I'll see if that's possible.

The line that learns almost fully at 40M have 32 of memory, while the line that jump upwards at 40M have 16 of memory. So increasing memory from 16 to 32 actually helped performance. Then, increasing memory to 64 reduced performance a lot.