Performance issues: possible CPU upgrade?

I'm doing a relatively simple project which requires an agent to traverse a maze like environment (it is not a maze as the agent knows where the exit it). The agent is more successful when its observations are stacked and it has some kind of memory to draw from, however this is computationally taxing and my system is struggling to keep up. I'm getting around 13 fps in unity while training 6 instances, with 15 stacked observations

My CPU is an I5 6600k, and my GPU is a 6gb 1060gtx. I have read that the GPU is under (or not) utilised in ML agents so I am looking to upgrade my CPU. Would I benefit from getting more cores, or from a higher clock speed? Ideally I would like to be able to run many instances of my training scenario, and possibly increase the agents available memory.

 

Thanks for the reply!