Big quality of life tip for tensorboard that was personally overlooked. If you click the gear icon in the top right corner and set Reload Data to true, the graph will continually update in real time as your RL Agents train. See attached screenshot. Side notes: If you have issues with it not updating, try uninstalling tensorboard from your developing environment and installing tb-nightly. You can also try setting the argument variables (--reload_interval=30 --reload_multifile=true) in tensorboard through the command line when you go to run it. If you want to look at the other arguments for tensorboard, type tensorboard --h in the command line.