Get gradients from model

Hi!

My model is learning poorly and I want to debug it. I'd like to make it explain its choices. I want to give it an action vector and make it produce gradients on all of the inputs, then I'll visualise those gradients in game, e.g. draw red and blue sensor lines. Would be best to do it during training, but inferrence time will do too.

I don't know where to start. The framework seems too black boxy.

Do I understand it correctly that the neural network training occurs on the Python side? Would it be hard to set up data exchange between Python and Unity? Or can I do it from Unity C# side via Barracuda?