Search Unity

Question Structure of the output neural network

Discussion in 'ML-Agents' started by ReinierJ, Aug 27, 2020.

  1. ReinierJ

    ReinierJ

    Joined:
    Jul 10, 2020
    Posts:
    10
    Hello all,

    Is there any documentation or rationale about the structure of the neural networks that result from training? For example, one of the networks I trained has the structure in the image below (as visualized by Netron).

    My current understanding is that the Sub node below the vector_observation subtracts the running mean, and the top left branch (Add, RealDiv, Sqrt, RealDiv) adds bias, thus serving as a normalization step. But what purpose do the Minimum and Maximum nodes serve, for example? And why does the top left branch consist of three calculation nodes instead of just a constant bias?

    I understand that the three Dense nodes with the Swish activation function represent the actual network calculations. But I also do not understand what happens after this. I would expect that if you want to get the output of the neural network, the final step would simply be to output the results from the last Dense node (possibly after an activation function). Instead, there is a large structure of operations. What is happening here? Also, why are there two output nodes, namely action and action_probs, while in my agent code I only have and need one output, namely the action vector?

    Is there any documentation about this, or can someone explain this?

    (The background for my question is that I would like to import ml-agents models in a tool which only supports simple multi-layer perceptrons. So I can only have the input node, a sequence of Dense nodes and activation functions and an output node, without branching. I am wondering if I can somehow convert the model to such a format, or maybe train such a model using (a slightly modified version of) ml-agents.)