Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Question Conversion from NEAT model to ONNX

Discussion in 'ML-Agents' started by KristianTv, Feb 21, 2023.

  1. KristianTv

    KristianTv

    Joined:
    Oct 11, 2022
    Posts:
    12
    Hey!

    I am training agents using neat-python to be used in Unity with MLAgents. The issue is that I have not found a way to convert the feedforward network (let alone the recurrent network) in NEAT.

    I have tried extensive googling, checking github and interrogating chatGPT without luck.
    Does anyone have any experience in performing such an conversion?
     
  2. hughperkins

    hughperkins

    Joined:
    Dec 3, 2022
    Posts:
    178
  3. KristianTv

    KristianTv

    Joined:
    Oct 11, 2022
    Posts:
    12
    Yeah, i have already trained the agent. I have a trained neat.FeedForwardNetwork which does not seem directly comparable to TensorFlow networks. I only want to run inference with this network.
    From what i just read about Barracuda it seems like documentation covers the usual neural net libraries like PyTorch, keras and TF.
    Does barracuda have support for such proprietary network types like NEAT?
     
  4. hughperkins

    hughperkins

    Joined:
    Dec 3, 2022
    Posts:
    178
    Yup. Because NEAT is not a type of network: it's a type of trainer.

    Other types of trainer include PPO, which mlagents can use, or simply using REINFORCE.

    The output of a trainer is a trained neural network that you can plug in for inference.

    Note that mlagents doesn't support inference on models not trained with mlagents. I doubt that's an insurmountable obstacle, especially given that mlagents is 100% opensource, but I reckon, if it was me, I'd just go with Barracuda.

    Barracuda doesn't cover pytorch, keras or TF. It has no awareness of any of these. Barracuda covers: ONNX. Or, a subset of ONNX. Here are the operators it covers: Supported ONNX operators | Barracuda | 1.0.4 (unity3d.com)
     
  5. hughperkins

    hughperkins

    Joined:
    Dec 3, 2022
    Posts:
    178
    Here are some network architectures that work with Barracuda:

    Supported neural architectures and models | Barracuda | 1.0.4 (unity3d.com)

    Note that you might need to map your NEAT network into the equivalent dense network, potentially.

    For example, imagine you have a very simple NEAT network like:

    Code (csharp):
    1.  
    2. input I:
    3. a
    4. b
    5. c
    6.  
    7. output O:
    8. d = a * 0.5 + b * 2
    9. e = c
    10.  
    So, this maps to the following matrix multiplication:
    Code (csharp):
    1.  
    2. O = WI
    3.  
    4. where:
    5. I = (a)
    6.      (b)
    7.      (c)
    8.  
    9. O = (d)
    10.        (e)
    11.  
    12. W = (0.5 2 0)
    13.         (0    0 1)
    14.  
     
  6. KristianTv

    KristianTv

    Joined:
    Oct 11, 2022
    Posts:
    12
    Ah I see, i was hoping there existed some library or methods that took the dirty work of translating the network structure in the FeedForwardNetwork by NEAT into onnx.

    I suppose going through the network iteratively and building the ONNX network by corresponding operators is my only choice. As the FeedForwardNetwork in NEAT keeps track of inputs / outputs per node and not having a layer specific structure it may be a bit tricky but should be possible.
     
  7. hughperkins

    hughperkins

    Joined:
    Dec 3, 2022
    Posts:
    178
    > As the FeedForwardNetwork in NEAT keeps track of inputs / outputs per node and not having a layer specific structure it may be a bit tricky but should be possible.

    Oh, right, good point. Hmmm. Yeah, you might need to create one dense layer per NEAT node.. Each node will have an output size of one. You can concatenate the outputs of various nodes, using `Concat` or similar, perhaps, and then feed them to the next dense mode.

    Just out of personal curiosity, what is attracting you to use NEAT over back-prop?
     
  8. hughperkins

    hughperkins

    Joined:
    Dec 3, 2022
    Posts:
    178
    > Ah I see, i was hoping there existed some library or methods that took the dirty work of translating the network structure in the FeedForwardNetwork by NEAT into onnx.

    I haven't actually googled around to see if there is, to be clear. And someone else might know of one. If there isn't, then there's an opportunity for you to make one :)
     
  9. KristianTv

    KristianTv

    Joined:
    Oct 11, 2022
    Posts:
    12
    I think a layer per node and then processing them all together might be a good solution.
    Making my own library for this would be pretty cool, i hope i can figure it out!

    The reason i am going for NEAT is because i am doing a explorative study, where i am comparing agents trained with NEAT against other algorithms to assess the effects on the player.
    Lastly it was a really interesting concept, intriguing me to learn and work with it. Reinforcement Learning would/will probably perform pretty well in the environment i am working with, as its of a more complex nature.
     
  10. hughperkins

    hughperkins

    Joined:
    Dec 3, 2022
    Posts:
    178
    Interesting. Thanks!