Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice

Question Dynamically loading Onnx file and setting as the model for an Agent

Discussion in 'ML-Agents' started by frieldhelm446, Nov 13, 2022.

  1. frieldhelm446

    frieldhelm446

    Joined:
    Sep 27, 2020
    Posts:
    28
    I have a .onnx file from a trained model that's stored on the application persistentpath. I want to load it at runtime and set it as the model for an Agent. Now, the Agent expects a NNModel in its SetModel method. This is what I've tried

    Code (CSharp):
    1.  
    2. var modelPath = NNHelper.GetOnnxPath(Species, selectedBrain.Brain);
    3. var modelData = File.ReadAllBytes(modelPath);
    4. NNModel model = ScriptableObject.CreateInstance<NNModel>();
    5. model.modelData = ScriptableObject.CreateInstance<NNModelData>();
    6. model.modelData.Value = modelData;
    7. GetComponent<FishAgent>().SetModel("Fishy", model);
    This results in the NotSupportedException: Format version not supported: 242 error since Barracuda doesn't support Onnx files apparently. Online I've read some people have used the ONNXConverter, however, that converts the byte[] to a Model, where a byte[] is required for the modelData.Value.

    Any suggestions?
     
  2. frieldhelm446

    frieldhelm446

    Joined:
    Sep 27, 2020
    Posts:
    28
    Fixed it. If anyone wants to know, take a look at the ONNXModelImporter.
     
  3. Pinguanec

    Pinguanec

    Joined:
    Jul 15, 2022
    Posts:
    1
    Thanks a bunch, man. Your hint came in clutch for me.

    But to kick the can a few meters down the streets and save people some time and confusion, I'll give a more specific answer for the folks in the future:

    Specifically, look at the implementation of the OnImportAsset(AssetImportContext ctx) method in the ONNXModelImporter class and do what they did there.

    I came up with this:
    Code (CSharp):
    1. NNModel LoadNNModel(string modelPath, string modelName)
    2. {
    3.     var converter = new ONNXModelConverter(true);
    4.     Model model = converter.Convert(modelPath);
    5.     NNModelData modelData = ScriptableObject.CreateInstance<NNModelData>();
    6.     using (var memoryStream = new MemoryStream())
    7.     using (var writer = new BinaryWriter(memoryStream))
    8.     {
    9.         ModelWriter.Save(writer, model);
    10.         modelData.Value = memoryStream.ToArray();
    11.     }
    12.     modelData.name = "Data";
    13.     modelData.hideFlags = HideFlags.HideInHierarchy;
    14.     NNModel result = ScriptableObject.CreateInstance<NNModel>();
    15.     result.modelData = modelData;
    16.     result.name = modelName;
    17.     return result;
    18. }
    :
     
    kokimitsunami likes this.
  4. kokimitsunami

    kokimitsunami

    Joined:
    Sep 2, 2021
    Posts:
    25
    It may depend on the version of ML-Agents and Barracuda you are using, but I was able to update onnx files for ML-Agents dynamically with the following code. The versions I used were ml-agents=v2.0.1 and barracuda=2.0.

    Code (CSharp):
    1. using Unity.MLAgents;
    2. using Unity.MLAgents.Policies;
    3. using Unity.Barracuda;
    4.  
    5. // Difficulty Levels
    6. public enum BattleMode
    7. {
    8.    Easy,
    9.    Medium,
    10.    Hard
    11. }
    12.  
    13. // Unity.Barracuda.NNModel class
    14. public NNModel easyBattleBrain = "BossBattle-easy.onnx";
    15. public NNModel mediumBattleBrain = "BossBattle-medium.onnx";
    16. public NNModel hardBattleBrain = "BossBattle-hard.onnx";
    17.  
    18. // Should match with Agent's Behavior Name in Unity's UI
    19. public string behaviorName = "BossBattle";
    20.  
    21. public void setDifficulty(BattleMode mode, Agent agent)
    22. {
    23.    switch (mode)
    24.    {
    25.        case BattleMode.Easy:
    26.            agent.SetModel(behaviorName, easyBattleBrain);
    27.            break;
    28.        case BattleMode.Medium:
    29.            agent.SetModel(behaviorName, mediumBattleBrain);
    30.            break;
    31.        case BattleMode.Hard:
    32.            agent.SetModel(behaviorName, hardBattleBrain);
    33.            break;
    34.        default:
    35.            break;
    36.    }
    37. }
    38.  
    We have also included a tutorial for the ML-Agents demo we developed at this URL. Please feel free to use it as a reference.
    I hope this information is helpful.
     
    Last edited: Apr 28, 2023
    Pinguanec likes this.