Question Dynamically loading Onnx file and setting as the model for an Agent

I have a .onnx file from a trained model that's stored on the application persistentpath. I want to load it at runtime and set it as the model for an Agent. Now, the Agent expects a NNModel in its SetModel method. This is what I've tried

Code (CSharp):

1.  
2. var modelPath = NNHelper.GetOnnxPath(Species, selectedBrain.Brain);
3. var modelData = File.ReadAllBytes(modelPath);
4. NNModel model = ScriptableObject.CreateInstance<NNModel>();
5. model.modelData = ScriptableObject.CreateInstance<NNModelData>();
6. model.modelData.Value = modelData;
7. GetComponent<FishAgent>().SetModel("Fishy", model);

This results in the NotSupportedException: Format version not supported: 242 error since Barracuda doesn't support Onnx files apparently. Online I've read some people have used the ONNXConverter, however, that converts the byte[] to a Model, where a byte[] is required for the modelData.Value.

Any suggestions?

 

Fixed it. If anyone wants to know, take a look at the ONNXModelImporter.

 

Thanks a bunch, man. Your hint came in clutch for me.

But to kick the can a few meters down the streets and save people some time and confusion, I'll give a more specific answer for the folks in the future:

Specifically, look at the implementation of the OnImportAsset(AssetImportContext ctx) method in the ONNXModelImporter class and do what they did there.

I came up with this:

Code (CSharp):

1. NNModel LoadNNModel(string modelPath, string modelName)
2. {
3.     var converter = new ONNXModelConverter(true);
4.     Model model = converter.Convert(modelPath);
5.     NNModelData modelData = ScriptableObject.CreateInstance<NNModelData>();
6.     using (var memoryStream = new MemoryStream())
7.     using (var writer = new BinaryWriter(memoryStream))
8.     {
9.         ModelWriter.Save(writer, model);
10.         modelData.Value = memoryStream.ToArray();
11.     }
12.     modelData.name = "Data";
13.     modelData.hideFlags = HideFlags.HideInHierarchy;
14.     NNModel result = ScriptableObject.CreateInstance<NNModel>();
15.     result.modelData = modelData;
16.     result.name = modelName;
17.     return result;
18. }

:

 

It may depend on the version of ML-Agents and Barracuda you are using, but I was able to update onnx files for ML-Agents dynamically with the following code. The versions I used were ml-agents=v2.0.1 and barracuda=2.0.

Code (CSharp):

1. using Unity.MLAgents;
2. using Unity.MLAgents.Policies;
3. using Unity.Barracuda;
4.  
5. // Difficulty Levels
6. public enum BattleMode
7. {
8.    Easy,
9.    Medium,
10.    Hard
11. }
12.  
13. // Unity.Barracuda.NNModel class
14. public NNModel easyBattleBrain = "BossBattle-easy.onnx";
15. public NNModel mediumBattleBrain = "BossBattle-medium.onnx";
16. public NNModel hardBattleBrain = "BossBattle-hard.onnx";
17.  
18. // Should match with Agent's Behavior Name in Unity's UI
19. public string behaviorName = "BossBattle";
20.  
21. public void setDifficulty(BattleMode mode, Agent agent)
22. {
23.    switch (mode)
24.    {
25.        case BattleMode.Easy:
26.            agent.SetModel(behaviorName, easyBattleBrain);
27.            break;
28.        case BattleMode.Medium:
29.            agent.SetModel(behaviorName, mediumBattleBrain);
30.            break;
31.        case BattleMode.Hard:
32.            agent.SetModel(behaviorName, hardBattleBrain);
33.            break;
34.        default:
35.            break;
36.    }
37. }
38.  

We have also included a tutorial for the ML-Agents demo we developed at this URL. Please feel free to use it as a reference.
I hope this information is helpful.