ML-Agents NullReferenceException on ONNX file

Hi,
I have finished training a model and seen the onnx file in the results folder but when I get it into the assets folder and drag and drop to the Model in the Behavior Parameters script I get a NullReferenceException. Cannot attach the model to the script. I'm using the latest ml agents version (1.0.7) in Unity 2019.4.24

 

I'm having the same problem.

 

Hi, what version of the python package are you using?
Could you please attach the text of your logs instead of images. It's hard to get the full story from images of text.

 

I'm using Python 3.7.7
Logs:
NullReferenceException: Object reference not set to an instance of an object
Unity.MLAgents.Inference.BarracudaModelParamLoader.CheckModel (Unity.Barracuda.Model model, Unity.MLAgents.Policies.BrainParameters brainParameters, Unity.MLAgents.Sensors.SensorComponent[] sensorComponents, Unity.MLAgents.Policies.BehaviorType behaviorType) (at Library/PackageCache/com.unity.ml-agents@1.0.7/Runtime/Inference/BarracudaModelParamLoader.cs:152)
Unity.MLAgents.Editor.BehaviorParametersEditor.DisplayFailedModelChecks () (at Library/PackageCache/com.unity.ml-agents@1.0.7/Editor/BehaviorParametersEditor.cs:108)
Unity.MLAgents.Editor.BehaviorParametersEditor.OnInspectorGUI () (at Library/PackageCache/com.unity.ml-agents@1.0.7/Editor/BehaviorParametersEditor.cs:68)
UnityEditor.UIElements.InspectorElement+<>c__DisplayClass58_0.<CreateIMGUIInspectorFromEditor>b__0 () (at <5984c823338e4ca69b7a0ca01115425e>:0)
UnityEngine.GUIUtilityrocessEvent(Int32, IntPtr)

 

Which version of the ml-agents pip package are you using?

 

pip ml-agents is 0.26.0

 

Please either upgrade your ml-agents c# package to the latest version or downgrade your Python package to the one that was released at the same time as the c# 1.0.7 version. You can find a table of recommended versions in our releases on GitHub. You’ll want to see the versions for release 2

 

Hello,

I'm getting a very similar error, but I'm not even able to import the *.onnx file without errors. Only when I

Treat Errors As Warning

in the imported assets inspector. When importing, I'm getting the following three errors:

I'm using

ML Agents Version 1.0.7
Barracuda Version 1.0.4
Python Version 3.8.8
mlagents (pip) Version 0.26.0
Unity Version 2020.3.7f1

Because of these errors, I'm unable to import the *.onnx file and therefore I can't use the training result as a model in the

Behaviour Parameters

component.

I tried a bunch of preview versions of the ML Agents package, but either ran into the same issue or in other version issues when importing the *.onnx file, when binding it to the model in the

Behaviour Parameters

or when starting to run the game.

For example, upgrading to version 2.0.0-pre.3 resulted in exactly the same errors together with new errors about methods I had overriden. Same errors on version 1.3.0-preview.

On version 1.4.0-preview I'm able to import and bind the imported file to the

Behaviour Parameters

model, but get the following error then instead:

Any ideas or workarounds for this would be appreciated. Thank you.

 

It's working now. Thanks a lot!

 

What combination of versions for Python, ML Agents Package, and mlagents (pip) are you using now?

 

I was now successfully able to use the generated model as well. I've downgraded the mlagents (pip) version to 0.20.0 and then retrained my model.

pip install --upgrade mlagents==0.20.0

Thank you @christophergoy for the hint in the right direction. I also have to admit, that I didn't see or ignored a warning in the Unity console stating, that I should be using a Python package version up until 0.20.0 but not higher, exactly because of these output format differences.

 

I think we will stop training with an error message to ensure that the message is more clear in the future.

 

I had a similar problem and was able to solve it by upgrading the ML Agents package. According to https://github.com/Unity-Technologies/ml-agents/releases the only mismatch using the latest release was the com.unity.ml-agents and com.unity.ml-agents.extensions packages. Assuming you are currently able to start training successfully, you can determine all of the version information except for the C# packages immediately when running mlagents-learn. The version of the C# packages is also outputted on the command line as soon as the scene is connected.

So I did this rather than downgrade python, as I had tried that and it said my yaml had a lot of invalid fields and I'd rather upgrade than downgrade. I installed com.unity.ml-agents through the package manager but had to enable pre release builds to get the latest version. You may also need com.unity.ml-agents.extension package, which must be installed via github; see https://forum.unity.com/threads/unity-does-not-recognise-mlagents-namespace.947286/#post-6577441.

Then, if you get a bunch of errors in your code, you'll have to do some editing since things change between versions, slight differences in function names, types and such:

https://forum.unity.com/threads/no-suitable-method-found-to-override.1105432/

See https://docs.unity3d.com/Packages/c...i/Unity.MLAgents.Actuators.ActionBuffers.html for documentation, the dropdown menu in the top left lets you select your version.

Assuming this works, you should simply be able to retrain and drag the generated onnx file into your agent model field without errors.

 

I got pre trained model for tensroflow and then converted it to model12.onnx. with the help of visual studio code.
The process how i converted is by following the below link(TensorFlow) :https://github.com/onnx/tensorflow-onnx
In unity I have :
unity 2021.3.16f1 LTS
ML- Agent 2.0.1
Barracuda 2.0.0
Now when i import model12.onnx model, I get following error :
Error1 :
OnnxImportException: Unexpected error while parsing layer Unsqueeze__35:0 of type Unsqueeze.
Couldn't find attribute axes of type Ints
Json: { "input": [ "StatefulPartitionedCall/sequential/bidirectional/forward_lstm/zeros_1:0", "const_axes__119" ], "output": [ "Unsqueeze__35:0" ], "name": "Unsqueeze__35", "opType": "Unsqueeze" }
at Unity.Barracuda.ONNX.ONNXNodeWrapper.FindAttribute (System.String name, Onnx.AttributeProto+Types+AttributeType type) [0x00010] in D:\Sonu Work\TensorFlowProject\TFLiiteProject\Library\PackageCache\com.unity.barracuda@2.0.0\Barracuda\Runtime\ONNX\ONNXNodeWrapper.cs:281
at Unity.Barracuda.ONNX.ONNXNodeWrapper.GetRequiredIntArray (System.String name) [0x00000] in D:\Sonu Work\TensorFlowProject\TFLiiteProject\Library\PackageCache\com.unity.barracuda@2.0.0\Barracuda\Runtime\ONNX\ONNXNodeWrapper.cs:329
at Unity.Barracuda.ONNX.ONNXNodeWrapper.get_Axes () [0x00000] in D:\Sonu Work\TensorFlowProject\TFLiiteProject\Library\PackageCache\com.unity.barracuda@2.0.0\Barracuda\Runtime\ONNX\ONNXNodeWrapper.cs:107
at Unity.Barracuda.ONNX.ONNXModelConverter+<>c__DisplayClass23_0.<UseStandardImporter>b__5 (Unity.Barracuda.ModelBuilder net, Unity.Barracuda.ONNX.ONNXNodeWrapper node) [0x0005f] in D:\Sonu Work\TensorFlowProject\TFLiiteProject\Library\PackageCache\com.unity.barracuda@2.0.0\Barracuda\Runtime\ONNX\ONNXModelConverter.cs:306
at Unity.Barracuda.ONNX.ONNXModelConverter.ConvertOnnxModel (Onnx.ModelProto onnxModel) [0x003a7] in D:\Sonu Work\TensorFlowProject\TFLiiteProject\Library\PackageCache\com.unity.barracuda@2.0.0\Barracuda\Runtime\ONNX\ONNXModelConverter.cs:2798
Unity.Barracuda.ONNX.ONNXModelConverter.Err (Unity.Barracuda.Model model, System.String layerName, System.String message, System.String extendedMessage, System.String debugMessage) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:3298)
Unity.Barracuda.ONNX.ONNXModelConverter.ConvertOnnxModel (Onnx.ModelProto onnxModel) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:2807)
Unity.Barracuda.ONNX.ONNXModelConverter.Convert (Google.Protobuf.CodedInputStream inputStream) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:155)
Unity.Barracuda.ONNX.ONNXModelConverter.Convert (System.String filePath) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:83)
Unity.Barracuda.ONNXModelImporter.OnImportAsset (UnityEditor.AssetImporters.AssetImportContext ctx) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Editor/ONNXModelImporter.cs:58)
UnityEditor.AssetImporters.ScriptedImporter.GenerateAssetData (UnityEditor.AssetImporters.AssetImportContext ctx) (at <11d97693183d4a6bb35c29ae7882c66b>:0)

Error 2 :
Asset import failed, "Assets/Models/model12.onnx" > OnnxImportException: Unexpected error while parsing layer Unsqueeze__35:0 of type Unsqueeze.
Couldn't find attribute axes of type Ints
Json: { "input": [ "StatefulPartitionedCall/sequential/bidirectional/forward_lstm/zeros_1:0", "const_axes__119" ], "output": [ "Unsqueeze__35:0" ], "name": "Unsqueeze__35", "opType": "Unsqueeze" }
at Unity.Barracuda.ONNX.ONNXNodeWrapper.FindAttribute (System.String name, Onnx.AttributeProto+Types+AttributeType type) [0x00010] in D:\Sonu Work\TensorFlowProject\TFLiiteProject\Library\PackageCache\com.unity.barracuda@2.0.0\Barracuda\Runtime\ONNX\ONNXNodeWrapper.cs:281
at Unity.Barracuda.ONNX.ONNXNodeWrapper.GetRequiredIntArray (System.String name) [0x00000] in D:\Sonu Work\TensorFlowProject\TFLiiteProject\Library\PackageCache\com.unity.barracuda@2.0.0\Barracuda\Runtime\ONNX\ONNXNodeWrapper.cs:329
at Unity.Barracuda.ONNX.ONNXNodeWrapper.get_Axes () [0x00000] in D:\Sonu Work\TensorFlowProject\TFLiiteProject\Library\PackageCache\com.unity.barracuda@2.0.0\Barracuda\Runtime\ONNX\ONNXNodeWrapper.cs:107
at Unity.Barracuda.ONNX.ONNXModelConverter+<>c__DisplayClass23_0.<UseStandardImporter>b__5 (Unity.Barracuda.ModelBuilder net, Unity.Barracuda.ONNX.ONNXNodeWrapper node) [0x0005f] in D:\Sonu Work\TensorFlowProject\TFLiiteProject\Library\PackageCache\com.unity.barracuda@2.0.0\Barracuda\Runtime\ONNX\ONNXModelConverter.cs:306
at Unity.Barracuda.ONNX.ONNXModelConverter.ConvertOnnxModel (Onnx.ModelProto onnxModel) [0x003a7] in D:\Sonu Work\TensorFlowProject\TFLiiteProject\Library\PackageCache\com.unity.barracuda@2.0.0\Barracuda\Runtime\ONNX\ONNXModelConverter.cs:2798
Unity.Barracuda.ONNX.ONNXModelConverter.Err (Unity.Barracuda.Model model, System.String layerName, System.String message, System.String extendedMessage, System.String debugMessage) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:3298)
Unity.Barracuda.ONNX.ONNXModelConverter.ConvertOnnxModel (Onnx.ModelProto onnxModel) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:2807)
Unity.Barracuda.ONNX.ONNXModelConverter.Convert (Google.Protobuf.CodedInputStream inputStream) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:155)
Unity.Barracuda.ONNX.ONNXModelConverter.Convert (System.String filePath) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Runtime/ONNX/ONNXModelConverter.cs:83)
Unity.Barracuda.ONNXModelImporter.OnImportAsset (UnityEditor.AssetImporters.AssetImportContext ctx) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Editor/ONNXModelImporter.cs:58)
UnityEditor.AssetImporters.ScriptedImporter.GenerateAssetData (UnityEditor.AssetImporters.AssetImportContext ctx) (at <11d97693183d4a6bb35c29ae7882c66b>:0)

 

I feel like this error is different than the title of this thread. I also feel you probably should create a minimum reproducable example. I also feel that you should start by investigating the issue strictly for Barracuda, without bringing mlagents into the mix. *mlagents do not support providing your own custom models*, i.e. Unity staff won't support this use-case, but barracuda itself should. https://docs.unity3d.com/Packages/com.unity.barracuda@1.0/manual/SupportedOperators.html#Unsqueeze

 

The thing is when i load onnx model for github it is working fine, only when i made a TFlite mode and converted it to .onnx format using visual studio code. it is throwing the import exception