Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

*.onnx file load script at runtime.

Discussion in 'ML-Agents' started by jhlee0918, Feb 18, 2022.

  1. jhlee0918

    jhlee0918

    Joined:
    Dec 18, 2018
    Posts:
    5
    Hi, I have some problem using MLagents.

    1. The onnx file was created through mlagents.
    2. and that file drag&drop to the agent(behavior param) , it works without any problem.
    3. I want to load and replace this onnx file at runtime(in script) , but the following message appears.

    Is there any other way to solve this problem?

    Thank you for read this...

    [my code]
    Code (CSharp):
    1. byte[] modelValues = File.ReadAllBytes(_modelPath);
    2. Unity.Barracuda.NNModel nnmodel = ScriptableObject.CreateInstance<Unity.Barracuda.NNModel>();
    3. nnmodel.modelData = ScriptableObject.CreateInstance<Unity.Barracuda.NNModelData>();
    4. nnmodel.modelData.Value = modelValues;
    5. nnmodel.name = "sa_agent";
    6.  
    7. rl_env.GetComponent<shooting_env>().AgentsList[0].Agent.GetComponent<BehaviorParameters>().Model = nnmodel;


    [Message]
    System.NotSupportedException: Format version not supported: 244
    at Unity.Barracuda.ModelLoader+<LoadAsync>d__11.MoveNext () [0x00106] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.barracuda@2.4.0-preview\Barracuda\Runtime\Core\ModelLoader.cs:223
    at Unity.Barracuda.ModelLoader.Load (System.IO.BinaryReader fileReader, System.Boolean verbose, System.Boolean applyPatching, System.Boolean skipWeights) [0x0002d] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.barracuda@2.4.0-preview\Barracuda\Runtime\Core\ModelLoader.cs:200
    at Unity.Barracuda.ModelLoader.Load (System.Byte[] stream, System.Boolean verbose, System.Boolean skipWeights) [0x00001] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.barracuda@2.4.0-preview\Barracuda\Runtime\Core\ModelLoader.cs:140
    at Unity.Barracuda.ModelLoader.Load (Unity.Barracuda.NNModel model, System.Boolean verbose, System.Boolean skipWeights) [0x00001] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.barracuda@2.4.0-preview\Barracuda\Runtime\Core\ModelLoader.cs:31
    at Unity.MLAgents.Inference.ModelRunner..ctor (Unity.Barracuda.NNModel model, Unity.MLAgents.Actuators.ActionSpec actionSpec, Unity.MLAgents.Policies.InferenceDevice inferenceDevice, System.Int32 seed) [0x00085] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Inference\ModelRunner.cs:72
    at Unity.MLAgents.Academy.GetOrCreateModelRunner (Unity.Barracuda.NNModel model, Unity.MLAgents.Actuators.ActionSpec actionSpec, Unity.MLAgents.Policies.InferenceDevice inferenceDevice) [0x00036] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Academy.cs:620
    at Unity.MLAgents.Policies.BarracudaPolicy..ctor (Unity.MLAgents.Actuators.ActionSpec actionSpec, System.Collections.Generic.IList`1[T] actuators, Unity.Barracuda.NNModel model, Unity.MLAgents.Policies.InferenceDevice inferenceDevice, System.String behaviorName) [0x00008] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Policies\BarracudaPolicy.cs:84
    at Unity.MLAgents.Policies.BehaviorParameters.GeneratePolicy (Unity.MLAgents.Actuators.ActionSpec actionSpec, Unity.MLAgents.Actuators.ActuatorManager actuatorManager) [0x000b9] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Policies\BehaviorParameters.cs:240
    at Unity.MLAgents.Agent.ReloadPolicy () [0x00023] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Agent.cs:648
    at Unity.MLAgents.Policies.BehaviorParameters.UpdateAgentPolicy () [0x00016] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Policies\BehaviorParameters.cs:276
    at Unity.MLAgents.Policies.BehaviorParameters.set_Model (Unity.Barracuda.NNModel value) [0x00008] in D:\project_junhyeoklee\GA_map_evolution\Library\PackageCache\com.unity.ml-agents@2.1.0-exp.1\Runtime\Policies\BehaviorParameters.cs:108
    at mapEvolvWithRL_SceneManager_controller.Initialized (System.String _loadMapFilePath, System.String _modelPath) [0x001af] in D:\project_junhyeoklee\GA_map_evolution\Assets\Scripts\SA\mapEvolvWithRL_SceneManager_controller.cs:337
    UnityEngine.Debug:Log (object)
    mapEvolvWithRL_SceneManager_controller:Initialized (string,string) (at Assets/Scripts/SA/mapEvolvWithRL_SceneManager_controller.cs:342)
    mapEvolvWithRL_SceneManager_controller:Start () (at Assets/Scripts/SA/mapEvolvWithRL_SceneManager_controller.cs:194)
     
  2. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    473
  3. ChillX

    ChillX

    Joined:
    Jun 16, 2016
    Posts:
    145
    This is a snippet of code from how I"m doing this.


    Code (CSharp):
    1.  
    2. if (CXMLTrainingManager.Instance == null)
    3. {
    4.     //Running in inference mode. Get the appropriate brain for each behavior based on type
    5.     switch (AIType)
    6.     {
    7.         case Enum_AIType.Player:
    8.             AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);
    9.             break;
    10.         case Enum_AIType.Enemy:
    11.             AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);
    12.             break;
    13.         case Enum_AIType.NPC:
    14.             AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);
    15.             break;
    16.     }
    17.     AIBehavior.BehaviorType = BehaviorType.InferenceOnly;
    18. }
    19. else
    20. {
    21.     // Training mode so null the NN model if training that agent. Else get the appropriate brain for each behavior based on type
    22.     switch (AIType)
    23.     {
    24.         case Enum_AIType.Player:
    25.             if (CXMLTrainingManager.Instance.Env_TrainPlayer) { AIBehavior.Model = null; }
    26.             else { AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);  }
    27.             break;
    28.         case Enum_AIType.Enemy:
    29.             if (CXMLTrainingManager.Instance.Env_TrainEnemy) { AIBehavior.Model = null; }
    30.             else { AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);  }
    31.             break;
    32.         case Enum_AIType.NPC:
    33.             if (CXMLTrainingManager.Instance.Env_TrainNPC) { AIBehavior.Model = null; }
    34.             else { AIBehavior.Model = CXGameManager.Instance.AI_GetBrain(this);  }
    35.             break;
    36.     }
    37.     if (ManualControl)
    38.     {
    39.         AIBehavior.BehaviorType = Unity.MLAgents.Policies.BehaviorType.HeuristicOnly;
    40.     }
    41.     else if (AIBehavior.Model == null)
    42.     {
    43.         AIBehavior.BehaviorType = BehaviorType.Default;
    44.     }
    45.     else
    46.     {
    47.         AIBehavior.BehaviorType = BehaviorType.InferenceOnly;
    48.     }
    49. }
    50.  

    This code is where I hold references to the various OMNX files.

    Code (CSharp):
    1.  
    2.         [SerializeField]
    3.         private NNModel m_AIBrainPlayer;
    4.         public NNModel AIBrainPlayer { get { return m_AIBrainPlayer; } }
    5.  
    6.         [SerializeField]
    7.         private NNModel m_AIBrainEnemy;
    8.         public NNModel AIBrainEnemy { get { return m_AIBrainEnemy; } }
    9.  
    10.         [SerializeField]
    11.         private NNModel m_AIBrainBackgroundNPC;
    12.         public NNModel AIBrainBackgroundNPC { get { return m_AIBrainBackgroundNPC; } }
    13.  
    14.         public NNModel AI_GetBrain(CXMLAgentBase AgentInstance)
    15.         {
    16.             switch (AgentInstance.AIType)
    17.             {
    18.                 case Enum_AIType.Player:
    19.                     return AIBrainPlayer;
    20.                 case Enum_AIType.Enemy:
    21.                     return AIBrainEnemy;
    22.                 case Enum_AIType.NPC:
    23.                     return AIBrainBackgroundNPC;
    24.                 default:
    25.                     Debug.LogError(@"Unknown AI Type in GameManager.AI_GetBrain");
    26.                     return null;
    27.             }
    28.         }
    29.  
    30.  
     
    Haneferd likes this.
  4. jhlee0918

    jhlee0918

    Joined:
    Dec 18, 2018
    Posts:
    5
    @mbaske Thank you. I will try.

    @ChillX Finally I'm trying something new, I'll refer to your code later. thanks for letting me know.
     
  5. jhlee0918

    jhlee0918

    Joined:
    Dec 18, 2018
    Posts:
    5
    @mbaske Thanks for comment this issue
    re-write my code follow your comment. but can work it :)

    Code (CSharp):
    1. Unity.Barracuda.NNModel nnmodel = (Unity.Barracuda.NNModel)AssetDatabase.LoadAssetAtPath("Assets/Models/v1.4.14/basic_agent.onnx", typeof(Unity.Barracuda.NNModel)); //nnmodel = null ...
    2.  
    3. Agent.SetModel("basic_agent", nnmodel);
    I put it directly into the agent in the Unity editor, there is no problem.
    It calls using scripts are not possible.
     
  6. jhlee0918

    jhlee0918

    Joined:
    Dec 18, 2018
    Posts:
    5
    After few hours late ... I solved.
    @mbaske gave me hint, I was setting the asset's path incorrectly. Thank you.