Question Does BERT work with Barracuda?

Hello!

※ Problem
I tried to get Barracuda to work using BERT(Bidirectional Encoder Representations from Transformers).
However, when importing ONNX, an error is output.



※ Question
1. Can I run BERT on Barracuda?
2. Can I use Transformer with Barracuda?

※ Environment
Unity 2020.3.29f1
Barracuda 3.0.0

Thanks.

 

I also need this information..

 

I was trying, for a university thesis, to introduce BERT within Unity or otherwise a social agent.
Do you have any suggestions?
I have tried several times to load the .onnx model but always had various problems..now I am in a dead end T_T

 

Hi there,
You could use the ONNX runtime ML.NET nuget package. Be careful to export your BERT model with a higher opt version. Then you will need an adapted tokenizer that works with subwords, no official package is available, but a simple online research will yield solutions in that regard.
Good luck !

 

Do you happen to have any particular guide that can help me?
If I could just get the model uploaded to unity, that would already be an achievement.

I saw this video:

Should I do as he did?


Thank in advance

 

Hey,
What is done in the video feels similar to what I did. My workflow is: use python transformers library (by HuggingFace) to load BERT model and train it on a downstream task, export it to ONNX, then load the .onnx model with onnx runtime & ML.NET, and use it for inference.
See Using Huggingface Transformers with ML.NET | Rubik's Code (rubikscode.net) for example.
HuggingFace has some documentation too (see : Export Transformers Models (huggingface.co)).
I did not find an exhaustive tutorial that suited my needs, but I believe the first link may help you. By the way, they are other ways of loading a .onnx model in Unity than what is presented in the first link.

Hope it helps

 

Nothing to do, i'm going crazy.

 

Check this document. I used it as a reference and successfully implemented inference for a BERT model.

https://onnxruntime.ai/docs/tutorials/csharp/bert-nlp-csharp-console-app.html

Here is my code:

Code (CSharp):

1. using BERTTokenizers;
2. using Microsoft.ML.Data;
3. using Microsoft.ML.OnnxRuntime;
4. using Microsoft.ML.OnnxRuntime.Tensors;
5. using System;
6.  
7. namespace MyApp
8. {
9.     internal class Program
10.     {
11.         static void Main(string[] args)
12.         {
13.             var sentence = "I'm sorry for what i said";
14.             //var sentence = "{\"question\": \"Where is Bob Dylan From?\", \"context\": \"Bob Dylan is from Duluth, Minnesota and is an American singer-songwriter\"}";
15.             Console.WriteLine(sentence);
16.  
17.             var tokenizer = new BertUncasedBaseTokenizer();
18.             var tokens = tokenizer.Tokenize(sentence);
19.             //Console.WriteLine(String.Join(", ", tokens));
20.             var encoded = tokenizer.Encode(tokens.Count, sentence);
21.  
22.             var bertInput = new BertInput()
23.             {
24.                 InputIds = encoded.Select(t => t.InputIds).ToArray(),
25.                 AttentionMask = encoded.Select(t => t.AttentionMask).ToArray(),
26.                 TypeIds = encoded.Select(t => t.TokenTypeIds).ToArray(),
27.             };
28.  
29.  
30.             var modelPath = @"F:\AI\Model\Bert\bert_goemotion_6500.onnx";
31.  
32.             using var runOptions = new RunOptions();
33.             using var session = new InferenceSession(modelPath);
34.  
35.             // Create input tensors over the input data.
36.             using var inputIdsOrtValue = OrtValue.CreateTensorValueFromMemory(bertInput.InputIds,
37.                   new long[] { 1, bertInput.InputIds.Length });
38.  
39.             using var attMaskOrtValue = OrtValue.CreateTensorValueFromMemory(bertInput.AttentionMask,
40.                   new long[] { 1, bertInput.AttentionMask.Length });
41.  
42.             using var typeIdsOrtValue = OrtValue.CreateTensorValueFromMemory(bertInput.TypeIds,
43.                   new long[] { 1, bertInput.TypeIds.Length });
44.  
45.             var inputs = new Dictionary<string, OrtValue>
46.             {
47.                 { "input_ids", inputIdsOrtValue },
48.                 { "attention_mask", attMaskOrtValue },
49.                 { "token_type_ids", typeIdsOrtValue }
50.             };
51.  
52.             // Run session and send the input data in to get inference output.
53.  
54.             //error here
55.             using var output = session.Run(runOptions, inputs, session.OutputNames);
56.             var predictedClassIndex = GetMaxValueIndex(output[0].GetTensorDataAsSpan<float>());
57.  
58.             var emotions = new List<string>
59.             {
60.                 "admiration", "amusement", "anger", "annoyance", "approval", "caring", "confusion",
61.                 "curiosity", "desire", "disappointment", "disapproval", "disgust", "embarrassment",
62.                 "excitement", "fear", "gratitude", "grief", "joy", "love", "nervousness", "optimism",
63.                 "pride", "realization", "relief", "remorse", "sadness", "surprise", "neutral"
64.             };
65.  
66.             var predictedLabel = emotions[predictedClassIndex];
67.  
68.             Console.WriteLine($"Predicted Emotion: {predictedLabel}");
69.  
70.         }
71.  
72.         public struct BertInput
73.         {
74.             public long[] InputIds { get; set; }
75.             public long[] AttentionMask { get; set; }
76.             public long[] TypeIds { get; set; }
77.         }
78.  
79.         static int GetMaxValueIndex(ReadOnlySpan<float> span)
80.         {
81.             float maxVal = span[0];
82.             int maxIndex = 0;
83.             for (int i = 1; i < span.Length; i++)
84.             {
85.                 var v = span[i];
86.                 if (v > maxVal)
87.                 {
88.                     maxVal = v;
89.                     maxIndex = i;
90.                 }
91.             }
92.             return maxIndex;
93.         }
94.  
95.  
96.     }
97. }

 

btw, this model is for text-classification task.