Barracuda roadmap and continuing TensorFlow support?

Hi all

TL/DR first : Barracuda looks like an awesome step forwards and we love it when Unity NAILS cross-platform features. It looks ML-agents centric though, but we have a yearning for a more general inference engine for models developed in Keras / TensorFlow. Any intel on the Barracuda roadmap and model support in tensorflow_to_barracuda.py? Also we'd love to feed back specific model examples and other observations if there's an appetite for it under the heading of 'general inference' rather than 'ML agents'.

We're currently working on some really interesting ML ideas for projects we're developing with Unity. As others have found, ml-agents is fairly narrowly focussed on its own subset of neural network architectures. We've had mixed results using tensorflow_to_barracuda.py on models built and trained externally with Keras / Tensorflow. Works a charm with a cleanly stripped model if we're very careful about what we use. Goes 'bang' (even on a frozen model) with things like dropouts, batch normalization (which tf2bc.py appears to try and handle) and LSTM. I appreciate that Unity might favour it's own 'spin' on ML / agents / inference engines, but IMO there's huge potential in servicing users who are coming from a "TensorFlow first" angle and just looking for a nice way of getting cross platform inference. Unity is so good at doing cross platform when it nails it that the thought of having to wrangle with TFLite, or the XLA TFCompiler, etc etc makes me sad ;-)

I suspect that making Barracuda "100% TF compatible" is unlikely to happen, but are there any good approaches for making a 'Barracuda compliant' model in Keras? Examples of 'compliant LSTM' perhaps, tips for dropout or B.N. layers?

I've had a dig around in ml-agents to see how tf2bc supports LSTM, and for examples of models constructed in, I assume, 'Barracuda compliant' fashion - e.g. tf.contrib.rnn.BasicLSTMCell being used in models.py, but info is fairly thin on the ground.

Ta

Andrew

 

Have you continued to use ML-Agents/Inference Engine/Barracuda?
And have you tried ONNX for your serialization format?

TY

 

Was there ever any official word back on this?

Shame to see a relevant, well-written question get no response from anyone at Unity.

 

Hi,

Have tried to import sk-learn random forest model in unity barracuda package. But by default it is going into layer loop. So I have three questions regarding this -
1. How to remove layer specification from barracuda package ?
2. How to load sk-learn model in barracuda ?
3. If Barracuda can not load model, then is there any other way to load random forest model in unity ?

 

Hello, I´m actually studing Game Programing, and for my thesis I´m trying to integrate a pre trained tensorflow model into unity, I first started using tensorflowsharp for my export, I froze a mnist test and could use it into unity, the thing was that I had to use tensorflow 1.5 instead of tensorflow 2.0, also people from the project in github told me it was probably not getting any updates soon. I actually prefer tensorflow 2.0 so I started reasearching a little bit more about how export a pretrained model with tensorflow 2.0 to Unity, I stumbled upon this post So i´m actually trying to use barracuda for the CNN deploy.

would you reccommend going down this path?

I just want to migrate a CNN that will diferenciate between drawings and paragraphs into unity, trained with tensorflow 2.0 in google collab with te NIST, Google QuickDraw, and other paragraph datasets. I haven´t used barracuda before, I actually wrote a question for this in stackoverflow and it got downvoted, so I´m asking for advice over here