Official Barracuda 4.0

any news on this ? months ? years ?

 

Perfect!

 

Here's some onnx files you can test it on.
https://huggingface.co/bes-dev/stable-diffusion-v1-4-onnx/tree/main
I don't think it works on all of them at the moment.

Also can you make it work with as many onnx in the "onnx zoo" as possible.

Any timeline for the new version? Like roughly middle of the year/end of the year/2024?

 

I have an idea. Why not just take Microsoft ONNX Runtime and make Unity RenderTextures one of the "execution providers" https://onnxruntime.ai/docs/execution-providers/.

 

This wouldn't make the task any easier I think. All the hard work is in implementing the execution provider.

 

Fair enough. Well I have abandoned Barracuda and am just using Onnx Runtime since I only require it to on on Windows.

 

Ah nice! Does it work well with Unity ? which execution provider do you use? does it support GPU on Windows? I have yet to find a good solution for GPU inference on Windows ( Barracuda is the best so far but has some compat issues hopefully some of them will be fixed with 4.0)

 

Yes, Onnxruntime has a dot net version.
CUDA is best but you need to install the runtime CUDA files of about 2GB!!! (Not ideal to package with your game) DirectML works (even with integrated graphics) but is 2x as slow as CUDA on some GPUs. CPU also works but it's just not worth bothering with.
Barracuda will be good once it is finished which may be never! (And might not necessarily be as fast as DirectML or CUDA) - but will hopefully work on more hardware.
So I would say if you need inference for a Windows game and Baraccuda doesn't work and it doesn't have to be super-top speed, DirectML Onnxruntime is an acceptable solution.

 

yeah fr

 

I'm excited to hear more about Barracuda 4.0. I expect the timing of this will be extremely important given the explosion of LLM interest recently. I hope it involves supporting a much larger set of ONNX.

 

Perhaps the explosion of interest in LLM has forced Unity to try and monetize their ONNX work, and thus Unity AI. We may never see version 4.

 

Code (CSharp):

1. 100a ¨100.000

 

@Aurimasp are there any updates on a release date?

 

I also hope developers to support large set of ONNX. opset 9 is out of date version....

 

My company is using generative AI tooling to our own art data for our project; we are evaluating what options we have to integrate.

Unity seems to be pushing Barracuda as the answer to embedded AI tooling (based on blogposts, etc); so is there an update on this? The original post was over a year ago and the release GitHub is a over a year since the last commit.

 

@unity_UmihkUVVzIbHtg

One of most annoying facts about Barracuda 3.0 is the garbage collection, it produce more than 240Kb each step , I was able to edit the package code and drop that 63kb per step, but still too much

Also, its not single thread friendly, if you target webgl for AI inference, all the engine process waiting for the inference step to complete which is very very slow.

Our game run roughly 30FPS on webgl while inference on, once its off, game jump to 60FPS .

My hope for Barracuda 4.0 is reducing the Garbage collection and improve performance for data layers between unity engine and python backend.

 

Any idea if Barracuda 4.0 will support MiDaS v3+ models?
The developer of DepthViewer (an AI 2D to 3D image/video converter) mentioned he was waiting for the
Barracuda 4.0 release to hopefully support newer/better models and maybe even improve performance significantly: https://github.com/parkchamchi/DepthViewer/issues/5#issuecomment-1657114890