Question Tensorflow brain in App

Hi guys,

I am not sure this is the right section, so please feel free to replace this Thread.

I am trying to use a pre-trained brain (Tensorflow) within an App that I developed.

I tried using IronPython2 to call my Python-Script but didn't work, since it can't handle NumPy and Tensorflow (throws error: unexpected token "from")

I tried converting my tensorflow brain to ONNX and use Barracuda to infere but I didn't really understand how to set my Input-Data (Input data should be localRotation.eulerAngles) to match the expected array (-1,291,3,1) from my brain.

Do you guys know any other way to do what I want to?
What about IronPython3? Does anyone know how to integrate it in Unity? Or is there another way to call a Python3 code from Unity?

Thank you very much for your help!

 

Hi @Aurimasp,

I have ML-Agents on this project, so I just used the Barracuda that was installed with it.
I trained my Tensorflow-Model with an input array shaped (-1,291,3,1).
The values are the localRotation as Euler Angles from GameObject.
This means I have 291 local rotation values from x, y and z.

I tried to create the tensor just as it is in this documentation, but I apparently I got it wrong.

Do you mean this would work?

var t = new Tensor (-1, 291,3,1)
for (int i =0; i<291; i++)
t[0,i,0,0] = localRotation.x
t[0,i,1,0] = localRotation.y
t[0,i,2,0] = localRotation.z

And what I also didn't understand is how to performe Inference to this tensor using my pre-trained model.

Thank you very much for your help!!

 

I am not training a ML-Agents.
I will try again with this Guide and then I will get back to you.

Thank you again!

 

Ok I got it running! Thank you very much!

Starting the Play Mode on Unity with the Oculus Quest attached to the PC, works fine.

Now I am trying to build the App to run it in a Oculus Quest.
The building works also fine, but when I try to open the App in the Oculus Quest nothing happens. All I see is the loading scene with the three dots...

 

@AugustoGartner you could run `adb logcat` and collect logs from the Oculus Quest device, maybe it will give you some hints what is actually happening.
https://developer.oculus.com/documentation/native/android/mobile-logcat/