Search Unity

  1. The 2022.1 beta is now available for testing. To find out what's new, have a look at our 2022.1 beta blog post.
    Dismiss Notice

Help Wanted Using the model in Unity

Discussion in 'Barracuda' started by James_Adey, Sep 9, 2021.

  1. James_Adey

    James_Adey

    Joined:
    Jul 4, 2018
    Posts:
    19
    Hi,
    I have my model imported into Unity and I have the basic code from the 'Getting Started' section of the Barracuda documentation (see code.jpg). However I am confused with how to actually set up the input tensor in the code. I figured that I just look at the UI on the model for the input data and then use that in the code but the batch number is an asterix (see Model info.jpg).

    I am wanting to try and take some of my time-series data and use it in Unity to test the model. My model is trained to detect certain emotions based off of ECG data (see ECG data.jpg). My model is a 1 dimensional CNN and it does not take image or video based data. I am stuck with what to try next as I can only seem to find information on using Barracuda with images and videos instead of text-based data.

    Any help or advice would be greatly appreciated!

    Thanks,
    James
     

    Attached Files:

  2. alexandreribard_unity

    alexandreribard_unity

    Unity Technologies

    Joined:
    Sep 18, 2019
    Posts:
    48
    Hey James!

    I'm assuming that you have a variable number of batches in your model.
    Usually we recommend that you fix that on the model architecture, but dynamic batch size can work.

    To create your input tensor, simply do:
    Code (CSharp):
    1. var input = new Tensor(128,42)
    In this case I am assuming you want to have 128 batches.
    If on the other hand you only care about 1 batch, then replace 128 with 1 in the examples

    Then you fill it up either with tensor accessors, ie
    Code (CSharp):
    1.  
    2. input[0, 4] = 5.0f; // here I am filling in the first batch
    3. input[32, 4] = 5.0f; // batch # 33 here
    4.  
    or if you have a buffer representing your data, you can build the tensor directly from it as follows
    Code (CSharp):
    1. float[] srcData = ...
    2. var input = new tensor(128,42, srcData);
    Then execution is the same as in the documentation
     
    James_Adey likes this.
  3. James_Adey

    James_Adey

    Joined:
    Jul 4, 2018
    Posts:
    19
    Thanks for the help I appreciate it!
    I have also noticed that Unity thinks some of the layers in my model are 2D when they are actually supposed to be 1D. For example, the Conv1D layers in my model are being shown as Conv2D layers.

    I also noticed that when I am using MaxPooling instead of GlobalMaxPooling, the keras2onnx converter does not like it. I end up getting a type error:
    "TypeError: must be str, not NoneType"
    I have tested every layer in my models and it works fine until I try using MaxPooling which is odd.
     
    Last edited: Sep 11, 2021
  4. alexandreribard_unity

    alexandreribard_unity

    Unity Technologies

    Joined:
    Sep 18, 2019
    Posts:
    48
    Yes, sorry for the potential confusion. But most of the 1D operators can be implemented with 2D ones (one dimension being 1) so we don't distinguish.
    But it should work fine.

    As for your error, the best I could find is the following:
    https://github.com/onnx/keras-onnx/issues/665
    I think it might be good for you to try to get the model to export GlobalMaxPooling as our implementation is for efficient than MaxPooling
     
    James_Adey likes this.
unityunity