Search Unity

  1. Get the latest news, tutorials and offers directly to your inbox with our newsletters. Sign up now.
    Dismiss Notice

Help Wanted Confusion on generating input in Barrcuda

Discussion in 'Barracuda' started by jwvanderbeck, Jan 10, 2021.

  1. jwvanderbeck

    jwvanderbeck

    Joined:
    Dec 4, 2014
    Posts:
    775
    PREFIX: I'm very new to all of this ML stuff. I've spent the last week learning the high level strokes of using GANs to generate images, and built a Generator and Discriminator in PyTorch, trained the models, and saved the generator to ONNX.

    I've now brought that trained model into Unity and following the basic example in the Barracuda docs have set up a simple GO that upon scene start attempts to setup and run a worker.

    However I can't seem to figure out the proper way to generate the input Tensor data. In my PyTorch code, I generate a noise sample for the generator as such:
    torch.randn(sample_size, 100, 1, 1)

    However the format of Tensor() in Barracuda seems to be different and the same inputs simply throws an error. Could any kind soul help me close this hopefully last bit of the loop?
     
  2. jwvanderbeck

    jwvanderbeck

    Joined:
    Dec 4, 2014
    Posts:
    775
    Worked out running the ONNX model in Python, and that works, so I just need to figure out how this translates to Unity.

    Code (CSharp):
    1. import torch
    2. import onnxruntime
    3.  
    4. ort_session = onnxruntime.InferenceSession("outputs/generator,onnx")
    5. device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    6.  
    7. def create_noise(sample_size, nz):
    8.     return torch.randn(sample_size, nz, 1, 1).to(device)
    9.  
    10. def to_numby(tensor):
    11.     return tensor.detach().cpu().numpy() if tensor.requires_grad else tensor.cpu().numpy()
    12.  
    13. ort_inputs = {ort_session.get_inputs()[0].name: to_numpy(create_noise(64, 100))}
    14. ort_outs = ort_session.run(None, ort_inputs)
    15. tensor = torch.tensor(ort_outs[0])
     
  3. Mantas-Puida

    Mantas-Puida

    Unity Technologies

    Joined:
    Nov 13, 2008
    Posts:
    1,860
    On Barracuda side you only need to remember that data layout is channels-last, so your input tensor creation could look like this:
    Code (CSharp):
    1.  
    2. var inputTensor = new Tensor(sample_size, 100);
    3.  
    4. // you can set tensor values via [] operator
    5. inputTesnor[0,0] = 1.0f;
     
  4. jwvanderbeck

    jwvanderbeck

    Joined:
    Dec 4, 2014
    Posts:
    775
unityunity