Search Unity

  1. Get the latest news, tutorials and offers directly to your inbox with our newsletters. Sign up now.
    Dismiss Notice

Help Wanted Can I "rewrite" an inference Python script to C# to run in Barracuda?

Discussion in 'Barracuda' started by samuelmorais, Apr 13, 2021.

  1. samuelmorais

    samuelmorais

    Joined:
    Aug 3, 2012
    Posts:
    52
    Hi,

    I want to run this inference script

    https://github.com/NVlabs/Deep_Object_Pose/blob/master/src/dope/inference/detector.py

    in Unity Barracuda.

    I have already converted the pth weights to ONNX, but now I needed to know if it is possible to run the inference based on the Python script linked above (of course if it was "converted" to C#).

    I know it is not an easy task, but if anyone can give me any advice regarding if it is minimally possible to start researching about how to run it in C# (maybe using OpenCV and/or other libraries).

    The idea of this network is, based on an image, predict an pre-defined object pose with 6 degrees of freedom (i.e. the location and rotation of object relatively to the camera).

    Any help would be apreciated.

    Thanks!

    Below is a part of the code extracted from the above github project.
    Code (Boo):
    1.  
    2.  
    3. class ObjectDetector(object):
    4.     '''This class contains methods for object detection'''
    5.  
    6.     @staticmethod
    7.     def detect_object_in_image(net_model, pnp_solver, in_img, config):
    8.         '''Detect objects in a image using a specific trained network model'''
    9.  
    10.         if in_img is None:
    11.             return []
    12.  
    13.         # Run network inference
    14.         image_tensor = transform(in_img)
    15.         image_torch = Variable(image_tensor).cuda().unsqueeze(0)
    16.         out, seg = net_model(image_torch)
    17.         vertex2 = out[-1][0]
    18.         aff = seg[-1][0]
    19.  
    20.         # Find objects from network output
    21.         detected_objects = ObjectDetector.find_object_poses(vertex2, aff, pnp_solver, config)
    22.  
    23.         return detected_objects
     
  2. amirebrahimi_unity

    amirebrahimi_unity

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    378
    samuelmorais likes this.
unityunity