Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We are updating our Terms of Service for all Unity subscription plans, effective October 13, 2022, to create a more streamlined, user-friendly set of terms. Please review them here: unity.com/legal/terms-of-service.
    Dismiss Notice
  3. Have a look at our Games Focus blog post series which will show what Unity is doing for all game developers – now, next year, and in the future.
    Dismiss Notice
  4. Join us on Thursday, September 29, for a day with Unity's SRP teams here on the forum or on Reddit, and discuss topics around URP, HDRP, and the Scriptable Render Pipeline in general.
    Dismiss Notice

Question Can I "rewrite" an inference Python script to C# to run in Barracuda?

Discussion in 'Barracuda' started by samuelmorais, Apr 13, 2021.

  1. samuelmorais

    samuelmorais

    Joined:
    Aug 3, 2012
    Posts:
    57
    Hi,

    I want to run this inference script

    https://github.com/NVlabs/Deep_Object_Pose/blob/master/src/dope/inference/detector.py

    in Unity Barracuda.

    I have already converted the pth weights to ONNX, but now I needed to know if it is possible to run the inference based on the Python script linked above (of course if it was "converted" to C#).

    I know it is not an easy task, but if anyone can give me any advice regarding if it is minimally possible to start researching about how to run it in C# (maybe using OpenCV and/or other libraries).

    The idea of this network is, based on an image, predict an pre-defined object pose with 6 degrees of freedom (i.e. the location and rotation of object relatively to the camera).

    Any help would be apreciated.

    Thanks!

    Below is a part of the code extracted from the above github project.
    Code (Boo):
    1.  
    2.  
    3. class ObjectDetector(object):
    4.     '''This class contains methods for object detection'''
    5.  
    6.     @staticmethod
    7.     def detect_object_in_image(net_model, pnp_solver, in_img, config):
    8.         '''Detect objects in a image using a specific trained network model'''
    9.  
    10.         if in_img is None:
    11.             return []
    12.  
    13.         # Run network inference
    14.         image_tensor = transform(in_img)
    15.         image_torch = Variable(image_tensor).cuda().unsqueeze(0)
    16.         out, seg = net_model(image_torch)
    17.         vertex2 = out[-1][0]
    18.         aff = seg[-1][0]
    19.  
    20.         # Find objects from network output
    21.         detected_objects = ObjectDetector.find_object_poses(vertex2, aff, pnp_solver, config)
    22.  
    23.         return detected_objects
     
  2. amirebrahimi_unity

    amirebrahimi_unity

    Unity Technologies

    Joined:
    Aug 12, 2015
    Posts:
    400
    samuelmorais likes this.
unityunity