Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question ScatterElements error

Discussion in 'Barracuda' started by DannyWoo, Dec 2, 2021.

  1. DannyWoo

    DannyWoo

    Joined:
    Oct 25, 2012
    Posts:
    24
    AlexRibard said " we added support for BackgroundMattingv2. It will be present in 2.3.0 It does require you exporting the model with fixed input size. But otherwise than that it works on the base model"

    i am using barracuda 2.3.1

    but i got still error when i import Model

    the message is "Asset import failed, "Assets/Onnx/onnx_mobilenetv2_hd.onnx" > OnnxImportException: Unknown type ScatterElements encountered while parsing layer 878."
     
  2. alexandreribard_unity

    alexandreribard_unity

    Unity Technologies

    Joined:
    Sep 18, 2019
    Posts:
    53
    Hi!
    Cross posting the answer from the github issue
    https://github.com/Unity-Technologies/barracuda-release/issues/136

    In order to be able to use BackgroundMattingV2 models, you'll need to export the .pth to onnx with fixed input size
    Here is the code to achieve that:

    Code (CSharp):
    1.     // https://github.com/PeterL1n/BackgroundMattingV2
    2.     // Download the github repo.
    3.     // Download the pth models here https://drive.google.com/drive/folders/1cbetlrKREitIgjnIikG1HdM4x72FtgBh
    4.     // Execute this script:
    5.        import torch
    6.    
    7.        from model import MattingBase, MattingRefine
    8.    
    9.        for model_backbone in ['resnet101', 'resnet50', 'mobilenetv2']:
    10.            model = MattingBase(model_backbone)
    11.            model.load_state_dict(torch.load('pytorch_' + model_backbone + '.pth', map_location='cpu'), strict=False)
    12.    
    13.            src = torch.randn(2, 3, 224, 224)
    14.            bgr = torch.randn(2, 3, 224, 224)
    15.    
    16.            # Export ONNX
    17.            input_names=['src', 'bgr']
    18.            output_names = ['pha', 'fgr', 'err', 'hid']
    19.    
    20.    
    21.            torch.onnx.export(
    22.                model=model,
    23.                args=(src, bgr),
    24.                f='MattingBase_' + model_backbone + '.onnx',
    25.                verbose=True,
    26.                opset_version=12,
    27.                do_constant_folding=True,
    28.                input_names=input_names,
    29.                output_names=output_names)

    Let me know if that works for you
     
    DannyWoo likes this.
  3. DannyWoo

    DannyWoo

    Joined:
    Oct 25, 2012
    Posts:
    24
    i have to using args.model_type == 'mattingrefine': to get best qulity of mask
    but barracuda 2.3.1 version also does not support ScatterElements by today
    i am waitting....
     
  4. alexandreribard_unity

    alexandreribard_unity

    Unity Technologies

    Joined:
    Sep 18, 2019
    Posts:
    53
    Ok, I checked the model with that option, indeed with static input shapes it generates a `ScatterElements`
    I'll create a task to support `MattingRefine`.
    But for now I'd suggest using `MattingBase` as that one works
     
    Last edited: Dec 7, 2021
  5. DannyWoo

    DannyWoo

    Joined:
    Oct 25, 2012
    Posts:
    24
    New
    `MattingBase` is of such a very very poor quality that i can't use as attached PIC.
    It's much better to use Keijiro's NNCam than `MattingBase`
    (https://github.com/keijiro/NNCam)
    Thanks :)

    Code (CSharp):
    1. import torch
    2.  
    3. from model import MattingBase, MattingRefine
    4.  
    5. for model_backbone in ['resnet101', 'resnet50', 'mobilenetv2']:
    6.     #model = MattingBase(model_backbone) #this is not a good
    7.     model = MattingRefine(
    8.         backbone=model_backbone,
    9.         backbone_scale=0.25,
    10.         refine_mode='sampling',                                           #default='sampling', choices=['full', 'sampling', 'thresholding'])
    11.         refine_sample_pixels=80000,                                      #default=80_000
    12.         refine_threshold=0.1,
    13.         refine_kernel_size=3,
    14.         refine_patch_crop_method='roi_align',                          #choices=['unfold', 'roi_align', 'gather']
    15.         refine_patch_replace_method='scatter_element')             #choices=['scatter_nd', 'scatter_element'])
    16.  
    17.     model.load_state_dict(torch.load('pytorch_' + model_backbone + '.pth', map_location='cuda'), strict=False)
    18.  
    19. .........
    20. .........
     

    Attached Files:

    • q2.JPG
      q2.JPG
      File size:
      40.1 KB
      Views:
      260
    • q3.JPG
      q3.JPG
      File size:
      39.9 KB
      Views:
      255
    Last edited: Dec 8, 2021