Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

[Released] Awesome ML Kit — Optimized real-time ML Models centered around computer vision

Discussion in 'Assets and Asset Store' started by leavittx, Jul 13, 2023.

  1. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    176
    Awesome ML Kit gives the Unity developers easy way to employ optimized real-time ML models centered around computer vision.




    The current set of models for the Unity Barracuda inference library includes
    1. High-quality background removal/background matting based on RobustVideoMatting (optimized for Unity)
    2. Depth estimation by single image
    And more is planned!

    Features
    1. Sample scene for each model
    2. Specially made visualizations with different post effects
    3. Select solid color / image / video as background replacement
    4. Different model versions for Quality / Speed tradeoff
    5. WebCamTexture support
    6. Unity VideoPlayer integration
    7. AVPro Live Camera integration (recommended)

    Ideal for
    • Media installations
    • Apps with video streaming (e.g. can be used with Agora.io)
    • Kiosks
    Performance
    NVidia RTX3070
    • Background removal: 300 FPS
    • Depth estimation: 400 FPS
    NVidia GTX1050Ti (mobile)
    • Background removal: 50 FPS
    • Depth estimation: 80 FPS


    To show what's possible to do with such asset, we've created a remix of the Disclosure - Hourglass ft. LION BABE music video where all the male dance masking is done using this asset (we just discard the color information from the webcam)

    Also Demolition Media Hap was used in the creation process of this experience (to sync with original video)

    Note: only desktop platforms currently supported (tested on Windows DX11 and Linux Vulkan)
    Unity versions: 2021/2022 and above

    Get it on the Asset Store: https://assetstore.unity.com/packages/tools/ai-ml-integration/demolition-ml-kit-249251
     
    Last edited: Feb 15, 2024
  2. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    176
    A small update for all existing and new users: in most cases the Gamma color space should be selected in the project settings in order for the colors to be correct on the video. You can do that with Edit->Project Settings->Player->Other Settings->Color Space set to Gamma
     
    Last edited: Jul 28, 2023
  3. leavittx

    leavittx

    Joined:
    Dec 27, 2013
    Posts:
    176
    We got a big update recently which is covered in this video: