Search Unity

Question Use external packages / libs (FFmpegInteropX) in Unity

Discussion in 'Windows' started by slice3984, Oct 26, 2023.

  1. slice3984

    slice3984

    Joined:
    Jul 24, 2023
    Posts:
    2
    Hello,
    i'm working on a Hololens 2 UWP app and need to decode frames i receive over network.
    For this task i wanted to use https://github.com/ffmpeginteropx/FFmpegInteropX but don't know how to use external libs / plugins in unity.

    I already tried to just download the source of the package and build it for ARM64 and just copy the DLLs and WINMD file and trying to make it available through the Editor and the Inspector (Switching platform to UWP and checking the "Editor" checkbox) but this did not make my package available as import in my scripts.

    The unity plugin NuGetForUnity also did not work, i was able to install the package using this plugin but got some warnings about missing libraries / wrong framework.

    Trying to get this work for a couple days now and hope someone can help me.
     
  2. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,438
    might be difficult to get many plugins working inside hololens..

    if no other solutions,
    can you use some server/desktop in between and decode there,
    then feed into hololens using some easier format?
     
  3. slice3984

    slice3984

    Joined:
    Jul 24, 2023
    Posts:
    2
    I already tested to just generate test frames (24bit RGB), write it to the texture and upload to GPU, issue here is the upload time / Apply texture calls.

    Gave me about 20ms just for the texture upload alone at a resolution of just 1920x1080, the goal is to actually stream in the native resolution and blit over the whole HMD.

    So the current idea is to decode the frames (H264/5 in this case) and use the decoded frame i got in GPU Memory directly as texture to blit.

    I've seen people already getting my mentioned package to work in a HL2 application but don't know how they did it.