Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice

Feedback Construction of AudioClip from NativeArray<byte> or byte* with length.

Discussion in '2023.1 Beta' started by nishikinohojo, Dec 4, 2022.

  1. nishikinohojo

    nishikinohojo

    Joined:
    Aug 31, 2014
    Posts:
    46
    I don't know if I should post this topic here or not.

    AudioClip.SetData exists but it only accepts byte[]. I personally, do not appreciate this.
    Now that we can create Textures and Meshes quite fast from native byte array on the fly, (Material is also possible, in a fast enough manner.) I really want to create AudipClip from bytes with minimum allocation.

    The reason why I need this is all about loading. Unity's AssetBundle and Addressables are, well, you know, just not good enough for some people.

    In many ways, it is much better for some people to have their own custom solution in Loading and Unloading assets using StreamingAssets. Because they want to take better performance and great responsibility of "What is loading/unloading" and "What is building assets".

    To achieve this efficiently, Unity's runtime assets need to have performant ways to convert themselves into native bytes and vice versa. As I said, Textures, Meshes and Materials do have their way of doing these.
    Why not AudioClip?


    Edit:
    Why am I talking specifically about Audio Clip?
    Other things like Scene, timeline, particle effect, animation or anything, actually can be entirely replaced with our solution in Unity by users. Which means we can make them Bytes-Compatible.

    But audio is something not easy for us to develop in Unity. Yes there are many good EXTERNAL solutions, but using external solution is somewhat invasive. I feel like it crosses the goddamn line.

    Maybe DSP Graph will become a thing in the future and solve everything, but will it?
    In the meantime using NativeArray for reconstrcution would ease the situation.

    Of course more precise interaction to internal system would be welcomed. Allocation of managed array should be just one of small problems here.

    01/03/2023 Edit:
    I've partially rewritten my post. My English was so poor.
     
    Last edited: Jan 2, 2023
    ModLunar likes this.
  2. TheZombieKiller

    TheZombieKiller

    Joined:
    Feb 8, 2013
    Posts:
    266
    This problem applies to other resource types too, unfortunately. Fonts and non-legacy AnimationClips being stand-out examples. If your audio clip comes from a file instead of in-memory bytes, you can technically use UnityWebRequestMultimedia with a local file path, but it's not a very satisfying solution.
     
  3. nishikinohojo

    nishikinohojo

    Joined:
    Aug 31, 2014
    Posts:
    46
    Oh dear, I've completely forgot about Fonts. Actually I do feel some small unique things might be OK with Resources, but yes, everything should be able to be treated more properly in the same way.

    Animation was yes, bloody serious and something I had to solve by my proprietary solution for many reasons. I mean not only for loading problems but also many performance issues. I had to replace entire skinning and animation. Of course it was blooody tough.
     
  4. ModLunar

    ModLunar

    Joined:
    Oct 16, 2016
    Posts:
    374
    This would be super helpful.
    Also thanks for sharing @TheZombieKiller , I had no idea that existed!