Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

AudioStreamNetCode | network audio based on NetCode for GameObjects

Discussion in 'Assets and Asset Store' started by r618, Aug 5, 2022.

  1. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,302
    Asset provides client/server audio comms over network using Unity Transport as part of NetCode for GameObjects
    [ https://unity.com/products/netcode
    https://docs.unity3d.com/Packages/com.unity.netcode.gameobjects@1.0/manual/index.html ]

    It's currently based on latest release of NetCode and uses Unity Transport only - | this means it doesn't support Relay and is thus OOTB suited only for LAN based solutions. |
    / NetCode supports 3rd party transports so it's possible to substitute for it.



    It can stream any AudioSource or AudioListener to all connected clients and clients are allowed to stream back into server mix with their own audio.
    (Audio is encoded using OPUS codec - this also means max. 2 channels per source)

    Network throughput for single audio source is up to ~30 kB/sec depending on settings.

    Please see demos here:
    Windows x64 | macOS | Android/ChromeOS

    Please see latest documentation here
    and for more information and demo builds Asset Store page
     
    Last edited: Apr 15, 2023
  2. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,302
    Asset is summitted, hopefully in a month or so it will be available.
    I set 50% discount for two weeks after launch.

    I will be taking reasonable requests after the first release.

    ~~
     
  3. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,302
    1.0 o92o22, Xcities
    - initial release
    - client/server audio mixer


    live (-:
     
  4. amitghimire

    amitghimire

    Joined:
    Jun 15, 2022
    Posts:
    12
    Hi, Can I use this as voice communication solution for netcode for gameobjects based unity apps?
     
  5. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,302
    Yes, that's its purpose
    It will work as is on local network (LAN)

    Internet streaming via Unity Relay is not supported - Unity explicitly prohibits transfer of audio data via their services - since they have their own Vivox solution (which is technically based on the same OPUS codec btw)
    - it's possible to replace relay (internet transport) with other 3rd party service, but this is not implemented yet

    Hope this answers the question! feel free to ask if something's not clear still
     
    Last edited: Apr 15, 2023
  6. MigolasUnity

    MigolasUnity

    Joined:
    Dec 13, 2021
    Posts:
    2
    i buyed the asset, it is very good , but i want capture microphone in the clients and send it to the other clients. the mixer in the server is no good for this . Is this posible?
     
  7. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,302
    Hi @MigolasUnity thanks !
    I think there are currently two pieces missing:
    -- transmission of client Unity microphone capture (in demo)
    Unity mic can be scripted by user and otherwise should be the same as capturing AudioSource in scene as it is now
    But adding this to a demo scene makes sense

    and
    -- when a client is transmitting its own mic input/audio that shouldn't be transmitted back to it from server mix
    Is that what you think the problem is ?
    I'll add a setting for this and moreover this behaviour should be even the default I think

    If you want, send me your invoice # to PM, I'll make sure to let you know about update as soon as it is ready / please do note that it will take some time though /

    Thanks once more!
     
  8. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,302
    Just submitted an update, it could be online on the store shortly:

    ==================================================================
    1.1 1o2o22
    - demo scene: + audio visualization textures on main listener
    ==================================================================
    1.2 o82o23
    - Unity NetCode ~1.5.x, requires min. Unity 2021.3 LTS
    (this should also fix macOS/M1+ unability to bind server at times)
    - clients transmission (when clients transmit back to server/host):
    - fixed/updated server's client mixer - each client receives its own specific audio w/o its own audio included
    - added possibility for each client to transmit Unity Microphone
    - demo scene:
    - added Unity Microphone transmission option to clients

    ----------------------------------------------------------------------------------------------------------------------

    this should now allow complete 'chat room' like scenes to be built, hope it can useful ~!
     
  9. yujiet

    yujiet

    Joined:
    Feb 13, 2018
    Posts:
    2
    super cool! does it work with iOS too?
     
  10. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,302
    Hi, yes, everywhere where NetCode runs / except WebGL currently /
     
  11. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,302

    -~~`
    December '23 SALE `~~-

    - For two weeks - until about ~28th - the asset is 40% OFF

    Don't forget to download demos and enjoy the holidays - \o|
     
  12. UnitDan

    UnitDan

    Joined:
    Feb 4, 2019
    Posts:
    12
    Hi. Thx for your asset. Can you say what latency should I expect If all ist setup correctly, around 20ms?
     
  13. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,302
    Hi thanks ! This currently doesn't use better (2D only) latency I realized - it uses full spatialization only so there's always additional overhead -
    Will get to adding 2D only client after the new year - with an option in demo scene; - but can't promise any ETA
    (as for precise ms I never measured it but it should be in ballpark over localhost interface then)
    Hope this answers the question !
     
  14. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,302
    managed to prepare an update, but it's not submitted yet ( demo builds are not yet updated, too)
    there was a bug applying volume for clients, and the new "2D mode" will help with latency
    it's still little bit more than OSC used in AudioStream, and this should be around 1oo's of ms rather than 20ms in most cases most likely

    there is one server CPU optimization which will be done later still pending,
    and if you were asking about latency mainly because of client's microphone: this will definitely help, but for better latency than default Unity Microphone, there's no other way than to use something like AudioStreamInput2D from AudioStream asset
    [it should be easier and probably much less work to just use both assets if/when needed compared to submitting a new store package which would integrate these two currently, too, but we'll see]

    thanks && happy new year everyone.

    ===============================================================================
    1.2.1 122o23

    Host/Server:
    - added new toggle to AudioStreamNetCodeServer 'use2DRecevingAudioSources':
    - when On all receving AudioSources (clients and clients' audio on server) are in 2D mode which improves latency over network - see Docs for more
    - fixed a bug with 'All clients mix audio volume/monitoring volume' where volume/monitoring volume was not applied correctly for clients' audio mix

    - demo scene:
    - added a toggle 'Use lower latency mode (2D only receivers)' for the above which is available before launching host/server
    - this applies only to receivers - original signal can still be spatialized - run at least two separate client instances demos to see how this behaves
     
  15. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,302
    update sent for review..

    - please note that demos download links will be updated on the store page only once this is live
    ~~