Search Unity

  1. New Unity Live Help updates. Check them out here!

    Dismiss Notice

AudioStream - an audio streaming solution for all and everywhere

Discussion in 'Assets and Asset Store' started by r618, Jun 19, 2016.

  1. unityvisualstudio2018

    unityvisualstudio2018

    Joined:
    Feb 8, 2020
    Posts:
    2
    Thanks for the reply.
    I will share the streaming URL once I get permission from the team.
    My use case for audio streaming is
    Users in my application will listen to the live streaming of audio. My current source of streaming is Url and the url is generated with the help of the streaming app Wowza. Live streaming of audio is working When I am running the Wowza generated Url in the VLC media player.

    Thanks.
     
  2. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    send me the link (feel free to PM) once you have it - I'll check the format / if its'possible to use it
     
  3. TuringSense

    TuringSense

    Joined:
    Sep 21, 2015
    Posts:
    7
    Hi @r618 , I found out that my app defaults to earphone audio when playing any Audio Clip. I would like the default audio to be through my phone speaker, and to be able to let the user switch to a bluetooth speaker or Ear Pods . Can AudioStream can help in this case? Thanks!
     
  4. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    Hi, the output selection is more restricted and only changeable by user on iOS:

    Tech House Guy Gerber - Circoloco Radio 118.png

    I could display this sheet from the plugin but it's not implemented currently - you can access this by pulling control center:
    Screenshot 2020-02-13 at 09.47.31 m.png

    moreover, when application is in recording mode (Unity has setting for this in iOS player), the earpiece/speaker can be selected:
    Screenshot 2020-02-13 at 09.54.59.png

    BT outputs/inputs are tied together by default (when BT mic is selected for recording, the connected BT headset output is activated automatically) - this can be overridden to a degree by the plugin, and by Unity with 'Override speaker' setting for iOS Player
    you can change outputs between speaker/phone at runtime when in recording mode with the plugin by changing the session, but it needs to be called manually (it's not in the demo I think) and usually the defaults in iOS Player settings work for most users for the lifetime of an application

    Hope this helps!
     

    Attached Files:

  5. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    I'm leaving here link of a testing APK build for Android I did in hopes to compare Unity/direct FMOD playback latency:
    https://www.dropbox.com/s/jfnkq4xx2vi4j6m/PlayAudioLatencyTest.zip?dl=1

    unity audio settings are set to Best Latency, the clip is trimmed at the beginning to start immediately
    first button plays via AudioSource
    second via fmod with DSP buffers set to the values below
    you should try to enter as small values as possible and press 'Update DSP buffers' ( e.g. 128 / 2 on my testing phone, defaults are populated on start )
    there is very small noticeable difference, but I can't tell how much this means also compared to other assets

    -- would require much of a rewrite too if gone via FMOD route and abandoned Unity audio/clips, so leaving this here so it might be useful for comparing for an Android developer
    but overall the latency on Android is not the best to say the least and FMOD just uses what's available
     
  6. TuringSense

    TuringSense

    Joined:
    Sep 21, 2015
    Posts:
    7
    Thanks, apparently my app defaults to earpiece output because the recording mode was set on my app. I tried setting the 'Override Speaker' option for iOS Player, but this prevents the Audio from playing on my TV when my iPhone is connected to it. So I need better solution to switch between audio output in my iOS app...

    Will AudioStream plugin support this feature for iOS any time soon?

    Another question- What does this plugin add to Unity-iPhone application if we exclude this feature?

    Thanks!
     
    Last edited: Mar 5, 2020
  7. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    What happens when the app is not in recording mode and you pull up current outputs in control center ? Does is allow you to switch to your TV (I suppose it's via AirPlay) ?
    If it doesn't I'd guess the default session set by Unity is not the correct one (see also below *; unfortunately I can't test with a TV with AirPlay support myself)
    - 'Override' - as the name suggests - just overrides whatever current setting OS thinks is optimal for given situation -


    * This feature (updating and changing AVAudioSession) is implemented in few objc/++ source files (as iOS plugin in Unity) and C# script
    Apart from that iOS FMOD native libraries + accompanying AudioStream scripts, demos and resources are included in Xcode project by default
    You can isolate it and use only the above sources if you remove everything else - except probably AudioStreamInput_iOS and demo which use it and possibly some common C# support script/s -
    but the whole functionality is isolated from FMOD and Unity
     
  8. TuringSense

    TuringSense

    Joined:
    Sep 21, 2015
    Posts:
    7
    My app requires recording mode to be set. Regardless, I was able to connect to Airplay and Speaker through the control center. It did not seem to be able to detect and play audio from Bluetooth speakers and AirPods though, and I had to use this hack to do so.
    Does the AudioStream support something similar to this- maybe one or two files from the plugin that I had to include in my app in order for the app to be able to detect and play Audio on Bluetooth devices through the Control Center?

    I'd prefer to eventually do the audio switching through Unity UI, but of course Control Center is a nice temporary solution for now...
     
  9. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    Right, yes it supports playing on BT connected devices - it's little bit buried in the description, but the screenshot of the recording scene running on device is there too:
    Screenshot 2020-03-06 at 05.46.06.png
    (the device 1 is AirPods)

    For just playback alone (e.g. in a separate scene from recording) you'd have to call the included AVAudioSession wrapper and update the session (it's enough to do this once somewhere near start) - afterwards it's possible to select BT connected device from the OS outputs selection and Unity will play all audio on it
    (- this is not included by default in AudioStreamBase, but it's worth adding I think - so it's not automatic right now - sorry! -)
     
  10. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    btw that 'hack' is not that horrible really - there is cca no other way to update audio session Unity is using on iOS otherwise
    (it's "just" a matter of doing it cleanly and in a way friendly to the user and to engine)
     
  11. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    I just submitted small update:

    v 2.3 032020 Corona+

    Updates/fixes:
    - AudioStreamOutputDevice : moved automatic output devices changes notifications to separate component (see below)
    - AudioStreamInput* : moved automatic intput devices changes notifications to separate component (see below)
    - AudioStreamInput2D : fixed incorrect handling of Mono output, corrected input signal retrieval so this further improves latency, Speex resampler output is improved as consequence, too
    - AudioStreamInput2D_iOS : updated same as above
    - AudioStreamBase, AudioStreamLegacyBase: automatically update audio session at start to include Bluetooth devices as possible system output on iOS

    New Components:
    - AudioStreamOutputDevicesChangedNotify : optional component can be placed in the scene to receive system's output devices changes updates
    - AudioStreamInputDevicesChangedNotify : optional component can be placed in the scene to receive system's intput devices changes updates


    So separate notification components (mainly useful when using just mixer plugin but want to be notified of system changes) and rather nice input/recording fixes

    However, I noticed that AudioStream - main streaming components - stopped working on iOS some time ago
    Legacy still works with the above improvements, so please use that if you don't need secure HTTPS transport for the time being
    Please contact me otherwise if you need streaming & playback on iOS to speed things up - I'll be most likely replacing FMOD - for streaming playback only, everything else - such as recording / device handling - is fine with it
    But streamed decoding with current approach is borderline unusable, especially when all supported various codecs are involved
    - I intent to use NAudio/cscore most likely, but am open to suggestions if there are any ( I personally am a fan of BASS/.NET but it's not free for commercial use unfortunately so I won't be including support for it by default )
     
    Last edited: Mar 8, 2020
  12. TuringSense

    TuringSense

    Joined:
    Sep 21, 2015
    Posts:
    7
    Thanks @r618 ! Bluetooth Input/Output works like a charm.
    By the way, is it possible and do you have any plan to support Audio Output selection from iPhone Control Center any time soon? Would be awesome to be able to switch to one of those devices in app as well!
     
    Last edited: Mar 10, 2020
  13. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    It's.. problematic; iOS doesn't provide user space with all available outputs and all interaction is handled on the OS presented selection sheet exclusively - so I can't just display outputs directly in the app (like when recording mode)
    The maximum I could do is to call device picker programmatically, but at that point there's little difference between this and users actually pulling control center by themselves
    It requires manipulating Unity View from inside the plugin - it's not impossible, but complicates things a bit - and due to the above not entirely worth it
    I recommend sticking to iOS way of handling things for the time being - I hope it's not that big of a problem
     
  14. JF_SuperSplendide

    JF_SuperSplendide

    Joined:
    Mar 10, 2020
    Posts:
    1
    Hello, I'm looking to understand if this is love : is your plugin the one I've been looking for?
    I wish to output 8 different audio sources from unity, going each to their own speaker (passing through a sound card in order to route to the 8 speakers I suppose). This is not for surround sound. Each audio source will play their own sound.

    What do you reckon?
     
  15. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    hi, it should work without problems when you set a custom mix matrix for each sound
    - have a look at the demo application (downloads are on the store page) and 'MultichannelOutputDemo' scene where a mono sound is directed on selected channel on selected output and simple automatic custom mix matrix is used
    you will have to setup your scene similarly and each sound separately in it
    (the matrix is simple for mono sounds - the one from the demo can be reused - but you can e.g. mix multichannel sounds to a single output channel with different one)
     
    Last edited: Mar 12, 2020
  16. paulsz

    paulsz

    Joined:
    Mar 25, 2010
    Posts:
    26
    Hello,

    I have a really weird issue, sorry if it came up in this thread but I haven't found that as I browsed through it.

    Only on some machines, when using an audio source that has a AudioSourceOutputDevice.cs component from Audiostream attached, the sound is played with a high pitch even though the clip is not pitched at all. If I disable the AudioSourceOutputDevice component, the sound plays normally without the pitch but the output device source is no longer the one I previously set (the one I want).

    I've been down in the dumps for more than a week trying to figure this out and it's eroding my morale :)

    This issue happens also in the demo scenes.

    Also on machines where this issue happens I get spammed with log messages like this

    PCMReadCallback 1222032920 underflow - requested 38400 while having 37888 (providing 37888), time: 5352.08533333333
    UnityEngine.Debug:LogFormat(String, Object[])
    AudioStream.FMODSystemOutputDevice:pCMReadCallback(IntPtr, IntPtr, UInt32) (at Assets/Vendor/AudioStream/Scripts/AudioSourceOutputDevice/FMODSystemOutputDevice.cs:85)

    On machines that do not have this issue, this message also appears but much more rare (like 40, 50 times as opposed to over 1000).

    Also I keep geting this error even on the PC where everything works fine and I don't have installed FMOD studio:
    FMOD Studio: The FMOD Studio project path must be set to an .fspro file.

    Is there a way to avoid this error without having to install FMOD studio separately ?


    I used unity 2018.3.5.f1 and after that I upgraded to 2018.4.19f1 so I can update to the latest fmod package and latest Audiostream package. The results were the same.

    It is very weird that this does not happen on my pc, it works perfectly fine but only happens on my laptopt and a colleague's laptop.

    Thank you in advance for your quick replies.

    If there is any other info that I haven't specified that you need, please let me know.

    Best Regards,
    Paul
     
  17. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    Hello Paul,

    first for FMOD Studio:
    you can delete most of the stuff from FMOD package after importing it (or alternatively select just needed stuff in the import dialog)
    the needed parts are (with package's default import location):
    'Plugins/FMOD/lib'
    - here you can select only platform you want support for
    and
    'Plugins/FMOD/src/Runtime/wrapper'
    you can delete everything else FMOD package related from the project, and all FMOD Studio messages will vanish immediately ;o)
    [ this is mentioned in README also if you want to have a look ]

    for the main issue:
    I will have a look at your device first - please run editor on problematic machine, select DEBUG log level on the component (it's in [Setup] section) and send me the console log ( you can open it directly from console in editor:
    upload_2020-3-17_23-20-8.png
    )

    the sound might play with severe problems due to performance, but the needed info should be there
    (you can PM the log if needed)

    Thank you !
     
  18. paulsz

    paulsz

    Joined:
    Mar 25, 2010
    Posts:
    26
    Thank you for the very quick reply,

    We managed to solve the issue by setting the Default Speaker Mode to Mono (in Unity Editor: Edit->Project Settings->Audio)

    Unfortunately we didn't have time to do the logs as you requested but if another issue appears or this returns we will promptly provide the logs.

    Thank you again for the assistance.
     
  19. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    Hm, I guess somewhere somehow is wrong # of channels set - by syncing unity default with the problematic output you fixed it that way; if you can stick with unity output being mono then it's good enough solution for now
    : will have to test this
    Logs would still help though )

    Just to be clear : are you using latest version currently on the store ? - should be 2.3. released cca week ago

    Anyway, thanks for letting me know !
     
  20. paulsz

    paulsz

    Joined:
    Mar 25, 2010
    Posts:
    26
    We are not using the latest version of Audiostream since we have to use Unity 2018 3.5f1 and the new FMOD requires Unity 2018 4.18 or higher. Anyway, if I ever get a chance to get the logs I will definitely send them to you asap.

    Thanks again.
     
  21. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    hm the '2018.4.18' or higher is rather specific - I would expect them to support at least LTS - which means all 2018.4 releases - but I might be wrong
    I recommend upgrading to LTS - should not be difficult going from .3 release - but I know these things can be problematic - just a suggestion -
    I was fixing wrong mono handling in latest - but for input, not for this component - there are no functional changes for output device component going couple of versions back - will look at how to replicate this if possible
     
  22. kotor322

    kotor322

    Joined:
    Mar 27, 2017
    Posts:
    5
    Hello there, I have problem with reapiting lag with live stream, when after starting stream in first "soundItem" I create another "soundItem" and put link to file from PC. After this I start audio from PC, but previous item with live stream has reapiting lag(like last 2 seconds repeats alltime and stream is not continue). When I put another live stream, then previous works fine.
     
    Last edited: Mar 24, 2020
  23. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    Hi @kotor322 , which version are you using ?
    In one of the previous ones a local file playback went through wrong codepath and didn't use local file system
    Are you on latest (currently 2.3) ?
    If yes give me some time to replicate and fix it
    Thank you
     
  24. kotor322

    kotor322

    Joined:
    Mar 27, 2017
    Posts:
    5
    Yes, I use latest version and Unity 19.2.21f, also testing it with AudioStreamMinimal recently and it's works fine, but I need AudioStream because must have my audiosource with customize range and controlled sound curve. And problem with live stream when i set in another "audiofile" audio from pc, but audio from pc works fine.
     
  25. kotor322

    kotor322

    Joined:
    Mar 27, 2017
    Posts:
    5
    And also when it happend I don't have any errors or event warning, if I have at least error when it happend, I just make replay "audioitem" with this problem.
     
    Last edited: Mar 24, 2020
    r618 likes this.
  26. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    Thanks, I'll investigate

    btw meanwhile: you can still use AudioStreamLegacy ( it sits in 'Scripts\AudioStream\_LegacyUsingFMODNetworking' ) for playback from network <- if you don't use/need HTTPS that is
     
  27. kotor322

    kotor322

    Joined:
    Mar 27, 2017
    Posts:
    5
    Thanks, I try AudioStreamLegacy and it's works. Also I had reapiting lag with AudioStream in live streams and videos from PC when I use OpenFileDialog for selecting file. I temporary solve it with pause "audioitems" when panel was opened and restarting "audioitems" when OnApplicationFocus is triggered. But in AudioStreamLegacy I don't have this problem too. I hope you fix this in AudioStream, because I need HTTPS not asap, but eventually.
     
  28. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    Thank you for the report

    I want to replace current network stream implementation - see also: https://forum.unity.com/threads/aud...ll-and-everywhere.412029/page-12#post-5562682 , needs some time to be done

    the issue you were getting with opened OpenFileDialog is most likely because main/UI thread is paused at that time and current component gets its updates from it - one more reason to try different approach -

    I'll update thread once I have something usable,
    Thanks !
     
  29. ECHO-425

    ECHO-425

    Joined:
    Feb 2, 2018
    Posts:
    3
    Hi there,

    I was hoping that someone could tell me if this asset pack could work for my project. Specifically, we are having a producer client in Unity which will stream audio and generate data for when certain lights flash to the audio being streamed, which would need to be streamed out with the audio in sync. We would then stream that through ShoutCast or IceCast. That would then be received by a different Unity consumer / client, whom would receive the stream and associated metadata to reproduce the music + light show on their end, in sync.

    Does AudioStream enable encoding/decoding of audio with attached metadata? This would need to be done with a high accuracy < 100ms.

    I'm a bit new to the subject of audio streaming, but looking for a solution and appreciate any help that I can get. Thank you!
     
  30. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    Hi, you have access to all audio as it's being produced or played back, but you would have to code all support for any custom metadata - inserting it into PCM stream and reading back (and skipping them on audio output)
    This can be done either on current network transport level (using NetMQ packets), or directly in OnAudioFilterRead callback/s. This way you can have <100 ms resolution, but note that since it would break any compression you would have to use PCM encoding only which would consume considerable bandwidth -
    The final latency when going through Icecast (difference between producer and final client output) will be very high though - you would have to use provided server/client components (and having extra client application) to skip Icecast to reduce it - if possible
    If you have an Icecast instance running you can test 'IcecastSourceDemo' scene in the demo application (download is on the store page) with it

    Hope this helps!
     
  31. kotor322

    kotor322

    Joined:
    Mar 27, 2017
    Posts:
    5
    Hi there, any news about AudioStream update? It wiil be really cool to have this feature asap for my project.
     
    Last edited: Mar 26, 2020
  32. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    868
    Hi, sorry to keep your hopes up, but this won't take days - this is range of weeks / months (I can't work on it exclusively - unfortunately!)

    If you're about to release definitely don't wait just for this

    Sorry about that !
     
unityunity