Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

OnAudioFilterRead event parameters?

Discussion in 'Developer Preview Archive' started by AFrisby, Dec 29, 2011.

  1. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    The new OnAudioFilterRead event appears to be a little underdocumented; I'm implementing a streaming audio decoder to hook up to that; such that the sample buffer from the stream is read out each time it is called.

    As far as I can tell; it is supposed to be called at 50Hz; I'm guessing, the 'data' parameter contains (44.1Khz / 50Hz * channels) data points; but I can't seem to determine from the documentation:

    • The actual sample rate (22,050? 44,100?) - since I'm generating the data; there's no AudioSource sample above it. (per "a clip isn't attached to the audio source this filter will be 'played'.")
    • How the number of data points is calculated (do the channel count multiply the number of data points; are these then interleaved or consecutive blocks? what happens if a parent audio source over/underruns, will OnAudioFilterRead pass multiple sample frames? [and are they interleaved with channel data?])
    • How a non-1 channel count works - for stereo: first channel left, second right?

    If anyone knows a little bit more about this function, some clarifications would be appreciated.

    Thanks!

    Adam
     
  2. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    After some testing; I can report:
    - It does look to be 44.1Khz by default
    - Its called every 23ms optimally; and returns 23ms worth of data at a time.
    - Unity passes 2048 chunks into 'data' at a time, with a channels = 2 (44.1Khz / 1024 = 23ms)
    - Not yet sure if it's interleaved or chunked though. (Update: It's interleaved. %0 = Left, %1 = Right)
    - Not yet sure how it behaves when over/underruns occur.
     
    Last edited: Dec 29, 2011
  3. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    Got it working eventually - it's fairly straight forward once you've figured out the parameters. (see above!)

    However; I have a follow-up question:

    How can this be used as a audio clip; so that positional audio/etc can be assigned to it?

    I gather I could control the gain manually by sourcing the distance from the audio listener; but I'd prefer to be able to use all the benefits Unity provides here (e.g. reverb zones, editor control of falloff, etc.)
     
  4. soren

    soren

    Joined:
    Feb 18, 2008
    Posts:
    123
    The default samplerate is dependent on the platform (typically 44.1k on OSX, and 48k on windows) . You can query (or set) it with AudioSettings.outputSampleRate. The buffersize (default 2048 as you found out yourself) is also accessible thru AudioSettings.Set-/Get-DSPBufferSize().
    Channels are interleaved and if you underrun the buffer, it will just stutter as no data is fed to the hardware.
     
    elenzil likes this.
  5. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    Thanks - any way to input this so the clip mechanics can apply? (Mainly interested in positional falloff)
     
  6. soren

    soren

    Joined:
    Feb 18, 2008
    Posts:
    123
    Yes, just a attach the script (implementing OnAudioFilterRead) to an audiosource with no AudioClip attached and .Play() that. Then your filter is treated as a clip and all the source properties (rolloff curves, pna level etc.) will be applied.
     
  7. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    Seems like the 3D positional properties are ignored - only the 2D Pan, and the master volume slider seem to have an effect.

    Edit: sorry - pan doesn't work either.
     
    Last edited: Dec 31, 2011
  8. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    Actually I have noticed something odd here; it appears that the sampling rate(?) varies when .Play() is used instead of the 'Play on awake' inspector field.

    Seems like it might be something like 96Khz when .Play is used, but 48Khz on "Play on awake" -- both with no audioclip loaded. Doing some more testing now and will see if I can narrow it down specifically. (Pitch and speed is doubled from my perception?)

    Edit: Nope; wrong - looks like the channels count switches from 2 to 4.
    Edit #2: Dont know why the above happened, seems to have mysteriously stopped - but the audio issue was still present, so I attached a System.Diagnostics.Stopwatch class to the call, and checked how regularly it's called -- funny results actually. When Play On Awake is set - it calls every 19-20ms. When it's not set - but I use .Play(); without an audioclip; it calls every 0.07ms[!] - this looks like some sort of bug?
     
    Last edited: Dec 30, 2011
  9. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    Another odd issue - this one seems to be a bit more of a showstopper.

    Unity crashed while working on this (seems to be a common occurance); but when it came back - the script's OnAudioFilterRead function is no longer being called at all and the VU meter is missing from the inspector; but otherwise the code is unchanged. Any ideas?

    Edit: Solution found - Removed a call to 'AudioSettings.SetDSPBufferSize(1024, 2);' inside the Start function and it now functions. (This was to get around the mysterious channels count jumping to 4 issue which periodically is occuring.)
     
  10. soren

    soren

    Joined:
    Feb 18, 2008
    Posts:
    123
    If you're seeing crashes I would love get a repro project. SetDSPBufferSize() in Start should be stable - if not, report a bug so I can have a look at it.

    Thank you,
    Søren
     
  11. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    I'll see if I can push my scene up - unfortunately I've begun integrating this with our other projects; so the repro project is several GB now. ;)

    That said, I have found a couple of repro'able bugs:

    - AudioSettings.GetDSPBufferSize() is a heisen-function; the act of measuring seems to change the channel count; before it is called, the channel count is 2. After calling it becomes 4.
    - AudioSettings.SetDSPBufferSize() will cause a MonoBehaviour with OnAudioFilterRead to stop being called entirely.

    I'm also getting very frequent crashes in the editor when using OnAudioFilterRead; although that's harder to repro. I suspect it might actually be Mono's fault; I know some of the buffers I'm handling can be large and there may be a leak and Mono exhibits bad behaviour when running low on memory.
     
  12. soren

    soren

    Joined:
    Feb 18, 2008
    Posts:
    123
    I can't reproduce this, and I found it hard to believe because GetDSPBuffer is just reading from 2 integers and returning them. Maybe you introduced a heisenbug yourself ;). Make sure to lock then channel count variable when assigning and showing it on the main thread (don't use Debug.Log on the mixing thread!)

    This is fixed in the next version (filters were bypassed after resetting DSP buffer sizes). Thanks for finding!

    If you can repoduce the crashes and list the steps here, that'd be awesome

    cheers,
     
  13. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    I found the issue - the out parameters in GetDspBufferSize can be edited in the Mono behaviour which is equivalent to calling SetBspBufferSize with the changes, sort of.

    Basically; you are passing a reference to an internal value into the Mono script where I have assumed I'm playing with a copy of it; not the source internal value.

    My code was this:

     
  14. soren

    soren

    Joined:
    Feb 18, 2008
    Posts:
    123
    I'm sorry but that doesn't makes sense ... like at all :). Can you send me the project showing this weird behaviour?
     
  15. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    Yep - but it'll have to be a bit later (12:30AM here now! ;))

    I guess I upload it as a "Report Bug" and put your name in the description?
     
  16. soren

    soren

    Joined:
    Feb 18, 2008
    Posts:
    123
    I looked into your project. And you have a Channels parameter which is serialized/stored as 2. Then you overwrite it with the number of buffers!? with GetDSPBufferSize( bufSize, numBuffers), which is 4 on windows.
    I'm not sure what you're trying to achieve? . But this is not a Unity bug afaics.
     
  17. clockwork_fromage

    clockwork_fromage

    Joined:
    Aug 2, 2011
    Posts:
    13
    ive been playing with OnAudioFilterRead and im trying to figure out how , if possible , to set the frequency of a data packet , id like to see if unity can make frequency modulation on its own

    currently i just try to manipulate the data variable directly but that only gives me a static buzz every time its the value is changed , could you help me out ?

    Code (csharp):
    1.  
    2. public var gain : float;
    3. var tick:float;
    4. var tok:float;
    5. function OnAudioFilterRead(data:float[],channels:int) {
    6. for (var i = 0; i < data.Length; ++i)
    7. data[i] =gain;  // data[i] * gain;
    8. tick++;
    9. gain=Mathf.PingPong(tick,tok); }
    10.  
     
  18. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    You can do synthesis with this fairly easily - the data float[] is just a collection of PCM-encoded analog signals (on the amplitude domain, not frequency.)

    What you have written there is going to produce a very high frequency triangle wave, if it produces anything at all. If you want to generate a specific frequency sound; you need to do a little reading on the basic building blocks of digital audio 'emitters' (square, triangle, sine, sawtooth, etc wave generators).

    Given all PC audio tends to be standardised around a very common set of patterns; you should in theory be able to cut and paste code written for something else into Unity and feed the "output line" into the "data" buffer as it is fetched.

    In the projects I've touched this on so far; I've built a small buffer class into Unity first (a queue containing data blocks to be fed - which are dequeued and picked up in the OnAudioFilterRead event), which is topped up by another thread to keep the OnAudioFilterRead event always returning quickly.
     
  19. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    215
    Hi Soren,

    I'll need to take another look at the code; been a few weeks since I touched that.
     
unityunity