Audio How many audio samples does AudioSource.GetSpectrumData() analyze to create the spectrum?

For reference, GetOutputData receives an array with size equal to the number of samples we want, so we can easily get a certain number of seconds based on the value of AudioSettings.outputSampleRate. Likewise, OnAudioFilterRead gives us an array of a certain size, so we can easily know which time period it corresponds to.

However, GetSpectrumData docs only say "Provides a block of the currently playing audio source's spectrum data."

What does "currently playing" mean? How many audio samples is this? How can we know how much time it corresponds to, based on outputSampleRate?

We're looking for an official answer if possible. We wanted it to be exact. Thanks!

Cheers!
Daniel Lobo

 

GetSpectrumData fetches the spectrum for RIGHT NOW and not for a time frame.
The spectrum data is a bunch of frequency data, so the first sample will be the lowest frequency and the highest sample will be the highest frequency.
The number of Samples fetched can be thought of like "resolution" in graphics.
Fetch a small number of samples, and your data will be more blurry. Fetch a high number of samples, and your data will have a high resolution, with the draw back being less efficiency.
The amount of samples you fetch always spans from the lowest to the highest frequency. The more samples you have, the more fine grained detail you get, because each sample represents a smaller change in frequency so you can differentiate frequencies to a higher level.

Here is probably a better explanation:
https://answers.unity.com/questions/157940/getoutputdata-and-getspectrumdata-they-represent-t.html

 

Hello and thank you for your answer, but unfortunately (or fortunately) we already know all that. We know how the spectrum works and how it is obtained from the FFT, what the frequency ranges mean, etc.

To calculate a spectrum Unity needs to use audio samples. A certain number of them. "Right now" is not a real time frame and does not help (that is already in the docs, in fact). It has to be something well defined in Unity, and this is what we need to know. As the audio thread probably has no definition of what a frame is, we assume it uses a certain number of audio samples, depending on the outputSampleRate, but don't know how many...

EDIT: For example, if we were to calculate the spectrum manually, we would call GetOutputData with a certain number of samples (therefore defining a time frame), calculate the FFT and the real values from that. Unity has to do something like that internally for GetSpectrumData, but it never tells us how many samples it uses.

 

It just takes whatever is currently being played in the buffer to satisfy windows size you pass in
There's nothing to be explicitly set samples size wise even while creating FMOD DSP effect which Unity uses for this

 

Hi and thank you for your answer, @r618.

We don't need to set the samples size, but only to know what that size is exactly. Is it dependent on the DSP Buffer Size option in the Audio settings, then? ("Best latency", "Best performance", etc) Is there any way to know what this size is or is it constantly varying?

At 60fps we are observing several consecutive frames (2 or even 3 at times) with the exact same spectrum data, for example.

Also, you meant to say "to satisfy windows type" instead of "size", right? Can the different window options change the used number of samples in any way?

Thanks,
Daniel

 

I know - better phrasing would be you don't have access to FMOD playback buffer from which the samples are read - better ?

Very likely - but it's hard to tell in what way exactly for GetSpectrumData calls. You'll get deterministically sized chunks of audio buffer in OnAudioFilterRead based on this setting, and their size does not change.

Yeah - that's hard to measure wrt audio buffer since this is framerate dependent when called from Update.

Nope - I really meant window size - the number you pass as buffer size to GetSpectrumData call

I wouldn't think so, but my knowledge of fourier transform and its FMOD implementation is limited - ]

If you want to better understand what's going on I would maybe play cyclic signal with known size and use simplest FFT type FFTWindow.Rectangular, see here: http://www.fmod.org/documentation/#content/generated/FMOD_DSP_FFT_WINDOW.html

but since it will be framerate dependent you'd likely need to balance window size, audio buffer size and framerate ( Update frequency ) to get intuitively 'correct' results

 

Thank you very much for the detailed answer.

Good thinking regarding OnAudioFilterRead. We might check that and see if we can learn something.

Sorry, I wasn't thinking that the FFT window size corresponded to the number of samples passed to GetSpectrumData, but maybe they are "equivalent"...?

We greatly appreciate your explanation! This puts us on the right path. Cheers!