Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Unity 3 Audio Spectrum Analysis

Discussion in 'Made With Unity' started by klaymator14, Aug 28, 2010.

  1. klaymator14

    klaymator14

    Joined:
    Feb 3, 2009
    Posts:
    153
  2. Jessy

    Jessy

    Joined:
    Jun 7, 2007
    Posts:
    7,325
    Looks like a mess!

    SYNC TO VBL

    :!:

    Why is it changing from red to green?!

    That music sounds completely broken! Does a person really listen to this? :eek:
     
  3. klaymator14

    klaymator14

    Joined:
    Feb 3, 2009
    Posts:
    153
    :oops: ... sorry about that. I can update it so it will Sync TO VBL, want me to change the music? I only picked something that you could actually see the changes being made.
     
  4. Jessy

    Jessy

    Joined:
    Jun 7, 2007
    Posts:
    7,325
    Who cares what anybody else wants! Let us all complain while you make something you think is good!
     
    jister likes this.
  5. klaymator14

    klaymator14

    Joined:
    Feb 3, 2009
    Posts:
    153
    Update: Ok, so I added Sync to VBL and I changed the music. Hopefully it's less annoying now. Anyone know a way to average these values so it's no so erratic? Thanks :D

    I didn't think it was that good, I was just showing the spectrum analysis. Since no one else has posted anything about it, I thought someone would think it was interesting... but I guess I thought wrong :/
     
  6. stefano.cecere

    stefano.cecere

    Joined:
    Jun 24, 2010
    Posts:
    28
    thank you!
    finally a first little demo/example of spectrum analysis
    it's my most wanted features needed for U3.. and i didn't know it was already into v3 until today!

    i have it.. but the docs are really a few words sentence!

    anyway (on imac 2.8) i see several bad rendering with U3 betas..
    v2.6 is perfect... so i hope there's still some bug somewhere

    pelase keep on with sharing prototypes!
    some code would be welcome also!

    ciao
    stefano
     
  7. klaymator14

    klaymator14

    Joined:
    Feb 3, 2009
    Posts:
    153
    :) Awesome, so someone does appreciate it! I'd be glad to share some code, which isn't the cleanest looking code, but it works!

    Placed on Main Camera, with an audio source in the scene playing the music:
    Code (csharp):
    1.  
    2. var Audio : AudioListener;
    3.  
    4. var colorStart : Color = Color.red;
    5. var colorEnd : Color = Color.green;
    6. var duration : float = 1.0;
    7.  
    8. var ball : GameObject;
    9. var ball2 : GameObject;
    10. var pointLight : Light;
    11. var channel : int;
    12. var numSamples : int;
    13. var freq : int;
    14. var number = new Array ();
    15.  
    16. function Update () {    
    17.  
    18. var number = Audio.GetSpectrumData(numSamples, channel, FFTWindow.BlackmanHarris);
    19. ball.transform.localScale.y = 5 + number[freq] * 5;
    20. ball.transform.localScale.x = 5 + number[freq] * 5;
    21. ball.transform.localScale.z = 5 + number[freq] * 5;
    22.  
    23. Camera.main.fieldOfView = 60 + number[0] * 15;
    24. pointLight.light.intensity = 0.7 + number[freq] * 1.2;
    25.  
    26. var lerp : float = Mathf.PingPong (number[freq], duration) / duration;
    27. ball2.renderer.material.color = Color.Lerp (colorStart, colorEnd, lerp);
    28. }
    29.  
     
  8. Vectrex

    Vectrex

    Joined:
    Oct 31, 2009
    Posts:
    249
    hey, that's really cool. Does Unity3 also allow you to write to the audio buffer like FMod does?
     
  9. DavidPlans

    DavidPlans

    Joined:
    Jul 29, 2009
    Posts:
    39
    Very nice!

    Hey has anyone used Prime31's AudioRecorder plugin to get audio fed to Audio.GetSpectrumData?

    I'm trying to look into it, but it looks like it's not trivial.

    David
     
  10. kenlem

    kenlem

    Joined:
    Oct 16, 2008
    Posts:
    1,630
    Why would you need a plug in to get that data?
     
  11. kenlem

    kenlem

    Joined:
    Oct 16, 2008
    Posts:
    1,630
    I can't download your project at work.

    What are some good settings for:

    var channel : int;
    var numSamples : int;
    var freq : int;
     
  12. Jessy

    Jessy

    Joined:
    Jun 7, 2007
    Posts:
    7,325
    It will be dependent on material. Run your audio through a spectral analyzer of some sort to get an idea of what you'l need. (Logic and Soundtrack would be my choices.) Channel should be easy enough to find; just pick the one that lights up prettiest when you hear what you want to trigger something. Freq will be in the middle of the hump that corresponds to what you're listening for. numSamples will have to be bigger, the lower freq is.

    I don't know anything about window functions but Wikipedia seems to have a good entry, so I'll start there when I have some time to read up.
     
  13. klaymator14

    klaymator14

    Joined:
    Feb 3, 2009
    Posts:
    153
    I use either 0 or 1 for channel, try both and test to see which works out better for you. Number of samples must be a power of 2, I have mine set at 256. Frequency you should just change values at runtime and adjust to your liking. Hope that helps?
     
  14. DavidPlans

    DavidPlans

    Joined:
    Jul 29, 2009
    Posts:
    39
    what I was saying is how to get this kind of thing to get audio spectrum in -real time- from incoming audio (mic), not as it is now, from playing buffers of audio...
     
  15. stefano.cecere

    stefano.cecere

    Joined:
    Jun 24, 2010
    Posts:
    28
    i'm about to look into the same topic.. any news?

    s
     
  16. sonicviz

    sonicviz

    Joined:
    May 19, 2009
    Posts:
    1,044
    Last edited: Sep 30, 2010
  17. sonicviz

    sonicviz

    Joined:
    May 19, 2009
    Posts:
    1,044
  18. gl33mer

    gl33mer

    Joined:
    May 24, 2010
    Posts:
    280
    So, Audio.GetSpectrumData() - That's a .net class?

    All the unity classes I've been able to find are under AudioSouce or AudioListener.

    Is there more goodness under Audio.?? :)?

    Another thing. Loading the new .mod files off of WWW . It seems there is no built in way to do it. (There is one for Ogg). Though there is a way to load bytes or text. But that isn't straight forward. Maybe through the AssetBundles? I was wondering are the assetBundle functions no longer pro only? The manual doesn't state that they are pro only, so I'm wondering.

    Thanks
     
  19. gl33mer

    gl33mer

    Joined:
    May 24, 2010
    Posts:
    280
    Ok. I now see. BuildPipeline.BuildAssetBundle seems to be PRO only, so building the assetbundles would need a pro license but not reading them in?

    hmm...I'm wondering how difficult it would be to take a .mod file and load it up from a server.
     
  20. elbows

    elbows

    Joined:
    Nov 28, 2009
    Posts:
    2,499
    Thanks for the example script. Here is a script that will allow you to see the audio spectrum as bars. Im new to unity and I havent finished this properly, basically Im trying to make it clever enough to deal with a range of different number of samples without lots of the bars disappearing beyond the camera, but its not right yet and Ive run out of time so I shall post it as it. It isnt too bad if you have the number of samples set to 64 and the camera z position set to around -12.

    Create a prefab that is just a basic cube. Attach this script to the camera. Drag your cube prefab to Abar. Put your audio Source near to the camera. Set numSamples. Run.

    Code (csharp):
    1. var Audio : AudioListener;
    2. var numSamples : int;
    3. var numberleft = new Array ();
    4. var numberright = new Array ();
    5. var thebarsleft = new Array(GameObject);
    6. var thebarsright = new Array(GameObject);
    7. var abar : GameObject;
    8.  
    9. function Start () {
    10.     var spacing = 0.4 - (numSamples * 0.001f);
    11.     var width = 0.3 - (numSamples * 0.001f);
    12.     for(i=0; i < numSamples; i++){
    13.         var xpos = i*spacing -8.0f;
    14.         var positionleft = Vector3(xpos,3, 0);
    15.         thebarsleft[i] = Instantiate(abar, positionleft, Quaternion.identity); 
    16.         thebarsleft[i].transform.localScale.x = width;
    17.         thebarsleft[i].transform.localScale.z = 0.2;
    18.         var positionright = Vector3(xpos,-3, 0);
    19.         thebarsright[i] = Instantiate(abar, positionright, Quaternion.identity);   
    20.         thebarsright[i].transform.localScale.x = width;
    21.         thebarsright[i].transform.localScale.z = 0.2;
    22.        
    23.     }
    24. }
    25.  
    26. function Update () {    
    27.  
    28. var numberleft = Audio.GetSpectrumData(numSamples, 0, FFTWindow.BlackmanHarris);
    29. var numberright = Audio.GetSpectrumData(numSamples, 1, FFTWindow.BlackmanHarris);
    30. for(i=0; i < numSamples; i++){
    31.         thebarsleft[i].transform.localScale.y = numberleft[i]*30;
    32.         thebarsright[i].transform.localScale.y = numberright[i]*30;
    33.     }
    34. }
    What you will likely notice is that the spectrum data is much weaker once you get beyond the first few samples. You can compensate for this by using a larger multiplier as you go through the array of samples, so for example you could change the last few lines of code to something like the following (not a very sophisticated example, Im only just starting this voyage):

    Code (csharp):
    1. thebarsleft[i].transform.localScale.y = numberleft[i]*(20*(i+1));
    2. thebarsright[i].transform.localScale.y = numberright[i]*(20*(i+1));
    Also what you really need to do to make this stuff visually appealing is to smooth out the values quite a bit over time so that the bars arent flickering so wildly every frame.
     
  21. LoTekK

    LoTekK

    Joined:
    Jul 19, 2010
    Posts:
    136
    Increasing the multiplier isn't really the way to go, imo. What you should be doing is progressively increasing the intervals between samples as you get higher up the frequency spectrum. I don't have the "proper" explanation handy, but you can take a look at a typical graphic equaliser for a concrete example of this. Here's a shot of foobar2000's spectrum analyser:



    As you can see, the lower frequencies are spaced at smaller intervals than the higher frequencies (ie. 50-94hz, 620hz-1.2khz, 7.7-14khz).
     
  22. elbows

    elbows

    Joined:
    Nov 28, 2009
    Posts:
    2,499
    Well I know virtually nothing about the maths behind spectrum analysers, I was just going by what I saw when I played with this stuff today. As far as I know a logarithmic scale is often used for frequency so that a wider frequency range can be shown within limited space. Im not convinced this has any bearing on the issue of the amplitude being much lower for higher frequencies though. But yeah my multiplier thing is likely not the correct way to fixt this issue, if it even needs fixing, it was just a quick fudge to somewhat even out the looks of things.

    I think we might need more documentation on the data coming from unity with this feature in order to get this stuff right, Im not even sure what the range of frequencies are, and with my code Ive made an assumption about what numSamples means, and I havent played with the different window settings yet either. I think I might actually be happier is numSamples had lower minimum setting than is currently possible, 16 might do me for a range of uses but the minimum is 64.
     
  23. MSylvia

    MSylvia

    Joined:
    May 19, 2009
    Posts:
    42
    Thanks elbows, helped me understand what was going on in GetSpectrumData.

    C# version

    Code (csharp):
    1.  
    2.  
    3. using UnityEngine;
    4. using System.Collections;
    5.  
    6. public class CSBars : MonoBehaviour {
    7.    
    8.     // Public Variables
    9.     public AudioListener Audio;
    10.     public int numSamples;
    11.     public GameObject abar;
    12.    
    13.     // Private Varaibles
    14.     float[] numberleft;
    15.     float[] numberright;
    16.     GameObject[] thebarsleft;
    17.     GameObject[] thebarsright;
    18.     float spacing;
    19.     float width;
    20.    
    21.     // Use this for initialization
    22.     void Start () {
    23.         thebarsleft = new GameObject[numSamples];
    24.         thebarsright = new GameObject[numSamples];
    25.         spacing = 0.4f - (numSamples * 0.001f);
    26.         width = 0.3f - (numSamples * 0.001f);
    27.         for(int i=0; i < numSamples; i++){
    28.             float xpos = i*spacing -8.0f;
    29.             Vector3 positionleft = new Vector3(xpos,3, 0);
    30.                 thebarsleft[i] = (GameObject)Instantiate(abar, positionleft, Quaternion.identity) as GameObject;
    31.                 thebarsleft[i].transform.localScale = new Vector3(width,1,0.2f);
    32.            
    33.             Vector3 positionright = new Vector3(xpos,-3, 0);
    34.                 thebarsright[i] = (GameObject)Instantiate(abar, positionright, Quaternion.identity) as GameObject; 
    35.                 thebarsright[i].transform.localScale = new Vector3(width,1,0.2f);
    36.         }
    37.     }
    38.    
    39.     // Update is called once per frame
    40.     void Update () {
    41.         numberleft = Audio.GetSpectrumData(numSamples, 0, FFTWindow.BlackmanHarris);
    42.         numberright = Audio.GetSpectrumData(numSamples, 1, FFTWindow.BlackmanHarris);
    43.         for(int i=0; i < numSamples; i++){
    44.             if (float.IsInfinity(numberleft[i]*30) || float.IsNaN(numberleft[i]*30)){
    45.             }else{
    46.                 thebarsleft[i].transform.localScale = new Vector3(width, numberleft[i]*30,0.2f);
    47.                 thebarsright[i].transform.localScale = new Vector3(width, numberright[i]*30,0.2f); 
    48.             }
    49.         }
    50.     }
    51. }
    52.  
    53.  
     
  24. Sasmaster

    Sasmaster

    Joined:
    Jan 19, 2010
    Posts:
    44
    Hi guys , After reading this whole thread I have got the following question. I created sound visualizer .The basic model is like one above.I read spectrum with GetSpectrumData and assign the samples values to transform properties of my objects.But I have some 24-45 objects ,in my case cubes which I tween based on the sample data.My scene becomes so heavy in terms of CPU that the FPS drops to around 25-35 fps and everything becomes jerky .All those examples above,do they run smoothly ,don't they overload the CPU>? I can see that there are quite few objects being moved above.So may be I should restrict tweening only a couple of meshes in total.?It seems strange as even in Flash I can manipulate many more objects animating them simultaneously without noticeable frame rate drop.
     
  25. Dantus

    Dantus

    Joined:
    Oct 21, 2009
    Posts:
    5,667
    sasmaster, if the other examples run smoothly, it is probably something in your code that produces the frame rate drop.
    What do you mean with tween? Do you smooth the movement using code? Maybe you mean http://www.unifycommunity.com/wiki/index.php?title=Tween? No matter how you do it, as you only have 24-45 objects, it should not have such a significant influence to the frame rate.
    If you post your code, it is very likely that someone can tell you, what the reason for the weak performance is.
    What do you
     
  26. SkipKickCrash

    SkipKickCrash

    Joined:
    Oct 17, 2010
    Posts:
    80
    I'm actually working on a project that uses Spectrum Wave Analysis to get input from a Mic and display the fundamental frequency and note.

    Heavily using the FMOD plugin and samples from http://www.squaretangle.com/blog/?p=46

    Here's a Windows/Mac(untested) builds:

    www.skipkickcrash.com/jason/PitchDetection.rar

    FMOD plugins work beautifully with Unity 3 now that the Fmod that unity uses has been properly named, there's no conflicts like there was in 2.6
     
  27. burnumd

    burnumd

    Joined:
    May 27, 2008
    Posts:
    367
    So, maybe I don't understand exactly what the spectrum data is. I'd expect each number in the output array to represent the power of a certain frequency, but that's not what I'm observing. If I play a pure tone, shouldn't I see just one or two consecutive indices into the array at a roughly consistent nonzero level? If that is what's being returned, how do I find the upper bound on the frequencies? I've read a little on audio spectrum analysis, namely this Wikipedia article, and a couple other Googled-up bits of information, but can someone who's more of an audiophile explain how to interpret Unity's spectrum data?
     
  28. MSylvia

    MSylvia

    Joined:
    May 19, 2009
    Posts:
    42
    I notice that with the latest update to Unity 3.1.0f3 (54715) GetSpectrumData is being depreciated, which broke the C# version I made so here's an update that works with the latest version. Enjoy :)


    Code (csharp):
    1.  
    2.  
    3. using UnityEngine;
    4. using System.Collections;
    5.  
    6. public class CSBars : MonoBehaviour {
    7.    
    8.     // Public Variables
    9.     public AudioListener Audio;
    10.     public int numSamples = 64;
    11.     public GameObject abar;
    12.    
    13.     // Private Varaibles
    14.     float[] numberleft = new float[64];
    15.     float[] numberright = new float[64];
    16.     GameObject[] thebarsleft;
    17.     GameObject[] thebarsright;
    18.     float spacing;
    19.     float width;
    20.    
    21.     // Use this for initialization
    22.     void Start () {
    23.         thebarsleft = new GameObject[numSamples];
    24.         thebarsright = new GameObject[numSamples];
    25.         spacing = 0.4f - (numSamples * 0.001f);
    26.         width = 0.3f - (numSamples * 0.001f);
    27.         for(int i=0; i < numSamples; i++){
    28.             float xpos = i*spacing -8.0f;
    29.             Vector3 positionleft = new Vector3(xpos,3, 0);
    30.                 thebarsleft[i] = (GameObject)Instantiate(abar, positionleft, Quaternion.identity) as GameObject;
    31.                 thebarsleft[i].transform.localScale = new Vector3(width,1,0.2f);
    32.            
    33.             Vector3 positionright = new Vector3(xpos,-3, 0);
    34.                 thebarsright[i] = (GameObject)Instantiate(abar, positionright, Quaternion.identity) as GameObject; 
    35.                 thebarsright[i].transform.localScale = new Vector3(width,1,0.2f);
    36.         }
    37.     }
    38.    
    39.     // Update is called once per frame
    40.     void Update () {
    41.        
    42.         AudioListener.GetOutputData(numberleft, 0);
    43.         AudioListener.GetOutputData(numberright, 1);
    44.        
    45.         //numberleft = Audio.GetSpectrumData(numSamples, 0, FFTWindow.BlackmanHarris);
    46.         //numberright = Audio.GetSpectrumData(numSamples, 1, FFTWindow.BlackmanHarris);
    47.        
    48.         for(int i=0; i < numSamples; i++){
    49.             if (float.IsInfinity(numberleft[i]*30) || float.IsNaN(numberleft[i]*30)){
    50.             }else{
    51.                 thebarsleft[i].transform.localScale = new Vector3(width, numberleft[i]*30,0.2f);
    52.                 thebarsright[i].transform.localScale = new Vector3(width, numberright[i]*30,0.2f); 
    53.             }
    54.         }
    55.    
    56.  
     
  29. lion-gv

    lion-gv

    Joined:
    Nov 12, 2007
    Posts:
    118
    Actually GetSpectrumData wasn't deprecated but there was an API change. Below is a slight change to lines 39 and 40 of your previous example which should fix it.

    Code (csharp):
    1.  
    2.  
    3. using UnityEngine;
    4. using System.Collections;
    5.  
    6. public class CSBarsSpectrumData : MonoBehaviour {
    7.    
    8.     // Public Variables
    9.     public AudioListener Audio;
    10.     public int numSamples = 64;
    11.     public GameObject abar;
    12.    
    13.     // Private Varaibles
    14.     float[] numberleft;
    15.     float[] numberright;
    16.     GameObject[] thebarsleft;
    17.     GameObject[] thebarsright;
    18.     float spacing;
    19.     float width;
    20.    
    21.     // Use this for initialization
    22.     void Start () {
    23.         thebarsleft = new GameObject[numSamples];
    24.         thebarsright = new GameObject[numSamples];
    25.         spacing = 0.4f - (numSamples * 0.001f);
    26.         width = 0.3f - (numSamples * 0.001f);
    27.         for(int i=0; i < numSamples; i++){
    28.             float xpos = i*spacing -8.0f;
    29.             Vector3 positionleft = new Vector3(xpos,3, 0);
    30.                 thebarsleft[i] = (GameObject)Instantiate(abar, positionleft, Quaternion.identity) as GameObject;
    31.                 thebarsleft[i].transform.localScale = new Vector3(width,1,0.2f);
    32.            
    33.             Vector3 positionright = new Vector3(xpos,-3, 0);
    34.                 thebarsright[i] = (GameObject)Instantiate(abar, positionright, Quaternion.identity) as GameObject; 
    35.                 thebarsright[i].transform.localScale = new Vector3(width,1,0.2f);
    36.         }
    37.     }
    38.    
    39.     // Update is called once per frame
    40.     void Update () {
    41.         float[] numberleft = AudioListener.GetSpectrumData (numSamples, 0,FFTWindow.BlackmanHarris);
    42.         float[] numberright = AudioListener.GetSpectrumData (numSamples, 1,FFTWindow.BlackmanHarris);
    43.  
    44.         for(int i=0; i < numSamples; i++){
    45.             if (float.IsInfinity(numberleft[i]*30) || float.IsNaN(numberleft[i]*30)){
    46.             }else{
    47.                 thebarsleft[i].transform.localScale = new Vector3(width, numberleft[i]*30,0.2f);
    48.                 thebarsright[i].transform.localScale = new Vector3(width, numberright[i]*30,0.2f); 
    49.             }
    50.         }
    51.     }
    52. }
    53.    
    54.  
     
  30. CHPedersen

    CHPedersen

    Joined:
    Mar 2, 2011
    Posts:
    63
    Heya,

    I'm willing to give that one a shot. I did a little signal analysis on guitar data read by connecting a guitar directly to the computer's microphone in. That was before my Unity days, though, so I had to implement the analysis myself instead of using an API method. That's why I came to know a little about the underlying math. ;)

    Not having played with the AudioListener myself, this is constrained to a qualified guess, so bear with me. But I'm guessing the GetSpectrumData returns a standard Fourier spectrum calculated using a common implementation of the Fast Fourier Transform. The documentation states the input array needs to be a power-of-2, which indicates the algorithm behind it is probably the Cooley-Tukey FFT, since that algorithm's divide-and-conquer approach requires power-of-2 input. See http://en.wikipedia.org/wiki/Cooley-Tukey_FFT if you're interested in the math.

    An FFT doesn't return zero output at any frequency at all unless the input data represent an artificially sampled mathematical function, such as a pure sine wave. In real-life data such as audio, there will always be some energy at all frequency bands, though for harmonic audio such as music, the frequency spectrum will be dominated by peaks. The peaks stick up at regular intervals, making the spectrum look like tent canvas held up by poles.

    To implement a standard pitch-detector, you're looking for the highest peak across the frequency bins; i.e. you need to traverse the output array for every fourier-transformed window of samples and keep track of the index-position of the highest decibel value - This denotes the dominant frequency in the signal. The frequency at that peak then equals the array index at which you found it, multiplied by half the sampling frequency. The reason you multiply by half the sampling frequency is due to the fact that audio doubles (folds) around that particularly frequency, which in the field of signal analysis is known as the Nyquist-frequency; peaks on the right side of it will mirror the peaks on the left and be caused by the same frequency.

    As a final note, be aware that spectra made by music, or the human voice, will contain harmonic series. The fundamental frequency is then the lowest-frequency peak in the spectrum, but this does not necessarily equal the highest-decibel, peak (!). The frequencies of the other peaks in a harmonic signal are simply integer-multiplications of the frequency of the fundamental...
     
    Last edited: Apr 11, 2011
    oxcamel likes this.
  31. HavocK

    HavocK

    Joined:
    Apr 16, 2011
    Posts:
    7
    Works fine^^

    I'd like to know how can i get the bpm from this data? Is it possible? Like spawning an object each beat. Any clue?
     
  32. Quickfingers

    Quickfingers

    Joined:
    Aug 29, 2009
    Posts:
    268
  33. jonw

    jonw

    Joined:
    Jul 10, 2010
    Posts:
    2
    quickfingers, I'm interested to know how easy it would be to do beat detection from an external mp3 using Unity. Does your example use an external sound source?
     
  34. kmeleon

    kmeleon

    Joined:
    Jul 20, 2011
    Posts:
    3
    Hello everyone,
    How can i analyze a song in "one shot", not in real time. and put data in a file.So after, i can use the file into my game.
     
  35. luisanton

    luisanton

    Joined:
    Aug 25, 2009
    Posts:
    325
    Has anyone tried this feeding it with microphone input?
     
    Last edited: Aug 7, 2011
  36. KyleKing.Dev

    KyleKing.Dev

    Joined:
    Aug 16, 2011
    Posts:
    1
    I am wondering the same thing. Could anyone shed some light on this? If so, that would be great.

    Thanks,
    Kyle
     
  37. johnboy007_2006

    johnboy007_2006

    Joined:
    Jul 14, 2011
    Posts:
    81
    so to get pitch detection i need to traverse through all the frequency bins and keep track of the highest decibel vale like below?
    Code (csharp):
    1.  
    2. /* GetSpectrumData returns information about the frequency spectrum */
    3.         AudioListener.GetSpectrumData(analData.Bin, 0,FFTWindow.BlackmanHarris);
    4.         /*The resulting frequency spectrum values range from 0 to roughly 1.414 (the square root of 2).*/
    5.            
    6.         for(int i=0; i < numSamples; i++)
    7.         {
    8.             if(Mathf.Abs(analData.Bin[i])> max)
    9.             {
    10.                 max = Mathf.Abs(analData.Bin[i]);
    11.                 maxpos = i;
    12.                
    13.             }
    14.         }
    15.  
    is this correct? then i have to do something like below to get the frequency?

    double Freq = maxpos * (numSamples*0.5f);

    I hope this is correct?
     
  38. dadako

    dadako

    Joined:
    Jul 17, 2010
    Posts:
    66
    Thanks for this, got working. When changing the Samples though it does freak out. Unity throws out a warning like this:

    Assets/CSBarsSpectrumData.cs(40,53): warning CS0618: `UnityEngine.AudioListener.GetSpectrumData(int, int, UnityEngine.FFTWindow)' is obsolete: `GetSpectrumData returning a float[] is deprecated, use GetOutputData and pass a pre allocated array instead.'

    Would be nice if there was an official code sample by Unity that we can base music to object interactions on. (would be nice if this was documented at all)
     
  39. soren

    soren

    Joined:
    Feb 18, 2008
    Posts:
    123
    As the warning states, you should pass a preallocated array instead (so we don't kill the garbage collector by allocating a managed array on every GetSpectrumData). Something along these lines:

    Code (csharp):
    1. class MySpectrum : Monobehaviour
    2. {
    3.    private float numberLeft[1024]; // 1024 should equal your numSamples
    4.    private float numberRight[1024]; // 1024 should equal your numSamples
    5.  
    6.   void Update()
    7.   {
    8.     AudioListener.GetSpectrumData(numberLeft, 0, FFTWindow.BlackmanHarris);
    9.     AudioListener.GetSpectrumData(numberRight, 1, FFTWindow.BlackmanHarris);
    10.     // use your existing code
    11.   }
    12. }
    (Haven't compiled the code)

    BTW using blackmanHarris windowing is probably overkill for visualization (it's more suited for precise pitch detection or the like, just use a simple one.)

    cheers,
    Søren
     
  40. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
  41. Ullukai

    Ullukai

    Joined:
    Aug 24, 2010
    Posts:
    746
    wonder why i see this altered ad everywhere on the forums now ?

    other than that > did you ever research audio and how it is used in unity ? i looked in the reference scripts, manual , etc.. forums and came up with my own way of doing it > and that is what i did to make my own music animator !
     
  42. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    I simply posted it to make the community more aware of the asset. I just found some posts that were discussing music visualization in Unity, so I figured people in those threads might find it interesting as well. It wasn't meant to spam or hijack any threads. I only posted it in 3 relevant posts, not everywhere on the forums. :)
     
  43. wakeupscreaming

    wakeupscreaming

    Joined:
    Jul 16, 2012
    Posts:
    5
  44. Gaiyamato

    Gaiyamato

    Joined:
    Nov 19, 2009
    Posts:
    95
    Gday. I have been trying to build a very simple primitive beat detector to spawn enemy units to, I have a flash actionscript version that works fine, but the problem I am having is that none of the code that any of you have posted even works.
    I get compile errors saying GetOutputData or GetSpectrumData are boith void. But using it as it wants me to, my float array is null so I get nothing.
    I have tried with AudioListener and AudioSource.

    Those who have put up sample projects etc. for use also refuse to compile and will not work.

    Is this feature pro-only or are you all completely re-writing the Unity classes so they work?

    EDIT:
    Scratch that. All of the code samples here still don't compile but this did work. I had to fiddle with some audio track settings and use an AudioSource NOT and AudioListener because my sound is 3D and it does not always pick it up.

    Code (csharp):
    1.  
    2. var currentvalues:float[] = new float[256];
    3.     for(var t:int = 0; t<256; t++)
    4.         soundtrack.GetOutputData(currentvalues, t);
    5.  
     
    Last edited: Sep 18, 2012
  45. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
    Can the Microphone work with GetSpectrumData?

    It would be useful if Unity just exposed the FFT algorithm.

    I tried taking the Audio clip from the Microphone and playing it with an AudioListener to get the spectrum data, but it doesn't actually change the passed array...
     
  46. neroziros

    neroziros

    Joined:
    Aug 25, 2012
    Posts:
    129
    Hi all! I'm looking for a way to create a script that turns a boolean to true when the music does a beat (in a dubster song). Is there a way to know when a particular instrument of a song have been played? (I'm absolutely noob in the sound area)

    Thanks in advance!
     
  47. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
  48. Cro007

    Cro007

    Joined:
    Aug 3, 2012
    Posts:
    2
    Interesting thread.
    I tried first script (because I need only light to react to beat) but I can't get it working.
    I am kinda new to scripting and I dont know how to change code to work.
    I placed code to main camera, attached audio listener, chose light but no result
     
  49. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
    There is Visualizer Studio:
     
unityunity