Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Microphone input plugin for Windows/MacOS

Discussion in 'Assets and Asset Store' started by boat365, Oct 1, 2011.

  1. boat365

    boat365

    Joined:
    May 22, 2011
    Posts:
    7
    I see people ask about this on here from time to time, so I hacked up a plugin over a weekend and figured I'd publish it. There's a native library so you'll need Unity Pro, and the scenes might need Unity 3.4.1; I've included all of the source code if anyone wants to hack on it.

    Here's a quick link to the Unity3D project; you can have a hunt around the google code site if you want to check out the source: https://code.google.com/p/unity3d-wavein/wiki/Unity3DProject
     
  2. elbows

    elbows

    Joined:
    Nov 28, 2009
    Posts:
    2,502
    Nice one!

    Afraid Im not having any luck getting this working on OS X though:

    Cannot get fft example to work. It says its found 1 input, but I see nothing, regardless of whether I have line-in or mic selected as input device in the OS. I don't see any error messages in unity, and the problem is the same with Lion and Snow Leopard. If I load an audio file into the audio player then I see bars at the bottom of the screen, but the central fft display never does anything.

    Record demo crashes unity editor when using OS X Lion. It doesn't crash if I use Snow Leopard, but Im not sure if its actually recorded anything.

    I haven't tried rebuilding the plugin, just using the downloaded binary version.

    Any ideas?
     
  3. boat365

    boat365

    Joined:
    May 22, 2011
    Posts:
    7
    I don't have Lion, so I don't know if the plugin works there. From what you've said, I guess maybe not. The Mac version is kind of an afterthought I'm afraid. The Windows version should be fairly sturdy :)

    Two things:
    1) You'll need to call StartDevice etc. yourself in the FFT scene; really there's a missing component that wasn't mine to release with the rest of the project. How that worked was, the missing component found a WaveInFFT component on the same object using GetComponent, then called StartDevice in 'Start' to start receiving samples. I guess you could call StartDevice from a 'Start' added to WaveInFFTDemo (be mindful of the order of calls there I guess, since WaveInPlugin needs to do some stuff in 'Start')

    2) Look at the console output in the record demo; it should include the name of the file(s) that raw samples have been written to.
     
  4. boat365

    boat365

    Joined:
    May 22, 2011
    Posts:
    7
    Just checked; the record demo works on my Snow Leopard Mac. Output is written in ~/Library/Caches/colinbarrettdotnet
     
  5. elbows

    elbows

    Joined:
    Nov 28, 2009
    Posts:
    2,502
    Many thanks for the info.

    If I just add the line waveInFFT.StartDevice(0); to the end of the Awake function in WaveInFFTDemo.cs then it works!

    Only on Snow Leopard though, the above crashes Lion in the same way the record demo did. I doubt I have the skills to fix this but I will investigate a bit anyway.
     
  6. boat365

    boat365

    Joined:
    May 22, 2011
    Posts:
    7
    I installed Lion to have a look at this:

    1) A standalone built with the 'record' scene doesn't crash (actually appears to work fine)

    2) The editor does crash; worse, the Unity bug reporter comes up but the editor hangs up and has to be force-quit so there's no call stack for me to look at. Even worse, for whatever asinine reason, I can't attach the Xcode debugger to the Unity application (I've noticed this before, and I suspect it's a DRM thing; there's no such problem using Windows and MSVC).

    So, you're not alone, but I don't know what to do to help you :(
     
  7. elbows

    elbows

    Joined:
    Nov 28, 2009
    Posts:
    2,502
    Thanks for trying. I spent some hours looking at the source for the plugin, but I was unable to spot anything really obvious. Looking at editor log in console wasn't terribly illuminating to me, but this is what I got:

    WaveInPlugin initialized; found 1 device(s)
    UnityEngine.Debug:Internal_Log(Int32, String, Object)
    UnityEngine.Debug:Log(Object)
    UnityEngine.MonoBehaviour:print(Object)
    WaveInPlugin:Start() (at Assets/Plugins/WaveInPlugin.cs:21)
    WaveInRecord:Start() (at Assets/WaveInRecord.cs:19)

    (Filename: Assets/Plugins/WaveInPlugin.cs Line: 21)

    WaveInRecord started
    UnityEngine.Debug:Internal_Log(Int32, String, Object)
    UnityEngine.Debug:Log(Object)
    UnityEngine.MonoBehaviour:print(Object)
    WaveInRecord:StartRecording() (at Assets/WaveInRecord.cs:53)
    WaveInRecord:Update() (at Assets/WaveInRecord.cs:36)

    (Filename: Assets/WaveInRecord.cs Line: 53)

    Receiving unhandled NULL exception
    Launching bug reporter

    *** Launching bug reporter due to crashbug.
    Obtained 60 stack frames.
    0 Unity 0x00fcb585 _Z17LaunchBugReporter13BugReportMode + 1077
    1 libmono.0.dylib 0x020de6c2 mono_chain_signal + 76
    2 libmono.0.dylib 0x02016a42 mono_sigsegv_signal_handler + 263
    3 libsystem_c.dylib 0x9449d59b _sigtramp + 43
    4 ??? 0xffffffff 0x0 + 4294967295
    5 libmono.0.dylib 0x021ede03 GC_mark_some + 480
    6 libmono.0.dylib 0x021e622a GC_stopped_mark + 467
    7 libmono.0.dylib 0x021e6605 GC_try_to_collect_inner + 369
    8 libmono.0.dylib 0x021e6793 GC_collect_or_expand + 134
    9 libmono.0.dylib 0x021e6d41 GC_allocobj + 268
    10 libmono.0.dylib 0x021eb127 GC_generic_malloc_inner + 176
    11 libmono.0.dylib 0x021eb22a GC_generic_malloc + 92
    12 libmono.0.dylib 0x021eb49f GC_malloc + 137
    13 libmono.0.dylib 0x021668b8 mono_object_new_alloc_specific + 93
    14 libmono.0.dylib 0x02166a13 mono_object_new_specific + 290
    15 ??? 0x0892741c 0x0 + 143815708
    16 ??? 0x0d69d91e 0x0 + 225040670
    17 ??? 0x0d69d80d 0x0 + 225040397
    18 ??? 0x0d69d6a2 0x0 + 225040034
    19 ??? 0x0d69c7f0 0x0 + 225036272
    20 ??? 0x0d69baad 0x0 + 225032877
    21 ??? 0x0c0481ef 0x0 + 201621999
    22 libmono.0.dylib 0x0201db6c mono_jit_compile_method + 1283
    23 libmono.0.dylib 0x02165360 mono_runtime_invoke + 133
    24 libmono.0.dylib 0x0216dad7 mono_runtime_invoke_array + 1711
    25 libmono.0.dylib 0x0211b2ae mono_class_fill_runtime_generic_context + 20415
    26 ??? 0x0cb9a568 0x0 + 213493096
    27 ??? 0x0cb99c95 0x0 + 213490837
    28 ??? 0x0d69b945 0x0 + 225032517
    29 ??? 0x0cb9661d 0x0 + 213476893
    30 ??? 0x0cb96587 0x0 + 213476743
    31 ??? 0x0d693e54 0x0 + 225001044
    32 ??? 0x0c0481ef 0x0 + 201621999
    33 libmono.0.dylib 0x0201db6c mono_jit_compile_method + 1283
    34 libmono.0.dylib 0x02165360 mono_runtime_invoke + 133
    35 Unity 0x008dab3e _ZN13MonoBehaviour22DoCallInputEventMethodEPS_P10MonoMethodR10InputEventiPPvbbb + 334
    36 Unity 0x008db9bc _ZN13MonoBehaviour5DoGUIER10InputEventbbbi + 124
    37 Unity 0x00fde3cb _ZN7GUIView12OnInputEventER10InputEvent + 315
    38 Unity 0x00fdf39d -[GUIOpenGLView renderRect:] + 237
    39 Unity 0x00fc668a -[OpenGLView drawRect:] + 330
    40 Unity 0x00fdcc3b _ZN7GUIView10RepaintAllEb + 427
    41 Unity 0x00ee5254 _ZN11Application11UpdateSceneEb + 484
    42 Unity 0x00ee5718 _ZN11Application19UpdateSceneIfNeededEv + 72
    43 Unity 0x00eefe00 _ZN11Application9TickTimerEv + 672
    44 Foundation 0x947673eb __NSFireTimer + 286
    45 CoreFoundation 0x999eb256 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 22
    46 CoreFoundation 0x999eabe7 __CFRunLoopDoTimer + 743
    47 CoreFoundation 0x999c9ce0 __CFRunLoopRun + 1888
    48 CoreFoundation 0x999c91ec CFRunLoopRunSpecific + 332
    49 CoreFoundation 0x999c9098 CFRunLoopRunInMode + 120
    50 HIToolbox 0x9af93487 RunCurrentEventLoopInMode + 318
    51 HIToolbox 0x9af9adc3 ReceiveNextEventCommon + 381
    52 HIToolbox 0x9af9ac32 BlockUntilNextEventMatchingListInMode + 88
    53 AppKit 0x986418ec _DPSNextEvent + 678
    54 AppKit 0x98641159 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 113
    55 AppKit 0x9863d4cb -[NSApplication run] + 904
    56 AppKit 0x988d0b54 NSApplicationMain + 1054
    57 Unity 0x00fe19a5 main + 837
    58 Unity 0x0003f36e _start + 216
    59 Unity 0x0003f295 start + 41
     
  8. boat365

    boat365

    Joined:
    May 22, 2011
    Posts:
    7
    That's consistent with what I see; the problem occurs somewhere in 'GC_mark_some'. It feels like this is a mono/Lion issue (the reverse p/invoke in the plugin is something that maybe not many other people are doing, and it's tricky with other threads calling into the mono runtime: it should work (and it appears to on other operating systems), but maybe there are problems on Lion. Anecdotally, I've heard that there are problems with older versions of mono on Lion. I can't find much in the way of detail about that though).

    I logged a bug here: http://fogbugz.unity3d.com/default.asp?421543_9fqc7j96
     
  9. XY01

    XY01

    Joined:
    Jul 15, 2011
    Posts:
    24
    Hey, thanks heaps for this, I am pretty stoked about getting it running since I am currently using a very round about method (Processing > UDP > Unity) to get the mic fft data.
    I downloaded the project, on windows, had to upgrade it to 3.5 and I get an error saying:

    DllNotFoundException: wavein
    WaveInPlugin.Start () (at Assets/Plugins/WaveInPlugin.cs:31)
    WaveInFFT.Start () (at Assets/Plugins/WaveInFFT.cs:106)

    I can't see the .dll in the project either. Any help would be ace, thanks man :)
     
  10. XY01

    XY01

    Joined:
    Jul 15, 2011
    Posts:
    24
    Actually I got it running, got the wrong version.
    Though in the FFT demo I have to add a wave/mp3 to the Audio Source to get it to work?
    Am I missing something for getting the mic to work?
     
  11. boat365

    boat365

    Joined:
    May 22, 2011
    Posts:
    7
    I think, if I remember how that demo is supposed to work; you hook up an AudioSource to play sound through the speakers, and the already running WaveIn samples the playing sound. Then an FFT is calculated for both the playing sound and the recorded sound (my idea was that both of those FFTs should look similar, since they are generated by the same sound or close to it). But you shouldn't need to play a sound; you should be able to make a sound yourself and at least the FFT display of the microphone input should update (in that case there's no other FFT to display, since the computer isn't making the sound).

    I thought- though I haven't checked- that microphone input was a built in feature of Unity 3.5? So maybe you wouldn't need my plugin at all anymore?