Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

WebGL and Microphone

Discussion in 'WebGL' started by sonicviz, Mar 7, 2015.

  1. sonicviz

    sonicviz

    Joined:
    May 19, 2009
    Posts:
    1,042
    Last edited: Mar 7, 2015
  2. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,106
    Can anyone from Unity say if microphone access is likely to be added in the 5.x cycle? (It's vague in that blog.)
     
  3. Schubkraft

    Schubkraft

    Unity Technologies

    Joined:
    Dec 3, 2012
    Posts:
    972
    Well, it is on our list but the priority is not high at all.
     
  4. alanbrahmrowe

    alanbrahmrowe

    Joined:
    Jan 14, 2015
    Posts:
    6
    Is there an update to this in 2015. Finishing a project that requires students to email recordings to their teacher but cannot find a way to access the mic in version 5 when building to webgl.
     
  5. gfoot

    gfoot

    Joined:
    Jan 5, 2011
    Posts:
    550
    If you need it in a hurry, and can invest some time and effort, you could make a jslib plugin to capture microphone input and present it to your C# code. It is not completely trivial but it's also not too hard if you approach it the right way. The biggest step is finding out how to do it in Javascript in the first place. So long as you can get your Javascript to record the audio data into a buffer, it is pretty simple to present that buffer to C# code.

    Here's an example of getting live audio data and analyzing the data in Javascript, which surely involves buffering it in memory at some point: https://github.com/cwilso/pitchdetect
     
  6. Schubkraft

    Schubkraft

    Unity Technologies

    Joined:
    Dec 3, 2012
    Posts:
    972
    There is no timeline associated with the mic support.
    So if you really need it now go the way pointed out by gfoot.
    Sorry.
     
  7. gfoot

    gfoot

    Joined:
    Jan 5, 2011
    Posts:
    550
    @alanbrahmrowe: I had a play with Javascript microphone support yesterday and it was pretty easy to capture the audio stream. I could put a Unity wrapper around it fairly easily if you are still interested.
     
    Psyco92 and twobob like this.
  8. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,106
    @gfoot : I too would be very interested in this.
     
  9. gfoot

    gfoot

    Joined:
    Jan 5, 2011
    Posts:
    550
    Here is a microphone plugin then: http://www.gfootweb.webspace.virginmedia.com/MicrophoneWebGL.unitypackage

    It is crude and basic. Look at TestMicrophoneWebGL.cs for a usage example.

    You can also try out that example here: http://www.gfootweb.webspace.virginmedia.com/TestMicrophoneWebGL/

    Note that the demo only works on Firefox due to limitations in Unity's audio playback code. The microphone side of things works fine on both - at least, I see sensible-looking data even on Chrome, I just can't get it into an audioclip to actually play it back again.

    As shown in the example, you should call Init(), then call PollInit() until it stops saying "pending". Hopefully it then says "ready", in which case you can carry on; otherwise it will say an error of some sort. One reason for the polling is that the user will be prompted to allow access to the microphone, so we need to wait for that.

    Once it is initialized you can call Start() to start recording and Stop() to stop recording. During recording, the plugin buffers up the microphone data in units of whatever buffer size you passed into Init(). You can query how many buffers it is holding using GetNumBuffers(), and you can pop the next buffer using GetBuffer().

    These functions should work fine both during recording and after stopping, and you could in theory record indefinitely, so long as you consume the buffers. If there are no pending buffers then GetBuffer() also returns false, so you could just poll it from your Update() function and ignore GetNumBuffers.
     
    Dantus and jeremy.johnston like this.
  10. gecko

    gecko

    Joined:
    Aug 10, 2006
    Posts:
    2,106
    Awesome, thank you so much!
     
  11. bkachmar

    bkachmar

    Joined:
    Mar 15, 2013
    Posts:
    37
    First of all, thanks gfoot for writing microphone wrapper for Firefox.
    I am very glad that now we can use microphone with WebGL.

    Can you help me to figure out what kind of data it is and how to play it back again?
     
  12. sonicviz

    sonicviz

    Joined:
    May 19, 2009
    Posts:
    1,042
    Nice work gfoot!
     
  13. gfoot

    gfoot

    Joined:
    Jan 5, 2011
    Posts:
    550
    It's just regular audio data - there are some options in the javascript to choose the format I think, when the microphone is initialized.

    The problem appeared to be that Chrome's Javascript interface for audio playback didn't support streaming in data like this on the fly, at least not through the interface that Unity was using to access it. I don't know whether anything has changed since then.
     
  14. bkachmar

    bkachmar

    Joined:
    Mar 15, 2013
    Posts:
    37
    Thatnk you very much =)
     
  15. bkachmar

    bkachmar

    Joined:
    Mar 15, 2013
    Posts:
    37
    Hi gfoot,
    I noticed that when playing the recording in Unity it changes pitch of audio to half tone.
    E -> D#
    D# ->D
    D -> C#
    And so on.

    Do you know what can cause this problem?
     
  16. dominicjackson

    dominicjackson

    Joined:
    Feb 11, 2013
    Posts:
    8
     
  17. nomadic

    nomadic

    Joined:
    Mar 4, 2010
    Posts:
    33
    Thank you @gfoot for sharing.

    I adapted the plugin to simply get microphone volume instead of creating a clip. I can share if anyone is interested. It was working great for me on Unity 5.1.1, except for occasional instability in Chrome.

    Building from any version after 5.1.1 results in the Init never completing on any browser. Anyone have an idea what changed in the WebGL preview to affect this plugin?
     
  18. dominicjackson

    dominicjackson

    Joined:
    Feb 11, 2013
    Posts:
    8
    I found this same issue and have posted a bug / request (it may not actually be a bug at all as I dont really understand the consequences and resons yet). However I do have some insights related to pointer and buffer management - it seems if you change the 2 pointer shifts as follows in these 2 functions, it will revert to working state (seems the pointers are now getting shifted unnecessarily twice now sometime between 5.1.1 and 5.1.3)

    MicrophoneWebGL_PollInit: function(resultPtr, resultMaxLength)
    {
    var sliceLen = Math.max(0, resultMaxLength/2 - 1);
    var str = microphoneWorker.initResult.slice( 0, sliceLen );
    //stringToUTF16(str, HEAPU32[resultPtr>>2], resultMaxLength); - removed
    stringToUTF16(str, resultPtr, resultMaxLength); // added
    },

    and

    MicrophoneWebGL_GetBuffer: function(bufferPtr)
    {
    if (microphoneWorker.buffers.length == 0)
    {
    return false;
    }
    //HEAPF32.set(microphoneWorker.buffers.shift(), HEAPU32[bufferPtr>>2]>>2); - removed
    HEAPF32.set( microphoneWorker.buffers.shift(), bufferPtr>>2 ); .// added
    return true;
    },
     
    nomadic likes this.
  19. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    175
    Bump 2016. Bump 5.3.4. Microphone is still not exposed natively.

    I don't even see it on the roadmap.
    https://unity3d.com/unity/roadmap

    Microphone support in WebGL is kind of important for in-game chat and of course voice-commands!
     
    Last edited: Apr 6, 2016
  20. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,635
    @theylovegames This thread shows a workaround to use the Microphone in WebGL today, so this should not prevent you from making anything. We cannot currently make Unity's built-in Microphone class work in WebGL, because the WebGL audio backend can not be made not match to it's usage, and I don't want an API which works only somewhat like on other platforms, that would create too much confusion. Once we get FMOD to work on WebGL (which would require threads) this situation may change, but that is still going to take a while.
     
    twobob likes this.
  21. ammarvohra

    ammarvohra

    Joined:
    Jan 24, 2016
    Posts:
    10
    Hello @gfoot I download and run it on WebGL platform but I cannot see the things happening as your build file.

    I am not getting the init actually...

    please help me out...
    init.PNG
     
  22. dakume

    dakume

    Joined:
    Feb 9, 2016
    Posts:
    1
    hey @gfoot do you have this code on github somewhere? i think the library may need to get tweaked because it's breaking. i don't mind working on it a bit.
     
  23. dawidk

    dawidk

    Joined:
    Jul 27, 2012
    Posts:
    2
    hey @gfoot would you mind sharing your work on github? the link mentioned above is dead. just as @dakume mentioned a few people would be interested in developing a solution for mic input in webgl and it seems like a good starting point.
     
  24. arnabghosh

    arnabghosh

    Joined:
    Sep 5, 2016
    Posts:
    4
    Hello everyone,
    Can you tell me where do i find the MicrophoneWebGL.unitypackage.
    Because above links are showing.
    "This site can’t be reached"
     
  25. arnabghosh

    arnabghosh

    Joined:
    Sep 5, 2016
    Posts:
    4


    Hello gfoot,
    Can you tell me where do i find the MicrophoneWebGL.unitypackage.
    Because above links are showing.
    "This site can’t be reached"
     
  26. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    175
  27. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    175
    I have a few more tweaks to go, but it's basically working. I still need to incorporate supporting multiple microphones and multiple sample rates in the WebGL plugin.

    The HTML5 media API is a beast. When you just want the raw wave data it gives you blobs. When you have a blob you need to run it through a request to extract the bytes. Or you can bypass that and get the direct float data and you have to handle the timing manually.

    There's another nice things like needing to run in HTTPS if you don't want to always see the Microphone access prompt.

    Anyway, I'm improving this over time. WIP demo: https://theylovegames.com/WordDetection_1_1/

    I have a C# interface that mirrors the regular Microphone API and uses the WebGL jslib when needed.
     
    twobob likes this.
  28. specweapons

    specweapons

    Joined:
    Dec 23, 2016
    Posts:
    7
    I am building a very small API C# interface to the WebGL JS AudioCapture Device for Unity3d to replace the missing microphone. It is in testing but the result is looking very pleasing right now. It has testing modes to show what is happening, and will be just bringing the data back into the standard Unity engine as a set of buffers being recorded, pointer to where the next buffer is going to, and buffer status information. This can be used by application programmers or applied to Audio Source for games. I want to sell it to fund important future motherboard GPU hardware improvements.

    How big of demand is there for this? I am going to use it for my own testing of some AI I have worked on for years and in school. Just using Unity3D to model it. Are there people out there willing to pay for a functional testable, "Unity3d WebGL Microphone Asset". I would probably version different Asset Products according to what their functionality is. I can get it ready for simple use as a (dll/js script) nuget package as an asset and get it on the Unity Asset store if anyone thinks it would be useful.

    Here is some of the Test Debug Info it outputs as a visual output to verify functionality, i.e. Integration Test.
     
  29. twobob

    twobob

    Joined:
    Jun 28, 2014
    Posts:
    1,785
  30. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,643
  31. Schubkraft

    Schubkraft

    Unity Technologies

    Joined:
    Dec 3, 2012
    Posts:
    972
    sama-van likes this.
  32. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,643
  33. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,643
  34. twobob

    twobob

    Joined:
    Jun 28, 2014
    Posts:
    1,785
    It isn't dead. I just downloaded it again. no issues grabbing it.

    And individually both of those other components you listed work "okay" on my builds here.
    Perhaps if you were a bit more explicit with your intentions people could be more helpful.
    There are some notes that the audio_analyzer is not "real-time ready" and other notes to indicate a code hook is required to get the mic working as you might expect. Perhaps it is one of those caveats. Probably a new question is appropriate at that point.

    The support for getUserMedia is spotty at best at the moment. - that possibly needs love https://stackoverflow.com/questions...gator-getusermedia-is-not-a-function/28991938

    Short answer. No. No native support yet
     
    Last edited: Jul 12, 2017
  35. sonicviz

    sonicviz

    Joined:
    May 19, 2009
    Posts:
    1,042
    I just tried another project to build to WebGL and getting the dreaded error CS0103: The name `Microphone' does not exist in the current context two and half years later.

    Is this likely to be in soon?
    WebAudio supports it no?


    ah...I see. Still at "The Microphone class is not supported in WebGL."
    https://docs.unity3d.com/Manual/webgl-audio.html
     
    Last edited: Oct 6, 2017
  36. Deleted User

    Deleted User

    Guest

    @Schubkraft Opened 2015, asking in 2019. WebGL Microphone support? Upcoming?
     
  37. Deleted User

    Deleted User

    Guest

  38. Psyco92

    Psyco92

    Joined:
    Nov 15, 2013
    Posts:
    21
    Clearly...
     
  39. xzesstence

    xzesstence

    Joined:
    Nov 23, 2019
    Posts:
    4
    Sorry Unity Team but as a User who switched from another Big Engine, i can not understand points like this.
    In 5 Years you didn't provide mic support for WebGL... wow
     
    dagnais likes this.
  40. Marvbr

    Marvbr

    Joined:
    Jul 2, 2017
    Posts:
    2
    Any news? Would be very useful for a project I am working on right now.
     
unityunity