WebGL and Microphone

Hi,

I'm testing a working audio project under WebGL that works in editor but when I try and build I get:

error CS0103: The name `Microphone' does not exist in the current context

I thought microphone was in WebGL?

EDIT: My mistake, it's not!

http://blogs.unity3d.com/2014/04/29/on-the-future-of-web-publishing-in-unity/

 

Can anyone from Unity say if microphone access is likely to be added in the 5.x cycle? (It's vague in that blog.)

 

Is there an update to this in 2015. Finishing a project that requires students to email recordings to their teacher but cannot find a way to access the mic in version 5 when building to webgl.

 

If you need it in a hurry, and can invest some time and effort, you could make a jslib plugin to capture microphone input and present it to your C# code. It is not completely trivial but it's also not too hard if you approach it the right way. The biggest step is finding out how to do it in Javascript in the first place. So long as you can get your Javascript to record the audio data into a buffer, it is pretty simple to present that buffer to C# code.

Here's an example of getting live audio data and analyzing the data in Javascript, which surely involves buffering it in memory at some point: https://github.com/cwilso/pitchdetect

 

@alanbrahmrowe: I had a play with Javascript microphone support yesterday and it was pretty easy to capture the audio stream. I could put a Unity wrapper around it fairly easily if you are still interested.

 

@gfoot : I too would be very interested in this.

 

Here is a microphone plugin then: http://www.gfootweb.webspace.virginmedia.com/MicrophoneWebGL.unitypackage

It is crude and basic. Look at TestMicrophoneWebGL.cs for a usage example.

You can also try out that example here: http://www.gfootweb.webspace.virginmedia.com/TestMicrophoneWebGL/

Note that the demo only works on Firefox due to limitations in Unity's audio playback code. The microphone side of things works fine on both - at least, I see sensible-looking data even on Chrome, I just can't get it into an audioclip to actually play it back again.

As shown in the example, you should call Init(), then call PollInit() until it stops saying "pending". Hopefully it then says "ready", in which case you can carry on; otherwise it will say an error of some sort. One reason for the polling is that the user will be prompted to allow access to the microphone, so we need to wait for that.

Once it is initialized you can call Start() to start recording and Stop() to stop recording. During recording, the plugin buffers up the microphone data in units of whatever buffer size you passed into Init(). You can query how many buffers it is holding using GetNumBuffers(), and you can pop the next buffer using GetBuffer().

These functions should work fine both during recording and after stopping, and you could in theory record indefinitely, so long as you consume the buffers. If there are no pending buffers then GetBuffer() also returns false, so you could just poll it from your Update() function and ignore GetNumBuffers.

 

Awesome, thank you so much!

 

First of all, thanks gfoot for writing microphone wrapper for Firefox.
I am very glad that now we can use microphone with WebGL.

Can you help me to figure out what kind of data it is and how to play it back again?

 

Nice work gfoot!

 

It's just regular audio data - there are some options in the javascript to choose the format I think, when the microphone is initialized.

The problem appeared to be that Chrome's Javascript interface for audio playback didn't support streaming in data like this on the fly, at least not through the interface that Unity was using to access it. I don't know whether anything has changed since then.

 

Thatnk you very much =)

 

Hi gfoot,
I noticed that when playing the recording in Unity it changes pitch of audio to half tone.
E -> D#
D# ->D
D -> C#
And so on.

Do you know what can cause this problem?

 

Thank you @gfoot for sharing.

I adapted the plugin to simply get microphone volume instead of creating a clip. I can share if anyone is interested. It was working great for me on Unity 5.1.1, except for occasional instability in Chrome.

Building from any version after 5.1.1 results in the Init never completing on any browser. Anyone have an idea what changed in the WebGL preview to affect this plugin?

 

I found this same issue and have posted a bug / request (it may not actually be a bug at all as I dont really understand the consequences and resons yet). However I do have some insights related to pointer and buffer management - it seems if you change the 2 pointer shifts as follows in these 2 functions, it will revert to working state (seems the pointers are now getting shifted unnecessarily twice now sometime between 5.1.1 and 5.1.3)

MicrophoneWebGL_PollInit: function(resultPtr, resultMaxLength)
{
var sliceLen = Math.max(0, resultMaxLength/2 - 1);
var str = microphoneWorker.initResult.slice( 0, sliceLen );
//stringToUTF16(str, HEAPU32[resultPtr>>2], resultMaxLength); - removed
stringToUTF16(str, resultPtr, resultMaxLength); // added
},

and

MicrophoneWebGL_GetBuffer: function(bufferPtr)
{
if (microphoneWorker.buffers.length == 0)
{
return false;
}
//HEAPF32.set(microphoneWorker.buffers.shift(), HEAPU32[bufferPtr>>2]>>2); - removed
HEAPF32.set( microphoneWorker.buffers.shift(), bufferPtr>>2 ); .// added
return true;
},

 

Bump 2016. Bump 5.3.4. Microphone is still not exposed natively.

I don't even see it on the roadmap.
https://unity3d.com/unity/roadmap

Microphone support in WebGL is kind of important for in-game chat and of course voice-commands!

 

Hello @gfoot I download and run it on WebGL platform but I cannot see the things happening as your build file.

I am not getting the init actually...

please help me out...

 

hey @gfoot do you have this code on github somewhere? i think the library may need to get tweaked because it's breaking. i don't mind working on it a bit.

 

hey @gfoot would you mind sharing your work on github? the link mentioned above is dead. just as @dakume mentioned a few people would be interested in developing a solution for mic input in webgl and it seems like a good starting point.

 

Hello everyone,
Can you tell me where do i find the MicrophoneWebGL.unitypackage.
Because above links are showing.
"This site can’t be reached"

 

Hello gfoot,
Can you tell me where do i find the MicrophoneWebGL.unitypackage.
Because above links are showing.
"This site can’t be reached"

 

Bah must've switched phone providers and lost the web storage.

Okay starting over... WebGL plugins...

https://docs.unity3d.com/Manual/webgl-interactingwithbrowserscripting.html

 

I have a few more tweaks to go, but it's basically working. I still need to incorporate supporting multiple microphones and multiple sample rates in the WebGL plugin.

The HTML5 media API is a beast. When you just want the raw wave data it gives you blobs. When you have a blob you need to run it through a request to extract the bytes. Or you can bypass that and get the direct float data and you have to handle the timing manually.

There's another nice things like needing to run in HTTPS if you don't want to always see the Microphone access prompt.

Anyway, I'm improving this over time. WIP demo: https://theylovegames.com/WordDetection_1_1/

I have a C# interface that mirrors the regular Microphone API and uses the WebGL jslib when needed.

 

I am building a very small API C# interface to the WebGL JS AudioCapture Device for Unity3d to replace the missing microphone. It is in testing but the result is looking very pleasing right now. It has testing modes to show what is happening, and will be just bringing the data back into the standard Unity engine as a set of buffers being recorded, pointer to where the next buffer is going to, and buffer status information. This can be used by application programmers or applied to Audio Source for games. I want to sell it to fund important future motherboard GPU hardware improvements.

How big of demand is there for this? I am going to use it for my own testing of some AI I have worked on for years and in school. Just using Unity3D to model it. Are there people out there willing to pay for a functional testable, "Unity3d WebGL Microphone Asset". I would probably version different Asset Products according to what their functionality is. I can get it ready for simple use as a (dll/js script) nuget package as an asset and get it on the Unity Asset store if anyone thinks it would be useful.

Here is some of the Test Debug Info it outputs as a visual output to verify functionality, i.e. Integration Test.

 

* Cough *
https://web.archive.org/web/2016051....virginmedia.com/MicrophoneWebGL.unitypackage

 

dead URL

@Unity :
Still not in build?
Microphone still not working in the last version of Unity...

 

Found this which is free :
https://github.com/tgraupmann/UnityWebGLMicrophone

It detects my microphones from the webGL player but didn't success to use it yet...

I am actually trying to use it with this :
- https://github.com/zombience/audio_analyzer

Wondering if someone made it works?

 

It isn't dead. I just downloaded it again. no issues grabbing it.

And individually both of those other components you listed work "okay" on my builds here.
Perhaps if you were a bit more explicit with your intentions people could be more helpful.
There are some notes that the audio_analyzer is not "real-time ready" and other notes to indicate a code hook is required to get the mic working as you might expect. Perhaps it is one of those caveats. Probably a new question is appropriate at that point.

The support for getUserMedia is spotty at best at the moment. - that possibly needs love https://stackoverflow.com/questions...gator-getusermedia-is-not-a-function/28991938

Short answer. No. No native support yet

 

I just tried another project to build to WebGL and getting the dreaded error CS0103: The name `Microphone' does not exist in the current context two and half years later.

Is this likely to be in soon?
WebAudio supports it no?


ah...I see. Still at "The Microphone class is not supported in WebGL."
https://docs.unity3d.com/Manual/webgl-audio.html

 

@Schubkraft Opened 2015, asking in 2019. WebGL Microphone support? Upcoming?

 

This looks like a good alternative for now:
https://assetstore.unity.com/packages/tools/input-management/microphone-webgl-library-79989

 

Clearly...

 

Sorry Unity Team but as a User who switched from another Big Engine, i can not understand points like this.
In 5 Years you didn't provide mic support for WebGL... wow

 

Any news? Would be very useful for a project I am working on right now.

 

Also looking for WebGL microphone support! Only found out after I got it working in windows, then tried to build with WebGL.

Is there a timeline for this feature? Would be much appreciated!

 

@jukka_j Heads up on this issue, request for microphone support since 2015 lol.

 

We're also looking for this for our application : )

 

Hey y'all, having the same need on my end. I found this. I have not tested it and won't be testing it right away, but I found that it does compile. It looks like you need to wrap all of your pre-existing Microphone code in platform directives and use this for WebGL. It's free and MIT.

https://github.com/tgraupmann/UnityWebGLMicrophone

 

If you finally fix Mic support for WebGL it will be epic. It's been years...

 

This is great news. There is a large group awaiting this feature. Any timelines when you think this might become available? Web makes corporates adopt solutions more easily due to higher security. In this absence of it my solution takes the standalone exe, or native app approach which is a hindrance. Looking forward to this feature in Unity.

 

How is the priority list coming along?
Interested in this feature..

 

Anyone have found a good sdk to use the microphone?

 

Seems close to what i am looking for, can you share the code or the asset?