Amplitude for WebGL

Hello, getting different readouts of values when analysing frequencies in the editor versus in the browser. In webGL it's picking up consistent 'noise' data across all frequency ranges and the peaks don't hit the same highs when compared with the analysis in the Editor window. See screenshots, both using same settings in demo scene (with boost set to 0) and recording readout at the 5th tone of the demo audio...
https://imgur.com/a/tUYLZaF

Is this normal / known? Is there a way to calibrate so the Editor behaves the same as webGL, or is that just a limitation of working across the two platforms?

 

Hi underseaaa,

The editor is using the underlying FMOD audio framework, that Unity uses across all platforms except WebGL. In WebGL, Unity uses a custom wrapper for the browsers underlying Web Audio API framework. They are two very different audio frameworks. The purpose of the editor is to give you an approximation of what you can expect in WebGL, but it can’t be demonstrated precisely because of the different frameworks.

Michael

 

Hey - for anyone else having problems getting data from Amplitude with Unity 2021, I believe we've found a fix!

Amplitude 1.2.2 attaches an Analyser to a GainNode in the Web Audio API, but either Unity's latest 2021 implementation or browser changes over time seem to have caused that node to only emit zeroes when sampled.

Grabbing one of raw audio buffers, creating a new source node from it, and attaching the analyzer to that node instead *seems* to get everything working again... though presumably at some additional performance cost.

I've emailed our working source to Crazy Minnow, and hope it helps for the next release!

 

@samuel-levine were you able to implement this fix by changing the Amplitude source code (such as AmplitudeLib.jslib), or did you have to make custom scripts yourself? I'm currently stuck with this same issue and am hoping to get it fixed as soon as I can, so any help is appreciated!
Additionally, not sure if this matters, but I also see this message when my audio has stopped playing:
Failed to delete analyzer (-458772) from source TypeError: analyzerObj.source.disconnect is not a function

 

I'm having the issue described in post #103 above-

Neither the provided demo scene or my own scene is able to get anything but zeroes in a WebGL build using 2021.3.15 when using Amplitude.average

Seeing that the whole point of this asset is to work in WebGL builds..it would be great to get a solution to this show stopping issue. 0_0

 

Can you post the code from your solution? Having the same issue.

 

No answer?

 

Hi imaginationrabbit,

We’re aware of this issue and working on a solution. We don’t have any control over Unity’s process, or when they introduce breaking changes. Amplitude is built as a JavaScript library and aims to provide access to Web Audio API capabilities that Unity chose to not support. It communicates directly with the browser’s underlying Web Audio API, but is also dependent on Unity’s difficult to navigate WebGL build/conversion process. It works as intended in previous Unity builds and seems to only have issues with certain point releases.

As soon as we have a work around to the effected point release, we will release an update.

Michael

 

Amplitude 1.2.2 works as intended on Unity up to build 2021.2. The Unity and/or Web Audio API changes were introduced in 2021.3.

 

Any update on the fix to the API changes? I'm using Unity 2022.2.

I also tried Unity 2021.2.19f1. The example project works in the editor, but not in a Build and Run test on a chromium browser.

 

I have a test build partially working, but it's not ready for release yet.

 

Here with a project using Salsa and amplitude in unity 2021.3.23 looking forward to the bug fix. Thanks for the effort!

 

Hi Crazy-Minnow-Studio:
Sorry to bother you. Just checking on the status of this fix. Last post mentioning the test build was partially working sounded very promising! Any ETD available yet?
Regards!

 

I'm still testing on builds, two releases will likely be necessary to cover the version range since Unity keeps flip/flopping something on the back end that's causing issues. Should be published next week.

Michael

 

Hey @Crazy-Minnow-Studio any updates on the status for these new builds? Hoping to ship an update to my app with this working next week if at all possible

 

Hello! Version 1.2.3 should now be available and should address the issues introduced in newer Unity version updates. Please give it a try and let us know how it works for you.

Please, consider this a test to fix the current problems and let us know if you still run across issues introduced by Unity version changes. If you still find issues, please let us know the problem in as much detail as possible along with pertinent browser console log errors, warnings, or notices. Please also include the full Unity version and browser you are using.

D.

 

Hey D,

Tried a clean empty project in Unity 2022.2.17, downloaded new version of Amplitude for WebGL, added it to project, opened example scene, did a Build And Run to Edge, but no data is coming through still.

Here is the Edge version:


And here is the console log:

navigator.mediaDevices support available

WebGL.framework.js.gz:10 [UnityMemory] Configuration Parameters - Can be set up in boot.config

WebGL.framework.js.gz:10     "memorysetup-temp-allocator-size-audio-worker=65536"

WebGL.framework.js.gz:10     "memorysetup-temp-allocator-size-background-worker=32768"

WebGL.framework.js.gz:10     "memorysetup-bucket-allocator-granularity=16"WebGL.framework.js.gz:10     "memorysetup-bucket-allocator-bucket-count=8"

WebGL.framework.js.gz:10     "memorysetup-bucket-allocator-block-size=4194304"

WebGL.framework.js.gz:10     "memorysetup-bucket-allocator-block-count=1"

WebGL.framework.js.gz:10     "memorysetup-main-allocator-block-size=16777216"

WebGL.framework.js.gz:10     "memorysetup-thread-allocator-block-size=16777216"

WebGL.framework.js.gz:10     "memorysetup-gfx-main-allocator-block-size=16777216"

WebGL.framework.js.gz:10     "memorysetup-gfx-thread-allocator-block-size=16777216"

WebGL.framework.js.gz:10     "memorysetup-cache-allocator-block-size=4194304"

WebGL.framework.js.gz:10     "memorysetup-typetree-allocator-block-size=2097152"

WebGL.framework.js.gz:10     "memorysetup-profiler-bucket-allocator-granularity=16"

WebGL.framework.js.gz:10     "memorysetup-profiler-bucket-allocator-bucket-count=8"

WebGL.framework.js.gz:10     "memorysetup-profiler-bucket-allocator-block-size=4194304"

WebGL.framework.js.gz:10     "memorysetup-profiler-bucket-allocator-block-count=1"

WebGL.framework.js.gz:10     "memorysetup-profiler-allocator-block-size=16777216"

WebGL.framework.js.gz:10     "memorysetup-profiler-editor-allocator-block-size=1048576"

WebGL.framework.js.gz:10     "memorysetup-temp-allocator-size-main=4194304"

WebGL.framework.js.gz:10     "memorysetup-job-temp-allocator-block-size=2097152"

WebGL.framework.js.gz:10     "memorysetup-job-temp-allocator-block-size-background=1048576"

WebGL.framework.js.gz:10     "memorysetup-job-temp-allocator-reduction-small-platforms=262144"

WebGL.framework.js.gz:10 Loading player data from data.unity3d

WebGL.framework.js.gz:10 Initialize engine version: 2022.2.17f1 (54cb9bda89c4)

WebGL.loader.js:1 Creating WebGL 2.0 context.

WebGL.framework.js.gz:10 Renderer: WebKit WebGL

WebGL.framework.js.gz:10 Vendor:   WebKit

WebGL.framework.js.gz:10 Version:  OpenGL ES 3.0 (WebGL 2.0 (OpenGL ES 3.0 Chromium))

WebGL.framework.js.gz:10 GLES:     3

WebGL.framework.js.gz:10  EXT_color_buffer_float EXT_color_buffer_half_float EXT_disjoint_timer_query_webgl2 EXT_float_blend EXT_texture_compression_bptc EXT_texture_compression_rgtc EXT_texture_filter_anisotropic EXT_texture_norm16 KHR_parallel_shader_compile OES_draw_buffers_indexed OES_texture_float_linear OVR_multiview2 WEBGL_compressed_texture_s3tc WEBGL_compressed_texture_s3tc_srgb WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_lose_context WEBGL_multi_draw WEBGL_provoking_vertex GL_EXT_color_buffer_float GL_EXT_color_buffer_half_float GL_EXT_disjoint_timer_query_webgl2 GL_EXT_float_blend GL_EXT_texture_compression_bptc GL_EXT_texture_compression_rgtc GL_EXT_texture_filter_anisotropic GL_EXT_texture_norm16 GL_KHR_parallel_shader_compile GL_OES_draw_buffers_indexed GL_OES_texture_float_linear GL_OVR_multiview2 GL_WEBGL_compressed_texture_s3tc GL_WEBGL_compressed_texture_s3tc_srgb GL_WEBGL_debug_renderer_info GL_WEBGL_debug_shaders GL_WEBGL_lose_context GL_WEBGL_multi_draw GL_WEBGL_provoking_vertex

WebGL.framework.js.gz:10 OPENGL LOG: Creating OpenGL ES 3.0 graphics device ; Context level  <OpenGL ES 3.0> ; Context handle 1

17The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

_JS_Sound_Init @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10  The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

tryToResumeAudioContext @ WebGL.framework.js.gz:10

WebGL.framework.js.gz:10 Input Manager initialize...

WebGL.framework.js.gz:10 UnloadTime: 0.500000 ms

WebGL.loader.js:1 [UnityCache] 'http://localhost:51743/Build/WebGL.data.gz' successfully downloaded and stored in the indexedDB cache

WebGL.framework.js.gz:10 Failed to delete analyzer (3474) from source TypeError: analyzerObj.source.disconnect is not a function

 

Thanks @Rom- ,
I was able to reproduce this. Mike (dev) is out until the end of the week. He tried to get this pushed out prior to leaving and it appears something happened to the package and his fixes don't seem to be implemented. I'm doing some further testing to provide him as much information as possible for when he returns.

Thanks for everyone's patience,
D.

 

Running smoothly on Unity 2021.3.24, Chrome, Edge, and Firefox. Thx!!

 

Hello, can you please confirm me that SALSA Lipsync now work for Unity 2021.3.1+ on WebGL?

I'm trying to find a runtime lypsinc addon but so far all the other ones didn't worked on web.

 

Hello, we are still trying to resolve some issues with the Amplitude package. It currently appears to work for some and not for others. I would recommend holding off until we can resolve whatever changes are required or have fleshed out build process requirements. We will post back here when we have resolved any existing issues. Without Amplitude, SALSA will not work in WebGL out of the box. It may work with other WebGL audio analysis solutions, but might require some additional work. We appreciate your patience.

Thanks,
D

 

Amplitude for WebGL v1.2.4 has been released.
This version supports Unity version 2019.4.1 - 2023.beta+

Hello everyone!
Mike and I did quite a bit of testing to figure out what happened with the Amplitude build and it appears version 1.2.3 was mapping to the wrong versions of Unity and was not being delivered correctly from the store to the project. We did a bunch of testing to finalize the build requirements for the different versions of Unity and have it nailed down to the chart below. There are 3 versions required to support Unity 2019.4.1.LTS thru 2023.beta. Please see the chart below for details. Unity should provide you with the appropriate version when you download it via the AssetStore/PackageManager. However, be aware, if you upgrade a project from one version to another, it will be necessary to update Amplitude if you cross the version boundaries. We have no idea why the builds stopped working mid-branch; it is obvious Unity changed things up in the middle of things.

Please also note the caveat for the Amplitude build 2021 that does not support audio looping.



We greatly appreciate everyone's patience while we sorted this all out and apologize for any frustration incurred.

We will load all 3 versions on our downloads portal. If Package Manager won't deliver the correct version to you, you can download them from there. We have tested the delivery around a version build cusp and the correct version is delivered. For example, using Unity 2021.2.15, package manager delivers v1.2.4 build 2020, while Unity 2021.2.16 receives Amplitude v1.2.4 build 2021.

ALWAYS BACKUP YOUR PROJECT BEFORE INSTALLING THIS OR ANY OTHER ASSET/ADD-ON.
If you experience issues, please confirm you have the correct version in your project. You can check the read-me file and cross-reference that version with the chart above. If everything looks correct, please send us an email with your Invoice number and a detailed description of the issue you are experiencing. We are more than happy to help resolve any problems.

Thanks,
The Minnow Team

 

Hello everyone, hopefully this reply reaches the forum users that were having issues with, or curious about using Amplitude with respect to various Unity releases. As was mentioned in our previous post, v1.2.4 was released last night after a long marathon of process of elimination with respect to where Unity changed course and broke existing processes in Amplitude mid-branch. We believe the previous v1.2.3 package was not being delivered correctly in PackageManager and the wrong builds were being delivered which is why Mike's fixes were not working for everyone. In the chart in the previous post, you can see where we found the exact points Unity made changes that broke Amplitude.

We have three builds to address the breakpoints. We refer to them as 'build 2019', 'build 2020', and 'build 2021'. Unfortunately, as mentioned the breakpoints are mid-branch so these build names aren't entirely intuitive, but they are names we can reference together if someone gets stuck. Likely, most customers will be using 'build 2021' as it supports the most modern releases. We have tested PackageManager delivery and it appears to be working correctly. For example, using Unity 2021.2.15 -- build 2020 was correctly delivered to the project. Using Unity 2021.2.16 -- build 2021 was correctly delivered.

For anyone who was experiencing issues with Amplitude working in various Unity versions, please Update/Re-download and Import the new version (1.2.4). You can confirm you have the correct version by viewing the readme file included in the package and if you need to, you can cross-reference with the chart in the previous post or in the Amplitude Overview documentation.

Thanks for being patient with us and we apologize for the frustration -- we were frustrated too. Please let us know if you have any other issues with the package. We understand the dissatisfaction during these trying times, but hope you might try the package again and let us know it works for you.

Thanks,
D.

 

Hello D.
Thank you very much for your effort on all recent versions, I appreciate it very much.
I have to report some mixed results after updating and testing the new version.
I have been testing using Unity 2021.3.22 and Pixel Crushers dialogue system, with SALSA with Random Eyes (ver 1.5.5), Amplitude (1.2.4 build 2021) and AmplitudeSALSA (1.2.0).
Our Unity scene setup is simple in most of our products, we have an instructor talking to a trainee (the user). In general, we pre-load the audio files to the scene before starting the webGL game.
After several tests, I have concluded that the compatibility of our product with our webGL based clients has decreased. This is because now the lip sync animation setup we have, using the before mentioned SALSA products versions, works most of the time when using a Chrome web browser (this means: it triggers character's mouth/facial animation) but it seldomly works for Firefox (depending on how the dialogue nodes are setup in each conversation tree, sometimes it only works for the first node). Maybe needless to say but before all these new versions, lip sync animation used to work 100% of the time in both web browsers.
There are some workarounds I have found to force better performance on Firefox, though.
1. Pausing the game (after resuming the game, the NPC start lip sync animation)
2. Adding empty nodes in between the actual populated dialogue nodes.
3. Also, adding a small delta time delay in the actual dialogue node in the form of: "AudioWait(SoundFile)@0.25;" (any of these workarounds guarantee's a 100% animation trigger).
However, any of these would require a quite vast rework patching the dialogue's databases with this workaround, something we are trying to avoid.
Is there any setting or workaround you can think of, to make the game run in Firefox as it does in Chrome?
(I'm curious, is this something we should be using your "WebAudioUrls add-on" , instead?)
*I have a test file (webGL test) I can send you but the systems warned me it was too large to upload (29MB).
Any light you can shed on this issue will be truly appreciated.
Thank you very much for your attention.

 

Hello David,
This sounds like a web resource async issue. Does this happen with all audio files or only the ones that you aren't preloading? I'm making the assumption your are fetching the remaining audio from the web. The fact that it works when you add a delay is a red-flag for an async problem. You could implement the WebAudioUrls example and make changes as necessary or simply roll-your-own. Keep in mind, WebAudioUrls is example, helper code. It hasn't been updated in a while, but the core mechanics should work. I think Unity has continued to make changes in their async implementation so you might find some obsolete calls in it. And the examples for WebAudioUrls are for SALSA LipSync Suite v2 -- you are on the old version of SALSA which hasn't been supported/updated for several years. All this is to say, I doubt you will have a plug-and-play experience. To sum it up, from your description, it definitely sounds like the audio isn't fully ready when you start using it. Adding an arbitrary delay appears to fix it, so it appears to be async timing from a 10,000 foot view.

We also aren't aware of any Firefox inconsistencies, we mostly use Firefox to test. But all browsers implement WebGL in their own way, so I cannot speak too authoritatively on that subject. We do know that Safari still does not work -- doesn't support the same implementation as Chromium and Firefox, although there are some indications this is potentially a MacOS or Safari permissions issue.

It could also be something indirectly in the old SALSA with RandomEyes code. v1 performed audio analysis very differently from v2. I am *not* saying an upgrade from v1 to v2 will resolve the issue, just pointing out the v1 code is different.

Async issues are not Amplitude/SALSA issues but rather Unity and Web implementation design issues. As mentioned earlier, Unity has made a lot of changes over time in their Web resource acquisition API. Another assumption I am making is you recently upgraded your project(s) to a newer Unity version, which required an Amplitude update in turn because of the changes Unity implemented in their underlying systems. As to why it is occurring only in Firefox, I don't have an answer for that. But if you can not find any other solution, perhaps you can extend your audio calls or use a new action to call the audio that includes the delay? That would probably be a better question to ask Pixel Crushers. They may even have a better way to handle async Web calls.

Hope this helps,
D

 

It does help. Thank you for the advice!
*It happens with all audio files, all dialogue audio files are preloaded before launching.

 

if i select my audio in unity, and in inspector i check mark the Override for webGl, and select load type compress in memory _, and after creating build, play on my browser that time Lip synch not working, load memory issue pops up on browser, any suggestion, and when i select my audio and load type decompress on load after i play on browser only audio play not lip sync, plz answer

 

Please send your SALSA and Amplitude for WebGL invoice numbers to assetsupport@crazyminnow.com. Thanks!

 

Hi, I'm trying Amplitude for WebGL analyzing multiple audio clips in sequence at runtime. For some reason, if I load an audioclip from the Resources folder it works properly, but if I load an audioclip that i received from an HTTP web request, it doesn't work with the error: "Could not find analyzer XXXXX".

In the editor it is working as intended, but I keep getting this error in the WebGL build. I cannot understand why the plugin cannot recreate the audio analyzer after it get deleted if the audio clip is downloaded at runtime.

 

C0lonnello,

Please send your Amplitude for WebGL invoice number to assetsupport@crazyminnow.com, and reference your forum name so we know who the email is coming from.

Streamed audio files must be properly handled with async operations to ensure the audio files are in a ready state before you play them. Non-ready state audio can be played, but cannot be analyzed. This is a requirement of your browser’s underlying WebAudio API, which is what Amplitude for WebGL uses to analyze audio. Here is an example async implementation, feel free to implement your own.
https://crazyminnowstudio.com/docs/amplitude/addons/webaudiourls/

The Unity Editor is not representative of the final WebGL result. Unity uses FMOD as the underlying audio framework in editor, and WebAudio API as the audio framework in WebGL builds.

 

Hi @Crazy-Minnow-Studio , I'm trying to get Amplitude for WebGL working with Salsa Lipsync but I keep getting the "Could not find analyzer" error in console.
We have the following setup:
- Unity 2022.3.12f1
- Amplitude version 1.2.4, build 2021
- Salsa v1.5.5

All the audio is preloaded before playing, with the following audio settings:


Any help would be greatly appreciated!

 

My first assumption is an issue with the audio compression. Please try wav format audio.

Michael