Audio Resonance Audio SDK for Unity: Deliver High-Fidelity Spatial Audio at Scale

Does support webgl?

 

Congratulations, great news!

Already downloading at the SoundFellas.com game audio labs!

Looking forward to start using it and be part of the community.

Cheers!

 

Exciting stuff! I really enjoyed the demos.

I have a question about pairing Google Resonance with Native Audio Plugins, such as the examples by Unity located here: ( https://bitbucket.org/Unity-Technologies/nativeaudioplugins ).

Is it possible to process native audio plugin effects with Google Resonance? (I may be phrasing this poorly, please bear with me.) For example, in the plugin demos I've linked above, there are some demo synthesizer plugins, such as the "DemoTeeBee303". If I understand correctly, they generate sounds as an effect – not your typical audio source with an audio clip. Can this be processed by the Google Resonance spatializer plugin?

I hope I've expressed that clearly enough, I'm still pretty new to all this. Any thoughts would be a huge help. Much thanks!

 

The audio positioning is incredibly good.

It's so good that even using JBL monitors, not headphones, I can pinpoint accurately where a sounds come from even from the back. Wow.

In a shooter or an RTS with a large map, locating threats is usually done with visual cues. With audio imaging this good, visuals can be replaced by audio, like in real life.

 

I tested v1.0.0 (the .package file) in Unity 2017.1.2p2, using the provided ResonanceAudioDemo.unity scene.
However, there was no audio was hearable first.

I had to open "Edit > Project Settings > Audio" and change "Spatializer Plugin" and "Ambisonic Decoder Plugin" from "None" to "Resonance Audio", as shown below.

This made the example demo work for me.

 

It's one of the tricks that are in the google documentation. I usually don't read those but this one is concise and complete...

...Except that I still wonder if occlusion objects can be given a material to provide an absorption rate. Baked occlusion have as a material linked to a material, not dynamic occluders it seems.

PS: dynamic occlusion! wow! to do that, change the resonanceAudioListener's occlusion mask to everything and add a cube in the room, then move behind the cube voila!

 

Hi!

Does it work with consoles and for traditional 3rd person games (non VR) ? I'd like to build my whole game with rich audio, and for larger levels....

Thanks.

 

It's really nice feature, already played with it

It's been implemented via Physics.RaycastNonAlloc, so... not sure how fast it is if many audio sources enable occlusion.

 

According to the blog-post, Consoles don't seem to be supported (yet?):

Non-VR applications seem to be supported. I built and ran the provided example project without VR hardware and it seemed fine to me (Windows 10 desktop).

 

I'd like official verification that all relevant consoles will be supported or it's a complete non starter for us.

 

I'm curious how this compares with SteamAudio, which I'm currently using. If I see any real advantages, I may switch over.

 

Question time:

1. Is the transition between overlapping audio probe lerped through time instead of position?
2. Audio profiler shows cpu% is at 8% on a 1700x, are these 8% on another thread?
3. Max distance has no effect on sound, is volume how you let the spatializer determine the sound's reach?
4. if distance has no effect on the sound's volume, how is spatial property affecting the sound with this plugin?
5. How do I simulate atmospheric muddying out the sound? Same trick of cranking up the "spacial" curve and adding a low pass filter or something specific to resonance?
6. looking through the code, a float is sent to resonance for occlusion and resonance does it magic, how can we tweak the effect of occlusion?

 

Just to add on to some of the points Michael answered:
3) When distance attenuation curve is selected as logarithmic, Unity does not zero out the distance attenuation at max distance, but rather stops attenuating sound at that point. That may possibly be what you are experiencing?
6) It is actually possible to tweak the occlusion value beyond 1.0, and also to use fractional values such as 0.5 as desired - by modifying the ResonanceAudio.ComputeOcclusion method accordingly. The resulting value will be used as an input value continuously mapped to the occlusion filter.

 

There's actually a demo in the native audio plugin examples (speakerrouting) that routes audio from a native audio plugin running on the mixer *into* an Audio Source using OnAudioFilterRead. Then the spatialization can run on that stream and route it to another Audio Mixer Group.
I had initially thought this wouldn't work because there's a bug in Unity where results from OnAudioFilter read aren't panned left and right with the default spatialization. But, if you use a spatialization plugin instead it works fine.

@slufter - FYI - adding this routing into Audio Helm in the next version.

 

57% audio cpu
19% unity cpu
so total Audio CPU isn't the hardware's cpu, anyone knows what it is?

 

Just to answer the transition question (1).

You are right, there is no positional-interpolation between overlapping probes. A switch of probes happens immediately when crossing boundaries of probes. See https://developers.google.com/reson...guide#understanding_overlapping_reverb_probes

However, in order to avoid artifacts, we let the reverb tail of the last applied probe "run its course" or die out. In that sense, yes, you may say there is a temporal transition.

 

It's a strange choice and doesn't sound very good. Is it a design choice or a first release limitation?

 

Thanks for the feedback! It was a choice mainly to be consistent with how Audio Rooms handle transitions, so as to not surprise old users when they start using Reverb Probes together with Audio Rooms. But definitely we can look into the transitions for both and potentially change them so they are consistent AND better. Could you create an issue on https://github.com/resonance-audio/resonance-audio-unity-sdk/issues and label it as a "feature request"?

Thanks again.

 

https://github.com/resonance-audio/resonance-audio-unity-sdk/issues/6
What is audio room?

 

Here is a bit of audio room related documentation...

Audio Room
https://developers.google.com/resonance-audio/develop/design-tips#audio_room
https://developers.google.com/resonance-audio/develop/unity/developer-guide#room_effects_in_unity

ResonanceAudioRoom API
https://developers.google.com/resonance-audio/reference/unity/class/resonance-audio-room

 

Hi,
Is it possible to sync an ambisonic audio file to a video.
With the FB360 plugin for the Gear VR, we can sync the TBE to the millisecond of a video, so its always sync.

Is there anyway to do the same thing with the resonance audiosourcefield ?

Thanks.

 

Hi,

About the occlusion effect, first of all, the acoustic materials we provide are for reflectivity only, and not for transmission. Basically for an input energy, we only model how much is reflected back. The part that is not reflected could be either absorbed or transmitted (passing through the object), and we don't yet have a distinction between the latter two. In the future we might consider providing this feature.

But before that, you can take a reference of our ResonanceAudio.ComputeOcclusion() and do something similar to compute an occlusion value (which is a positive floating point, with 0 meaning no occlusion and higher values meaning more occlusion) that better suits your need.

Thanks!

 

Do you know how use .tbe with Google resonance?

 

Resonance Audio works for traditional 3rd person games (non VR) on mobile/desktop platforms, using stereo output via headphones. However, it does not currently support consoles because that relies on different CPU architectures and would require channel-based audio output for the majority of users, who don't wear headphones. If RA were to support consoles, which RA features and console platforms would you find most relevant?

 

It is indeed possible to sync the Ambisonic Soundfield recordings made by Resonance Audio from within Unity with video playback in Unity. If you know the timestamp of the video playing back, you can check that against the 'time' field on the Unity AudioSource which is playing the ambisonic soundfield, and reset it if it is too far out of sync.

For example:

if (Mathf.Abs(videoPlayer.timeStamp - audioSource.time) > syncThreshold)
audioSource.time = videoPlayer.timeStamp;

of course, this will cause a pop in the audio, so, syncThreshold should be set to a high number so it only really happens if there was a buffering problem with the video (and when the video first finishes buffering).

 

Hello! I would like to support PS4 and Xbox One. Switch would be nice but as it's quite portable, so no loss.

Am I wrong to think this would enhance ordinary non VR experiences? - perhaps I am baking up the wrong tree as I didn't realise headphones would be mandatory, sorry

 

You're not wrong, even with speakers on each side of my monitor I notice a much improved positioning, it's even helpful in an RTS type camera

 

do you use it?

 

I wonder, if that possible that the soundfield (or entire sdk) does not work in Unity 2017.3?
Actually, on iOs, I had to update the pods manually via the terminal to make the sdk working,
but on Android no luck so far.

Using:
Unity 2017.3.0f3
Resonance Audio SDK for Unity v1.1.1

I get those errors in Android Device Monitor:
Unable to find libaudiopluginresonanceaudio

Audio effect Resonance Audio Renderer could not be found.
Check that the project contains the correct native audio plugin
libraries and that the importer settings are set up correctly.

In Unity 2017.2, everythings worked perfectly!

Any idea?

 

Are Unity Mixer Groups supported on the default ResonanceAudioMixer?

I have a few groups setup on it, and sliders to adjust it down, pretty standard issue volume controls for music/sfx/master, etc. The only one that seems to have any effect is the Master group.

I have a fire that is playing resonanceaudiosource with the audiosource component output set to the SFX group. Turn down the SFX group and it does nothing (despite seeing the levels change in the editor). But gets quieter when I turn down the master volume group.

 

Same thing.

 

I’ve tried to use Mixer Groups as well. Yes, only the Master Group (=the Group with the spatializer renderer effect) seems to change audio volume. I guess this is due to the way Resonance Audio handels spatialization. According to this blogpost (https://blogs.unity3d.com/2017/11/0...io-high-fidelity-sound-across-mobile-desktop/) it converts all clips into ambisonic and then spatializes all of them only once - for efficiency reasons. I imagine that only after using the spatializer renderer effect in the respective Mixer Group, the clips leave the Resonance Audio pipeline and can be handled the usual way in the Mixer.
You could try using multiple Groups each with a spatializer renderer effect and mix them. I haven’t tried this yet. Even if it works it might be less CPU efficient...

 

1) According to the documentation (and the prefab in the unity package) the Spatialize checkbox must be disabled in the AudioSource component when adding an Ambisonic soundfield. Tried this with an Ambisonic clip. Only worked when the checkbox was ENabled. Otherwise everything is silent. Or am I missing some information?
EDIT: Sorry, I was wrong. After disabling and reenabling the Ambisonic checkbox in the AudioClip import settings (clip recorded with the Soundfield Recorder of the Resonance Audio Listener component) the soundfield works as expected with the Spatialize checkbox disabled in the AudioSource component.

2) In the reverb baking demo scene: If I put a spatialized AudioSource somewhere in the church, the reverb is applied as long as the Listener is inside the building. When leaving the church and waiting at the door the sound from inside the church is audible, but has no reverb, which is not the natural experience. I’ve read the paragraph about overlapping probes and don’t see a way to apply the reverb to sources inside a probe when the Listener is outside but near the respective probe. Any ideas (except putting an extra effect on the AudioSource)?

 

Hi,
Resonance works fine in a Unity project, however it does not seem to work when exporting the Unity project as a webGL application. Is this supported ?

This is the error message in the Firefox console :
Audio effect Resonance Audio Renderer could not be found. Check that the project contains the correct native audio plugin libraries and that the importer settings are set up correctly.
Audio source is playing an ambisonic audio clip, but an ambisonic decoder could not be created successfully. Make sure an ambisonic decoder is selected in the audio settings.

Thanks

 

pretty sure it's only for desktop applications.

 

So I understand that now Resonance Audio is built into Unity 2018. However I'm a bit confused how to use this built-in version. I see you can choose Resonance Audio as the Spatializer and Ambisonic Decoder, but None of the Components seem to exist.

Do you still need to import the older SDK to get access to the components, or how do you go about this? or is this a stripped down version that works through Unity's standard components?

 

I have noticed that if scene has resonance audio source, but it doesn't play (not needed right now), it breaks a stereo effect on all sounds.

 

Hello, for some reason on Mac, although everything works fine inside Unity, I lose both Resonance Audio Room and Resonance Audio Reverb Probes when exporting as an app. This is highly frustrating, I also tried with the demos and it was the same, with both baked and/or not baked reverbs. I haven't found anyone complaining about it yet though, so I was wondering if anyone else could make it work...? Thanks in advance!

 

Hey,

If anyone can help me, please.

I installed Resonance Audio yesterday and since then every time I click on a component with Resonance Audio Source my terrain goes black and I get this giant crosshair that mirrors the Listener and Source Directivity icons from the Resonance Audio Source component.

I'm sure there is something obvious I'm missing but at the moment it's not obvious. Any help would be appreciated.

looks like this:


Thanks in Advance
Nathan

 

It's one of the traps that are in the google documentation. I as a rule don't peruse those yet this one is succinct and finish...

...Then again, actually despite everything I think about whether impediment s can be given a material to give a retention rate. Prepared impediment have as a material connected to a material, not dynamic occluders it appears.

PS: dynamic impediment! amazing! to do that, change the resonanceAudioListener's impediment cover to everything and include a 3D shape in the room, at that point move behind the 3D shape

 

So... I see above that folks have asked about the fact that the included version in Unity 2018 doesn't seem to expose any of the components previously mentioned such as the Room, acoustic materials, etc but the github repo for the Resonance SDK hasn't been updated in a year. Do you need to still import the SDK separately to get those features? Does it cause a collision with the included Resonance spatializer/ambisonic plugin?

 

I never got an answer to that. but it seems you can just import the repository and set up as usual and everything is fine. So I honestly don't know what benefit being 'built in' brings or even if that version gets used. I guess there must be some performance benefit to them doing that or something, but I wish they would explain and clarify.

 

Devs never appear here, and not update long ago. But I'm still using Resonance (from Github) in unity 2019 with HDRP and everything works (exept some minor bugs). Of course, I would like a more friendly solution.

 

I'm getting two copies of the framework in my iOS builds but I just delete one every time. I do get the feeling this is another case of "open-sourced" == abandoned.

 

Hi it's possible to rec 3D sound and export ? here
thank's

 

No, as far as I know.

 

ok thank's i will check that's later