Audio Playable Under the Hood

I'm trying to figure out how the Audio tracks on the timeline work. Unfortunately, even if you open up the timeline DLL, all the Audio playable stuff is internal calls, so there's no way to see what's actually being called by e.g. ProcessFrame.

As near as I can figure, when an Audio clip plays, it triggers a weird kind of OneShot or similar to play the clip. This is really a strange design decision, and a strange implementation:

* If you change parameters on the AudioSource while the clip is playing, it updates the audio. For instance, if you increase the volume or change the spatial blend, the playing clip changes accordingly.
* Certain fields seem to be ignored; for instance, changing the pitch on the AudioSource has no effect on playback
* There is no way to tell if the AudioSource is currently being used to play a timeline clip. AudioSource.isPlaying returns false, and AudioSource.clip returns null.

I have to ask: why was this the choice for how audio should work in the timeline? Why do the AudioMixerPlayable, AudioPlayableAsset, etc. rely on so many internal calls? And why isn't there a
PlayableBehaviour for audio clips like there are with the other, sample track types?

 

Having said all that, here's my current conundrum:

We have an internal audio management system for our application. It takes care of ongoing sounds (e.g. environment audio), one shot sounds, and important 2D / VO tracks. When we play important tracks, we duck ongoing sounds & do some other things.

I want to use the timeline for a tutorial sequence, but I cannot for the life of me figure out how to connect the audio track there to our audio management system. I need to know when the timeline starts playing important audio so I can do what I need to do elsewhere.

If I could just have access to more of the code, I could just create a derived track type, say ImportantAudioPlayableAsset, that would then talk to our audio system when the timeline plays an audio clip. However, everything is so hidden and not-meant-to-be-read that I've run into a brick wall here.

So is there some other way that I can elegantly tell when the timeline is playing audio on one or more particular audio tracks & let our audio management system know, so that it can appropriately duck other sounds, cancel any ongoing 'important' audio tracks, etc.?

 

Have you worked with Audio Mixers? A simple solution would be to make an Audio Mixer with an "important" audio channel that other audio channels duck under. You would then set the Output audio channel of your important AudioSources to the important audio channel and all non-important AudioSources to other channels. This will cause the other sounds to duck when the important sounds are playing. If this solution works for your, here are a couple tutorial to get your started:

 

Thanks for the suggestion!

That solves part of the problem. However, there are a number of other issues. For instance, one thing our audio manager does is rank important audio. When you load a new audio clip, it checks its importance vs any currently playing important audio and decides whether to skip playing it or to stop the current audio and play the new audio instead.

There are other things that our system manages too - for instance, controlling volume of playing clips during scene fade. With the way the timelines are put together, I can't see how I can get enough information out of it to tie it into our audio manager. So my problem is not a lack of knowledge about the audio system; what I really need is either:

* Callbacks for all timeline events (track start/end, clip start/end, clip process) that I can register for
or
* Proper access to the existing track scripts and the ability to inherit from them - meaning things should be marked virtual, no classes should be sealed, no classes should be marked internal in a DLL, and actually having access to all the logic (which really should be in C# - there's no speed reason I can see to hide big chunks of this in C++)
or
* A sample template that acts just like the existing audio tracks that I can modify - I really don't have time to try to reinvent the wheel here

 

I do not believe you can get all the features you're looking for from the built-in AudioSource Track. You can use the clip's weight to see when the Timeline enters and exits the clip (start/end). These are "IsDone()", "IsActiveAndEnabled()" and "GetTime()" functions that can be used to determine the state of your Timeline and its TrackAssets. But ultimately, Timeline--and out-of-the-box audio support for Timeline--is still really early.

It is possible to write your own custom track to trigger audio objects that your audio manager processes. With this approach you only use the Timeline as an event system (for easy timing implementation) and let your audio manager to the heavy lifting.

 

(Note: the post from Optera above was me - I was on a different machine and logged into a different account)

Indeed! And that, really, is my rant: that this system is built where things are hidden and not modifiable / extensible by the user. I'm hoping that someone from Unity sees this and we can get a conversation going on what users are trying to do and how to make the timeline clips more customizable / extensible for various use cases.

I could certainly write my own custom track - and maybe I'll consider that eventually. But the current audio tracks have a lot of nice features that would be very hard to replicate: they show the waveform, for instance, and snap to the end of the clip length. I've written a few custom tracks - for pausing the timeline, for kicking off events from the timeline - and I have no idea how to replicate those audio features yet.

So I'll keep an eye on this thread, and see if anyone comes along with better ideas.

Thanks for the input!