Koreographer - Audio Driven Events, Animation, and Gameplay

Hiya - I got Koreographer to handle highlighting text when speech is said. I'm trying to modify the karaoke controller but it seems to be set up to use multiple lines and animation. Is there a simple way to have a block of text that doesn't move, but have a highlighted overlay in time to the audio?

Cheers!

 

Great! Thanks for picking up Koreographer! It should definitely help you with that problem!

Hmm... it would be helpful to get a better picture of what you're trying to build. From what you've written, it seems like the Karaoke Demo included with Koreographer effectively does this already.

The line-feed animation portion is controlled by the KaraokeController MonoBehaviour while the word painting is controlled by the KaraokeTextLine MonoBehaviour. How these are built and how they work is outlined in the Karaoke Demo Overview documentation included with the demo. Basically, you should only need the TextRow prefab included with the demo as well as some custom control scripts (something to replace KaraokeController, which was built to handle the animation portion).

Once you have your custom control script, you would only need to create your Koreography assets which contain all the events and payloads to drive the highlighting. Does this make sense?

 

How do I set the music player to start 2 minutes in?

I have a 5 minute music track and testing events 3 minutes in gets tiresome when having to listen to the first 3 minutes over and over.

Is this possible?

 

Sure is possible! You can write a simple script that, in Start, will adjust the location of the music player to your desired location. The process is outlined in this forum post.

This is the main code outlined in that post:

Code (CSharp):

1. using UnityEngine;
2. using SonicBloom.Koreo;
3. using SonicBloom.Koreo.Players;
4.  
5. // ...
6.  
7. // "Public" so it can be set in the Inspector.
8. public SimpleMusicPlayer simplePlayer;
9. public float startTimeInSeconds;
10. public Koreography playbackKoreo;
11.  
12. // ...
13.  
14. void Start()
15. {
16.     // Convert seconds to samples with the sample rate of the audio in the Koreography.
17.     int startTimeInSamples = startTimeInSeconds * playbackKoreo.SampleRate;
18.  
19.     // If you do not set a Koreography to the SimpleMusicPlayer in the Inspector, nothing will play.
20.     //  This will instruct the player to play.
21.     simplePlayer.LoadSong(playbackKoreo, startTimeInSamples);
22. }

While the above example uses the LoadSong API, you can just as easily use the Music Players' SeekToSample API, which will allow you to jump to a specific location in the currently playing music at any time.

 

Most excellent. Thanks a bunch!

 

Hi! Very cool looking asset. I'm seeing that Koreographer pro can generate events on the fly. Does this mean it could work for a rhythm game where users could use their own music? For example, for iOS if I had a user import a track using something like https://assetstore.unity.com/packages/tools/integration/ios-music-31128 could Koreographer parse the beats and set up events automagically in real time (or perhaps with a slight loading wait time)? Just basic downbeats is what I'm looking for and would be awesome if players could import their own tracks. Thank you! Apologies if this has been asked and answered before -- looked through the forums but didn't spot it.

 

Thanks for reaching out! Unfortunately, Koreographer actually can't generate events on the fly. The Analysis features that you are likely referring to only run at edit-time in the Koreography Editor. As for generating a rhythm game using the user's music, I would point you to this FAQ entry on the Koreographer forums.

You could build your own runtime analysis functionality and use it alongside Koreographer (analysis would be a preprocess step to generate Koreography data used by the system). At present, such analysis algorithms are highly targeted towards a specific use case (e.g. genre of music, gameplay/visualization, etc.), which is why we haven't provided anything yet. We are looking into it at present (though have nothing to announce) and would be happy to help you integrate a custom analysis solution with Koreographer for runtime playback. I hope this is helpful!

 

Okay cool -- appreciate the thoughtful reply and the ref to the detailed FAQ. It does sound like a pandora's box! The needs for my project are pretty simple and I think I could fudge some things. If I *did* manage to parse some faked beats from a track at runtime, could you point me to any docs on how I could procedurally create koreographer events on the fly? Is any of that exposed? What I'm seeing so far seems very locked to the editor...

 

Yes! That's entirely feasible. Here's the process:

1. Player loads music.
2. Analysis algorithm performs its magic.
3. Koreography and Koreography Track(s) Assets are created.
4. Detected timing and any metadata is used to generate KoreographyEvents which are added to Koreography Track(s).
5. Music (and Koreography) is loaded.
6. Music plays and Koreography sends out event triggers.

Unfortunately we don't yet have a browsable API, but rest assured all of that is 100% doable. The Rhythm Gameplay demo does something fairly similar, albeit without steps 1 and 2.

Hope this helps!

 

Very cool! Thanks for the quick reply. Just bought the pro version of Koreographer -- look forward to putting to it work. Also, you see this? Looks interesting... https://medium.com/giant-scam/algorithmic-beat-mapping-in-unity-intro-d4c2c25d2f27

 

Thanks very much! We hope it is of great help with your projects!

I hadn't previously seen that. Very good article! It looks like the algorithm used in that article is based on Spectral Flux - one of the best current [non-AI based] algorithms for finding onsets. If you're interested in learning more about it (and loads more related stuff), check out these pages (Warning: this is an entrance to a massive rabbit hole):

1. VAMP Plugins (for use with applications like Sonic Visualiser, Sonic Annotator, and others).
2. Librosa (Python).
3. Essentia (C++).

Each of those pages has links to various papers, references, and source code. Have fun!

 

Great! Plugin is working great and is a lot of fun to use. Thanks again!

 

Really enjoying using your plugin! Thank you so much for such a great tool. I wondered if I could make a feature request? I think being able to set the color of your keyframes in the koreographer interface would be super useful. Especially for the creation of something like a rhythm game. That way one could see the payloads visually as you create them. Right now it's a little difficult to tell what's going on at a glance without rolling over every key to see what's happening. Just a thought -- something simple I would imagine that could make a big difference to dev workflow using the tool! Thank you.

 

Thank you very much! And of course!

That's a very interesting idea! A few questions to get a better idea of what such a feature implementation would look like:

1. Do you envision tagging each Koreography Event with an Editor-only color? Or are you already tagging with Color Payloads and want to see what those are?
2. How do you envision being able to set the color of one or more Koreography Events in the User Interface?
   • One example might be setting up some "rules" for Koreography. E.g. "if the Int Payload is 6, set it to [Color]". Other ideas or intuitions?

Any more information you can provide us on what an implementation of such a feature would look like would be very helpful!

Also, for quick checking of Payloads, we recommend using the "Select All" feature ([Ctrl/CMD+A] in the Koreography Editor). It will select all events and show the "peak" UI for each configured Koreography Event.

 

Thanks for your awesomely quick reply! Yes -- I was hoping to tag int events with editor only colors. I don't mind setting them myself manually in the editor (ie maybe just a little color swatch when you click on an event allows you to set the Editor color). If there's a way to set them automatically by int, even better! But that sounds more complicated to pull off from a UI perspective. Certainly allowing users to manually just set keyframe colors would be an amazingly helpful feature...

Ah -- select all is certainly useful. I think seeing them as colored keys could make dev even simpler and more enjoyable (you could then 'see' your rhythms at a glance)! Very grateful for your reply!

 

That's extremely helpful! I'll make sure all of this info makes it into the writeup in our feature request tracker!

 

Hi there

I need to get a reference to AudioSourceVisor to add it programmatically and assign the Target Koreographer from a reference in my script, is there a way to this? I am having trouble declaring an instance of AudioSourcesVisor in my project.

Thanks

 

There are two parts to this question.

1. Accessing the AudioSourceVisor type in scripts
To do this, you will need to ensure that you are using the SonicBloom.Koreo.Players namespace. This namespace provides access to the player components in Koreographer. See:

Code (CSharp):

1. using UnityEngine;
2. using SonicBloom.Koreo.Players;  // <-- Add this!
3.  
4. public class Test : MonoBehaviour
5. {
6.     public AudioSourceVisor visorCom;  // Works!
7. }

2. Assigning the Target Koreographer reference in scripts
Further, Koreographer's API does not currently provide direct access to the targetKoreographer field (they are marked private). This is due to ambiguity with what transitioning between Koreographer components does - if the player is a Music Player do you transition "Music Player status" between the two (breaking the connection to the original)? Do you leave it? Also for Music Players, do you unregister the Koreography that might have been auto-registered and then re-register with the new Koreographer component? It is not knowing the answers to these questions in advance that kept us from making the targetKoreographer field public by default.

As things stand today, updating the field directly with reflection would also simply not work. What you would actually need to adjust is the koreographerCom field of the AudioSourceVisor's internal AudioVisor field (named visor). If everything were public it would look like this:

visorCom.visor.koreographerCom​

You could update that value with reflection yourself and it should work fine. If you want to attempt this, we would be happy to show you how to do so using reflection.

That said, there may be a simpler solution.

Why don't you just have two separate AudioSourceVisors with different Target Koreographer settings configured that you switch between? You could easily emulate overriding a Target Koreographer in this manner by simply pointing two AudioSourceVisor components at the same AudioSource component and then switch their enabled states (enable one; disable the other). This could be expanded to work with any number of Koreographer components.

Is there a reason that you would prefer not to take that approach? Could you explain a bit more about your specific use case?

 

This was probably asked before, but is there a way to trigger Koreograph events on the Unity TImelineso that I can see the waveform there? I am trying to sync an audio driven events in Koreograph with some camera movements and other stuff in the Unity Timeline. Is there a way to easily align these two timelines? Right now, I am triggering the Koreograph timeline in the Unity timeline by simply animating the Audio Source in the Music Player to turn on at a certain time, but beyond that, it's tricky to see where the things happen in the audio. Here's a screenshot.



See, the events start there, but beyond that, I'm not sure how I can accurately sync themwith the unity timeline. Can there be some kind of Koreographer track? Just an idea.

Another unrelated question: I emailed you a bit ago about getting access to the Wwise integration. It's from a Davidm address. Did you ever receive it?

Thanks!

 

I'm not sure I'm clear on the goal. If you would like to align Timeline clips along the audio waveform in the Timeline UI, you can add an Audio Track to your Timeline asset (in the track dropdown select "Add From AudioClip" to pick the AudioClip you want) and then time events against that. When you're done with timing other clicks to the music, you could just delete that Audio Track. Would this help you accomplish your goal?

Currently, it doesn't look like it's possible to add custom Track UI to Timeline Clips along Timeline Tracks (at least according to this forum post), so, at time of writing, this may be a non-starter :/

Yup! We received it! Sent a response as well on the same day! Check your spam folder, perhaps? (We recently saw an issue with our mail configuration that we've since addressed.) I'll resend our response, just in case.

 

ALright... LOL OH YEAH! That actually makes a crap ton of sense. Good idea! I wish Unity would allow you to hear the audio outside of the game though. AH well it's still new.

ALso, ignore my email asking you to resend the email, I found it. It was hidden pretty deep in the spam folder.

 

I'm not sure they have scrubbing as Koreographer does (hold ALT/Option while moving the mouse over the timeline for a preview), but it will actually play the audio if you "play" the timeline...

Got it! Thanks for looking and sorry about the spam situation. We've adjusted our mail server setup so that that should no longer occur! Glad to hear we've figured out where it went, though!

 

Hello sir!

Quick question:
Does the .GetMusicBeatTime take the audio delay, that is assigned in .EventDelayInSeconds, in account?

(Awesome plugin btw)

 

Hello!

It doesn't, unfortunately. The Koreographer.GetMusicBeatTime method uses the raw audio time reported by the underlying AudioSource (or equivalent in integrations), rather than one the value that's been offset.

Thanks very much for pointing this out to us! We were under the mistaken impression that this was already handled properly. Luckily, there's a quick workaround that you can implement.

To get the current Music Beat Time while taking into account the configured Event Delay In Seconds value, do the following:

Code (CSharp):

1. Koreography koreo = ... // Get reference to music Koreography.
2. double beatTime = koreo.GetBeatTimeFromSampleTime(koreo.GetLatestSampleTime());

The first step is to access the Koreography object that drives the Music Time APIs. How you do this will change depending on your setup. At runtime, the Koreography object keeps track of the latest update sample positions (both start and end). You can access the latest sample time processed by using the GetLatestSampleTime() method and then convert it to beats with the Koreography's GetBeatTimeFromSampleTime method. When fixed, this is exactly how the Koreography component's API will function. As good as workarounds can get!

At time of writing, only the Koreographer.GetMusicSampleTimeDelta and Koreographer.GetMusicSecondsTimeDelta respect the Event Delay In Seconds value. We will make sure this is addressed in a future release.

Hope this helps!

(Thanks very much!)

 

Thank you for your quick response!
Helps a lot with our upcoming game so we can let the players adjust the delay to be more than deltaTime between beats!
To be honest I first had my own beatTime counter that counted from the events and then changed it to get the BeatTime directly from the koreographer when I noticed there was that kind of option.
Will be using your workaround now on because I think its more reliable than counting from events .
Keep up the good work!

 

Hi. We're having an issue in Koreographer Pro with synchronizing multiple sounds: a) A Beat and b) A Metronome sound. We used the solution you posted earlier (see below). The intervals between events are right but every time we schedule a Metronome beep it come in earlier than it should (about 1/16th earlier than the Beat). This continues throughout the track.

Here is our code followed by the solution on which we based our code provided by you on this forum.

OUR CODE:

Code (CSharp):

1. //Update current frame sample time
2.             currentFrameSampleTime = Koreographer.GetSampleTime();
3.  
4.             //Clear event list and get events within specified range
5.  
6.             koreoEvents.Clear();
7.  
8.             koreoTrack.GetEventsInRange(
9.                 lastFrameSampleTime + scheduledSampleOffset,
10.                 currentFrameSampleTime + scheduledSampleOffset,
11.                 koreoEvents);
12.  
13.             //If event count is not equal to zero, schedule a beep with the AudioSource component
14.  
15.             if (koreoEvents.Count > 0)
16.             {
17.                 foreach (KoreographyEvent koreoEvent in koreoEvents)
18.                 {
19.                     //Get time from current moment to execution of event
20.                     eventTimeFromNow = (koreoEvent.StartSample - currentFrameSampleTime) / levelKoreography.SampleRate;
21.                     //Calculate time on DSP timeline to play the beep
22.                     timeToPlayBeep = AudioSettings.dspTime + eventTimeFromNow;
23.                     //Schedule the beep
24.                     beepSource.PlayScheduled(timeToPlayBeep);
25.                 }
26.             }
27.  
28.             lastFrameSampleTime = currentFrameSampleTime;

YOUR CODE:

Yes. This is doable with a little extra work. In order to do this you will need to make use of the AudioSource's PlayScheduled feature. Here is an outline of how you might accomplish this:

1. In Start(), initialize a "scheduleSampleOffset" value such that:
   scheduleSampleOffset = dspBufferSize * (numDSPBuffers + 1)
2. In Update(), call levelKoreo.GetTrackByID(shootEventID).GetEventsInRange() with a startSample location of lastFrameSampleTime + scheduleSampleOffset and an endSample location of currentFrameSampleTime + scheduleSampleOffset.
   • "levelKoreo" is a reference to your Koreography data that contains the "shooting" events.
   • currentFrameSampleTime can be grabbed from Koreographer.GetSampleTime(). At the end of Update() you would set lastFrameSampleTime to this value.
3. If any KoreographyEvents are returned by the above call, do the following for each KoreographyEvent koreoEvent (assuming OneOffs):
   1. Calculate the time-in-seconds of koreoEvent in the future:
      eventTimeFromNow = (koreoEvent.StartSample - Koreographer.GetSampleTime()) / levelKoreo.SampleRate.
   2. Calculate the precise time on the DSP Timeline at which to start the audio:
      timeInTheFutureToPlay = AudioSettings.dspTime + eventTimeFromNow.
   3. Grab an AudioSource from your pool and ensure the AudioClip is loaded into it.
   4. Call AudioSource.PlayScheduled(timeInTheFutureToPlay).
4. Check your scheduled sound effect AudioSources to see if they've completed playback. If they have, return them to the pool (you actually may want to do this before steps 1-3 above - this would free them up for immediate reuse).

 

It turns out there's a very minor bug in your code. Specifically, in this line:

Code (CSharp):

1. eventTimeFromNow = (koreoEvent.StartSample - currentFrameSampleTime) / levelKoreography.SampleRate;

I will assume that the eventTimeFromNow field is a double type. All of the other variables on that line are int. As the division in that line of code should typically result in a value of roughly 0.1..., it is being rounded down to 0 and then cast to a double. To fix this you need to cast the integers to the double type before dividing:

Code (CSharp):

1. eventTimeFromNow = (double)(koreoEvent.StartSample - currentFrameSampleTime) / (double)levelKoreography.SampleRate;

You also do not need to check the koreoEvents.Count prior to the foreach loop. If there are no elements in the List over which to loop, the loop is short-circuited (it checks for that case internally).

Put it all together and it should look something like this:

Code (CSharp):

1. //Update current frame sample time
2. currentFrameSampleTime = Koreographer.GetSampleTime();
3.  
4. //Clear event list and get events within specified range
5.  
6. koreoEvents.Clear();
7.  
8. koreoTrack.GetEventsInRange(
9.     lastFrameSampleTime + scheduledSampleOffset,
10.     currentFrameSampleTime + scheduledSampleOffset,
11.     koreoEvents);
12.  
13. //If any events are found, schedule a beep with the AudioSource component
14.  
15. foreach (KoreographyEvent koreoEvent in koreoEvents)
16. {
17.     //Get time from current moment to execution of event
18.     eventTimeFromNow = (double)(koreoEvent.StartSample - currentFrameSampleTime) / (double)levelKoreography.SampleRate;
19.     //Calculate time on DSP timeline to play the beep
20.     timeToPlayBeep = AudioSettings.dspTime + eventTimeFromNow;
21.     //Schedule the beep
22.     beepSource.PlayScheduled(timeToPlayBeep);
23. }
24.  
25. lastFrameSampleTime = currentFrameSampleTime;

This worked perfectly in our tests! Please let us know if it resolves your issue!

 

Works now. Thank you very much for the help.

Really great asset!

 

Hi there,

I'm sure this has been asked many times before, but here's our situation:

Code (CSharp):

1. SimpleMusicPlayer.LoadSong(Clip, 0, false);

By doing that, we want to just load the given clip in memory to reduce delays when playing, but don't want to have it playing at that moment. However, the clip starts sounding right away, which leads me to think that the autoPlay flag is either doing nothing or not being used by me as intended.

How could I achieve such a simple behavior? I just want to load a clip in memory but not start it until I do SimpleMusicPlayer.Play();

 

Hmm... we've been unable to reproduce the issue you describe. Do you perhaps have a call to the SimpleMusicPlayer.Play() function nearby that is triggering?

Here is our test script:

Code (CSharp):

1. using UnityEngine;
2. using SonicBloom.Koreo;
3. using SonicBloom.Koreo.Players;
4.  
5. public class AutoPlayTest : MonoBehaviour
6. {
7.     public Koreography koreo;
8.     public SimpleMusicPlayer player;
9.  
10.     void Update()
11.     {
12.         if (Input.GetKeyDown(KeyCode.Space))
13.         {
14.             player.LoadSong(koreo, 0, false);
15.         }
16.     }
17. }

To test with that script, perform the following steps:

1. Create a new C# script called "AutoPlayTest.cs" and replace its contents with the code above.
2. Create a new scene.
3. Create an empty GameObject.
4. Add the following components to the GameObject:
   1. Koreographer
   2. Simple Music Player
   3. Auto Play Test
5. Add a Koreography asset reference to the Auto Play Test component.
6. Connect the Simple Music Player component to the Auto Play Test component (drag and drop or select from the Object Picker).
7. Play the scene.
8. Press the Spacebar key at any point.

You should see the Koreography load into the SimpleMusicPlayer as well as the Koreographer component in the Inspector. You should also not hear the audio begin to play.

Is there perhaps something else going on?

 

Okay, thank you for that! I'll try to just use your script following your steps to see if that works and, in case not, I'll get back to you.

 

Hey everyone! We just released version 1.5.1 of Koreographer and Koreographer Professional Edition to the Unity Asset Store! Here's a rundown of the changes:

• [NEW] [Pro] Custom MIDI Conversion support! Building on the Custom Payload system added in the previous release, the Custom MIDI Conversion system allows you to hook into the MIDI Converter's conversion process. Details are included in the Custom Payload Demo documentation. The demo content includes a new payload type example: the MIDIPayload.
• Remove legacy GUI components from Demo Scenes to remove warnings in the console in recent versions of Unity!
• Moved the Koreography and KoroegraphyTrack icons to locations expected by Unity 2018.1.0+.
• Adjust the Rhythm Game Demo's LaneController.Restart() API to take an optional sample time to which to restart to.
• [Pro] Fix external file loading logic [used by certain integrations] for Unity 5.4.0+.
• [Pro] Fix attempting to load the empty string ("") as an audio file path when using external audio file loading.
• [Pro] Fix handling paths with spaces (" ") in recent versions of Unity when using external audio file loading.
• [Pro] Fix GradientPayload rendering in Unity 2018.3+.
• [Pro] Fix warnings in recent versions of Unity with the source code package.
• [Pro] Fix Analysis Panel resizing once when changing tabs.
• [Pro] Fix tooltips not appearing when hovering over the range selector in the Analysis Panel on recent versions of Unity.
• [Pro] Fix integration compatibility with Master Audio 4.2.0+.
• [Pro] Fix integration compatibility with Wwise 2017.1.0+.
• Fixs for several minor API documentation bugs.
• Fix manual Tempo Section editing to correctly track the section being modified.
• Fix Koreographer Music Time APIs to all respect configured delay offsets.
• Fix spelling mistakes in the Help Panel.

If you purchased a previous version of Koreographer or Koreographer Professional Edition, then v1.5.1 is a free upgrade!

 

How do I access different instances? When I do GetComponent<Koreographer>() it shows the correct Koreographer, but the instance is always the first one created and not the second one I created on this GameObject. And my events never fire since that instance never seems active. I have the Simple Music Player setup with the correct settings, it is playing the song in my Koreography but I cannot register the events. Well I can register them but they never fire.

GetComponent<Koreographer>().RegisterForEvents("accuracyStartTrack", fireaccuracyStartTrackEvent);

But the fireaccuracyStartTrackEvent never fires. And when I debug it shows the Instance is some other instance.

 

There are a few ways to handle using multiple Koreographer components at once. Here are some options:

GetComponents
The standard GetComponent API will return the first component of the type you specify that is attached to a GameObject. The GetComponents version will return an Array of all of the Components of the type you specify, rather than just the first. You can then iterate over that array to find the one you care about.

Use Multiple GameObjects
If you have distinct GameObjects and limit yourself to one Koreographer component per GameObject then you can continue to use the GetComponent<Koreographer> approach that you've been using. You would simply need to make sure that the GameObject reference you're using actually contains the Koreography component that your music player is targeting.

Set in the Inspector
You can specify a Koreographer field to register with directly on your custom event listener component. The script looks something like this:

Code (CSharp):

1. using SonicBloom.Koreo;
2.  
3. public class AccuracyDetector
4. {
5.     // Shows a Koreographer field in the Inspector.
6.     public Koreographer targetKoreographer;
7.  
8.     void Start()
9.     {
10.         targetKoreographer.RegisterForEvents("accuracyStartTrack", fireaccuracyStartTrackEvent);
11.     }
12. }

With the above code you will be able to specify which Koreographer component to Register for events with by using the Inspector. This is precisely what the included audio player component scripts.

If you have multiple Koreography components on a single object then it can be challenging to "pick" the correct one in that field. The best solution here is to use multiple Inspector views at once. You can then click and drag the header of the specific component you care about from one Inspector view to the field you wish to add it to in the other Inspector view.

I hope this is helpful!

 

Hi guys

I need to assign Target Koreographer tracks (in Audio Source Visor) to a bunch attached to a list of audio sources.

I wish to do so with a list of Koreographer I have but I am not sure how to get component Audio Source Visor and perform these assignments?

Any ideas?

Thanks

 

Hmm... this may need some clarification. Do you need to assign Koreographer components or Koreography assets to the Audio Source Visor components?

 

I need to assign Koreography assets to the Audio Source Visor components programmatically.

Thank you!

 

I need to assign Koreographer assets here, see screenshot.

It says Koreographer in the Audio Source Visor so that's why I wrote 'list of Koreograher'

 

The Audio Source Visor component is designed to be as simple as possible. The goal with this component is that you simply add it to a GameObject that also has an Audio Source component on it. The Audio Source Visor will then report audio timing updates to the Koreographer in your scene (or the one optionally specified). This means that you do not load Koreography into it. Rather you load the Koreography assets directly into the Koreographer component. The Audio Source Visor will say "Hey, 'Some Cool Song.wav'' has a time of 34.239 seconds!" and then the Koreographer component says "Thanks! I'll check if any Koreography for that song is loaded and then let any listeners know if an event happened!"

To manage the Koreography loaded at any given time via script you can use the Koreographer.LoadKoreography(Koreography koreo) and Koreographer.UnloadKoreography(Koreography koreo) methods.

Behind the scenes, the music players use those two APIs to manage "playing Koreography" for you.

Keep in mind that the Koreographer component lives in the SonicBloom.Koreo script namespace.

 

Ahh, got it. The Target Koreographer field allows you to specify which Koreographer component you wish to use to track events. This is only really used in advanced circumstances (e.g. if you have split screen multiplayer and need to manage loaded Koreography and tracked events separately for each player...).

 

Thanks for the response.

How do I store a list of this type of Koreo elements? See screenshot.

In my previous script I was doing this: Koreographer.Instance.RegisterForEvents("events_name", FireContinuousEvent); and it works great, but I need to do it so I am not hardcoding "events_name"

Is Koreographer.Instance.RegisterForEvents("events_name", FireContinuousEvent);
the same as
Koreographer.Instance.LoadKoreography(Koreography koreo)

I am getting the Koreography terminology down

 

So in my implementation I am trying to store a list of "events" that I could plug in in: Koreographer.Instance.RegisterForEvents("events_name", FireContinuousEvent);

My first stab at this was to grab the name of the event like this: KoreoEvents.name;
Koreographer.Instance.RegisterForEvents(KoreoEvents.name, FireContinuousEvent);

What do you recommend?

Thanks again

 

INITIAL APROACH:

Code (CSharp):

1. string[] temp = _player.KoreoEvents[i].GetEventIDs();
2. Debug.Log(temp[0]);

I can now hook up each RegisterForEvents like this:

Code (CSharp):

1.  
2. Koreographer.Instance.RegisterForEvents(temp[0], FireContinuousEventForStems)
3.  

So that works for me, but wondering if there is an easier or more elegant way.

Thanks

 

So my previous message solves the event registration but I am still unsure how to hook up my Koreographer to the Target Koreographer procedurally in a prefab I am instantiating. See screenshot

Thanks

 

Those are Koreography Track assets. Koreography Track assets should be bundled into Koreography assets. You can make a list of those as follows:

Code (CSharp):

1. public List<Koreography> koreoList = new List<Koreography>();

When that code is part of a component you will be able to add Koreography to the list in the Inspector. This is basic Unity functionality.

The string Event ID that you are locking into should make sense for whichever script is registering for events. You don't have to hardcode the value. The included demos show an option for allowing Inspector configuration of Event ID using a custom Attribute provided by Koreographer that is "Event ID aware". At the very least you can create a string variable and pass that to the Register/Unregister functions.

This appears to be what you're doing in your custom script, which should be fine.

You do not need to use the Target Koreographer unless you are doing something really strange. Again, if you have only a single Koreographer component in your scene, the global singleton reference is all you should need. Your custom Player Script (the thing behind your "_player" reference in your scripts above, presumably) should be doing the Koreography asset registration/unregistration. It should reach out to the Koreographer component and Load/Unload Koreography as necessary. Those should look something like:

Code (CSharp):

1. // Load a Koreography Asset
2. Koreographer.Instance.LoadKoreography(_player.koreoList[i]);
3.  
4. // Unload a Koreography Asset
5. Koreographer.Instance.UnloadKoreography(_player.koreoList[i]);

Your _player script should then directly play an AudioClip through the AudioSource component that has a connected AudioSourceVisor next to it. The AudioSourceVisor is a "Set and Forget" Koreographer component helper. When you use it you are specifically in charge of:

1. Loading/Unloading Koreography to the scene's Koreographer component.
2. Playing/Stopping AudioClips in the associated AudioSource component.

The idea is that you hand a bunch of Koreography assets to the Koreographer component and then go about your business playing audio through any number of AudioSource components. IF one of those AudioSource components plays an AudioClip that the Koreographer component is aware of (through its list of loaded Koreography assets) then it will send out event notifications and updates as they occur to any script that has registered for any Event ID defined in any Koreography Track asset associated with the playing AudioClip (by way of the Koreography).

This is documented in the Koreographer User's Guide included with the Koreographer package (also available here). Please see pages 9-13 for an overview of the Koreography data structure and page 32 for an overview of the Audio Source Visor component.

 

Check this out.

Initially I was trying to do this:

_player is a scriptable object used as a data source exclusively (see screenshot)

Code (CSharp):

1. GameObject temp;
2.         for (int i = 0; i < _stemAmount; i++)
3.         {
4.             AudioSource asTemp = new AudioSource();
5.             SonicBloom.Koreo.Players.AudioSourceVisor koreo; // because I added AudioSourceVisor I thought I could assign it like this
6.  
7.             temp = Instantiate(_prefab);
8.             temp.transform.SetParent(transform);
9.             asTemp = temp.GetComponent<AudioSource>();
10.             asTemp.clip = _player.Stems[i];
11.             asTemp.outputAudioMixerGroup = _player.Mixer[i];
12.             temp.name = i.ToString();
13.             Debug.Log("Awake on MYOM_StemManager 1");
14.             koreo = (SonicBloom.Koreo.Players.AudioSourceVisor)temp.GetComponent<SonicBloom.Koreo.Players.AudioSourceVisor>();
15.             Koreographer.Instance.LoadKoreography(_player.KoreoEvents[i]); // I added this later and it seems to be working
16.             _audioSources.Add(asTemp);
17.             Debug.Log(koreo);
18.             //var arr = asTemp.GetComponents(typeof(Component));
19.             //Debug.Log(arr[2]); // this is where I got SonicBloom.Koreo.Players.AudioSourceVisor
20.  
21.         }

As you can see I thought you could grab a reference to AudioSourceVisor and assign Target Koreographer as a usual Unity process. Even after I hooked up SonicBloom.Koreo.Players.AudioSourceVisor koreo to my code it wasn't possible. This is where you said "The Audio Source Visor component is designed to be as simple as possible" but I still didn't get it.

The reason for my disconnect even after using Koreographer.Instance.LoadKoreography(_player.koreoList) I was still not seeing anything updated in the inspector, but after a bit it dawned on me that it probably didn't need to. Is that the case?

My updated code looks like this:

Code (CSharp):

1.  
2. GameObject temp;
3.         for (int i = 0; i < _stemAmount; i++)
4.         {
5.             //Debug.Log(_stemAmount);
6.             AudioSource asTemp = new AudioSource();
7.  
8.             temp = Instantiate(_prefab);
9.             temp.transform.SetParent(transform);
10.             asTemp = temp.GetComponent<AudioSource>();
11.             asTemp.clip = _player.Stems[i];
12.             asTemp.outputAudioMixerGroup = _player.Mixer[i];
13.             temp.name = i.ToString();
14.             Koreographer.Instance.LoadKoreography(_player.KoreoEvents[i]);
15.             _audioSources.Add(asTemp);
16.         }
17.  

Thanks again!

UPDATE
I am using 1.5.0, should I update to 1.5.1?

 

Wait. What is your goal here? It looks like you have a song broken out into stems and you would like to synchronize those together with each stem having its own Koreography. Is that the case?

Sure! It shouldn't hurt anything - unless you've modified the built-in Demo Scripts: the update may try to overwrite those changes. That would be the only reason I'd suggest caution. Otherwise, the 1.5.1 update had lots of little bug fixes and offers better compatibility with more recent (and upcoming) versions of Unity.

It shouldn't affect anything we've been discussing in this latest exchange!

 

Yes, that is correct. Am I doing this the correct way?

I feel lame not explaining that initially. Man I suck

 

I think you should take a long, hard look at the Multi Music Player. It was built for exactly this purpose.

This is an example that was created with the demo content included with Koreographer:

Things to note:

1. This can be turned into a Prefab (Asset).
2. You could alternatively create a ScriptableObject with a List of List<MusicLayer> lists to create a playlist that you could load into a Multi Music Player instance at runtime (see: MultiMusicPlayer.LoadSong).
3. Koreography is specified as a music source in all "Music Layers" above but you may optionally specify a raw AudioClip instead (e.g. if there is no Koreography associated with that layer/stem).
4. There are three Audio Source components on the example above. They were individually dragged onto the "Audio Source" fields in the inspector such that each layer had its own pre-configured Audio Source component.
   • IMPORTANT: This allows you to specify a per-layer Mixer! In the example above, the first Audio Source component is assigned to the "Harmony" layer. You could expand the Audio Source component and connect the Mixer you want to use for the Harmony layer directly onto the Audio Source. Such settings can be captured in a Prefab (Asset).

Please see pages 28-29 of the Koreographer User's Guide [in documentation] for a detailed writeup.

 

Hey, I'm using Koreographer for a rhythm game, and using the Rhythm Game demo scene as a base. I was hoping to loop the track, so once it ends, it just starts again - Do you know how I could achieve this?

The system should show the upcoming notes (as your current demo does), and when it reaches the end of the track, it should show the final notes of the current track, as well as the starting notes of the "next" track (which is just the current track playing again). I'm trying to figure out the best way to do this, as in the current demo system, the notes move according to the current track, and there's no way to look ahead into the next track :/