Search Unity

Koreographer - Audio Driven Events, Animation, and Gameplay

Discussion in 'Assets and Asset Store' started by SonicBloomEric, Sep 15, 2015.

  1. kapaakea2

    kapaakea2

    Joined:
    Sep 4, 2013
    Posts:
    7
    Hi,

    Purchased Koreography Pro a few days ago.

    Iʻm getting an array index out of range on
    float endPos = (charInfos[endIdx].cursorPos.x + charInfos[endIdx].charWidth) * scaleOffset;
    in KaraokeTextLine.cs

    When I play in Unity. Cant play more than a second without the exception being thrown.

    This didnʻt use to occur...there was some error which occured..something about arrays and increasing max size of something. After the Unity restart (it crashed after clicking the prompt) at is the Koreography is destroyed and the audio file became unlinked. The tracks also become lost and I have to reload all 3 (1 audio file + 2 tracks). I was editing the audio and timing for a new audio file I put in place. Perhaps around 50 tags per track.. I was fixing the last 2 of them and was verifying the completed work when this occurred.

    Now I get the above error..debugging CharInfos.Length is 1 where endIdx is 39. I recreated the Koreography by loading the two tracks and audio file but suspect something is wrong with the koreography file or tracks now.

    I canʻt share the files here..wondering if I can send them to you.

    Kalani

    Update: Canvas.ForceUpdateCanvases() fixed the issue but I had to put it in the KaraokeTextLine code.

    UICharInfo[] charInfos = baseText.cachedTextGenerator.GetCharactersArray();

    was returning the wrong # because it was too soon after the start of the scene..why I have no idea..as it was working fine until then. Possible update to codebase for Karaoke Demo.
     
    Last edited: Oct 4, 2017
    SonicBloomEric likes this.
  2. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    @kalani Woah! Thanks for the report and the update! A few notes:
    I'm very sorry to hear that you lost some work. When editing Koreography, we recommend clicking "File→Save Project" pretty frequently. The standard "[CTRL/CMD]+S" to save only saves the active scene (it's kind of confusing, but so it goes). Koreography are separate assets and therefore must be saved with the project. The same thing is required if you modify a prefab directly and want to actually save the changes to disk (I should note that Unity auto-saves all modified assets [e.g. Prefabs, Koreography, etc.] when you quit normally).

    I believe I recall seeing that Unity did some kind of optimization recently that allowed them to update the UI Canvas(es?) less frequently. Perhaps this change is at the source of the issue...

    Yup! That's what we're currently thinking! What version of Unity are you using? We'd love to reproduce the issue and then get a fix into the code!
     
  3. kapaakea2

    kapaakea2

    Joined:
    Sep 4, 2013
    Posts:
    7
    Good to know.

    Good tip and possible cause. Wondering where to find out more info on that.

    The issue only started occurring after a bad crash (didn't update after crash, just restarted). "2017.1.1f1 Personal" is my version. Up until then it worked perfectly...after the crash I needed to have that Canvas in there...every call actually otherwise at some point in time it would stop. I can't explain it. I can try to send my project over to you (though I canʻt post here).

    The canvas thing fixed it but I donʻt know why I didn't need it until the crash.

    ---Possibly related, possibly not (the crash occurred during the timing/playback on the last few lines)--
    I am also encountering an issue where no matter what I do the last 3 lines of Karaoke donʻt ever render..i.e. they donʻt fill in with red, just show as white. I've checked the start time, ensured they have exact text. Iʻve confirmed issues of timing/text had similar symptoms didn't have the same cause.... I havenʻt been able to figure out the cause..only last few lines before completing the song...darn...but they wont work.

    Update - had an extra ending space in the lines track on the Text event prior to the problem track. The corresponding lyric track didn't have the ending space..hence the issue...thought I had found everything. Was able to get to the end of the song.
     
    Last edited: Oct 5, 2017
  4. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    The best location would be the release notes (available at the Download Archive). I did a cursory search before posting about it and couldn't locate the information. It's possible I encountered that tidbit elsewhere (e.g. conversation) or am simply misremembering.

    Did you perhaps try deleting the Library when you crashed? It is possible that the metadata generated by Unity and stored in that directory was corrupted in some way that is causing this issue (especially if it didn't happen before the crash). The Library contains metadata generated from the project and is safe to delete. It will only require that the project assets be re-imported the next time the project is opened.

    Very glad you were able to figure this out! The issue about whitespace is a tricky one... perhaps we can try to detect situations like this while in the editor and log a warning(?). Either way, glad you got things working here!
     
  5. Arturotron

    Arturotron

    Joined:
    Jul 7, 2017
    Posts:
    11
    Hi. Is there a list of properties and methods of Koreography and KoreographyEvents?
     
  6. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Hi! Thanks for reaching out! There are two ways to see the list of Koreography and KoreographyEvent properties and methods:
    1. Use the IntelliSense or AutoComplete features in MonoDevelop/Visual Studio/etc. These source the XML files that are included next to the assemblies (DLLs)
    2. Search the XML files directly (in a text editor like Sublime Text, Notepad++, etc.).
    We are working to improve the API documentation. We're investigating several packages to process the xmldoc and generate other formats. This includes Doxygen, DocFX, and a few others. Our challenge is how to best clearly support both Koreographer and Koreographer Professional Edition as their APIs are slightly different (Professional Edition has more content available).

    I hope this is helpful!
     
    Arturotron likes this.
  7. Arturotron

    Arturotron

    Joined:
    Jul 7, 2017
    Posts:
    11
    Is there any inbuilt way to handle variable bmp in a song?
     
  8. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Yes. Koreographer supports multiple Tempo Sections which allows you to set the tempo for specific regions of a song. See the Tempo Section Settings (pictured below) to configure these. The buttons highlighted with the red box manage adding/removing sections.

    TempoSectionSettings.png

    I should note that the start position of a Tempo Section is set in terms of Sample Time so you'll need some large numbers to set the position. To set a tempo section to start at exactly 10 seconds, you'll need to input 10*44,100=441,000 (assuming a sample rate of 44100 for the song).

    There is no "end sample" in tempo sections because it is assumed that the last one set continues through the end of the song.

    Please note that you can click on the "Start Sample" text and drag left/right to adjust the value as you can with any normal number field in the Unity Editor UI.

    I hope this is helpful!
     
    Arturotron likes this.
  9. RyanNguyen

    RyanNguyen

    Joined:
    Jul 20, 2016
    Posts:
    8
    Hi,
    I bought Koreographer a few days ago and having a same problem like thanhle
    How does Koreographer keep working while I'm not using MSourceClip in Koreography?
    I'm very interested in the solution about creating a new Visor class. I've read the document on page 28 but still didn't understand how to start writing this class, can I have more info about creating a new Visor class or any prototype codes, video tutorials about this solution?
    Thanks.
     
  10. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Do you have any further information about your setup that you could share? Are you playing music in some way? If so, what does the setup look like?

    To create your own Visor class, you extend the SonicBloom.Koreo.Players.VisorBase abstract class. This will require that you provide an implementation for the abstract methods defined in that class. This includes everything required to get the status of the audio that Koreographer can use to "pump" the time system. The methods follow:
    Code (CSharp):
    1. /// <summary>
    2. /// Gets the name of the audio currently playing back.  This is used to identify Koreography for
    3. /// event triggering.
    4. /// </summary>
    5. /// <returns>The name of the currently playing audio.</returns>
    6. protected abstract string GetAudioName();
    7.  
    8. /// <summary>
    9. /// Gets whether or not the audio is currently playing back.
    10. /// </summary>
    11. /// <returns><c>true</c> if the audio is playing, <c>false</c> otherwise.</returns>
    12. protected abstract bool GetIsAudioPlaying();
    13.  
    14. /// <summary>
    15. /// Gets whether the audio is set to loop or not.
    16. /// </summary>
    17. /// <returns><c>true</c> if audio should be looping, <c>false</c> otherwise.</returns>
    18. protected abstract bool GetIsAudioLooping();
    19.  
    20. /// <summary>
    21. /// Gets whether the audio looped this frame or not.
    22. /// </summary>
    23. /// <returns><c>true</c> if the audio looped, <c>false</c> otherwise.</returns>
    24. protected abstract bool GetDidAudioLoop();
    25.  
    26. /// <summary>
    27. /// Get the current time in samples of the current audio (the playhead position in samples).
    28. /// </summary>
    29. /// <returns>The current sample time.</returns>
    30. protected abstract int GetAudioSampleTime();
    31.  
    32. /// <summary>
    33. /// Gets the first *playable* sample position of the current audio.
    34. /// </summary>
    35. /// <returns>The first playable sample position.</returns>
    36. protected abstract int GetAudioStartSampleExtent();
    37.  
    38. /// <summary>
    39. /// Get the last *playable* sample position of the current audio.
    40. /// </summary>
    41. /// <returns>The last playable sample position.</returns>
    42. protected abstract int GetAudioEndSampleExtent();
    43.  
    44. /// <summary>
    45. /// Get the number of samples that were played in the last frame.  Be sure to consider current settings
    46. /// and playback state.
    47. /// </summary>
    48. /// <returns>The delta time in samples.</returns>
    49. protected abstract int GetDeltaTimeInSamples();
    The values for much of these are based on samples so if the time you're using is in "solar time" then you'll need those functions to convert into samples (a simple multiplication of solar time by sample rate). You'll also need to have a way to set the internal koreographerCom reference. This is the connection that allows the Visor to talk to a specific Koreographer component instance. The AudioVisor class does this in the constructor (e.g. new AudioVisor(audioSource, someKoreographerComponent)) - but it does make the parameter optional. If a value isn't passed in, it attempts to use the Singleton (global) instance.

    We may be able to provide better insight with more specific details if we get a better sense of your specific use-case. Could you provide any further details as to what your setup looks like?
     
    RyanNguyen likes this.
  11. RyanNguyen

    RyanNguyen

    Joined:
    Jul 20, 2016
    Posts:
    8
    Hi @SonicBloomEric
    Thanks you for your quick reply.
    I use Koreographer MIDI Converter to export Koreography Track from MIDI audio files, then I use Koreography Track and MIDI audio file in gameplay.
    If I don't put an MP3 Audio file into Koreography MSourceClip then the gameplay it will stop, so I'm looking for a solution to provide audio clip information for Koreographer from MIDI audio files without adding MP3 track to MSourceClip.
    Hope you can provide more information on how to setup interaction between a custom Visor Class and Koreographer. How to use a Custom Visual Class to solve the problem of stopping gameplay when MP3 Audio Clip is not attached to Koreography.
    Any advise would really help.
    Thanks.
     
    Last edited: Oct 16, 2017
  12. Arturotron

    Arturotron

    Joined:
    Jul 7, 2017
    Posts:
    11
    Hi. I saved my project and exported it to another computer and now its saying the following:

    error CS0246: The type or namespace name `KoreographyEvent' could not be found. Are you missing an assembly reference?

    error CS0246: The type or namespace name `DeltaSlice' could not be found. Are you missing an assembly reference?

    All the koreographer folder is inside the project, do I need any other configuration or something? its the same unity version: 5.6.3p2

    Ps. I'm sorry i'm using this post as a general faq and I realized this might not be on topic. Is there any better way to ask this questions?
     
    Last edited: Oct 16, 2017
  13. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    When you exported the project did you include the Assets/Plugins and Assets/Editor Default Resources directories? Koreographer installs important content into those special folders - generally you can delete the Assets/Koreographer folder and everything will continue to work because functional content is installed into those special folders.
     
  14. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Your setup would definitely benefit from a custom Visor. What you want to do is create a Visor that takes timing information from your MIDI file player and funnels it to the Koreographer component to perform the triggering.

    What are you using to drive MIDI File playback?
     
  15. Arturotron

    Arturotron

    Joined:
    Jul 7, 2017
    Posts:
    11
    Yes, I have the Assets/Plugins and Assets/Editor Default Resources :C
     
  16. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Did you include the .meta files when you transferred the project?

    Could you describe the steps you took to export the project?
     
  17. Arturotron

    Arturotron

    Joined:
    Jul 7, 2017
    Posts:
    11
    I put all the folder of the project in a rar transfered via usb port to another machine, and put the file inside the rar into the project folder. And opened unity, unity recognize the project, and the only error are that it doesnt recognize koreographer and koreographer related classes
     
  18. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Did you have "Visible Meta Files" set? When you look in the Assets directory in Windows Explorer/Finder do you see files with the ".meta" extension at the end of them?
     
  19. Arturotron

    Arturotron

    Joined:
    Jul 7, 2017
    Posts:
    11
    Yes, the meta files are there. The second pc is working offline, could it be a license problem?
     
  20. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Hmm... Two more questions:
    1. Are there any non-Koreographer-related compiler errors?
    2. Are you sure that you properly included the namespaces with a statement like "using SonicBloom.Koreo;" at the beginning of the troublesome script?
     
  21. Arturotron

    Arturotron

    Joined:
    Jul 7, 2017
    Posts:
    11
    The full errors are:
    Assets/Koreographer/Demos/Scripts/CubeScaler.cs(34,20): error CS0246: The type or namespace name `KoreographyEvent' could not be found. Are you missing an assembly reference?

    Assets/Koreographer/Demos/Scripts/CubeScaler.cs(34,75): error CS0246: The type or namespace name `DeltaSlice' could not be found. Are you missing an assembly reference?
    Assets/Koreographer/Demos/Scripts/EmitParticlesOnSpan.cs(39,31): error CS0246: The type or namespace name `KoreographyEvent' could not be found. Are you missing an assembly reference?
    Assets/Koreographer/Demos/Scripts/MusicalImpulse.cs(38,19): error CS0246: The type or namespace name `KoreographyEvent' could not be found. Are you missing an assembly reference?
    Assets/Koreographer/Demos/Scripts/UIMessageSetter.cs(18,3): error CS0246: The type or namespace name `KoreographyEvent' could not be found. Are you missing an assembly reference?
    Assets/Koreographer/Demos/Scripts/UIMessageSetter.cs(38,19): error CS0246: The type or namespace name `KoreographyEvent' could not be found. Are you missing an assembly reference?

    Assets/Koreographer/Demos/Scripts/UIMessageSetter.cs(38,74): error CS0246: The type or namespace name `DeltaSlice' could not be found. Are you missing an assembly reference?
    Assets/Scripts/EstadoJuego.cs(26,2): error CS0246: The type or namespace name `Koreography' could not be found. Are you missing an assembly reference?

    Assets/Scripts/EstadoJuego.cs(43,20): error CS0246: The type or namespace name `KoreographyEvent' could not be found. Are you missing an assembly reference?
    Assets/Scripts/EstadoJuego.cs(47,15): error CS0246: The type or namespace name `KoreographyEvent' could not be found. Are you missing an assembly reference?

    Assets/Scripts/Item.cs(21,2): error CS0246: The type or namespace name `KoreographyEvent' could not be found. Are you missing an assembly reference?
    Assets/Scripts/Item.cs(22,2): error CS0246: The type or namespace name `Koreography' could not be found. Are you missing an assembly reference?

    Assets/Scripts/Item.cs(86,26): error CS0246: The type or namespace name `KoreographyEvent' could not be found. Are you missing an assembly reference?
    Assets/Scripts/Item.cs(90,29): error CS0246: The type or namespace name `Koreography' could not be found. Are you missing an assembly reference?

    Assets/Scripts/Rythm.cs(15,20): error CS0246: The type or namespace name `KoreographyEvent' could not be found. Are you missing an assembly reference?

    And i would be like 98% sure of the namespace inclussions as it is working on my first machine perfectly, the scripts are unchanged on both computers.
     
  22. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Got it. Would it be possible for you to send the project to us to investigate? (support@sonicbloomgames.com)
     
  23. Arturotron

    Arturotron

    Joined:
    Jul 7, 2017
    Posts:
    11
    Sent.
     
  24. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Received!

    We were able to unpack the project and open it with Unity 5.6.4 on macOS with zero issues. Out of curiosity, what platform are you build settings?
     
  25. Arturotron

    Arturotron

    Joined:
    Jul 7, 2017
    Posts:
    11
    I'm sorry, The bug apparently was a missinstalation of unity, my second computer was not running 5.6.2 but 5.5.something and showed 5.6. I uninstalled and installed the latest version of unity and everything is working now. Sorry for the trouble. Awesome plugin and awesome support by the way :)
     
    SonicBloomEric likes this.
  26. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Thanks for reporting! That actually makes a lot of sense, too. Koreographer was built with Unity 5.6 and the version you had installed was only compatible with Unity 5.6 and newer. You'd have needed to update Koreographer from the Asset Store in order to resolve the issue (as it would download the version built against Unity 5.4).

    Very glad to hear that you got the issue resolved! Best of luck with the project :D
     
  27. Dreamcube017

    Dreamcube017

    Joined:
    Dec 20, 2009
    Posts:
    253
    Hello, I have two questions.

    1. Was this video made with the standard or pro version of your asset?


    2. If I get the standard, is an upgrade price available?

    Thanks.
     
  28. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Hi! Happy to answer your questions:
    The video was made with the Professional Edition of Koreographer. However, only one Pro-only feature was used to create it: the Gradient Payload. The Gradient Payload was used to smoothly change the color of the lighting as the music played. All other effects were implemented using features that are common to both the standard and Professional Edition of Koreographer.

    Yes! The upgrade price is the difference between the two packages when not on sale ($80). That said, at time of writing, Koreographer Professional Edition is on sale for 50% off at $58 - which is less than the upgrade price.
     
  29. Dreamcube017

    Dreamcube017

    Joined:
    Dec 20, 2009
    Posts:
    253
    I just got it, thanks!
     
    SonicBloomEric likes this.
  30. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Hah, great! Happy to help :) Best of luck with your projects!
     
  31. Sujeto

    Sujeto

    Joined:
    Jun 30, 2015
    Posts:
    11
    Hi! i have koreographer (not pro) and i am trying to use it with Bolt (visual programin plugin) but i am totally lost i dont know how to translate the code to bolt. Thanks!
     
  32. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Thanks for reaching out! Just wanted to let you know that we've reached out to the Ludiq developers about this issue (we're not familiar with Bolt at this time). I will let you know once we've heard back from them about this!
     
  33. Sujeto

    Sujeto

    Joined:
    Jun 30, 2015
    Posts:
    11
    Thank you! I really appreciate it!
     
  34. Dreamcube017

    Dreamcube017

    Joined:
    Dec 20, 2009
    Posts:
    253
    I'm pretty interested to hear what they say. I have both of these plugins and it'll be interesting to see what happens.
     
  35. CurialLloses

    CurialLloses

    Joined:
    Oct 20, 2017
    Posts:
    8
    Hello, I've just arrived to this interesting asset and I wanted to ask for some questions before fall to buy it:

    - I want to make a game which can play multiple song tracks synchronized at the same time. The game begins playing the first track but as the player follows the beat correctly with the input (pad or keyboard) the other tracks unlock/unmute progressively as a reward. Can I play multiple synchronized song tracks and play/pause/stop/reset them with Koreographer?

    - For what I am reading in the documentation, it is possible to compare the beat events timing with the players input and check if player made a "GOOD/GREAT/PERFECT" timing as it happens in Dance Dance Revolution. Is this input time range customizable via Koreographer to know whats a "GOOD" or "GREAT" or so on?

    Thank you very much for your help!
     
  36. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    I still haven't heard back from Ludiq about this but I saw a helpful response on a forum topic in the Ludiq forums. (I suspect that @Sujeto will find it familiar ;D)

    Based on the information in that response, it seems as though Bolt doesn't support a fully-visual integration out of the box at this time. However, it seems that it should be fairly straightforward to forward Koreographer Events into the Bolt visual scripting flow. While I've not tried this, my guess is that it would look something like this:

    Code (CSharp):
    1. using UnityEngine;
    2. using SonicBloom.Koreo;
    3. // using appropriate Bolt namespace?
    4.  
    5. public class EventSubscriber : MonoBehaviour
    6. {
    7.     [Tooltip("The Event ID for which to register for events.")]
    8.     public string eventID;
    9.  
    10.     // Initialize Koreography Callback
    11.     void Start()
    12.     {
    13.         Koreographer.Instance.RegisterForEventsWithTime(eventID, TriggerBoltEvent);
    14.     }
    15.  
    16.     void TriggerBoltEvent(KoreographyEvent evt, int sampleTime, int sampleDelta, DeltaSlice deltaSlice)
    17.     {
    18.         // Not sure if "gameObject" is correct or if something else would be better...
    19.         CustomEvent.Trigger(gameObject, evt, sampleTime, sampleDelta, deltaSlice);
    20.     }
    21. }
    If you try this out and get it working, I'd love to hear if this works and, if not, what does!

    I will of course update again if/when we hear back from Ludiq.

    UPDATE
    Lazlo from @ludiq responded on their own forums with a much more complete version of my hypothesized code above. Please check out this response for a detailed explanation on how to get the two assets to work together!
     
    Last edited: Feb 12, 2018
    Akshara likes this.
  37. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Yes. Koreographer ships with the Multi Music Player component that does precisely this. It works by playing all layers simultaneously. You can cut out/bring in different layers by adjusting their volume. Read more about it on page 28 of the Koreographer User's Guide!

    Yes*, with an asterisk. You can compare the beat timings with the current frame time. Unity's input system does not [currently] provide timestamps for sub-frame timing: the most you get is "the user pressed a button within the last 30ms." With this in mind, you don't actually have a "time" at which the user pressed a button, but a "time window" within which the user pressed a button. There are different ways to handle this and it would be up to you to decide how best to deal with it for your game. Note, also, that this is a limitation of Unity itself, not Koreographer.

    [I added "currently" in brackets because Unity is working on an Input System update that promises to include timestamps.]

    It is very important to note that you would not use Koreographer's Event System to compare user input to events. The event system triggers callbacks only when events actually occur, not before. This means that you wouldn't be able to easily detect a "too early!" hit. You can configure the event data itself so that this would work, but you'd end up making the data more difficult to work with in general.

    Instead, you would access the event lists (packaged as KoreographyTracks within Koreography data) and compare them with the current time. This is how the included Rhythm Game Demo works. For more details on how the Rhythm Game Demo works, check out the documentation here.

    I hope this helps! If you have any further questions, please don't hesitate to ask, whether that be here or on Koreographer's forums! :D
     
    Last edited: Sep 30, 2021
  38. Dreamcube017

    Dreamcube017

    Joined:
    Dec 20, 2009
    Posts:
    253
    Is it possible to do something like what Mario Odyssey, 3D World, and Galaxy does with the interactive arpeggiators. See ths video.



    I am thinking some other logic such as something to actually play the sounds or manage the arpeggiation such as a built in synthesizer would need to be there as well, but figured I'd ask. I have the pro version, so I do have access to the MIDI capabilities.
     
  39. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    This is a difficult question to answer as there are lots of parts to it. A short answer would be that you could emulate what Nintendo does pretty easily with Koreographer, provided you didn't need to transition between two songs. Unity's built-in audio system, upon which Koreographer's default music player components are built, is rather simplistic - it doesn't support dynamic, synchronized transitions well.

    Nintendo, as you may know, typically relies upon MIDI playback to handle much of their music (I assume that they work with extremely top quality Soundbanks). This provides them with the ability to dynamically swap instruments or duck layers as needed at runtime. Unity, of course, does not support MIDI playback out of the box. Koreographer, too, doesn't support MIDI playback. [Koreographer's MIDI Converter takes MIDI files and enables you to convert elements from the composition within the MIDI file into Koreography data.] That said, there are several solutions for the MIDI playback issue. See:
    1. Wwise - See this document. My understanding is that Wwise could easily handle the experience you're looking to create. Koreographer Professional Edition also supports integration with Wwise.
    2. Fabric? - Based on this set of release notes it seems that Fabric may very well support MIDI playback. Koreographer Professional Edition also supports integration with Fabric.
    3. Elias - I would provide a link, but their server currently appears to be down(?). They were messaging MIDI support pretty strongly last year.
    4. MIDI Unified - We have not used it ourselves, but if your target platforms match the supported platforms, then it may be an option to consider.
    There are likely other solutions available as well.

    Koreographer does not specialize in audio playback, but rather using the audio that is played to drive other effects or systems in-game. We refer to this as Proactive Audio (whereas interactive/adaptive audio would be Reactive Audio as the music is constantly reacting to events in or changes in state of the game). We try to provide integrations with popular Reactive Audio systems (e.g. Wwise and Fabric in the list above) to help you gain the best of both worlds. :)

    I mentioned earlier that you could emulate the effect with Koreographer. To do so you would break your music into at least two layers (the arpeggio/harmony and the mainline) and then use the Multi Music Player to control the volume level of the different tracks.

    I hope this is helpful!
     
  40. CurialLloses

    CurialLloses

    Joined:
    Oct 20, 2017
    Posts:
    8
    Thanks Eric for your help. I finally fall into koreographer and bought it :p

    I think I've understood how the events systems trigger callbacks and investigating a little bit for myself I've seen that is possible to create span events. Could work as a solution to create several span events before, during and after my desired beat time (before: span for GOOD, span for GREAT, span for PERFECT; after: span for GREAT, aspan for GOOD) in order to check if player pressed a button during an span event? So this way I could know by "brute force" my accuracy and if there's an Input off some span event, mark it as a "FAIL".

    If I understood it well, span events are triggered during a range of ms so I could check if there's and input while the event is being triggered.

    Don't want you to loose your time so with a simple "ummm, haven't tried but could work" or a "NO, you are wrong because..." works for me ^^.

    Thanks again!
     
  41. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Haha, thanks very much! I hope you enjoy working with it!

    Yes! This would indeed work as a "brute force" solution. Configured this way, you would get a callback each frame that a span event exists. You could check to see if the user pressed a button with the associated Event ID (assuming you've broken up button-specific event data into their own separate KoreographyTracks). In your callbacks you would simply check to see if the user had pressed the associated button.

    The one thing to watch out for is that if you have Span events that run up next to each other, you are likely to see frames where you get multiple callbacks: one for the span that ended within the current "frame" and one that began in the current "frame". There are ways to handle this (e.g. cache the "hit" information in the callback and then process it in the main game logic Update function) but I leave it up to you to decide what is best for your game!

    Best of luck with your game and thanks again for picking up Koreographer! Please keep us posted on how things go! :D
     
    CurialLloses likes this.
  42. Dreamcube017

    Dreamcube017

    Joined:
    Dec 20, 2009
    Posts:
    253
    That was VERY helpful actually! I have not really used Wwise, but seeing as it has an arp AND a synth built into it that I can use (was thinking that'd be another issue) this will do nicely! I in fact do have the Pro version of Koreographer and will give this a try.
     
  43. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Fantastic! Glad to hear it was helpful!

    Let us know when you're ready to integrate Koreographer with Wwise. We provide the integration as a separate package upon request :)
     
  44. LazloBonin

    LazloBonin

    Joined:
    Mar 6, 2015
    Posts:
    813
    Hi Eric! Lazlo from Bolt here.
    This is indeed exactly the right way to do it at the moment.
    In the future, Bolt will have delegate/callback support, but for now, using CustomEvent.Trigger is the way to go.
     
    SonicBloomEric and Akshara like this.
  45. gegagome

    gegagome

    Joined:
    Oct 11, 2012
    Posts:
    392
    Hi there

    I need to look ahead of currently-playing audioclips, find silence of 1, 2 or say 5 seconds in length.

    Does Koreographer do that?

    In other words I need to know when there is silence in everyone of my clips, and make logic when this silence is 5 seconds long.

    This is my forum question: https://forum.unity.com/threads/look-for-audio-silence-ahead-of-time.516136/

    Thanks
     
  46. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Koreographer can indeed do this for assets that you have at develop time. You would create Koreography data for each such AudioClip and provide events for the periods of silence manually (there are some things that you could automate, but the tooling for that would be something you'd need to develop).

    If you need lookahead at runtime (e.g. to show a gap in some game element that arrives at the player at a given time), then you could take a look at the included Rhythm Game Demo to see how you can use the Koreography data directly at runtime, rather than the event system itself.

    I hope this is helpful!
     
  47. Tiollo

    Tiollo

    Joined:
    Mar 14, 2015
    Posts:
    23
    Hi there,

    I bought Koreographer yesterday and it is an amazing tool - I'm really looking forward to use it in several future projects :)

    I write here because I followed the basic tutorial and I managed to achieve the results that I wanted, but I'm still having a little problem.
    I can't make the audio source loop and I can't find any option to do it in the editor. I saw that the "Multi Music Player" component has a "Loop" checkbox but I don't know how to use it with the Simple Music Player.
    Any help?

    Thanks a lot!
     
  48. SonicBloomEric

    SonicBloomEric

    Joined:
    Sep 11, 2014
    Posts:
    1,090
    Thanks so much! We look forward to seeing (and playing!) what you create!

    When you add the Simple Music Player to a GameObject in your scene, it will automatically add an Audio Source component. Check the "Loop" checkbox on the Audio Source and the Simple Music Player will handle the looping internally.

    The Multi Music Player does things a little differently because it internally manages several Audio Source components. The Loop checkbox on this component is a convenience method in the editor (and provides scripting functionality - its a very flexible player component and the script API helps simplify control of the various Audio Source components under its watch).

    I hope this helps!
     
  49. Tiollo

    Tiollo

    Joined:
    Mar 14, 2015
    Posts:
    23
    Cool - you are right, thanks a lot!
    I didn't realize it because the same game object already had already some audio sources (but in the editor it was still showing the old ones).
    Thanks again! :)
     
    SonicBloomEric likes this.
  50. akbar74

    akbar74

    Joined:
    Nov 16, 2017
    Posts:
    21
    hello mr eric.
    i know its noob question but it make me crazy.:(
    i just wanna check if pressed button is either after or before the current event.
    but i dont know how to do it.
    i just cant understand the sample time and beat time.:(