Maestro Midi Player Tool Kit - Good news for your rhythm game !

Hello Robert
Thank for your interest for MPTK, your support and encouragement are very useful!!!
I am a bit bothered. For sure what you try cannot work, the method
MPTK_Play() takes an entry from the Midi DB not from an external byte array. Of course, it's possible but with another prefab (MidiExternalPlay) ... available with the Pro version ;-) See here https://paxstellar.fr/midi-external-player-v2/
All my best
Thierry

 

Hi Thierry,

I've started using the free version so far and very happy with it. Really great thing to have for free so thank you!

I have a problem that I would like to solve in the best and most precise way and I lack knowledge about audio and midi latency, threading etc.

I would like to make something like the 'CatchMusic' demo but without collisions or world interaction and where the sound is triggered more precisely than in fixedUpdate. So I show the notes and then they are played when they reach a certain point. Basically, what I want is like Synthesia (https://synthesiagame.com/).

I think what I need Is to receive an event precisely so many milliseconds after a note is played from the MidiFilePlayer. As if there was a copy of each MIDI Event running behind. I also want to allow tempo changing whilst the track is playing.

Are there any suggestions you could make about how to approach this?

Thanks,
Bruce

 

Hello Bruce
First of all, thank for your interest with MPTK.
For sure, processing with fixedUpdate is not accurate.
On which OS do you want to publish your app ?
Have you tried with OnEventNotesMidi ?
Look in the script TestMidiFilePlayerScripting.cs

Best regards
Thierry

 

Thanks for the quick response .

Windows is the main and only vital target platform.

I have managed to generate the notes visually by adding a listener to OnEventNotesMidi. That part works beautifully. I don't know how to make the music play precisely 2000ms in the future.

Is it a matter of creating some precise timer when I receive OnEventNotesMidi? If so how do I do that? Or can I take advantage of the audio thread that MPTK is working in?

 

Why not let MidiFilePlayer playing the note ?


Another point to lower the latency is to decrease the buffer size:

 

I wanted to have a MidiFilePlayer creating the coloured note blocks as shown below.

So now I have tried a setup with 2 MidiFilePlayers which I have tried to sync.

The first one creates the notes the coloured blocks and the second one starts itself when the first one has the position property go past a certain value. It doesn't always seem to sync very precisely.

 

I have tried setting the buffer size to less than default (256) but it's completely distorted. I have also changed the DSP buffer size in 'Project Settings > Audio' to 'Best Latency'. What is the difference between these two buffer size settings?

 

It's the same parameter. And yes, below 256 the sound is distorded. I don't know why.

Perhaps we could exchange by email ?

 

Thank you. Yes, good idea. I have sent you a message.

 

Below see how to use the event triggered for a more accurate music generation:
public void Play()
{
lastMidiTimePlayCore = 0f;// System.DateTime.Now.Ticks / 10000D;
timeMidiFromStartPlay = 0d;
timeSinceLastBeat = 999999d; // start with a first beat
CurrentBeat = -1; // start with a first beat
if (IsPlaying)
midiStream.OnAudioFrameStart += PlayHits;
else
midiStream.OnAudioFrameStart -= PlayHits;
}

private void PlayHits(double synthTimeMS)
{
if (lastMidiTimePlayCore <= 0d)
lastMidiTimePlayCore = synthTimeMS;

// Calculate time in millisecond since the last loop
double deltaTime = synthTimeMS- lastMidiTimePlayCore;
lastMidiTimePlayCore = synthTimeMS;
timeMidiFromStartPlay += deltaTime;

// Calculate time since last beat played
timeSinceLastBeat += deltaTime;

// Is it time to play a hit ?
tempotemp = CurrentTempo = 5000d / SldTempo.Value;
if (IsPlaying && timeSinceLastBeat > CurrentTempo)
{
timeSinceLastBeat = 0d;
// your specific code here for generating music based on MPTKEvent of course ;-)
}
}

 

Hi
Want to build music with MPTK ? read this new post :
https://paxstellar.fr/2021/01/01/get-an-accurate-generated-music/

 

Hi
Quick noob question, I am making a simple guitar app and I want to play a guitar sound when a user touches a specific part of the fretboard, can I use this to generate a guitar sound for each note on the guitar? I don't want to have to record all the guitar notes individually and include them as audio files in my app, so it seems if I can generate them with this sdk that would be best

 

Yes ! I will replay more in detail soon.

 

You can use the MidiStreamPlayer prefab to select instrument (guitar in your case) and play midi note within your script in relation with user action on the fretboard. More over, the free version of MPTK could be enough for your need ;-)
Have a look in the demo TestMidiStream the methods midiStreamPlayer.MPTK_ChannelPresetChange, midiStreamPlayer.MPTK_PlayEvent, midiStreamPlayer.MPTK_StopEvent to understand how select an instrument, play a note, stop a note.
All my best
Thierry

 

This looks awesome. Can't wait to dive in when I get a moment.

 

OK it didn't even take a moment. This is fantastic! Now I can get maximum timing performance with ease. Thank you!

 

Hi Everyone, I just started using the Midi Player Toolkit. Super awesome by the way! I have a question I was hoping I could get some help with.

Right now I'm experimenting with using the midi tools to create a VR Theramin. I need to be able to do EXTREME pitch bending in real-time more than just bending a single semitone up or down. I'm trying to use the PitchWheelChange command but it isn't working.

Code (CSharp):

1. midiStreamPlayer.MPTK_PlayEvent(new MPTKEvent()
2. {
3.         Command = MPTKCommand.PitchWheelChange,
4.         Value = pitchChange,
5.         Channel = synthChannel
6. }); ;

Is there a way to just directly control the pitch of the synthesizer similar to how volume is controlled? Right now the only way I can get it to work is to clear the sequencer and then restart at the next pitch but this ends up sounding pretty bad.

 

Hi
Thank to share your work. It's very interesting.
Look at the source file TestMidiStream.cs (demo TestMidiStream) how to use PitchWheelChange.
With pitchChange between 0 and 127 (64 no pitch), don't forget to shift left the value with 7 bit (Midi standard) like this :Value = pitchChange<<7
But yes, pitch change variation is limited in amplitude.

The other solution (but with MPTK Pro) is using the method MTPK_ModifySynthParameter, see here.

Say me if you need more information.

All my best
Thierry

 

Thierry,

I'm making an app for musicians and they are very sensitive to timing. They want their metronomes to be perfect.

Right now, my app is using a metronome based on the audioFilterRead metronome script that unity has provided.

When I play want to play sounds on a beat, I use the playScheduled method using the audioDSP Time of the next metronome click. It seems to work very well.

I think the articulation and velocity capabilities of midi would yield a more musical result than stitching together series of notes using playScheduled, but for my purposes, consistent beat timing would take precedence over musicality.

My questions... Does MPTK provide a default audible metronome? If so ( or even if not and it must be built just playing some sound on every downbeat ), how accurate / precise do you think would be?

I think I read a while back that you were planning to get part of MPTK off the main thread to help with timing issues. Do you think my concern is related to that work? What's the status of that work?

Thanks for your great work!

 

Hello MarkRob
That to apprecaite MPTK! Great question you have ! but not obvious to answer.

Does MPTK provide a default audible metronome?
No, but i take the idea (for another product in progress able to generate music, but it's a secret!)

How accurate?
For my point of view, it will depend on the buffer size and the System Sample Rate : latency_ms = buffer_size/rate*1000
So with buffer size = 512 and rate = 44100, the maximum latency is 11.6 ms, which is very correct for me! What your thought about that ?

you were planning to get part of MPTK off the main thread to help with timing issues
yes! it's already available in the current vrsion: the synth is build around the OnAudioFilterRead and the Midi reader is build in a system thread independent of the Unity Update loop.

All my best

Thierry

 

Thanks for you quick reply Theirry.

I think 11.6 ms is also very good. I was reading up on musicians' ability to perceive latency and most articles suggest that around 10ms is where they really start to notice. So if at the common buffer size you reference, if 11.6ms is the maximum, I say well done.

I think MPTK will serve my purposes, but you're the architect so you would know better than me. Here's my use case:

I envision having a large library of 1 measure midi files for which I'm only interested in the rhythm, articulation, and velocity information available in the midi. At a steady beat ( with a metronome playing ), let's say 4/4 at 90bpm, before I play each midi file at the beginning of each new measure/bar, I want to swap out the midi file's pitch information with what I desire at the time. So, pre-configured rhythm, articulation, velocity, with in-the-moment provided pitch information ( and probably sustain ).

Do you think I'll be able to do this successfully while maintaining a steady beat with a some manner of metronome playing ( it sounds like I'll have to have a parallel midi file always playing to serve as the metronome )?

 

If I well understand, you want to apply a melodic line to a predefined rhythm ? Have you already this library ? i could be interested ;-)
maintaining a steady beat from a midi file ? yes, indeed, but when the midi will loop (restart from the beginning) perhaps some break in the rhythm will be heard.
Have you seen this video ? The rhythm is generated by an algo (available in MPTK Pro)
And the beats are triggered from the event OnAudioFrameStart at each 10 ms (with the parameter 512 / 48000). But remember : you can't call Unity SDK in this event but MPTK API is available to build and play notes.

Also, you could look at my post here.

 

You understand correctly. I don't have the rhythm library yet. If I go the midi route instead of the playScheduled route, I will create it.

Thanks for your thoughts, demo, and related post. All together, things looks very promising. It's definitely worth prototyping out the solution; the potential musical benefits gained with midi I think are worth it.

Thanks for the quick response again! I expect to purchase to MPTK in the future to prototype.

 

BachmannT, sorry for disturbing but I have strange behaviour of the player when I want to get two identical notes in a row. This is very simple script:

public void EndLoadingSynth(string name)
{
Debug.LogFormat("Synth {0} loaded", name);
StreamChannel = 0;
midiStreamPlayer.MPTK_PlayEvent(new MPTKEvent() { Command = MPTKCommand.PatchChange, Value = CurrentPreset, Channel = StreamChannel, });
int Velocity = 127, currentNote = 48;
float noteDuration = 1f;
List<MPTKEvent> lEvents = new List<MPTKEvent>() {
new MPTKEvent()
{
Command = MPTKCommand.NoteOn,
Value = currentNote,
Channel = StreamChannel,
Duration = Convert.ToInt64(noteDuration * 1000f), // millisecond, -1 to play undefinitely
Velocity = Velocity, // Sound can vary depending on the velocity
Delay = Convert.ToInt64(0f ),
},
new MPTKEvent() /// Even I delete this event the result is the same
{
Command = MPTKCommand.NoteOff,
Value = currentNote,
Channel = StreamChannel,
Duration = Convert.ToInt64(noteDuration * 1000f), // millisecond, -1 to play undefinitely
Velocity = Velocity, // Sound can vary depending on the velocity
Delay = Convert.ToInt64(2f * 1000f),
},
new MPTKEvent()
{
Command = MPTKCommand.NoteOn,
Value = currentNote,
Channel = StreamChannel,
Duration = Convert.ToInt64(noteDuration * 1000f), // millisecond, -1 to play undefinitely
Velocity = Velocity, // Sound can vary depending on the velocity
Delay = Convert.ToInt64(3f * 1000f),
}};
midiStreamPlayer.MPTK_PlayEvent(lEvents);
}

Let me comment this script - I prepare list from three evens (1: start note la - duration 1 sec, delay 0, 2: stop note la, delay 2 sec, 3: start note la again ...)
But only the one note sounds ... not two ...

Thanks in advance

 

Hello
Sorry, it's a lack on the documentation. Delay apply only on NoteOn event. But you can set the Duration on the NoteOn rather using a noteoff.
I will update the documentation ASAP.

All my best
Thierry

 

Thanks for you answer. but even I exclude event with NoteOff command I have the same result - Note sounds once, not twice.

To be sure that we are talkinkg about the same - this is part of code (list of events) :
List<MPTKEvent> lEvents = new List<MPTKEvent>() {
new MPTKEvent()
{
Command = MPTKCommand.NoteOn,
Value = currentNote,
Channel = StreamChannel,
Duration = Convert.ToInt64(noteDuration * 1000f), // millisecond, -1 to play undefinitely
Velocity = Velocity, // Sound can vary depending on the velocity
Delay = Convert.ToInt64(0f * 1000f),
},
new MPTKEvent()
{
Command = MPTKCommand.NoteOn,
Value = currentNote,
Channel = StreamChannel,
Duration = Convert.ToInt64(noteDuration * 1000f), // millisecond, -1 to play undefinitely
Velocity = Velocity, // Sound can vary depending on the velocity
Delay = Convert.ToInt64(3f * 1000f),
}};

I checked the result in Unity Editor and on android .

 

Try to uncheck "Release same note" in MidiStream prefab inspector

 

Thank you very much! Everything works!

 

Good to know !
But I'm a little perplex, do you want to create all your music by this way ? Delayed notes must be reserved for short delay and not for all notes of a song !

Explanation of the issue : A real keyboard can't played the same note at the same time: you need to release the note before playing a new note. It's also the default behavior with a Midi Synthesizer like fluidsynth. But I added the possibility to disable this behavior in MPTK.

Also, you didn't hear the first note because when the synth is started the volume is set to 0 and increase to the max in 0.5 second. I will add the possibility to change this behavior in the next version.

 

Thanks for your comment. I don't create "music" - I just create small excercises in polyphony less then 100 notes (2-3-4 voices). I check one voice in 50 (~) notes. Everything works correctly. May be I will have the problem which you are talking about when I add second voice (the same note in other voice) - but I think that is normal and I work around the problem programmatically.

 

Ok! don't hesitate to contact me by email if you need a more private exchange.
Please, don't forget to quote MPTK on the store! It's really important for publisher.
Thank

 

Is there an option to check if MidiStreamPlayer has finished playing a list of MPTKEvents?

 

Hello
MidiStreamPlayer is useful when you want to process midi messages in your application. So you will be aware when the end of the list is reach!
Perhaps you speak about MidiFilePlayer ? in this case, the list of midi events is read from a midi file. You can use some event as OnEventEndPlayMidi
Look here for the full MPTK API Description.

 

Thanks, Bachmann!

My problem is a bit wild, as I am actually using MidiStreamPlayer in a way where I send a lot of midi messages to my application at once, each with a predefined offset (delay), so while I can calculate when the last note in the stream has finished playing, I was wondering if there was a quick isStreamPlaying boolean or something similar, that I might have missed. I can't use MidiFilePlayer as first creating a midi file from my midi events would be too slow.

For now, I did figure out a custom timer, though. So my problem is fixed. Thanks again!

 

good ! more information : a delayed event is not an event in a stream. All events will be checked at each cycle to verify if it's time to play the sample, it's like an extension of ADSR function something like DADSR, delay, attack, sustain, release ;-).
The next version will propose an enhancement of MPTK : building a list of events and playing this list in the internal midi sequencer of MPTK without creating an intermediate file. But you will have to be little patient ...
Ally my best

 

Hello, thanks for the toolkit, the free version is extremely generous!

I tried looking and couldn't find it, but is there any kind of function to get the timing position of a MIDI event in linear time (not ticks) without first playing the song?

I am working on an application that creates animations based on arbitrary MIDI files, but all done in advance at load time/preprocessing, not just "reacting" in real time to playback. For this purpose, I need to get the timing of each MIDI event in advance in seconds, without actually playing the song. The timing would be relative to the start of the song, so events at tick 0 would be 0.000s, and the rest starting from there.

The calculation from ticks to real time is simple enough when you know the tempo and the quarter note values etc., but if the tempo changes in the middle of the song, you also need all the information on the changes. MPTK_MicrosecondsPerQuarterNote apparently only holds the current value, which is always just the song's initial tempo if the song hasn't been played yet, and I need a way to account for the tempo changing before it happens.

It also seems the MPTKEvent SetTempo meta event does not carry the actual tempo data. All relevant seeming fields (Info, Value ...) are empty/zero, and using MPTKEvent.ToData() on these events also seems to have forgotten what the tempo was in the original tempo event, it outputs just 255 or FF. Or am I missing something here?

So in summary:
1. Getting the real time of MIDI events in advance without playing the song, accounting for tempo changing mid-song? (my primary goal)
2. If the above doesn't exist, getting the tempo data for mid-song tempo changes (again in advance without playing the song)?
3. If all else fails, getting the raw hex MIDI packet data for events so I can parse the tempo out myself? Could this maybe be somehow saved in MPTKEvents that were read from an existing MIDI file?

My use case is probably relatively niche, but it would be nice to be able to do these things. And again, thanks for the work on the toolkit!

 

Hello
Hard questions!
A very quick answer: have you look at MidiFileLoader ?
This class is able to load a Midi file but not playing it ... and in the free version of MPTK, you are lucky ;-)
The MPTK_RealDuration method takes into account the tempo change inside the midi. But it's for the whole midi. There is nothing for a specific event. No chance here.
How you select the event to calculate the time ?
You want for all ?
At a specific tick ?
I find this function interesting, could be for a next version

 

Thank you for the quick reply!

Yes, I've looked at MidiFileLoader, and noticed the RealDuration - but I am looking for a way to calculate the real time for events. So, a conversion from any tick value in the song into linear real time in seconds since the beginning of the song. For my animations it will be mostly NoteOn events, but obviously if you can do the conversion, it will then work for every type of event.

Right now I can do the conversion calculation myself if the song has no tempo changes, because the initial tempo is all I need. But for songs that do have tempo changes, I have been unable to get the actual tempo value from the SetTempo meta MPTKEvents, they have 0 for Value and an empty Info. I'm not sure if I'm missing something there.

 

Hello
Look the Duration properties in MPTKEvent, it contains the quarter per minute value for a MetaEventType.SetTempo event. Not really obvious!
I will also add the MicrosecondsPerQuarterNote in the Value properties for a next version.
All my best

 

I will add for the next version in MPTKEvent something like "public float RealTime;" with the absolute time in millisecond of the event. It's not yet ready, tempo change are complex to check in midi format 1. I need to take a reflection time!
Some info here: https://web.archive.org/web/2012043...usstopia/MIDI/~jglatt/tech/midifile/tempo.htm

 

Done in the future version 2.88!!! that was not so obvious when set tempo event was unsync with noteon. But it's ok now. A new properties will be available for each events with it's calculated time.
All my best

 

Hi,

I'm new to the asset and just getting started but have some pretty ambitious goals for it.
Just trying to wrap my head around it's ability and limitations.
I have the Pro version for reference.

I'm very interested in building my own soundfonts from synthesizers and drum machines I already own.
I see in the Sound Effects section of your website there is a screen shot from Viena, see below.

• Is there a way in MTKP to control the parameters listed, such as the various Envelopes and LFOs, listed in the photo below?

• If so can these controlling actions be recorded in to the MIDI file to be replicated/performed on play back?

My sound track will mostly be synths so this would add quite a bit to the feel if there is a way for me to do this.
Also for reference, my app will be mostly on mobile, specifically Android, and I plan on using multi-track generative music.

Thanks so much and looking forward to really utilizing the asset to it's fullest!

Jim

 

Hello
You have a very interesting project!
Yes there is a way in MTKP to control the parameters. See the demo MidiStream. With the pro version you can control the value of generator in realtime:


Another example with the demo EuclideanRhythm

Generator value can be modified in real time with a screen touch control.

The method to use is MTPK_ModifySynthParameter in MPTKEvent class. See documentation here. Generator list and example.

That was the good news...

For your second question:
If so can these controlling actions be recorded in to the MIDI file to be replicated/performed on play back?
The answer is no with a standard Midi. You can modulate some sound parameter with the Midi norm but not the SoundFont generators. See here. Some controllers could be reused but they will not be compatible with all synthesizers.


All my best
Thierry

 

Thanks for the quick reply!

So my understanding then is that there is not a way to automate the parameters like envelope, LFO, etc. on playback during the game?

Also last question (for now lol), how resource intensive are the parameter adjustments on say an Android device?
Is it going to bog down the device if a few adjustments are happening at the same time?

Thanks again, very much appreciate the quick support

 

Specific controllers (not standard) will not be recognised outside MPTK.
But all controllers are not used, it is certainly possible to record some generator change with standard Midi event CC (see my previous link here, all CC are not used) and apply them when reading the Midi ... into MPTK of course.

Not so intensive, these real time generators change apply on the the current parameters of the synthesizer, not for example, on the whole sample. But I have no idea what you want to do! A proof of concept could be useful.

 

OK, thanks for the CC idea and idea on resource usage
I'll be sure to report back as we have more of a proof of concept.
Thanks again for the quick responses!

 

Nice but I am bad at music making

 

Ah ah ! There is millions of midi file ready to play! No script need.

 

Good to know! I'll try it out in the future!