Maestro Midi Player Tool Kit - Good news for your rhythm game !

Hello, have you make some progress ? need more information ?

 

Hi Thierry,
I already had pro and I came across an issue about track volumes.

Please download the zip file and play test2.mid file with MptkFilePlayer
All tracks will be played in the same volume.

Piano has 10% volume
Marima has 100% volume
Drum has 0% volume (it shouldn't play)

Now, go to https://signal.vercel.app/edit page an import the file and play it. It plays as expected.

Is this a bug or a problem with midi file?
Thanks!

 

hello
Sure, there is something strange, I look at this more in detail and let you inform as soon as possible.
Regards

 

Hello again,
This issue is only when playing the MIDI file from the editor in the "Midi File Setup". That will be corrected with the next version. Do you need a patch in the meantime?

I checked also when playing in run mode. There is no issue, the volume by channel is what expected.
But could you check that "Apply Modulator" is set in the inspector ?



Regards
Thierry

 

Hello Thierry,

Thank you for quick response.
I updated to the latest version and in fact the problem is gone for play mode (as you said). I have no urgency for editor mode.

Thanks
Cihad

 

Hi again Thierry,

I have another question. Players can create midi files in my game and I use the following code:

Code (CSharp):

1. MidiFileWriter2 mfw = new MidiFileWriter2();
2. // ...
3. MidiFile midiToSave = mfw.MPTK_BuildNAudioMidi();
4. var bytes = MidiFile.Export(midiToSave.Events);

I somehow see the following exception on crashlytics. Any idea why?

 

Hello, before you need to sort the MIDI, like this:
mfw.MPTK_SortEvents();

Regards
Thierry

 

I have checked the source code. You are already doing it in `MidiFile.Export` method (See screenshot)
Do I still need to call `mfw.MPTK_SortEvents()` ?

 

Exact, you are right. Could you send me a simple extract of your script with this issue. I will test it. Thanks

 

I confirm you need to call mfw.MPTK_SortEvents(); before.
Have a look to the demo Assets\MidiPlayer\Scripts\Editor\MidiSequencerWindow.cs
method:

Code (CSharp):

1.  
2. private void WriteMidiSequenceToFileAndPlay(string name, MidiFileWriter2 mfw)
3. {
4.             // build the path + filename to the midi
5.             string filename = Path.Combine(Application.persistentDataPath, name + ".mid");
6.             Debug.Log("Write Midi file:" + filename);
7.  
8.             // Sort the events by ascending absolute time (optional)
9.             mfw.MPTK_SortEvents();
10.             mfw.MPTK_Debug();
11.  
12.             // Write the MIDI file
13.             mfw.MPTK_WriteToFile(filename);
14.  
15.            // ...
16. }
17.  

 

Oh thank you very much @BachmannT
I'm releasing an update with SortEvents function today and let you know if there is any exceptions thrown.

 

Hello!

I'm working on a musical compositional rpg. I'm trying to proof of concept currently by simply loading a midi and adjusting values in live time.

Currently I cannot figure out how to play through my List<MPTKEvent> that I've loaded through the midiFileLoader. I know I need to use midiStreamPlayer but all the examples are just one-shot notes.

Help is super appreciated. Thanks!

 

Hello, that sound very interesting! What could be a "musical compositional rpg" ?

I will try to answer to your question.
The midiFileLoader is just a MIDI loader, useful only to analyses the MIDI events from a MIDI file. No playing capabilities.

For playing music by script there are two ways depending your needs:

• Playing music you are building in real time: The prefab MidiStreamPlayer is the good choice. It take MPTKEvent for input. But you need to manage time at which each notes (or other MIDI events) must be played. Typically generated in your Update() function.
• Playing music built by code but delayed: The class MidiFileWriter2 is a better choice (with the MPTK Pro). Ypu are able to define time for each MIDI events to be played in MIDI ticks or in milliseconds.

See global explanation and examples in the pages above.

Others sources are the demos available:




Have a look also to the detailed API description.

• MidiStreamPlayer
• MidiFileWriter2

If your need is simply loading a midi (from a MIDI file) and adjusting values in live time, you could also use the prefab MidiFilePlayer, it's "just" a MIDI player from a MIDI file but you can modifying MIDI events in real time, before they are played.
Thanks to the OnMidiEvent event (MPTK Pro). The OnMidiEvent is used also in the demo "Play Midi With Script"




... perhaps another way with a major next version, but that need a lot of works!

Regards
Thierry

 

Thank you very much for such a fast response!

I have made some progress with the help from your post and have managed to a selected MIDI into the file loader by using:

Code (CSharp):

1.  
2.  private void LoadMidi()
3.     {
4.         if (midiFileLoader.MPTK_Load())
5.         {
6.             midiSequence = midiFileLoader.MPTK_ReadMidiEvents();
7.         }
8.         else
9.             Debug.Log($"Loading '{midiFileLoader.MPTK_MidiName}' - Error");
10.     }
11.  

I am then executing the sequence through a coroutine as such:

Code (CSharp):

1.     private IEnumerator PlayWholeMidiTrack()
2.     {
3.         for (int i = 0; i < midiSequence.Count; i++)
4.         {
5.             Debug.Log(i);
6.             midiStreamPlayer.MPTK_PlayEvent(midiSequence[i]);
7.             yield return new WaitForSeconds(0.1f);
8.         }
9.     }

Now I know you mention this: "But you need to manage time at which each notes (or other MIDI events) must be played. Typically generated in your Update() function".

Does this mean there's no way of obtaining the note's pause data? As you can see in the coroutine, currently it just waits for 0.1 seconds and sounds very awkward.

To answer your question about the musical compositional RPG, I'm trying to create a battle system where the player can change the key with their attacks. This is why I want to load a MIDI sequence, have it play exactly as loaded and then change it dynamically later.

Again, thank you so much for your help! My project is for my Master's so the help is super appreciated.

Napsta

 

A very good idea! Could be a fun game!
There is no "pause" in midi norm, each events MIDI Note-on contains a time to be played (attributes ticks) and another MIDI events Note-off for the time to release the note (like a pianist on a keyboard).
MPTK create a singleMIDI event Note-on with these attributes : Ticks and RealTime (for time in millisecond) and Duration in milliseconds:


What you want to do is to re-create the internal MIDI sequencer available in MPTK, not so easy! That could work for a very simple MIDI but will be ... ankward!
What do you mean by change the "key" ? Change the value of the note ?

 

Thanks for the reply. it's related to Java and C#. The data is the param from Java to C#.I will see the site carefully. I'm sorry,my English is very bad.

 

really?
no choice to add soundfonts on the free version?

 

Really no, sorry !

 

Have you ever solve this problem?
I met the same problem.
Waiting for reply.
Regards

 

Hello, I just ran some test with the last version of Maestro and Oboe, all works fine on my tests devices. Unluckily, I can't check for every hardware.
On which platform have you this issue?
Also could you check with this demo?

 

I want to make a rhythm game.

I already can produce note by midi file so far.
And didn't make sound until note move to the specified position.
Like this video.

But I got a problem.

In Unity, sound tempo is not meet midi file's BPM.
My midi file is 170BPM.

But Its tempo is wrong, not 170BPM.
And And the sound rhythm is not stable. Ragged speed.

My note moving code is here:

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4.  
5. public class GameObjectMove : MonoBehaviour
6. {
7.     public static float Speed = 15f;
8.     public float translation;
9.     // public GameObject MusicView3;
10.     // Start is called before the first frame update
11.     void Start()
12.     {
13.         // MusicView3 = GameObject.Find("MusicView3");
14.     }
15.  
16.     // Update is called once per frame
17.     void Update()
18.     {
19.      
20.     }
21.  
22.     void FixedUpdate()
23.         {
24.             // Move the note along the X axis
25.             translation = Time.fixedDeltaTime * Speed;
26.             transform.Translate(translation, 0, 0);
27.         }
28. }

How do I fix it?

 

Hello,
For the issue with the tempo, could you send your MIDI file?
For the second question, I don't see anything related to Maestro in your code. Could you send me more details?
Thank

 

Thank you.
How do I send file to you?
E-mail or Do you have discord server?

 

As you want! go to the web site, you will find all contacts information.
regards

 

Galaxy S21 with android 11.I have tested your new demos,and they all sounded weird.If you can borrow one form anyone of your friends, that will be show you directly。

 

Hey @BachmannT,
Another small question...
Is there a way to change event data (eg. change note value) in `OnEventNotesMidi` callback?

When I checked the code, it was possible in legacy version. Isn't it possible now with core mode?
Thanks

 

Hello
Not with OnEventNotesMidi but with OnMidiEvent
https://mptkapi.paxstellar.com/d7/d...player.html#a1ff7a431b64a01a3ce800351461e2241

regards
Thierry

 

Oh thank you for providing that. Unfortunately it works in another thread and I can't use transform and other unity objects. I'll try to find a way. Thanks!

 

Hello, another approach would be to use the two methods!

• OnMidiEvent (run in a dedicated thread): able to change the MIDI events in real time before they are playing. This can be done also in relation with your app. All static data in your application can be used if they are updated by your Unity Update() loop in parallel.

• OnEventNotesMidi (run in the Unity thread): to change what you want in your Unity scene.

Regards
Thierry

 

Hello,

I am a student working on a mobile (Android) implementation of a project called the MidiFilePerformer. To put it simply, it consists in a user being able to replay a MIDI file in a simple rhythmic fashion without knowing how to play music.

Now, for technical reasons in this project (i.e. needing to consider notes separated only by a few milliseconds as simultaneous), I need to be able to convert the entirety of the events' ticks to milliseconds. This, of course, is a fairly well-known algorithm (using the header TicksPerQuarter resolution and the tempo events strung along the MIDI file) but I was wondering if something similar already exists in the library to save me the work of copy-pasting and adapting my old code, written to do this job.

To try and clarify, the process would look something like this :

- Look through the List<MPTKEvent> obtained via the MidiFileLoader's ReadMidiFile() method,
- Exclude any events except for note and tempo
- Use the last tempo event as a reference to convert the tick of each note.

I was wondering if this is already part of the library somehow.

While I'm at it, I just want to make sure : ReadMidiFile *does* return a flat list containing all events regardless of track in chronological order, right ?

Thank you very much in advance for your help !

 

Hello,
You have an interesting project!
Yes, MPTK_ReadMidiEvents returns an array of all MPTKEvent ordering by time and each MPTKEvent contains the time in milliseconds of the MIDI event in relation with the tempo change.

Regards
Thierry

 

Merci !

Just to make sure, the field you're talking about is MPTKEvent.RealTime, right ?

I might also take the chance to ask, but : do you think MidiInReader will work on an Android app ? It'd be nice if we could take input through a MIDI keyboard connected to the phone via USB.

Thanks again!

 

Yes ! it's the attribute RealTime
https://mptkapi.paxstellar.com/d9/d..._event.html#a39a42055ea47ddc0db185f9b10345f67

Unluckily, MidiInreader (and especially the plugins MidiKeyboard) is not running on Android, only MacOS and Windows. Obviously, MPTK_ReadMidiEvents and RealTime will not work with a real-time MIDI interface!

Regards
Thierry

 

wow this is good news... I wish I can edit midi note on unity..
is that posible to us for edit parameter (midi cc, velocity, etc) ? @BachmannT

 

Yes that's the goal! And also manage other event like Control Change. But it's a long road. For the last two months, I have had to works on other aspect of Maestro (and others projects), so I got behind my plan. I will soon publish a version 2.9 but with a very light version of the sequencer.

 

That's awesome!

FYI, I made my own simple midi editor for my mobile game.

 

That feels very interesting! That will be an app available on a store?

 

Yes, search "melody run" on appstores.

I have a new question though, How can I get the exact milliseconds of currently played midi?

I tried `MPTK_Position` but it's just the last played midi real time so it jumps from note to note.
For example it jumps from 0ms to 1000ms but I need the exact time in every update loop such as 0ms, 16ms, 33ms for a game runs at 60fps.

 

Hello
OnMidiEvent is available for the MidiStreamPlayer and MidiFilePlayer. Look here:
https://mptkapi.paxstellar.com/d3/d..._synth.html#a1ff7a431b64a01a3ce800351461e2241

You get each MIDI events ready to played.
DateTime.Now could provide you with current time.

Regards
Thierry

 

Thanks for the answer. I guess I couldn't tell what I want to achieve.

Let's say I have a MIDI file with 3 NoteOn events:
Time (seconds) | Event
1.0 | NoteOn_1
5.0 | NoteOn_2
10.0 | NoteOn_3

So, my player will freeze and jump at 1.0 seconds and will freeze until 5.0 seconds
Then it will jump at 5.0 seconds and will freeze at 10.0 seconds
Then it will jump at 10 seconds and will freeze there

Is there an easy way to get the real midi play time in terms of milliseconds?

 

@BachmannT
Anyways, I solved the problem with defining new variable called real_msec inside MidiLoad.cs
real_msec gives the realtime at a tempo change



so I can calculate realtime whenever I needed by the following code

(midiFilePlayer.midiLoaded.real_msec + (midiFilePlayer.midiLoaded.cur_msec - midiFilePlayer.midiLoaded.start_msec) * midiFilePlayer.MPTK_Speed)

Thanks!

 

Yes, tempo_map could be useful. I have in my backlog the idea to add tempo_map and method like convert tick to time in the MPTK API.
In your case, the properties RealTime of each MPTKEvent was not useful?
https://mptkapi.paxstellar.com/d9/d..._event.html#a39a42055ea47ddc0db185f9b10345f67

 

Hi Bachmann! Thank you for making this great application. I am a musician in part myself, and I would like to use MIDI with Soundfonts to handle both the background music and sound effects for my app.

1) MIDI Creation Workflow
I am struggling to figure out a good workflow for creating MIDI files. I have numerous DAW programs like Steinberg Cubase 10, Ableton, Reaper, LMMS, previously Cakewalk Sonar when it existed.

However, in most DAW programs you create one track per synthesizer instance, and you will load in the soundfont into a synth for that particular track, with the bank/patch settings then made in the synth instance per track (rather than the piano roll editor or track itself).

I am unfamiliar with any good workflow for creating a single unified MIDI file where each individual track (of the 16 we are allowed per MIDI file) already has the bank/patch settings coded into it.

Do you know of any decent programs for doing this? I also just tried downloading a bunch of MIDI editors like Sekaiju, SynthFont2, MidiEditor. To get SoundFont working with those, I had to set up QSynth and connect it via LoopBe MIDI routing. But then yet again, it seems I must set the bank/patch settings in QSynth and save this as a preset.

Perhaps that is the right workflow though? Per song, create a MIDI file using say Sekaiju, while routing to QSynth. In QSynth, make a preset for each song also with the matching bank/patch assignments per channel.

I am just wondering if you have any suggestions or how you have generally approached things. Do you find any usual DAW can create a useful MIDI output? Or do you use or recommend a specific MIDI editor?

2) Music & Sound Effects
I want to have both my music and sound effects (power up sounds, jump sounds, etc.) handled via MIDI.

I presume only one MIDI file can be routed to the synth on playback - is that correct? Ie. If I have one instance of the MidiFilePlayer running for background music, can I send another MIDI file (or series of programmatic MIDI notes if needed instead) to it on command in addition to the MIDI it is playing (ie. for sound effects)?

Or would I need a second MidiFilePlayer, and if so, can two instances run side by side and is this reasonable or still efficient for mobile? How would you recommend handling this?

3) Looping or Seeking
If I have say 3 sections to a MIDI soundtrack file and I want the playback to loop between them or go from one part to another in sequence (like say in a pop song: intro, to verse, to chorus, to verse, to chorus, to bridge, to chorus, to chorus, to outro) is there any way to easily trigger moving between the sections of a MIDI file in code?

Sorry for the many questions but I have been struggling to figure out a good workflow or how to implement all this for the past few days. I would appreciate your guidance to hopefully find a good approach more efficiently.

Thanks again.

 

Hello,
Ok! thanks to appreciate Maestro! It's a huge work.
I will read carefully your mail and answer ASAP.
Regards
Thierry

 

I start to answer by your last question, the easier!
Yes, you can. MPTK_TickCurrent do the job. Also, have a look to the scene demo MidiLoop
It's easier also to convert bar to tick with something like that (with 4/4 signature!):
long ConvertBarToTick(int bar)
{
return (long)(bar * midiFilePlayer.MPTK_MidiLoaded.MPTK_NumberQuarterBeat * midiFilePlayer.MPTK_MidiLoaded.MPTK_DeltaTicksPerQuarterNote);
}

 

You can mix on your Unity project any number of MidiFilePlayer and MidiStreamPlay with no limit other than capacities of the device.
Each MidiFilePlayer are able to play only one MIDI file at a time but multiple MidiFilePlayer can play together.
MidiStreamPlayer don't take a MIDI file in input but real time command from your script to play notes and sounds. Example here with MPTK_PlayEvent.
Look also at this demo:


Each MidiFilePlayer and MidiStreamPlayer have their own synth (MidiSynth) and their independent parameters (current bank/preset by channel, effect, ...) but only one SoundFont can be loaded at a time by all the synths. It's a setting done globally by the soundfont setup.

That will introduce my answer for your first question, just I need more time to figure out a solution.

 

Okay that's very helpful.

How might you recommend monitoring one's position in playback? Ie. if you know that after beat #160 in a MIDI file you want to loop back to beat #28, and maybe you are running a playback coroutine to monitor the position. Is there a cheap variable or function I can check per frame to see the current MIDI tick?

At 60 fps on Update(), doing a MIDI seek based on current position will give up to a ~15 ms error (1/60f) but I think this might probably still be acceptable. ~30 ms error on 30 fps will be a bit more possibly noticeable. (If it doesn't seem smooth I will instead just loop out the MIDI in the expected pattern in advance and only loop back every few minutes to minimize transitions.)

That's very helpful. Thanks. I will have to see how much overhead two concurrent instances add. Perhaps it is not too bad as the same SoundFont is used for everything.

I am thinking for MIDI production the best approach is to write the music as one normally would in a standard DAW, then output each of the up to 16 tracks as individual MIDI files and load them all together into a MIDI dedicated program like MidiEditor where one can set the bank/patch settings per channel and output a final composed MIDI file. I am working toward trying that. (I still appreciate any thoughts you might have.)

Audio Sample Buffer Access
Just one more question: Regarding the access to audio DSP level code, is there any place in your design where one can directly get to the audio sample buffers?

One thing that would help me tremendously is being able to have simple math on each channel's audio output for level mixing (ie. Channel1FloatValues * 1.2f, Channel2FloatValues * 0.8f) or even submixing and adding effects like simple dynamic compressors that can run in real time at the audio sample rate. I have done some audio DSP before so this is not too hard for me if I can access the place it happens.

Currently otherwise the only way to control channel volumes (I believe) is through note velocity, which can utilize different sample layers in the SoundFont that are likely not desirable (ie. I might want strong note hits from one instrument, but just not drowning out the other instruments).

Do you know of any place in your system where the sample buffer is generated or could be accessed ideally on a per channel basis? In theory, I presume such a place must exist somewhere as the synthesizer must take the audio samples from each WAV and read them as directed by MIDI, then sum them together into a final audio buffer, then send that to a higher Unity function for output.

If I can get into the code before it sums the channel buffers somehow, I can gain mixing control.

What do you think? Is this even theoretically possible? If so, any ideas on where might I have to look to get to that data? What functions are creating the per channel or summed audio buffers from the SoundFont WAVs? Or if it's not within your code, if you know where they exist, might they be accessible say through another means like Reflection or the Unity Native Audio SDK?

Thanks again. Sorry for such detail oriented questions. But I do think this is a fantastic potential way to handle the music for my app. With computing power getting better every year it seems even more logical to let the system handle playback and even some basic aspects of mixing on the fly programatically rather than in the old fashion way of mixing down to MP3 from a DAW. Even the SoundFont you included sounds remarkably good for only 35 MB.

 

Hello
You have the choice!
https://mptkapi.paxstellar.com/d7/d...player.html#aa9814f63623a1f8d67dcb0630e033a0a
https://mptkapi.paxstellar.com/d7/d...player.html#aabd956ce6d15dca46ca0d53d9be40fa7

But all these methods are updated only when a new MIDI event is ready to be played.

You can use this one for real time value:
https://mptkapi.paxstellar.com/d7/d...player.html#a154e08aaba43883d20c016b2b388d3f5

But the best method in you case could be a callback.
For setting a callback function for each synth frame:
https://mptkapi.paxstellar.com/d7/d...player.html#a0e95cf3ffd055114f8d5bdfb06e7609b

Or for the MIDI reader
https://mptkapi.paxstellar.com/d7/d...player.html#a1ff7a431b64a01a3ce800351461e2241

These two methods are running on dedicated thread, not the Unity thread, so not dependent of the FPS. The counterpart is you can't use Unity API (except Debug.Log) but obviously you can read and write your own object instance.

In a Unity Maestro project there is 3 threads

• The main Unity thread with the Update()
• The synth thread
  
• The MIDI reader thread

Right! I do like MidiEditor, simple to use and close to the MIDI norms. it's easier to control the MIDI flow.
It inspired me to write the a Midi Editor inside Maestro (not yet available)

Of course, Maestro synth is based on audio buffer, thanks to OnAudioFilterRead from Unity AudioSource. But there is no official access to these buffers. Perhaps in the future ... I'm figure out how integrate VST3 components in Maestro. It's not easy and will take a long time, I'm not sure to succeed ;-)

Level mixing by channel is on the shelve!
https://mptkapi.paxstellar.com/d7/d...player.html#a1e26b13b816bfe148d65d18dfcfe3f32

Look at this demo:

Maestro synth is based on audio buffer, thanks to OnAudioFilterRead from Unity AudioSource. But there is no official access to these buffers.
Perhaps in the future ...

All the Maestro source is available (except the MIDI interface). Look at the method OnAudioFilterRead in MidiSynth.cs

That's the idea for Maestro MPTK: be able to fully, deeply interact with MIDI and Synth or not ... to simply play a song.
I have a lot of idea (and you too, it seems !) My backlog is growing more quickly than I can produce! Nevertheless, don't hesitate to propose enhancements, new functions.
For now, I hope to publish a 2.9 version soon with a beta version of the MIDI editor and a full rewrite of the class MidiFileWriter2.

Regards
Thierry

 

Hello
Once again, I will need time to answer you.
Just for now, if you change the internal file of Maestro, you will lost the ability to update with the next version!
Also, perhaps we could continue our exchange in a private way, like email ?

 

Hey Bachmann! No sweat. I figured it out. I had a theory last night that it must be using two fluid_voices for stereo channels (one left and one right) given that each fluid_voice seemed certainly to be only reading mono samples. I just tested that. It does appear to be the case.

So I will just need to change some of the functions in MidiSynth and fluid_voice to change the way it sums the channels/voices in order to gain full mixing control.

I just want to say thank you again for this great project and for making it open source like this. Music and synth production is how I got into coding in the first place years ago. I haven't had much time for that anymore since I've been swallowed by game dev the past few years. I love game dev, but I've missed audio DSP and synthesis.

This has given me a perfect entry point to get back into music and audio DSP. I had no idea it was so accessible in Unity and with this great framework for handling MIDI data it should be quite easy to get what I want out of it. I will see if modern mobile CPU's can yet keep up with my imagination.

I also appreciate the design in terms of how you (or Fluid) coded it with passing arrays and Lists for the audio buffers, etc through functions as references, as it will allow me to hijack functions with external static class functions and thus allow minimal changes to your synth. So if there is an update in the future, it won't be any challenge to update and then just re-implement the few lines of changed code I need in terms of switching the functions called.

Thank you again sincerely. It's a huge gift. If there's anything else technical I'm wondering I will email instead. But I think I should be okay now. I deleted my last long message just to avoid clutter. Good luck with your future changes and to do list.

I don't need any Pro functions I don't think yet but I will buy a copy just to say thanks. Cheers.