Maestro Midi Player Tool Kit - Good news for your rhythm game !

Hi
I need some answer to check more in detail.
What soundfont are you using ? Is it the same soundfont in FL Studio ?
And the last: which wave and patch are you testing ? I Can't see in the video.
BR
Thierry

 

The soundfont I'm currently testing with is called GMGSx. It is the same soundfont I loaded into the FL Studio soundfont player.

I'm not sure exactly what wave I'm using, but in FL studio if you load the soundfont and select the standard drum kit (Patch 129), you can find the sounds with F#6 and G6 notes.

 

Hi
It's the sample 000232. BElow a screenshot from Viena


and here from MPTK:


Same rate 22050.

The issue is that you are comparing a sample playing by Unity with a Note playing with a sequencer.
Check this with MPTK: load the demo MidiStreamPlayer (it's a very simple sequencer), and set parameters as defined below. You will get something very close to FL Studio.



BR
Thierry

 

Thanks for the explanation, however, playing it with the sequencer still sounds significantly lower in quality than when I used the soundfont in FL Studio.

 

each sequencer sounds different, the algo are complex. Viena, Muse Score, .... all sounds different. Perhaps, the difference, I have not yet implemented effect with MPTK ...

 

I received a notifcation that you have posted a message but I can't get it ... strange.
For your error: delete the previous version of MPTK in your project before importing the new one.
BR
Thierry

 

Thanks for your reply.

As soon as i posted the message, i realizedwhat was causing the problem. Thats why i deleted it.

 

I will update the readme. it's not so obvious !

 

you are pro

 

Hi,

Before I purchase the PRO version I have some questions:

1. Can I send midi events to the audio player?

I am using my own midi-player.
Does your audio player have an API so I can send it midi events?
I want to send the following events:

Code (CSharp):

1. NoteOn(int velocity, int noteNumber)
2. NoteOff(int noteNumber)
3. ControlChange(int value)     // (sustain pedal)

2. Is this a wrapper of FluidSynth?

I do not want to use FluidSynth as I do not want to deal with Legal issues.
Is this library 100% written by you?

3. Is there any input delay?

I tried using RTMidi on Unity, but get ~100ms delay ( when I try with a usb piano).
Does this library have significant latency on the midi input port?

 

Hello
Thank for your interest for MPTK.
Can I send midi events to the audio player? No, only midi input areprocessed, you can't send midi events from MPTK to an external midi device.
Is this a wrapper of FluidSynth? Yes, but there is no legal issue: GNU LESSER GENERAL PUBLIC LICENSE
Is there any input delay? Hard to measure, but I have the feeling that there is a delay. Creating virtal instrument is not a use case for MPTK, it's my feeling.

So, don't buy MPTK !

Best regards
Thierry

 

No, I meant to ask:

can I send midi events to MPTK?

midiEvent ----> MPTK api ----> play audio


I ask this because I already built a midi player, but I need to output audio. So I want to send the midi events, individually, from my midi player to MPTK so it can play the soundfonts.

 

Yes ! you can. This is one of the major feature of MPTK.
see API example:

Code (CSharp):

1. midiStreamPlayer.MPTK_PlayEvent(new MPTKEvent() { Command = MPTKCommand.ControlChange, Controller = MPTKController.Pan, Value = PanChange, Channel = StreamChannel });
2.  
3. midiStreamPlayer.MPTK_PlayEvent(new MPTKEvent() { Command = MPTKCommand.PatchChange, Value = index, Channel = StreamChannel, });
4.  
5. midiStreamPlayer.MPTK_PlayEvent(
6.                             new MPTKEvent()
7.                             {
8.                                 Command = MPTKCommand.NoteOn,
9.                                 Value = CurrentNote,
10.                                 Channel = StreamChannel,
11.                                 Duration = duration,
12.                                 Velocity = Velocity,
13.                             });

And these videos. All are using the prefab MidiStreamPlayer able to play midi events from an algo.

Have Fun!

 

Hi, this might be the perfect asset for what I'm working on, but I'm having trouble getting it to work. I understand that adding soundfonts is only possible in the pro version. I've downloaded the free version and there is no soundfont so I can't test it out. If it works, I'll definitely upgrade to pro version; I just want to be able to test it first.

 

Hello
its' ok on my side. What's tour version of unity ?

 

Hello,

I am completely new to MIDI and music in general, but will try to explain what I need as precise as I can. I would like to know how can I read all tacts from a midi track, then analyze their sizes (i.e. 3/4, 4/4, 7/4 etc), and then I need to insert an empty tact inbetween some tacts with the same size for achieving pause (i.e. no sound).

 

Hi, tracks are always played in parallele. You can't used a track to pause others tracks. Br

 

Right, so there are several parallel tracks. Can I add an empty part to all of the tracks at the same exact position?

 

Hi!
I am working on a project where I need to play a few MIDI files but not resulting in audio but other events.
Is it possible to expose the note numbers and their velocities in real time to be used by other scripts as the midi file is playing?
I have a custom script that generates visuals instead of triggering audio samples, and I would like this script to be able to access all the notes and their velocities when the midi file is playing.
I though it would be simple but my coding skills are quite limited and it ended up giving me a headache
Btw, great work, your product is fantastic.
Kind regards

 

Sorry for the delay, seems that notification are break.
That will change nothing. Sorry.

 

Sorry for the delay, seems that notification are break.
Yes you can. Have a look to the prefab MidiFilePlayer. Uncheck the properties "Send To Synth", notes are read but not played.



You can use the event to receive note in your script and process it. Sorry for this you have to write some script.


Have a look to the demo TestMidiFilePlayerScripting
Best Regards
TRhierry

 

Hello TRhierry,

Thank you very much for your answer, it is really appreciated.

That is very informative!

Unfortunately, I am still getting lost in all the different scripts.
I managed to realize that all the MIDI messages are created by NAudio, but I'm still struggling to write a proper script that would "listen" to appropriate type of inputs from NAudio.

The only thing I want to achieve is triggering one VFX Graph by certain note, so that different VFX Graphs are triggered by different notes.
For now, they are triggered with live MIDI input from a keyboard thanks to MidiJack by Keijiro (https://github.com/keijiro/MidiJack).
For this, every VFX Graph Game Object has one script attached that goes like this:

Code (CSharp):

1. using UnityEngine;
2. using MidiJack;
3. using UnityEngine.Experimental.VFX;
4. [ExecuteInEditMode]
5.  
6. public class MIDIgraph : AbstractReactiveNote
7. {
8.     public int noteNumber;
9.     public MidiChannel channel;
10.     public VisualEffect vfxGraph;
11.  
12.     void Update()
13.     {
14.         if (MidiMaster.GetKey(channel, noteNumber) > 0.0f)
15.         {
16.             vfxGraph.SetFloat("SpawnRate",MidiMaster.GetKey(channel, noteNumber) * 1000f);
17.         }
18.         if (MidiMaster.GetKey(channel, noteNumber) == 0.0f)
19.         {
20.             vfxGraph.SetFloat("SpawnRate", 0.0f);
21.         }
22.     }
23. }

Probably it's clear that the MidiMaster.GetKey(channel, noteNumber) returns the velocity (float from 0.0 to 1.0) of the indicated note. That's exactly what I would need coming from MPTK.

Could I kindly ask you for directing me towards editing my script so that it picks up the MIDI from MPTK?

Thank you again, best regards,
Antek

 

First, Thanks for this great asset. I am looking to do the same as Ajancel. While I was able to trigger a public void in a script, I have no idea how to pass info such as note or velocity to said function.

 

Hello
No problem !

• Add a prefab MidiFilePlayer (read Midi file) or MidiInReader (read from connected Midi device)
• Uncheck the properties "Send To Synth", notes are read but not played.
  
• Add Event to the prefab

​

• In the script, write something like that:

Code (CSharp):

1.   /// <summary>
2.         /// Event fired by MidiFilePlayer when a midi notes are available (set by Unity Editor in MidiFilePlayer Inspector)
3.         /// </summary>
4.         public void MidiReadEvents(List<MPTKEvent> events)
5.         {
6.             foreach (MPTKEvent midievent in events)
7.             {
8.                 switch (midievent.Command)
9.                 {
10.                     case MPTKCommand.NoteOn:
11.                         Debug.LogFormat("Note:{0} Velocity:{1} Duration:{2}", midievent.Value, midievent.Velocity, midievent.Duration);
12.                         break;
13.                     case MPTKCommand.MetaEvent:
14.                         switch (midievent.Meta)
15.                         {
16.                             case MPTKMeta.TextEvent:
17.                             case MPTKMeta.Lyric:
18.                             case MPTKMeta.Marker:
19.                                 // Info from http://gnese.free.fr/Projects/KaraokeTime/Fichiers/karfaq.html and here https://www.mixagesoftware.com/en/midikit/help/HTML/karaoke_formats.html
20.                                 //Debug.Log(midievent.Channel + " " + midievent.Meta + " '" + midievent.Info + "'");
21.                                 string text = midievent.Info.Replace("\\", "\n");
22.                                 text = text.Replace("/", "\n");
23.                                 if (text.StartsWith("@") && text.Length >= 2)
24.                                 {
25.                                     switch (text[1])
26.                                     {
27.                                         case 'K': text = "Type: " + text.Substring(2); break;
28.                                         case 'L': text = "Language: " + text.Substring(2); break;
29.                                         case 'T': text = "Title: " + text.Substring(2); break;
30.                                         case 'V': text = "Version: " + text.Substring(2); break;
31.                                         default: //I as information, W as copyright, ...
32.                                             text = text.Substring(2); break;
33.                                     }
34.                                     //text += "\n";
35.                                 }
36.                                 infoLyrics += text + "\n";
37.                                 break;
38.  
39.                             case MPTKMeta.Copyright:
40.                                 infoCopyright += midievent.Info + "\n";
41.                                 break;
42.  
43.                             case MPTKMeta.SequenceTrackName:
44.                                 infoSeqTrackName += midievent.Info + "\n";
45.                                 break;
46.                         }
47.                         break;
48.                 }
49.             }
50.         }

Best Regards

 

Hi, see here code example https://forum.unity.com/threads/new...ccuracy-more-demos.526741/page-4#post-5326011

 

Hi, thank you very much for your detailed answer!
I have added the lines you've suggested to a new script attached to a new Game Object I have created in the scene but the Debug Log is not working, the notes are not being read to the console.
I also tried adding your additional script to the existing MidiFilePlayer.cs but also with no success.

I attach two screenshots under this link: https://imgur.com/a/bDXmcbM . They show how I added your proposed code, maybe I made some mistakes? I received no errors though.
Also, I was only interested in the NoteOn, so I removed everything belonging to MetaEvent.

Thank you again and sorry for the never ending questions...

Kind regards,
Antek

 

Hello Antek
It's a good start ! You have also to add the call of this method in the event "On Event Notes Midi" of the prefab MidiFilePlayer else MidiReadEvents will be never executed.

BR

 

Can we import a soundfont at run time ? My goal is to make a game with some musics in a database, associated with a sf2 file, download them from a server and use it, all in runtime. (So I can't use import with inspector in Editor)

 

Hello
No sorry, you can't import SF at runtime, only Midi.
Best Regards

 

Hello,

Thank you very much for your quick reply.
I did what you have explained in the picture, I am actually testing everything on the TestMidiFilePlayerScripting scene, as advised by you, that already contains those three calls, so it looks exactly like on your screenshot.
Yet the console is not reading the notes played.
Here is what it does read:

And then it reads the start and the end if stopped:

But then again, nothing else is read.

I really hope it can work, it seems that not much more is needed!

Best regards,
Antek

 

Sorry, but I can't help more, I have no idea. But, could you create a package of your project. I will have a look.

 

Hi,

Thank you for baring with me.
I am testing it in a new Unity 2019.2.11f1 3D project within the TestMidiFilePlayerScripting scene with only your MPTK asset imported.
The reading of the notes didn't work out of the box, so I added extra GameObject called ReadNotes with a custom WriteNotes script (with the code suggested by you) attached to it.
Please find the link to the package here: https://we.tl/t-7t9nkobCfM

Again, I am really thankful for your help and effort.

Best regards,
Antek

 

I opened your project:

there is no "MidiFilePlayer" prefab, I added one
I removed "TestMidiFilePlayerScript" (no use in this case)

Set event midi in MidiFilePlayer Inspector (from the Hierarchy not from the Project !)
Select the GameObject with your script ReadNotes

Select the method to call in your script

And run ...


It's ok now !

PS: you can removed "Missing Prefab"

 

Hi there!

Thanks for this asset! It fits my use case very well!

I am using the free version and I'm currently having an issue sending a "PatchChange" command event to MidiStreamPlayer. I'd like to do something like:

Code (CSharp):

1. _midiStreamPlayer.MPTK_PlayEvent(new MPTKEvent
2. {
3.     Command = MPTKCommand.PatchChange,
4.     Value = midiPatchIndex,
5.     Channel = streamChannel,
6. });

in Start or Awake of my game object that has a "MidiStreamPlayer" component, but I'm getting an:
"MPTK Error Object reference not set to an instance of an object"

I'm using Unity 2019.1.8f1.

Thanks!

 

Hello, thank to appreciate MPTK, it's a lot of work !
The reason is the Midi Synth is not yet started a at this time of the starting process.
Have a look yo the demo TestMidiStream, there is exactly what you need.

this method is called when the Synth is started and able to process Midi message:

Code (CSharp):

1.  
2. public void EndLoadingSynth(string name)
3. {
4.             Debug.LogFormat("Synth {0} loaded", name);
5.             midiStreamPlayer.MPTK_PlayEvent(new MPTKEvent() { Command = MPTKCommand.PatchChange, Value = CurrentPreset, Channel = StreamChannel, });
6. }

You can defined the call to this method with the MidiStreamPlayer Inspector:


Or by script:


Have a look to the script TestMidiStream.cs and ... choose the method !

Please, let a comment on the Unity Store if you are happy with this product. This is very help full for us.
Best Regards
Thierry

 

Thank you so much for that, it solves the problem! It was just a matter of selecting a different object in the inspector, one with the custom script attached.

Your help was invaluable, I appreciate it a lot.

Kind regards,
Antek

 

Thank to say that. Unity is not so simple to use, but with a few time with it, all is clear !
Best Regards
Thierry

 

Hi Thierry,

Firstly, thank you for the awesome MIDI system for Unity!

I just have a quick question, which I have been struggling to find a solution to.

I am trying to create a rhythm game of sorts, as follows:

1. I have created a stave on the screen.
2. I use the MIDIFilePlayer (without to send to synth enabled) to read the MIDI events as they occur. Depending on the event, a note is created on the stave in the correct position.
3. I then run a routine to move the note across the screen until it hits a bar, where I use a MIDIStreamPlayer to play the note.

This all seems to work fine.

However, I am trying to synchronise the notes to a backing track (MP3) using a standard Unity AudioSource. It appears that the two fall out of sync after around a minute or so of gameplay. I am guessing due to a latency somewhere.

I have tried many things:

1. I have figured out the delay in milliseconds between the MIDI file starting in the MIDIFilePlayer and the time taken for the notes to cross the screen.
2. I have tried synchronising the MPTK_Position with the AudioSource.time, using the above offset, which seems to create a lot of jumping around and still doesn't actually solve the problem. The MIDI playback still falls out of sync with the MP3 backing track.
3. I have tried various physics options, including adding velocity to the note, and creating a routine that intends to move the sprite in the exact amount of time needed. The problem still exists.

I am kind of stuck, and was wondering if you had any idea how best to synchronise the MIDIFilePlayer with an MP3 playing through an AudioSource?

Thanks in advance!

Joe

 

Hello Joe
Thank to appreciate MPTK !
To better understand your need. Why do you run an MP3 ? Is it the same music as the Midi but in MP3 format ?
BR
Thierry

 

Hi Thierry,

Thanks for your response, I am working on a rhythm game, where the backing tune is comprised of an MP3 file, with a MIDI score that is played over it.

For debugging, I also just tried to simply play the MIDI file using the MIDIFilePlayer at the same time as playing the MP3 to make sure it wasn't some issue in my code. The two still fall out of sync during playback. For reference, I am using a standard piano MIDI score at 80BPM and a metronome tick at 80BPM as the backing track in MP3.

 

ok, which platform are you using ? Mac ?

 

Hello, good news !
I thought it was impossible ... but I found a solution ! That will be integrated in the next version (Pro only).
It seems there is a limitation: i detected CPU overload with soundfont with huge sample size.
Best Regards
Thierry

 

Yes I am

 

There is an issue with MacOS. The timing is not accurate. That will be corrected with the version 2.7
But what you want to realize is quite complex : synchronize an MP3 and a Midi is difficult. They are so different by conception.

Try this to wait the next version.
It seems that Convert.ToInt64 have a different behavior between Windows and Mac :-(
Comment these lines in MidiLoad.cs, method ReadMidiEvents

Code (CSharp):

1. // The BPM measures how many quarter notes happen in a minute. To work out the length of each pulse we can use the following formula:
2. // Pulse Length = 60 / (BPM * PPQN)
3. // Calculate current pulse to play
4. CurrentTick = Convert.ToInt64(timeFromStartMS / TickLengthMs);
5. LastTimeFromStartMS = timeFromStartMS;

and replace with these lines:

Code (CSharp):

1. long deltaTick = Convert.ToInt64(((timeFromStartMS - LastTimeFromStartMS) / TickLengthMs) + 0.5d);
2. CurrentTick += deltaTick;
3. LastTimeFromStartMS = CurrentTick * TickLengthMs;
4.  

BR
Thierry

 

Hi Thierry,

Quick update, things are going well so far, and I have gone ahead and purchased the Pro version, it's going great!

I have managed to build out a large portion of the app, and it is working pretty well on iOS.

Just one quick question, which I hope you may be able to help with.

I am using a MIDIFileReader at the moment to read in a MIDI file, and programatically create notes on a stave. The timing seems spot on when the application is running, but when the application is paused, it appears that the MIDIFileReader does not pause promptly on the current MIDI tick. Instead, when the application is resumed, it appears to spit out all of the notes that would have otherwise appeared if the application was left running in one go.

In terms of mobile applications, and the user experience, I am guessing that users will frequently hit the home button or swipe the app away in order to resume later.

Is there any way that you know of (off the top of your head) to work around this?

I have tried the OnApplicationPause(bool pauseStatus) method in Unity, and executing MPTK_Pause() if the application is paused, and MPTK_Play() if the application is resumed. However this gives the same result.

Appreciate your input!

Joe

 

Hello Joe
I'm happy to know that your issue is resolved.
The case you are describing is very interesting. Helas, the MIDI sequencer is based on the time (DateTime) which is not a good idea!
I'm rewriting the sequencer with a different approach. I hope to deliver a new version in February. But if I have an idea for you before I will inform you.
BR
Thierry

 

Great thank you Thierry!

So are you looking into basing the sequencer on something like AudioSettings.dspTime? If you point me to the code in question I can also have a look myself.

Joe

 

Hello
No something else. Send me an email, I could make you participate to the beta. If you want.
BR
Thierry

 

Hey, just letting you know I (think) I found a solution to my problem from a few months ago! I was looking at the MIDI player settings and changed the "Interpolation Methode" from "FLUID_INTERP_NONE" to other options, all of which sound way better than the default. I haven't done much more testing beyond this, but I'm hoping changing this setting won't cause any new issues!

 

Hello
Thank for this information ! It's a good idea and there is no issue to changing this parameter, just more CPU need.I will set this value by default.
For now only FLUID_INTERP_NONE and FLUID_INTERP_LINEAR are implemented. I will add FLUID_INTERP_4THORDER and FLUID_INTERP_7THORDER more later.

Thank you a lot !
Thierry