[Official] Integrated Midi Support.

I was just watching the talk on YouTube about audio, was hoping MIDI would eventually be supported, but didn't hear any mention. It's very good to hear it's at least being talked about. Having virtual instrumentation would really help a lot of people I am sure. So essentially yes, this would be incredibly useful.

I don't have a specific plan, as I am nowhere near audio in my current project... though I can find it useful in almost any project I do. Ranging anywhere from a variation in pitch when attacking (think Skyward Swords hit sounds with musical notation), without taking up more space creating a ton of small changes. To just being able to change up the mood in a multitude of ways. I'd need to experiment to really see what can be done, but somehow being able to change little nuances in sound based off actions of my characters in my game just like how the effects will now be handled is perfect enough for me. I don't think you'd really need anything that isn't the most basic or standard practices for midi either, as long as you stick with similar conventions to how audio will be done in 5.x series.

I think it could change dialog in a way to. I always imagined it being cool, instead of cutting into a new sound bit based off where your character is in a talk or cutscene, you'd be able to fluidly change up the sound when going through text. That way you can detect what note is being played... and play back the correct section, or consecutive notes after to transition perfectly, no matter where the song is, or when you press the button where those changes will begin.

Say for instance in a scene, where you must click the button to proceed to the next speech bubble or movement of a character in any RPG or Adventure game, you could allow for a series of notes to be played in variations while the character waits to proceed, and easily transitioning into bigger sound and hits as the action moves forward. So it's not sounding all cut off, and it flows just perfect. Or where the song is just 1 song, one style, one way. A lot can be done though.

I create more action/adventure/rpg/sidescroller style games. Current project is a 2.5D Side Scroller.

That's how I imagine it would be useful for now.

 

I can't think what I might use it for. Having said that, I've no experience with it so I'm not familiar with its full range of potential capabilities, and none of my work has been particularly musical.

When you say "MIDI support", what exactly do you mean? Running virtual instruments in real time, generating output for external music devices, taking input from external music devices..? (This goes back to my lack of experience in the area, I guess.)

 

Not sure how it works both legally and on the backend, but ability to use preowned or free vst's, along with a very basic piano roll with snapshots of some sort would feel good. I don't think it has to be crazy or complex. Basics would be suitable in an interface for game design. Just something where you can either hand map notes, or possibly allow midi keyboard support. The rest I think should be done through scripting.

 

Thing is it'll start moving more and more towards a full DAW suite, when IMO Unity have a metric ton of other things they need to get on top of. I'm not sure what use Midi would be without VST's / AU's / RTAS plugs and a midi editor? I suppose it could be cool to integrate in an FMOD style solution.

 

Oh I definitely agree, I know they need plugins too, just have no clue how engines work on the fundamental backend. For legal reasons, plugins not necessarily intended or developed for unity. Say I used something by the major vendors, like EWQLSO or Kontakt products.

In the case of something like FMOD, I'd be 100% cool with that. Would speed up development and not take away too much from all the other necessities. Right now MIDI isn't number one, I'm very happy where the audio is already headed. Would make gaming, and development just that much more fluid. Audio has always seemed to be the least talked about thing, yet it can potentially be something huge for any given game. Almost necessary to be as important for some.

 

I would absolutely use it.

I can only bundle a handful of mp3 in a mobile application. The size of the apk matters a lot. So my game ends up having almost no music.

MIDI would be perfect for me, I could finally add several minutes of music without blowing up the apk size.

The way I would use it, is have a ton of short clips that can seamlessly transition form one to another, and I would play them randomly. Certain clips would be played when there is danger, certain clips would be played in specific areas.

 

Couldn't you do that with normal sounds?

 

Nope, 5 mins of music and I am either talking about a few megabytes or I have to reduce the sample rate to where it sounds awful.

 

Yes. MIDI, beyond turning notes on and off and with a decent soundbank of instrument samples sounding pretty good, can be used for many other purposes. For example it can drive animations or DMX controllers or Arduino setups or in my case for the next three major apps I am coding in Unity will be based around configuring and controlling MIDI instruments and sample banks. For an example of how much bandwidth MIDI can save you I just download Mussorgsky's Night On Bald Mountain which was 23 kb, whereas the mp3 is 13.5mb and the wav is 156mb. Those on this thread saying to not put resources into MIDI integration really do not know it's capabilities I assume. I want it yesterday.

 

Hipsters? Particularly if it were part of a Unity Retro bundle featuring a Sound Blaster Pro card?

 

One word: YES!

My current music workflow involves writing MIDIs then using something like Synthfont to render an OGG. Sadly, while OGG sounds slightly better than pure MIDI, I think for retro-style works like the kind myself and @TylerPerry do, native MIDI capability would be better.

 

No. Focus your (audio) team to clean the possible bugs of the new audio system, and only IF you have (some) time, why not?

 

In our installations we try to hook up Midi and Unity through the numerous MIDI interfaces of the music industry. The continuous controls which MIDI offers are the most important signals for us. Time is of the essence: So optimized code in the Unity Engine is a plus for us.

 

Midi is clearly not needed. You need to install appropriate sound patterns onto the sound card in order to get at least some decent sound out of midi songs (which weights much in memory space). Classic sound file formats should be supported and the problems on that matter should be fixed.

Keep concentrated on REALLY useful stuff and fixes...

 

I thought that MIDI was just a format for data exchange - to hear anything, there needs to be some sort of sound synthesizer or audio sample data in the sound card. (Thinking back to the late 80s when we had Steinberg Pro24 software, and racks of midi synths and samplers to generate the sounds from given midi data).

 

I'm not currently mobile focused -- but if I were MIDI support would be pretty killer. I had to yank one of my soundtracks because the file size went over the 'download over the wire' for Apple and i was afraid it would hurt sales. A MIDI soundtrack would have been negligible in size and could have in fact been a much longer song .. and multiple songs.

Also MIDI would be pretty awesome for a lot of retro games.

For desktop I don't think I would use it right now, fully composed music tracks both on the asset store and elsewhere are very high quality at decent prices and dont inhibit my product in any way.

 

I think it's worth clarifying what you mean by "MIDI support".

Do you mean:

• Unity will be able to receive MIDI data
• Unity will be able to send MIDI data
• Unity will be able to play a midi file as music

 

I would mainly like MIDI support in the form of official OSC support and Input mapping support (Assuming that U5 Input is going to be beefed up).

Not too interested in MIDI file as music playback, but Soundtracker support in Unity 5.x would be nice

 

Try and find a decent editor or composer. No such luck. MIDI OTOH is ubiquitous.

 

For those thinking retro or you need a sounblaster card. Not so. You can load high quality samples and if you only need sax and violins then only load those two sample sets. I posted above that the MIDI file for playing a 19 minute orchestral piece was only 43kb. Let me emphasize that F-O-R-T-Y-T-H-R-E-E KiloBytes..for 19 minutes.. and a meg or two for good orchestral samples.

 

Like Superpig said, there is already support for tracker music formats which are better than MIDI (for playing music). The annoying thing is that there really isn't much tracker music out there to buy, that isn't retro-styled so you'd have to compose your own. I'd think something like an hours worth of music that only takes up a few megabytes would do well on the asset store.

There is already support for that, hopefully they won't be removing it in 5.0 which would suck.

 

I probably wouldn't use it. Although if I did, I would more than likely be trying to be mixing and manipulating things on the fly.

I suppose I lean toward wanting a plugin rather than something integrated, unless what's integrated is purely just read support that could be fed on the fly.

 

Yes, it's a file format. But to be played, it relies on the synthesizer of the sound card (which pretty often is not quite good). Therefore, this format is most often used in professional sound setups where instruments are either played via Midi-connections (connect a midi plug to the computer,...) or via so called Soundbanks which, to keep things simple, is some kind of real life sound recordings which are put into the sound card midi memory and then played accordingly to the midi instrument/notes setup. (NB: I'd say that almost 100% of current dance floor music is produced with artificial sound banks.)

Good Soundbanks (or good midi instruments) cost a lot of money. And it's not something that you can easily hand over to a customer to install it on their sound card.

This is the reason why good OGG or mp3 compression is used to transport sound data to the end user... much less hassle and much less MB to transport...

 

Since when? It's just a signal messaging system that carries 16 channels of information, it's usually event driven and carries such information as velocity / pitch etc.

The soundcard is for AD / DA plus the soundcard controls sample rate / bit depth (Nyquist). Back in the day they used synthesised (sine) frequencies (orginally FM 8-bit) dependant on channel information. It moved on to wave table (DB) and now is CPU bound / processed then sent to DAC with the resultant introduction of VST / etc.

Studio AD/DA interfaces, the likes of Apogee / Lynx / Orpheus etc. has nothing really to do with Midi nowadays. As you can just plug in a USB cable from a keyboard and record midi notation with no sound card involved in the process. All you're doing is manipulating notation in a DAW to then sending the channel bank to a VST / AU / RTAS plugin.

Nothing is installed on a soundcard..

 

MIDI channel routing, expression control, MIDI note assignment and on/off. This format controls tens of millions of real world devices. It needs the basics to do that in the interface with piano roll playback and scripting control. What can it do beyond making file sizes minuscule? I have seen a robotic metal band playing guitars and drums and bass and the only data given the bot servos was MIDI data. I have seen entire stage lighting shows preplanned and synch with the show music. In game music can go from snails pace to speed metal frantic using just the MIDI file and changing tempo. Shift the oboe to a cello midstream, take the sax and now make a horn section by adding trombone one octave below with a trumpet a fifth above the sax root..then change the chord from a major to a minor by shifting the 3rd from the fourth halfstep to the 3rd half step above the root. Play Night on Bald Mountain on a tinkertoy piano or tuned garbage can lids. Let a gamer plug his entire studio of keyboard and digital drums into the computer and have it play the games soundtrack.

 

God yeah, MIDI support would be great for package size management on mobile.

 

Yes, I need MIDI support:

- Reading MIDI files and playing it, so the user can interact with the music on a note level. For example as play along or he/she can entering and changing the score in the game while playing.

- The user can load up their own midi files.

- The user can connect MIDI devices as a kind controller. There are lot of such (inexpensive) devices out there made for Ipad/iphone which could be used for that.

- In short it's about interacting with the music not just playing a soundtrack.

I want to you use it for a kind of music learning game.


Yes for a good sound bank you have to spend some money, but then you have samples (wav files) which will be played by the synth. Nothing to install on a sound card. I guess most of the better music apps like pianist pro or sample tank with midi support do it that way. (And you only need to provide the instruments which are used in your music.)

 

One thing no one has mentioned so far is that a MOD file is (to oversimplify everything) basically a MIDI file that includes its own samples. So the question that follows that: If you import a MIDI file, where would the samples come from? Would Unity have its own library? Could you assign your own?

Let's assume that a MIDI file's importer contains 16 slots into which you can drag AudioClips to be used as samples. At that point, as far as the engine is concerned, it should be basically identical to a MOD file, correct? Unless I've misunderstood it.

So all we need is a MIDI file importer.

 

I was completely unaware of these file formats and what they could do. This really helps, thanks!

 

I wish Unity has a feature that, when i import a midi to the project, it will convert it to MOD/XM/IT based on the sound banks in my computer. But anyway, I feel fine if Unity play midi with user's default soundbank, because the main reason i use midi is just to reduce size.

 

No, converting MIDI to a tracker format doesn't work well. Usually easier to just remake it from scratch.

 

I would certainly like MIDI support, I already see myself using it for controlling light sources and maybe for school projects

 

Would like to see some kind of low level support. The data structure of the decoded midi file (type 0 SMF) should be accessible (read/write) for example to change the Tempo, TimeSignature or assigning Instruments to channels. A controllable timer (Start/Stop/Reset/Duration) for real time processing of the midi data should be available. The note event which is going to be played, should be accessible - so synchronization with other events should be possible. Receiving midi data from a midi device/controller and store it in a data structure would be nice. Saving these data structures to a midi file or as persistent object would also be a nice to have. Choosing/importing a different sound bank should be possible (sf2).

Best

 

I agree with everyone who said yes, with the best examples of use given by ippdev. Unity isn't only useful for making 'typical' games

 

I think that while there are certainly plenty of people who would use a native MIDI implementation, this seems more like something for Asset Store developers. It's a kind of niche thing. If it wouldn't take too much effort away from other things though, go for it.

 

Yes, please.
Midi is a communication protocol widely used for interfacing with hardware.
While using midi is not a solution for games music, it is useful for :

- people creating music software ( there are at least 2 synths made with unity in app store )
- people creating art installations
- game development ( control various game parameters via sliders and knobs yeah! )
- hackers an tinkerers
- it might be the reason to see again Taumel in the forums

That said, http://keijiro.github.io/ contains several solutions for using midi now, in Free and Pro versions of Unity. Unity Japan rules

 

I honestly don't know much about Midi, and I never needed it. So I have no personal need. If it let's us use VST and instruments in realtime somehow though, it might be different and more interesting (even if still in a niche capacity to me). Overall I'd rather the audio system gets improved and expanded in other ways first with bug fixing being focused upon.

 

From the research I've done, doesn't Unity already support a viable Midi-alternative? I believe the audio system currently in Unity has support for loading in small, uncompressed samples and then manipulating them through coding in order to create various forms of music or ambient audio? I believe they are called Tracker Modules?

Doesn't this basically serve the same needs that MIDI generally addresses. Traditionally, MIDI has been highly dependent on the specific hardware running it. But something like Tracker Modules wouldn't have that limitation.

 

That link is amazing! How do i find more such things?

Everyone should read this post btw, good i quoted it, people seem to have a very closed mind as to what Unity can be used for and even more relevantly, what midi can be used for. No, a different format 'that does the same thing as midi inside unity' is not a proper answer because midi isn't only used and useful for seemingly a really narrow conception of music generation

 

Try to google combinations of "gist", "unity 3d", "github", I do it regularly and there are nice things to find every month or so.

 

I am aware of one meager editor for these that is available..very retro and hard to get a grip on. IOW./The tools for this format are very limited and the tools for MIDI are in the hundreds to thousands for any style of composing or level of authorship and control. I was excited when Unity announced this formats inclusion..until I tried to find tools to convert my soundtracks. I did not get one soundtrack converted.

 

Yes it does..but tracker mods are basically useless..Show me one app using it. No tools to create them worth a huff and a puff..

 

http://skaven252.bandcamp.com/album/bejeweled-2

http://skaven252.bandcamp.com/album/bejeweled-3

 

I'm not saying that there isn't merit in integrating MIDI support into Unity, just that it may not be the best fit for a game engine.

One of Unity's biggest strengths is it's cross-platform reach. Over the years, they've managed to get it running on more and more platforms. And the pipeline for shifting it from one platform to the next is very convenient.

But MIDI playback is dependent on the hardware of whoever is playing it. You get higher-quality playback depending on what manner of sound card you're playing it off of, and how good the MIDI support in that card is. Very few computers these days have dedicated sound cards. I don't think any tablets or smartphones do. So no matter what you do, having MIDI in your game is going to have the lowest-common-denominator playback. A few lucky power-users who invest in high-quality sound cards will be able to enjoy the full range and depth of the MIDI tracks that are created. Everyone else is going to get a lesser experience. That's potentially a lot of effort for not very much return.

 

I would love to see this but with OSC support also. I have found Unity lends itself pretty well to tasks that are not strictly game orientated. Great to see some diverse ideas being floated!

 

This is incorrect. This has nothing to do with soundcards. I can play midi files and do not have a "sound card". I wish folks would do the requisite research before coming in and naysaying what s going to be a widely used protocol for all kinds of stuff..not just music. I can for example map a phrase to each of 127 notes of each of the 16 channels and play a conversation of 2032 phrases..Or I could break them into sentence parts and words and have a million different sentence structures via midi...dynamically and different every time based on a few simple rules. Any midi control surface can be used to control any mechanic in a game. From the midi.org FAQ page

"Application areas for MIDI include music, lighting control, show control (and themed events), machine control (audio and video), robotics, and more. MIDI is often used to operate live shows such as the fountain at the Bellagio Hotel, and the Pirate Battle at Treasure Island (both in Las Vegas). Many acts and performers use MIDI-controlled stage lighting. There are also MIDI messages specifically for video performance and machine control."

more at link...http://www.midi.org/aboutmidi/midi-osc.php

Now.. tell me that tracker mods can do this type of stuff. It cannot. And has virtually no tools that have any value to speak of. MIDI on the other hand..take a look at the video here and tell me that this is soundcard driven and has nothing to do with games.

http://www.animusic.com/index.php


GuitarHero and RockSmith both use midi. But they are not games right??.. Just FPS's and RPG's and flappy birds are.