Question Using Unity as Midi Controller

Hi,

I'm new with Unity and I just want to do a simple application that can send out midi message. More precisely, I use my cellphone like controller with the touch function and when I move my finger, it changes a value (0 to 127) depending on the x,y axes.

Now this works fine but I just know how to use this values internally in my app but I would want to use it like a midi controller in other existing applications that can take midi input. I hope I'm clear enough.

I don't want to make sound with this app just send out the numeric values. Is somebody have any idea where to start with this kind of thing ? You would be life savers

Thank you all!

 

More information is needed.

(1) What type of cell phone is this? If Iphone what version of IOS are you running? If Android what version of Android are you running?

(2) How are you connecting to the midi device or what methods do you want to support? (e.g. Usb cable and/or Bluetooth?).

(3) What version of Bluetooth does your cell phone support?

(4) What midi device or device(s) are you connecting to specifically? If they are usb are the usb class compliant? Note if you are connection a midi usb class compliant device it expects power from the device in question. This can really drain your mobile device very quickly.

(5) Bluetooth is a nicer method to connect since you never have to worry about wires. If your midi device does not support Bluetooth natively then you can get some external things to plug in. Here are some examples of products that Yamaha has available to provide devices with bluetooth v4.x support. There are other companies making similar products.

NOTE: The first one is for the 5 pin din traditional midi device connector and the second one is for usb devices.

midi 5-pin din
https://usa.yamaha.com/products/music_production/accessories/md-bt01/index.html

usb midi
https://usa.yamaha.com/products/musical_instruments/pianos/accessories/ud-bt01/index.html

I have talked to friends who have the used the md-bt01 (5 pin traditional) version with a Yamaha piano and works fine. It was used in conjunction with an iPad mini 4 device. But it works equally on iPhones too.

Apple developed bluetooth midi and it is now a midi standard and you can read more about it here if you are not familiar with it here:
https://medium.com/@thomasgerbrands/the-truth-about-bluetooth-midi-54a3dc633052

Back to your question Midi and Unity.

You should look at Unity with desktop applications (e..g windows) for midi support first. It would be a good overview of what you need to support overall. You should look at the midijack program for unity here (https://github.com/keijiro/MidiJack)

There is discussion here about using this Unity application (which is a midi input) to support midi output. There appears to be a patch but I have not tried it myself. This issue has more discussion that you will find useful.

MIDI output?
https://github.com/keijiro/MidiJack/issues/3

If you already familiar with developments for mobile devices without using Unity you might consider just building an app for the mobile device directly. Then you can figure out what is needed from a core application perspective first and then bring that back to unity if you still want to wrap something within unity.

You will need to dig into the respective developer kits for IOS or Android to find the required midi APIs and work with them directly to output what you need on your device. If working with USB then you also need to understand how that works on your particular device and how it is supported in the SDK.

If you have an Iphone you are generally in much better shape since they have plenty of examples of using iphone or ipad as a midi controller in the app store.

The USB-IF publishes USB Device Class Specification for MIDI Devices v2.0. More details on the USB website ... it a long way to go but it is backwards compatible to midi v1.0.
https://www.usb.org/

For Apple you should be become an official apple developer if you are not already.
https://developer.apple.com/programs/

You can start here for midi topics on what you should use.

https://developer.apple.com/documentation/coremidi/#topics

Android requires that you have a specific version of the NDK. But you can start here to see what your device requires. I am not sure what they have in the google app store for examples but that might be a good thing to look at. Since cubase 3 came out on Android recently I expect it only

https://source.android.com/devices/audio/midi

I expect you already know about this website but I am just including it here for completeness.
https://www.midi.org

 

Hi thank you for your answer!

(1) Actually, we would want to make the app work on android and IOS, but for starting, we would at least make it work for IOS. I got an Ipad pro 13.5.1 for testing purposes.

(2) Ideally, for all the technical issues midi has with the USB cable that you said (and because I want it to be easily working on cellphones), I kind of took a different direction and go for sending OSC messages via Wifi instead exactly because it would be working more fluently on android and IOS. I wanted to use Unity because I'm making a kind of game with it (and I'm not at all familiar with mobile development, it's my first). Still your solution with midi-Bluetooth is really interesting!

I used midijack in/out for an earlier project I did and it's great but doesn't work for android or IOS, and all the purpose of my app is using my cellphone as a controller. I know there's an extension for the keijiro plugin for cellphone but it's just for input : https://github.com/momo-the-monster/MidiJack (I'm putting this here for anyone that would be searching for this). Now I was looking at keijiro OSCJack https://github.com/keijiro/OscJack and another OSC plugin: https://github.com/jorgegarcia/UnityOSC but I can't seem to make it work. I receive OSC on my computer (I test it with a monitoring app), but I have difficulty to make it work. I think it might not be the place for this specific issue, but if somebody knows how to use the OSC-Jack plugin that would be nice, I don't know how to start the server and everything.

I know that I kind of diverged somewhere else with all that and I'm sorry but thank you a lot for taking the time to answer me! It's really appreciated.

 

If would not use OSC at first and just make sure the normal midi path is working as expected without any additional overhead.

Before we consider getting into wireless side we should make sure the USB side works as expected for a wired midi connection with a simpler application. My understanding of Android is very limited. On the IOS side I would start looking at these examples. Some of these examples work on both iOS and MAC OS/X so it should make testing easier.

http://goodliffe.blogspot.com/2010/10/using-coremidi-in-ios-example.html

https://github.com/petegoodliffe/PGMidi

and also this looks interesting too.

https://github.com/mixedinkey-opensource/MIKMIDI

If want something more robust and commercial oriented you might look at something like JUCE. (It supports windows/mac/linux/ios/android). Read the license agreement carefully to make sure you understand the terms.

https://github.com/juce-framework/JUCE

 

Hi Did you Manage to Achieve this? I'm trying to find out how to send Midi CC 0-127 values out of an iOS app made with unity

 

I did this in Unity for an android app (should work on iOS) over wifi - trying to figure out USB but not sure it's in the cards

https://github.com/domportera/Reaper-LAN-OSC-MIDI-Faders-Android

 

Thanks, I remember seeing it and finding out it only does reaper. I found Another One Which I am yet to setup. https://42noir.com/es/

 

Hi,

Just wondering if there are any updates?

I want to be able to handle MIDI input events in my iOS game and am wondering if anyone has found a way to do this in Unity?