Search Unity

  1. Unity 2020.1 has been released.
    Dismiss Notice
  2. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Help Wanted Using Unity as Midi Controller

Discussion in 'Audio & Video' started by KillerPlankton, Jul 21, 2020.

  1. KillerPlankton

    KillerPlankton

    Joined:
    Jul 14, 2020
    Posts:
    3
    Hi,

    I'm new with Unity and I just want to do a simple application that can send out midi message. More precisely, I use my cellphone like controller with the touch function and when I move my finger, it changes a value (0 to 127) depending on the x,y axes.

    Now this works fine but I just know how to use this values internally in my app but I would want to use it like a midi controller in other existing applications that can take midi input. I hope I'm clear enough.

    I don't want to make sound with this app just send out the numeric values. Is somebody have any idea where to start with this kind of thing ? You would be life savers :p

    Thank you all!
     
  2. longroadhwy

    longroadhwy

    Joined:
    May 4, 2014
    Posts:
    1,483
    More information is needed.

    (1) What type of cell phone is this? If Iphone what version of IOS are you running? If Android what version of Android are you running?

    (2) How are you connecting to the midi device or what methods do you want to support? (e.g. Usb cable and/or Bluetooth?).

    (3) What version of Bluetooth does your cell phone support?

    (4) What midi device or device(s) are you connecting to specifically? If they are usb are the usb class compliant? Note if you are connection a midi usb class compliant device it expects power from the device in question. This can really drain your mobile device very quickly.

    (5) Bluetooth is a nicer method to connect since you never have to worry about wires. If your midi device does not support Bluetooth natively then you can get some external things to plug in. Here are some examples of products that Yamaha has available to provide devices with bluetooth v4.x support. There are other companies making similar products.

    NOTE: The first one is for the 5 pin din traditional midi device connector and the second one is for usb devices.

    midi 5-pin din
    https://usa.yamaha.com/products/music_production/accessories/md-bt01/index.html

    usb midi
    https://usa.yamaha.com/products/musical_instruments/pianos/accessories/ud-bt01/index.html

    I have talked to friends who have the used the md-bt01 (5 pin traditional) version with a Yamaha piano and works fine. It was used in conjunction with an iPad mini 4 device. But it works equally on iPhones too.

    Apple developed bluetooth midi and it is now a midi standard and you can read more about it here if you are not familiar with it here:
    https://medium.com/@thomasgerbrands/the-truth-about-bluetooth-midi-54a3dc633052

    Back to your question Midi and Unity.


    You should look at Unity with desktop applications (e..g windows) for midi support first. It would be a good overview of what you need to support overall. You should look at the midijack program for unity here (https://github.com/keijiro/MidiJack)

    There is discussion here about using this Unity application (which is a midi input) to support midi output. There appears to be a patch but I have not tried it myself. This issue has more discussion that you will find useful.

    MIDI output?
    https://github.com/keijiro/MidiJack/issues/3

    If you already familiar with developments for mobile devices without using Unity you might consider just building an app for the mobile device directly. Then you can figure out what is needed from a core application perspective first and then bring that back to unity if you still want to wrap something within unity.

    You will need to dig into the respective developer kits for IOS or Android to find the required midi APIs and work with them directly to output what you need on your device. If working with USB then you also need to understand how that works on your particular device and how it is supported in the SDK.

    If you have an Iphone you are generally in much better shape since they have plenty of examples of using iphone or ipad as a midi controller in the app store.

    The USB-IF publishes USB Device Class Specification for MIDI Devices v2.0. More details on the USB website ... it a long way to go but it is backwards compatible to midi v1.0.
    https://www.usb.org/

    For Apple you should be become an official apple developer if you are not already.
    https://developer.apple.com/programs/

    You can start here for midi topics on what you should use.

    https://developer.apple.com/documentation/coremidi/#topics

    Android requires that you have a specific version of the NDK. But you can start here to see what your device requires. I am not sure what they have in the google app store for examples but that might be a good thing to look at. Since cubase 3 came out on Android recently I expect it only

    https://source.android.com/devices/audio/midi

    I expect you already know about this website but I am just including it here for completeness.
    https://www.midi.org
     
  3. KillerPlankton

    KillerPlankton

    Joined:
    Jul 14, 2020
    Posts:
    3
    Hi thank you for your answer!

    (1) Actually, we would want to make the app work on android and IOS, but for starting, we would at least make it work for IOS. I got an Ipad pro 13.5.1 for testing purposes.

    (2) Ideally, for all the technical issues midi has with the USB cable that you said (and because I want it to be easily working on cellphones), I kind of took a different direction and go for sending OSC messages via Wifi instead exactly because it would be working more fluently on android and IOS. I wanted to use Unity because I'm making a kind of game with it (and I'm not at all familiar with mobile development, it's my first). Still your solution with midi-Bluetooth is really interesting!

    I used midijack in/out for an earlier project I did and it's great but doesn't work for android or IOS, and all the purpose of my app is using my cellphone as a controller. I know there's an extension for the keijiro plugin for cellphone but it's just for input : https://github.com/momo-the-monster/MidiJack (I'm putting this here for anyone that would be searching for this). Now I was looking at keijiro OSCJack https://github.com/keijiro/OscJack and another OSC plugin: https://github.com/jorgegarcia/UnityOSC but I can't seem to make it work. I receive OSC on my computer (I test it with a monitoring app), but I have difficulty to make it work. I think it might not be the place for this specific issue, but if somebody knows how to use the OSC-Jack plugin that would be nice, I don't know how to start the server and everything.

    I know that I kind of diverged somewhere else with all that and I'm sorry but thank you a lot for taking the time to answer me! It's really appreciated.
     
  4. longroadhwy

    longroadhwy

    Joined:
    May 4, 2014
    Posts:
    1,483
    If would not use OSC at first and just make sure the normal midi path is working as expected without any additional overhead.

    Before we consider getting into wireless side we should make sure the USB side works as expected for a wired midi connection with a simpler application. My understanding of Android is very limited. On the IOS side I would start looking at these examples. Some of these examples work on both iOS and MAC OS/X so it should make testing easier.

    http://goodliffe.blogspot.com/2010/10/using-coremidi-in-ios-example.html

    https://github.com/petegoodliffe/PGMidi

    and also this looks interesting too.

    https://github.com/mixedinkey-opensource/MIKMIDI

    If want something more robust and commercial oriented you might look at something like JUCE. (It supports windows/mac/linux/ios/android). Read the license agreement carefully to make sure you understand the terms.

    https://github.com/juce-framework/JUCE
     
unityunity