How to trigger musical notes with collisions in iOS game?

I am making an iOS app, where I would like a ball to bounce against various objects, each collision sounding a different musical note. Like a ball bouncing across a xylophone.

What's the best way to achieve this?

It seems like the simplest way would be to import individual audio files for each note, and then have them played when the ball collides with the associated object. However, this isn't very flexible. I'd love to be able to add functionality that could change the timbre (i.e. texture) of the note.

Is there a way to do this using MIDI?

Thanks!

 

MIDI is not directly supported by unity. Also, modern audio cards, as far as I know, normally do not include midi synthesizer at all.

You can start sounds by adding a handler for collision, like usual.

Also, you can synthesize any sound you want by overriding OnAudioFilterRead.
https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnAudioFilterRead.html

This allows you to make procedural music. However, this is also a bit tricky to work with, because of latency, and because audiosystem uses its own time.

Additionally 3rd party libraries like FMod allow you to have sounds with parameters. So you could expose pitch as a parameter.

 

Yep, cost-cutting is the darndest thing and there isn't really any incenitive for hardware MIDI synths in computers anymore.

Perhaps this might help: https://docs.unity3d.com/ScriptReference/AudioSource-pitch.html

 

Forgot to ask: are you planning on adding MIDI input functionality to your application? If not, perhaps the individual files approach is not that bad. Regardless, all you need to do is to detect the collision and if a collision is detected, an appropriate sound is played. If your objects are visually different, you may try to include the sound into the prefab (not that you can't if they are all the same but it just makes sense to).

 

Thanks! I am not planning on doing any MIDI input. It's really just a circle sprite bouncing against other sprites. But I would like to add functionality where if the circle collides with certain game objects, it would produce more sprites. Each sprite would have a different shape to it, and produce a different sound. I'm thinking 5-6 types of "instrument" sprites.

So it sounds like I should make each shape as a prefab, complete with its own collection of sounds?

I'm a beginner, so any advice would be appreciated. Thanks!

 

Gotcha. I just thought that because you've mentioned MIDI that for some reason you explicitly need to use it or your projects logic is somehow centered around it.

Sprites in sprites as "effects" or brand new circles that behave the same way that the initial circle does?

That depends. The thing is that there are many different ways to accomplish this, but all of them have collision detection in common:
1. You can make the surfaces that the ball is going to collide with into their own prefabs containing different sounds. On collision, a sound is played.
2. You can make the ball prefab contain all the sounds and on collision with the surfaces, the sound is played.
3. You can store all the sounds separately (in a list for example) in a non-ball/non-surface GameObject and when a collision occurs, play the sound by the sounds index in the list/sounds name/etc.
4. You can make a state machine using cloud AI with supercomputing Web Services socket 2.0 RayTracing with a Docker hosted microservices switch using Electron to call your sounds randomly D).
I suggested "unifying" the surface prefabs with the sounds, as it seems to be the most "xylophone-like" way to accomplish this.
Take this image of a xylophone as an example:

Each and every key on a xylophone is tuned a certain way and has specific properties. When, what essentially is a ball on a stick hits a key, a sound occurs. While you could try to make a xylophone simulation for your project and consider things like the size of the surface, the shape, the material the surface is made out of and so on, this goes against the KISS principle and is most likely overkill for your task. What you can do is to translate the same concept that xylophone uses into your projects logic and treat each surface as a key. This means that your key prefab outside of physics-related components, such as a Collider or a Rigidbody needs to also contain a reference to the sound/sounds that you want it to make. Whatever script you're going to make should play the sound/sounds that the prefab stores a reference to.

No worries. In the context of prefabs, i highly advise you to check this out:

If you have any more questions/etc, ask away!

 

Yes, you can do it this way.

 

You are a godsend, thank you!! I'll most likely NOT use option 4... cuz I have no idea what that means .

 

I brought up MIDI because I wonder if there's a way to use sampled instruments in a more sophisticated way (rather than just triggering individual wav files or whatever). For example, if I wanted the timbre of the note to sound different with different strike velocities -- i.e. maybe the virtual instrument instrument has multiple samples per note, played at different velocities.

I've seen a few MIDI plug ins on github, but I'm struggling to understand what they actually do. I will keep digging into those, but will do it the KISS way first.

Sprites as in the 2D shapes that can be created within unity.

 

You can do that with FMOD.

You cannot do that simply without FMOD. Timbre is a complex effect.

You could try looking for a software midi synthesizer, but it will be probably a paint to integrate and many of those require a sample bank. Which can be copyrighted.

 

That was a silly joke that i've made just to pick on all of those overly complex solutions to various tech related problems.

As @neginfinity has mentioned, it's going to be a bit complex to do that and it's going to take you an awful lot of time as a beginner to do that completely on your own, without using any sort of an asset.
Right away, i think you can kinda "simualte" the velocity by adjusting the loudness of the emitted sound in relation to the force of the ball hitting the surface.

As for the shapes, i mean do you need the shapes generally speaking (which are going to be used in whatever way you want to later on) or for some specific purpose? I'll advise to avoid anything related to creating audio/visual assets dynamically/procedurally/whatever else way there is at the beginner stage and stick with pre-made solutions (just drop your assets into Unity and use them as they are).

The MIDI plug-ins that you've seen are most likely related to input, allowing Unity to recognize MIDI signals sent from a MIDI-compatible device (a digital piano keyboard for example).

The best thing you can do is just to create a less "sophisticated" version of your app. Unity allows you to change things at any time pretty easily and see the results almost immediately, meaning that you can create the most basic version of your project and if needed expand later on.