Search Unity

  1. Improved Prefab workflow (includes Nested Prefabs!), 2D isometric Tilemap and more! Get the 2018.3 Beta now.
    Dismiss Notice
  2. The Unity Pro & Visual Studio Professional Bundle gives you the tools you need to develop faster & collaborate more efficiently. Learn more.
    Dismiss Notice
  3. Improve your Unity skills with a certified instructor in a private, interactive classroom. Watch the overview now.
    Dismiss Notice
  4. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

Audio Any code/plugin to read & play tracker (mod/it/xm) music files?

Discussion in 'Audio & Video' started by JoeStrout, Oct 14, 2017.

  1. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    7,189
    Yes, I'm aware that Unity has built-in support for importing Tracker Modules as audio clips.

    But I'm thinking about making a rhythm game, where I need to turn the note events in the file into visual objects and game events in my game. So just turning it over to FMOD to play out the speaker isn't good enough. I need to know how many channels there are in the file, and what each of those channels is doing.

    I suppose I can pick one format (probably .xm), dig up the spec, and write my own importer. And then make my own mixer code to play it... but all that sounds like a fair amount of work, so I thought I'd ask here.

    Does anybody know of any Unity (or generic C#, I suppose) code for reading these mod files, and playing them in a way that lets me get info about the event stream?

    (And for that matter: would anybody else be interested in such a thing? Unlike the built-in mod support, this would let you do things like display level indicators for each channel, easily change the volume of individual channels, etc.)
     
    mbbmbbmm likes this.
  2. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    I'm not of any help here unfortunately, just want to say that I would be very interested in such a thing. Module trackers are really powerful and (imo) nice to use and the potential interactivity here would be awesome for games as well as experiences and art pieces.
     
    JoeStrout likes this.
  3. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    38
    A Tracker Module player with per-channel mixer access is something that I've spent years searching for — I literally found this thread searching for it again. It's such a shame that Unity's Tracker Module support fell by the wayside as quickly as it was introduced, but Unity's implementation never did enough for it to be useful to anyone. Most people seem to either make their own purpose-built implementations, or turn to [other] [solutions].

    The closest I've found to a solution for tracker modules is this exchange on Unity Answers. It isn't a plug-and-play solution, but hopefully it'll get you closer. https://answers.unity.com/questions/577402/multi-channel-audio.html
     
    Last edited: Nov 21, 2017
    mbbmbbmm likes this.
  4. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    7,189
    Well, it's certainly something I could make (I've dug into the common formats, and they're pretty simple and well-documented). I'm not certain whether there's enough demand to make it worthwhile... but maybe!

    Let's see, so far it looks like we have two people (besides me) who would be interested. I wonder how we could find more?
     
    mbbmbbmm and HeyItsLollie like this.
  5. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    There is at least a certain number of people interested in such a feature:
    https://feedback.unity3d.com/sugges...-of-tracker-module-formats-dot-xm-mod-etc-dot
    68 votes could mean anything between 6 and 60 people, right? Trackers are not mainstream, but I think there are still a few people using them and some people might even think about learning one for the advantages this could give (small file size, reactive audio etc.)
     
    JoeStrout likes this.
  6. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    7,189
    Yes. Particularly the reactive audio seems like a compelling reason to me. Of course there are other ways to accomplish that — in High Frontier, in design mode, we have a complex multi-track score for Design Mode, and we fade tracks in and out based on how complete your design is. But it's a heckuva lot of data, and still limiting in some ways — for example, there is no good way to tell when the various instruments are being played.

    I'll play around with this a bit over the holidays and see how far I get.
     
    HeyItsLollie and mbbmbbmm like this.
  7. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    609
    I also like the idea.

    Edit: sid support ;)
     
    JoeStrout likes this.
  8. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    Supercool :)
     
  9. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    609
    @JoeStrout and, did you do anything on this front?
     
  10. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    7,189
    I did... I got on open-source mod player up and running within Unity, and it has the ability to turn individual channels on and off.

    That's about all it provides, though; I was about to dig into getting feedback on the level of each channel (so you can provide level meters or whatever), when a squirrel ran by and I got distracted.
     
    HeyItsLollie likes this.
  11. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    609
    i hate those squirrels...
     
    JoeStrout likes this.
  12. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    609
    I had a little play with this and ended up with a very crude prototype of a script that syncs the original notes to the audio playback of Unity:
    • read the raw mod file
    • store the individual rows per channel of the song in memory
    • align playback time of audioclip to row position (visual output: just like it looks in a player, highlights current row)
    • allows to use instructions / effects that are not implemented by the Unity audio import (i.e. jump to specific repeat location)
    • note: this is not a mod player - it uses the native Unity audio playback --- i.e. this will not allow you to change playback at runtime, i.e. switch off a channel or instrument, etc.
    can mainly be used for audio visualization effects based on the original notes, rather than assessing samples / spectrum.
     
    HeyItsLollie likes this.
  13. Pinkuboxu

    Pinkuboxu

    Joined:
    Mar 20, 2014
    Posts:
    45
    Did everyone get burned out worrying about Tracker Music files, or is it still interesting? I hope they aren't going extinct, they are far more useful than people know it seems. I still don't know why they don't have comparable control with it to FMOD because the feature is almost useless in Unity without some API for control and as far as I can tell the almost part is only because of the size of the files. The fact that I can double click the file in unity and it opens ModPlug Tracker has been really good too. I just hope they add more of the control features that makes them so powerful.

    I've been using Tracker files for some time, before Unity was a thing even, and though I'm glad it can use them It doesn't take full advantage of the potential. In fact it barely does anything to make them more useful than a standard sound file and I'm not entirely certain it uses any less resources other than the space the asset takes up.

    I use it for music. In the past when working with my own FMOD and "tiny mod player" setup, I could control the tempo(ramp it up), volume of each channel(add the electric guitars when stuff gets exciting) and had precision timing as to what song patterns to run and when, but also can send data from a row to the playback and such. That much is super robust.

    I also use it for sound effects. It's super valuable for this, and I think it gets over looked, as you can have one file stand for a highly variant sound effect, multiple sound variations to kill monotony, and have the help of envelopes and other effects. All of that would work without the need of a fancy Audio Authoring Tool. Granted you have to take time to learn how to use a Tracker like ModPlug Tracker but it probably would take just as long to learn another Tool and even longer to code your own system.

    Being able to change more intimate things on the fly, like samples and instrument data isn't as necessary but I got that far in the past on my own. Technically you can do that to the file in memory I suppose. I sort of think they should concentrate on the native ModPlug Tracker format as well, given the new features. Then again, I imagine this stuff is all possible through an Asset if someone where to flip the ModPlug source into a .dll and give it C# hooks for use in Unity. It must not be interesting enough or valuable enough to anyone to do so?
     
    HeyItsLollie likes this.
  14. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    609
    I think @JoeStrout got something working.

    There are a few open source C# based mod players that I guess could be made to work in Unity. When I looked at those a few months back, the lack of maintenance and documentation turned me off (i.e. too much work)

    I agree though, it would be great to have better mod support. I only wrote a basic reader to allow for precise effect timing based on the tracker position, playback is through Unity native mod support (so no control)
     
  15. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    7,189
    I did make some progress, but then yeah, I got distracted by something shiny (and shifting requirements in my own projects meant I no longer had any immediate need for it).

    It's a surprisingly deep subject, with a lot of formats and possible control points to think about.

    If anybody wants what I have, PM me and I'll zip it up and send it along, with no docs or implied fitness for any purpose whatsoever. Or pester me enough, and I might be convinced to polish it up and make a proper asset out of it. :)
     
    HeyItsLollie likes this.
  16. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    I'll do both :D
     
  17. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    I agree this alone would be very nice to have! Possibility to change effects like arpeggio settings or things like that would also be super nice. It would open even more ways to procedurally tie things happening in the game to music parameters

    That sounds interesting, how do you do this? Do you have just different pitches, effects etc. using the same sample?

    I don't know but it's maybe the worry that people might be a bit scared to use a music tracker. It's not exactly a shiny new UX that the trackers provide. But I think it's a bit of a chicken-egg situation. If you could do powerful things in Unity with it, more people would probably be inclined to learn it.
     
  18. MrWald

    MrWald

    Joined:
    Dec 7, 2016
    Posts:
    2
    Hey, anyone had success with this subject? I'm currently writing a dissertation about the benefits of using tracker-based adaptive music on modern game engines and I'm planning to make a prototype for it. However, I couldn't find a way of using its advantages (change patterns, mute channels, change tempo, etc.) inside Unity.

    Playback-only isn't interesting enough for these formats so at the time when I started with this project I kinda changed directions. I searched for the Renoise format (.xrns) and discovered that it's xml-based, then I parsed the data from the xml and started playing the notes in a patch I made in PureData. Once this patch is done (still needs some work) it is meant to be implemented in Unity, using libpd or heavy if possible.

    Although this solution seemed interesting for me, I still want to support legacy tracker formats, such as .s3m, .xm, .mod or .it. Today I found this post and now I have some hope again! I'd love to know if there's any progress.

    Cheers!
     
    HeyItsLollie likes this.
  19. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    I played around with JoeStrout's implementation a bit and it works well. I haven't had the time to dive deeper into it yet though. But muting channels is easily possible for instance. Sounds like an interesting dissertation by the way!
     
    HeyItsLollie and JoeStrout like this.
  20. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    38
    A recent episode of DF Retro turned my attention towards a mod-player library called Libxmp-lite. It supports the main four module formats (MOD, S3M, IT, XM) and intended for use with games and embedded apps. It seems to have been written in C, which makes me wonder what it'd take to port it to C# (if that'd even be necessary) and get it functioning in Unity. It feels like a relatively-sensible solution.

    I use Renoise, and I've thought about Renoise's XRNS format a bit. I was surprised to find that it's basically just a zip container - Samples can be easily unpacked and accessed, song/instrument data is XML-based as you've said. I've seen a couple cases of people using XRNS as an "event" sequencer (count 'em: one, two), where the audio is pre-rendered, but events are time-synced.

    It'd be awesome to have support for a modern tracker format like Renoise, especially if its DSP effects were included (no VST support, of course). We'd probably never see something so fully-featured unless the Renoise devs themselves got involved.
     
  21. Pinkuboxu

    Pinkuboxu

    Joined:
    Mar 20, 2014
    Posts:
    45
    That is pretty much right as well as a mixture of sound effects for dynamic ambience. I was mostly inspired by how sound effects get created for homebrew games on early consoles, like in famitracker for nes roms. You can get a ton of UI sounds from tiny samples, but you can also have a couple loops of birds/crickets chirping or traffic and crowd ones sound like it never repeats by spacing them out more. Yeah, I know you can always just do ambience programmatically, but having the tracker play back take care of it seems intuitive from an artistic point of view .
     
  22. MrWald

    MrWald

    Joined:
    Dec 7, 2016
    Posts:
    2
    Wow, I searched a lot and never found these projects... this seems to be the thing I am trying to do. Thanks for the references!
     
  23. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    609
    If you are after a parser, that is exactly what I did (I was unaware it has been done before) - I was parsing a mod file (not xml) - mod format was well documented.

    Very easy to use for visualization but very limited control (it’s not a player) it just knows the mod notes & effects at any point of time
     
  24. Omita

    Omita

    Joined:
    Dec 25, 2013
    Posts:
    11
  25. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    7,189
    Nope. The trouble with GitHub (for me, anyway) is that it only supports Git.
     
  26. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    38
    I have no prototype, sorry if I gave that impression. Only countless hours of thoughts.
     
  27. EvilDingo

    EvilDingo

    Joined:
    May 7, 2011
    Posts:
    188
    I was suprised when I saw Unity supported .mods. I had never created anything but loved the format way back in day. So anyway, I had a go at making a simple .mod to see how Unity worked with them and to my surprise, the playback was excellent. Actually - better than excellent. Having never created music with a tracker, I was immediately drawn to the benefits.

    I found this thread when I started looking for visualizations sort of like Star Control 2 had.

    It seems to me that a mod is essentially a sampler. Particularly for the old format, low res 8-bit samples (which sound awesome.) How does Unity convert them to an Audio Clip without re-implementing a tracker?

    I guess you could load the samples from the file as individual audio clips and parse the file for the performance details. You'd have to pitch them and whatever other features are used in the format. Hmm.

    It sounds like a fun project but I don't really want to rabbit hole down here and kill my actual project.
     
  28. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    609
    If i recall correctly they use fmod- i have only played with this a while ago for fun (visualization only)

    Unity only exposes very limited parts of the api. But there are some open source c# mod players around and I think there is also something on github for fmod/unity.

    I forgot all the details, but there is enough out there to get something going relatively quickly - not sure if it’s worth the effort. If I hd more time, I would do it just for the fun of the old days...but I don’t ;(
     
  29. Undertaker-Infinity

    Undertaker-Infinity

    Joined:
    May 2, 2014
    Posts:
    54
    I've been tempted to rabbit hole into this myself many times. My guess is everyone is in the fence too.
    Proper mod support, with looping, jumping to patterns and toggling channels would go a long way towards a proper music system. Old games used it and we've sort of regressed in dynamism.
    Also, think of the space savings. Most games use up their space mostly in audio!
    If we could have a substance-workalike for audio samples, coupled with modules, we'd regain so much space!

    Edit: just an API for basic module control is the first step though!
    -Enable/disable loop (how did mod loops work? I don't even remember)
    -Jump to pattern
    -Toggle channel
     
    HeyItsLollie and sngdan like this.
  30. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    38
    I mentioned it in a longer post above, but porting Libxmp / Libxmp-lite to Unity seems like a genuinely feasible option. Libxmp-lite in particular, as it is specifically developed to be lean and lightweight for game/app integration.

    Literally the only reason why I haven't attempted to tackle it myself, is because my strengths are in art and music, not programming. Porting feels a bit too far above my C# knowledge and skill-level.