Search Unity

  1. Going to #UniteBerlin? Bring a buddy & get 25% off the 2nd ticket! More info here.
    Dismiss Notice
  2. Famed game designer and creator of The Sims, SimCIty, and Spore is looking for a 3D artist to join their team. Join the challenge.
    Dismiss Notice
  3. Unity 2017.4 has arrived! Read about it here.
    Dismiss Notice
  4. ARCore is out of developer preview! Read about it here.
    Dismiss Notice
  5. Magic Leap’s Lumin SDK Technical Preview for Unity lets you get started creating content for Magic Leap One™. Find more information on our blog!
    Dismiss Notice
  6. We've opened a new World Building sub-forum to faciltate all your world building discussions! Drop in and ask any ProBuilder, ProGrids, or PolyBrush questions.
    Dismiss Notice
  7. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

Audio Any code/plugin to read & play tracker (mod/it/xm) music files?

Discussion in 'Audio & Video' started by JoeStrout, Oct 14, 2017.

  1. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    6,463
    Yes, I'm aware that Unity has built-in support for importing Tracker Modules as audio clips.

    But I'm thinking about making a rhythm game, where I need to turn the note events in the file into visual objects and game events in my game. So just turning it over to FMOD to play out the speaker isn't good enough. I need to know how many channels there are in the file, and what each of those channels is doing.

    I suppose I can pick one format (probably .xm), dig up the spec, and write my own importer. And then make my own mixer code to play it... but all that sounds like a fair amount of work, so I thought I'd ask here.

    Does anybody know of any Unity (or generic C#, I suppose) code for reading these mod files, and playing them in a way that lets me get info about the event stream?

    (And for that matter: would anybody else be interested in such a thing? Unlike the built-in mod support, this would let you do things like display level indicators for each channel, easily change the volume of individual channels, etc.)
     
    mbbmbbmm likes this.
  2. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    I'm not of any help here unfortunately, just want to say that I would be very interested in such a thing. Module trackers are really powerful and (imo) nice to use and the potential interactivity here would be awesome for games as well as experiences and art pieces.
     
    JoeStrout likes this.
  3. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    34
    A Tracker Module player with per-channel mixer access is something that I've spent years searching for — I literally found this thread searching for it again. It's such a shame that Unity's Tracker Module support fell by the wayside as quickly as it was introduced, but Unity's implementation never did enough for it to be useful to anyone. Most people seem to either make their own purpose-built implementations, or turn to [other] [solutions].

    The closest I've found to a solution for tracker modules is this exchange on Unity Answers. It isn't a plug-and-play solution, but hopefully it'll get you closer. https://answers.unity.com/questions/577402/multi-channel-audio.html
     
    Last edited: Nov 21, 2017
    mbbmbbmm likes this.
  4. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    6,463
    Well, it's certainly something I could make (I've dug into the common formats, and they're pretty simple and well-documented). I'm not certain whether there's enough demand to make it worthwhile... but maybe!

    Let's see, so far it looks like we have two people (besides me) who would be interested. I wonder how we could find more?
     
    mbbmbbmm and HeyItsLollie like this.
  5. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    There is at least a certain number of people interested in such a feature:
    https://feedback.unity3d.com/sugges...-of-tracker-module-formats-dot-xm-mod-etc-dot
    68 votes could mean anything between 6 and 60 people, right? Trackers are not mainstream, but I think there are still a few people using them and some people might even think about learning one for the advantages this could give (small file size, reactive audio etc.)
     
    JoeStrout likes this.
  6. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    6,463
    Yes. Particularly the reactive audio seems like a compelling reason to me. Of course there are other ways to accomplish that — in High Frontier, in design mode, we have a complex multi-track score for Design Mode, and we fade tracks in and out based on how complete your design is. But it's a heckuva lot of data, and still limiting in some ways — for example, there is no good way to tell when the various instruments are being played.

    I'll play around with this a bit over the holidays and see how far I get.
     
    HeyItsLollie and mbbmbbmm like this.
  7. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    511
    I also like the idea.

    Edit: sid support ;)
     
    JoeStrout likes this.
  8. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    Supercool :)
     
  9. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    511
    @JoeStrout and, did you do anything on this front?
     
  10. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    6,463
    I did... I got on open-source mod player up and running within Unity, and it has the ability to turn individual channels on and off.

    That's about all it provides, though; I was about to dig into getting feedback on the level of each channel (so you can provide level meters or whatever), when a squirrel ran by and I got distracted.
     
    HeyItsLollie likes this.
  11. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    511
    i hate those squirrels...
     
    JoeStrout likes this.
  12. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    511
    I had a little play with this and ended up with a very crude prototype of a script that syncs the original notes to the audio playback of Unity:
    • read the raw mod file
    • store the individual rows per channel of the song in memory
    • align playback time of audioclip to row position (visual output: just like it looks in a player, highlights current row)
    • allows to use instructions / effects that are not implemented by the Unity audio import (i.e. jump to specific repeat location)
    • note: this is not a mod player - it uses the native Unity audio playback --- i.e. this will not allow you to change playback at runtime, i.e. switch off a channel or instrument, etc.
    can mainly be used for audio visualization effects based on the original notes, rather than assessing samples / spectrum.
     
    HeyItsLollie likes this.
  13. Pinkuboxu

    Pinkuboxu

    Joined:
    Mar 20, 2014
    Posts:
    27
    Did everyone get burned out worrying about Tracker Music files, or is it still interesting? I hope they aren't going extinct, they are far more useful than people know it seems. I still don't know why they don't have comparable control with it to FMOD because the feature is almost useless in Unity without some API for control and as far as I can tell the almost part is only because of the size of the files. The fact that I can double click the file in unity and it opens ModPlug Tracker has been really good too. I just hope they add more of the control features that makes them so powerful.

    I've been using Tracker files for some time, before Unity was a thing even, and though I'm glad it can use them It doesn't take full advantage of the potential. In fact it barely does anything to make them more useful than a standard sound file and I'm not entirely certain it uses any less resources other than the space the asset takes up.

    I use it for music. In the past when working with my own FMOD and "tiny mod player" setup, I could control the tempo(ramp it up), volume of each channel(add the electric guitars when stuff gets exciting) and had precision timing as to what song patterns to run and when, but also can send data from a row to the playback and such. That much is super robust.

    I also use it for sound effects. It's super valuable for this, and I think it gets over looked, as you can have one file stand for a highly variant sound effect, multiple sound variations to kill monotony, and have the help of envelopes and other effects. All of that would work without the need of a fancy Audio Authoring Tool. Granted you have to take time to learn how to use a Tracker like ModPlug Tracker but it probably would take just as long to learn another Tool and even longer to code your own system.

    Being able to change more intimate things on the fly, like samples and instrument data isn't as necessary but I got that far in the past on my own. Technically you can do that to the file in memory I suppose. I sort of think they should concentrate on the native ModPlug Tracker format as well, given the new features. Then again, I imagine this stuff is all possible through an Asset if someone where to flip the ModPlug source into a .dll and give it C# hooks for use in Unity. It must not be interesting enough or valuable enough to anyone to do so?
     
    HeyItsLollie likes this.
  14. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    511
    I think @JoeStrout got something working.

    There are a few open source C# based mod players that I guess could be made to work in Unity. When I looked at those a few months back, the lack of maintenance and documentation turned me off (i.e. too much work)

    I agree though, it would be great to have better mod support. I only wrote a basic reader to allow for precise effect timing based on the tracker position, playback is through Unity native mod support (so no control)
     
  15. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    6,463
    I did make some progress, but then yeah, I got distracted by something shiny (and shifting requirements in my own projects meant I no longer had any immediate need for it).

    It's a surprisingly deep subject, with a lot of formats and possible control points to think about.

    If anybody wants what I have, PM me and I'll zip it up and send it along, with no docs or implied fitness for any purpose whatsoever. Or pester me enough, and I might be convinced to polish it up and make a proper asset out of it. :)
     
    HeyItsLollie likes this.
  16. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    I'll do both :D
     
  17. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    I agree this alone would be very nice to have! Possibility to change effects like arpeggio settings or things like that would also be super nice. It would open even more ways to procedurally tie things happening in the game to music parameters

    That sounds interesting, how do you do this? Do you have just different pitches, effects etc. using the same sample?

    I don't know but it's maybe the worry that people might be a bit scared to use a music tracker. It's not exactly a shiny new UX that the trackers provide. But I think it's a bit of a chicken-egg situation. If you could do powerful things in Unity with it, more people would probably be inclined to learn it.
     
  18. MrWald

    MrWald

    Joined:
    Dec 7, 2016
    Posts:
    2
    Hey, anyone had success with this subject? I'm currently writing a dissertation about the benefits of using tracker-based adaptive music on modern game engines and I'm planning to make a prototype for it. However, I couldn't find a way of using its advantages (change patterns, mute channels, change tempo, etc.) inside Unity.

    Playback-only isn't interesting enough for these formats so at the time when I started with this project I kinda changed directions. I searched for the Renoise format (.xrns) and discovered that it's xml-based, then I parsed the data from the xml and started playing the notes in a patch I made in PureData. Once this patch is done (still needs some work) it is meant to be implemented in Unity, using libpd or heavy if possible.

    Although this solution seemed interesting for me, I still want to support legacy tracker formats, such as .s3m, .xm, .mod or .it. Today I found this post and now I have some hope again! I'd love to know if there's any progress.

    Cheers!
     
    HeyItsLollie likes this.
  19. mbbmbbmm

    mbbmbbmm

    Joined:
    Dec 28, 2013
    Posts:
    55
    I played around with JoeStrout's implementation a bit and it works well. I haven't had the time to dive deeper into it yet though. But muting channels is easily possible for instance. Sounds like an interesting dissertation by the way!
     
    HeyItsLollie and JoeStrout like this.
  20. HeyItsLollie

    HeyItsLollie

    Joined:
    May 16, 2015
    Posts:
    34
    A recent episode of DF Retro turned my attention towards a mod-player library called Libxmp-lite. It supports the main four module formats (MOD, S3M, IT, XM) and intended for use with games and embedded apps. It seems to have been written in C, which makes me wonder what it'd take to port it to C# (if that'd even be necessary) and get it functioning in Unity. It feels like a relatively-sensible solution.

    I use Renoise, and I've thought about Renoise's XRNS format a bit. I was surprised to find that it's basically just a zip container - Samples can be easily unpacked and accessed, song/instrument data is XML-based as you've said. I've seen a couple cases of people using XRNS as an "event" sequencer (count 'em: one, two), where the audio is pre-rendered, but events are time-synced.

    It'd be awesome to have support for a modern tracker format like Renoise, especially if its DSP effects were included (no VST support, of course). We'd probably never see something so fully-featured unless the Renoise devs themselves got involved.
     
  21. Pinkuboxu

    Pinkuboxu

    Joined:
    Mar 20, 2014
    Posts:
    27
    That is pretty much right as well as a mixture of sound effects for dynamic ambience. I was mostly inspired by how sound effects get created for homebrew games on early consoles, like in famitracker for nes roms. You can get a ton of UI sounds from tiny samples, but you can also have a couple loops of birds/crickets chirping or traffic and crowd ones sound like it never repeats by spacing them out more. Yeah, I know you can always just do ambience programmatically, but having the tracker play back take care of it seems intuitive from an artistic point of view .
     
  22. MrWald

    MrWald

    Joined:
    Dec 7, 2016
    Posts:
    2
    Wow, I searched a lot and never found these projects... this seems to be the thing I am trying to do. Thanks for the references!
     
  23. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    511
    If you are after a parser, that is exactly what I did (I was unaware it has been done before) - I was parsing a mod file (not xml) - mod format was well documented.

    Very easy to use for visualization but very limited control (it’s not a player) it just knows the mod notes & effects at any point of time