Aleatoric Music

[Section8]

So first of all, what the fuck is "Aleatoric Music"? There are a few different meanings dpending on who you talk to, but basically it's "music with an element of chance".

And what the fuck does that mean? Basically that instead of an absolute and incontrovertible composition, there are random elements. I've seen a few variations on what people consider to be aleatoric music.

One way is to mimic something in the world and plot it on a score without consideration for musical context. An example I saw was a guy who plotted notes so they resembled spots of chewing gum on a footpath. Basically anything that has a fairly linear pattern could be "adapted" to music in this way.

Another is improvisation. Just playing whatever comes into your head. Obviously you're likely to get a fairly musical result out of a talented soloist, and something altogether unpleasant from someone who can't play an instrument.

A third way is to use various mechanical/electronic means, like twiddling knobs on an analogue synth. The classic example would be Pete Townshend's grand plan for performing his Lifehouse Project - taking quantifiable characteristics from an audience member, using those values to set parameters on a synthesizer and seeing what plays. It never really came to fruition, but he did collaborate to create an internet version of the concept.

And that's the one I'm really thinking about as I write this, because potentially, it has applications in gaming to create bizarre and unique soundscapes. There's even potential for using synthesised audio to convey gameplay data, and minimising the visual requirements of a user interface.

So I have a few ideas of my own on tying various synths and patches to the context of the game, but I'm wondering if anyone has seen it done at all before, and especially interested if someone has seen it done to good effect. I'm also curious to see if anyone believes there's any advantage to this system over a professional human composer.

I think there's vast potential. As long as you stick pretty close to the basic "rules" of music, you can avoid anything too cacophonous. And in general, my favourite game soundtracks are generally the very ambient ones that tend toward musical minimalism, instead favouring the texture of instrumentation and dominant rhythm - Silent Hill, Quake, Thief 3, Fallout - and I feel pretty confident you could get some decent results, and definitely something creepy to have in the background of a horror themed game. I'm just not sure about mood.

"Instrumentation" and rhythm are easy, so it would be pretty simple to establish a set of abrasive sounds and fast tempos for "adrenaline pumping" moments, and some nice slow, warm pads for quieter times, and it's easy enough to switch between major and minor keys for a broad "happy/sad" division, but the nuances that separate an evocative piece from an emotional indifferent piece aren't quite so simple to pin down.

So what do people think? Anyone have examples of music derived from game context? Anyone have good examples? How much free reign should a procedural generator be given? Is the idea preposterous? Should I go and listen to Deionarra's theme to scrub such impure thoughts from my brain?

A (multiheaded) penny for your thoughts.

 

[DarkUnderlord]

You mean context like how in Oblivion, when you get attacked, the music turns from "hey, you're out exploring" music into "hey, you're being attacked" music? Or like when you kill the thing that was attacking you, the music changes back because contextually, you aren't killing anything anymore and so like, the context has changed so the music does too.

Yup, I saw Oblivion do that.

 

[Müg]

So what do people think? Anyone have examples of music derived from game context? Anyone have good examples?

Meteos for the DS, actually, has the best done example of this ever I think. Specifically the Yort? Yurg? Whatever the cloud planet with a Y is. It's based off of classical music so the context fits even better than the rest of the levels.

 

[Section8]

You mean context like how in Oblivion, when you get attacked, the music turns from "hey, you're out exploring" music into "hey, you're being attacked" music? Or like when you kill the thing that was attacking you, the music changes back because contextually, you aren't killing anything anymore and so like, the context has changed so the music does too.

Not quite. Let me try to explain it a bit further. Imagine the tempo of the music is dictated by let's say... average DPS output from the player. There are two zombies and two pirates fighting against the player. The zombies each are represented by a distorted tuba sound, the pirates by accordions. Each monster has a random series of notes forming a "melody" and the pitch is determined by proximity to the player, the tempo determined by how much health the critter has lost. It all mashes together and gives you a "composition" that only really exists for the very specific circumstances at the present time.

As the circumstances change, so does the music. Kill a monster and there's less accompaniment. Do more damage and it speeds up. Let yourself get surrounded and you've suddenly got high-pitched wailing and intensity. At its peak, the music is probably sheer chaos, but as the fight dies down, the music morphs gradually into something more sedate.

That's what I mean by music driven by context.

 

[MF]

Not gonna work. We perceive coherent pitch-shifts in harmonic pieces outside of the key as modulations. If you have distorted tubas and accordions playing different notes, you will have a certain harmony. If one of the instruments shifts their pitch, it will sound dissonant and awkward. The only way to fix this is to make a discrete distance-to-pitch relation and use fifths, major/minor thirds and octaves for set distances, with the occasional dissonant thrown in by miscellaneous events. That would, however, means that you have dictated the root of the harmony. If you have done so, you have some control over how the music will be composed, albeit very little. Using this scheme, any free-form change in pitch other than a direct part of the chord triad will either have to be in the harmonic key of the root, or the listener will perceive a modulation. Considering modulations are either cleverly hidden behind chromatics (Chopin) or very dramatic in nature (Beethoven - or Johnny Cash, if you will) , the listener will be overloaded with 'mood-swings' and changes of pace to the point were you can no longer discern any coherent harmony, which results in cacophony.

You could do something like this, but in order to make the result remotely interesting or even bearable, you'd have to set up a lot of restrictions. Entirely context driven music will not be music if you don't stick to an equal frequency division over western harmony, unless you use single line melodies everywhere (like Fallout). Unfortunately, that would make the music very dull -Morgan put a lot of thought in the spacing of those melodies-, especially if you're going the Zombies-Tuba, Pirates-Accordion route.

 

[Kingston]

What MF said. It would most likely turn into an incoherent mess.

 

[Section8]

I think I have a habit of habitually under-explaining things, which is exactly why I make posts like these.

I'm reasonably music savvy. It's been a long time since I've brushed up on my theory, but I understand the need to tie pitch shifts to a global root for chords, and equivalent key for "melodies". And I think that's simple enough. As long as the procedure knows the rules of harmony, it can't really err in a technical sense.

As for my specific zombies/pirates example, that was just a fairly facetious way to illustrate what I meant by context. I probably shouldn't have even mentioned pitch, since the majority of dynamics would be changes that don't require as much control - ie timbre changes, such as drive, flange, chorus, delay and so forth. And obviously rhythm is another safe option, and the pitch of your percussion samples doesn't matter quite so much.

And to backtrack across the accordions and tubas bit - I wouldn't really want either going anywhere near the soundscape I want to create. Lots of pads and drones, dancing sine waves and whatever strange percussion I can dredge up to make it all seem more organic.

 

[toroid]

There's a console game called Rez that does this, but isn't quite as dynamic as what you describe. It plays a basic musical background and your actions contribute to the "music".

It's not easy finding examples of a skilled player who can create an attractive tune, but here's something, anyway.

http://www.youtube.com/watch?v=SUa9SZ1760k
http://www.youtube.com/watch?v=Ip7wpGkAQgE

 

[kingcomrade]

Well, I know that in Phantasy Star 3 the music that plays on the overworld theme depends on how many players you have in your party. More players = more parts to the thing, where when you are alone it just plays the melody, when you have everyone there are like 6 parts and they play the whole thing. Not realtime dynamic :S

I would not recommend finding the chiptunes for it, Phantasy Star 3 had possibly the worst game soundtrack of all time.

 

[Twinfalls]

Section8, do you recall 'Moondust' on the c64? I can't for the life of me remember how to play it, but you'll quickly appreciate the soundtrack. It's of the type you describe, with a very atmospheric result. You need a C64 emulator if you don't have one yet. The best I've found by far is Ccs64

Here's the rom for Moondust.

Its author, Jaron Lanier, went on to make something of a name for himself in synthesized music. He may be worth contacting for a discussion on the techniques he used.

 

[Section8]

Well, I know that in Phantasy Star 3 the music that plays on the overworld theme depends on how many players you have in your party. More players = more parts to the thing, where when you are alone it just plays the melody, when you have everyone there are like 6 parts and they play the whole thing. Not realtime dynamic :S

That can be a pretty cool effect in itself. Super Mario World had a simpler version again, if you were riding a Yoshi, it would add a bongo drum accompaniment to the theme for whatever level you were on. I'd like to see it taken further.

Section8, do you recall 'Moondust' on the c64? I can't for the life of me remember how to play it, but you'll quickly appreciate the soundtrack. It's of the type you describe, with a very atmospheric result.

I never actually played it but cheers, that's exactly the sort of thing I was looking for. That still manages to sound pretty cool just pumping out the little thin trebly sounds of a C64 chip. If that were tied into a fatter polyphonic synth it would be phenomenal.

Seems music isn't the only thing on Jaron Lanier's resume. Interesting fella, might have to read more about his work.

And I might have to track down a copy of Rez too. I think there's a Dreamcast around here somewhere.

 

[Astromarine]

other games that might possibly work as inspirations are Lumines and SSX Tricky, where the music becomes more sedate or more dynamic depending on how well you're doing.

 

[Twinfalls]

Aaaaand.... its inverse.

 

[Section8]

Aaaaand.... its inverse.

That's really fucking cool.

Let's Play Space Oddity, or Mr Blue Sky.

It doesn't look life-changing or anything, but it looks like a fun distraction and it's a whole lot less retarded than Paul Steed's shitty dancing stripper visualisations.

[edit] I'm also glad one of the beta testers rickrolled the game to save me the trouble.

 

[Kingston]

Its in open beta. I gave it a shot, great fun. Unfortunately it soon put a "purchase to continue playing" D;

 

[galsiah]

Not gonna work... no longer discern any coherent harmony, which results in cacophony.

Does music need to be about "coherent harmony"? Is that the only type of pattern which can be meaningful to a listener? Is "cacophony" as in "not harmony" necessarily a bad thing? I'm inclined to think that's a load of small-minded bollocks.
Obviously you can keep things on the harmonic straight-and-narrow by tethering things to a key (or a few over time), and following "the rules", but is that the only way to go? Clearly anything which isn't thus tethered would need to be radically different from harmonic music - but why shouldn't it be?

Don't try driving a car into a lake, then telling me I can't sail. (not that I can sail, mind you - but that's because I haven't learned, not because someone drowned in a volvo)


Clearly there's every possibility that almost entirely free-form music (or sound if you'd rather) would turn into a horrible mess. However, that's not the only possibility. Dismissing it out of hand because it doesn't work like Chopin or Beethoven seems absurd. [and I'm aware that it certainly wouldn't sound clean, since you'd have a load of unusual frequency ratios floating around - but does that need to be a bad thing?]

Entirely context driven music will not be music if you don't stick to an equal frequency division over western harmony...

Whyever not? Bending a guitar string doesn't all of a sudden turn things into non-music - even where it's bent for the entire note, and not to any equal frequency notch.

Music is about forming patterns with sound. Harmony is just one way to do that. "Modulations" are only a good way to look at things when the music is in a clearly defined key for any length of time. Music that's constantly modulating isn't sensibly viewed through the lens of modulation - just as a car that's swerving all over the road isn't thought to be repeatedly changing lanes: the lanes are simply irrelevant to understanding the car's movements. That's not to say there aren't interesting patterns.


That said, Section8 seems to be thinking of sticking more closely to the rules, so there's no trouble with all this anyway.


I've been screwing around with procedural sound for a while now. I think it has great potential - and could ideally go well beyond layering. One of the troubles is that music is a temporal beast, naturally involving patterns that run over time. Gameplay, on the other hand, can be both temporally incoherent, and unpredictable. It's difficult to build up coherent patterns when a composition algorithm has little clue where the gameplay will go, or how fast it'll get there.
I think you could get reasonable results for a consistent mood. The trouble would be in switching between moods with some satisfying contrast - particularly where the type, timing and degree of mood switch is dictated by player action. Of course that's largely the point in the first place, but what can you do.

It's probably a good idea to have a backup plan either way. Vast potential and possibility don't always translate into timely high-quality results.

 

[MF]

snip

Many people have tried. Microtonal movements, atonal compositions. To me, it sounds like shit. Zappa did some nice things with microtonal movements layered on top of a harmony creating the illusion of something happening, but I still wouldn't call that enjoyable music. It was rather tense, and it might suit some ambient purpose but only because of the underlying harmony, not despite it.

Anyway, try bending a guitar string between a quarter note and a half note. It'll sound like a wobble, not like a bend. Bend it a half note, and it's just a legato way of playing a sharp note. Bend it a full note..you get the idea. Quarter-bends are used in, for example, Black Sabbath songs, but they're used to imply descending legato from a half note.

I fully agree that there are ethnic types of music that stray a little from Western rules, but they stray in terms of relative note positions and spacing, not in terms of 'let's ignore the rules'.

 

[psycojester]

You know as i've read this thread not a single word has sunk in, i've just been fascinated by the idea of roving bands of zombies with tuba and pirates carrying accordions.

 

[TheLostOne]

You mentioned in the other thread that you're having various survival requirements: Sustenance, Social Interaction, Security, Relaxation. I would suggest adjusting the score based on that as well. When one of the aspects of survival drop too low have that affect the music, so for example if relaxation gets too low and people are on edge make the music slightly dischordant and faster paced (unless relaxation means physical rest... I may be misinterpreting). If security drops too low, the music could become more ominous. This way you're almost hearing the current mood of the room.

It sounds like a really good idea if you pull it off. I can imagine spiders with very frantic staccato piano, and Undead with a deap very slightly dischordant cello. Bosses could become completely symphonic or just something grand on a pipe organ.

I really think it needs to be prerendered to be decent though. Just following the rules of music theory isn't enough to make something worth listening to. It needs an emotional direction which only a human can give it. Keep in mind, I haven't really kept up with technology in this aspect, so maybe it's more advanced than I'm giving it credit, but I just can't see getting anything moving out of a formula driven piece of music.

Instead of formulaicly generating the harmonies when a different monster enters the fray, you could possibly start with a basic melody, and then layer it into a symphonic piece. Attach layers to certain triggers which activate in game. That way there's really no way for it not to work.

 

[boynextdoor]

snip

I fully agree that there are ethnic types of music that stray a little from Western rules, but they stray in terms of relative note positions and spacing, not in terms of 'let's ignore the rules'.

The musics other than western concert music do not just stray a little from western rules but they have completely different rules or in some cases they just have guidelines and no rules at all. Especially harmony... most musics do not use harmonic relations but they use melodic movements.

BTW my keyboard is randomly changing language so i cant use punctuation so use your imagination.

 

[MF]

snip

I fully agree that there are ethnic types of music that stray a little from Western rules, but they stray in terms of relative note positions and spacing, not in terms of 'let's ignore the rules'.

The musics other than western concert music do not just stray a little from western rules but they have completely different rules or in some cases they just have guidelines and no rules at all. Especially harmony... most musics do not use harmonic relations but they use melodic movements.

BTW my keyboard is randomly changing language so i cant use punctuation so use your imagination.

I've been thinking about this lately. Western music was melody driven as well. Gregorian chant evolved from single note melody lines like everything else. It is exactly this 'evolution' that implies that the rules are universal. If there are no harmonic rules in a given musical culture, that means that the culture hasn't evolved to a harmonic structure yet. Some African Tribal singing, for example, is like a drone over basic percussion. If more than one person chants the drone, they tend to gravitate to the same note. Unfortunately, tribal singing is almost always either a male-only endeavor. Men who can't join the drone due to their voice being too high simply don't participate. This frustrates the evolution of harmony in vocal tradition, and from the vocal tradition to the entire musical culture. Some of them evolved quite elaborate forms of polyrhythm, despite having no defined harmony at all.

This lead me to believe that it is possible to simulate the evolution of a musical tradition. If you can measure in the available voices, the macro-acoustics (lots of tribal music is based on jungle echo) and the ever-so-hard to consider social-cultural phenomena, you can create an arbitrary set of musical rules not unlike the robot-algorithms generated by evolutionary AI. I wonder if they will all eventually gravitate towards the western concert music system.

 

[galsiah]

I've been thinking about this lately.

Me too.

It is exactly this 'evolution' that implies that the rules are universal. If there are no harmonic rules in a given musical culture, that means that the culture hasn't evolved to a harmonic structure yet.

I wouldn't say that the rules are universal exactly - just that the factors upon which the rules are based are universal.

In any polyphonic context, consonance/dissonance is going to be a factor. It's inherent in the physics, and the human ear is going to pick it up. Any harmonic structure codifies a response to this consonance/dissonance (amongst other things). I don't think that response needs either to be universal or to be rigid - though responses are likely to have a lot in common, being based on the same fundamental factors.

Some African Tribal singing, for example, is like a drone over basic percussion....Men who can't join the drone due to their voice being too high simply don't participate.

This could be viewed as a very simple harmonic rule-set though - i.e. only allowing the highest possible consonance (unison). A simple response, but a response nonetheless. [and complications aren't always progress]

Some of them evolved quite elaborate forms of polyrhythm, despite having no defined harmony at all.

I still don't know that it makes sense to say that they have "no defined harmony". Only allowing unison is an extremely rigid "harmony"; allowing any combination of melodies without prejudice is an extremely relaxed "harmony". Of course neither total freedom nor total structure allows any harmonic creativity, so it's natural that these extreme responses push the creativity elsewhere.

While I agree that grey is usually more interesting than black or white, I'd hesitate to call a move to grey objective progress/evolution. Of course it's progress if you presuppose that harmonic patterns are a goal - but that's subjective.

I wonder if they will all eventually gravitate towards the western concert music system.

Even if they all developed non-trivial harmonic structures, I think the specifics would depend quite a bit on the timbres of available instruments (and it's worth noting that the sounds of voices are certainly not constant across cultures).

I'm not sure it makes sense to think that they would/should gravitate towards one centre anyway. Surely the most evolved musical state is to include many different types of music within one culture - and not necessarily to force them all together into one monolithic construct.
Of course you could force the issue in a computer simulation by defining the problem in terms of a search for one "global maximum" - but I think that'd be daft. I'd guess that a western concert system would evolve from many contexts - but alongside many other systems too.


Some questions (anyone feel free to chime in):
In general, what factors (if any) would you say determine the rules of harmonic systems apart from consonance/dissonance? If they exist, are they measurable? (if not why not?)

What factors would you say determine melodic "rules" / common patterns / ideas...?? Clearly you've got harmonically related "rules" like resolution of dissonance to consonance (or the reverse), but what other guidelines would you think are worth mentioning? Are these based on any objective underlying factors? Only symmetries/invariants/contrasts in patterns, or anything specifically acoustic??

When one of the aspects of survival drop too low have that affect the music...This way you're almost hearing the current mood of the room.

I imagine the aim would be to have almost everything affect the music (at least potentially). I'm sure stats would be a factor, as would any other significant context.

I really think it needs to be prerendered to be decent though. Just following the rules of music theory isn't enough to make something worth listening to. It needs an emotional direction which only a human can give it.

The problem here lies in the combination of pre-crafted content with procedural content.

While you're right that a human is better able to hit a particular emotional target, the problem is that he's firing blind. He can only aim for a few pre-decided broad brush-strokes targets - none of which might fit the current game context particularly well. On the other hand, the computer might not be a great marksman, but it at least knows exactly where it's aiming.

The more a game constructs its gameplay context procedurally, the higher the penalty for firing blind. No-one's going to care that a piece captures a mood precisely when it's entirely the wrong mood. One of the main reasons to use a procedural system is to be sure of getting an approximate fit to any context - even those which aren't predicted ahead of time.

Another point to consider is the feedback advantage of a fine-grained procedural approach. With pre-rendered stuff there's much less scope to communicate the specifics of a situation.
Consider also that it's not entirely necessary for such feedback to use conventional musical moods to put over information. Any game sound-effect with great gameplay significance creates its own mood once the player recognises it. There's no need for the sound to be inherently disturbing/pleasant/hopeful... for the player's association of that sound to be disturbing/pleasant/hopeful (pleasant sounds with horrible associations can even add to the horror through their "wrongness").

Similarly, when a player comes know intuitively that [musical quality X] has [gameplay significance Y], X will naturally come to have the feel of Y - whether or not there's any natural a-priori association.
Once the player has played for long enough, any consistent set of musical rules with gameplay significance will generate their own moods. That's not to say that they can't also aim for more natural "hard-wired" moods - just that there's no need to rely on them entirely.

When the music is wired directly to the gameplay, there's less need for:
Music---->Rich Emotional Associations---->Emotional Response

Since you've got:
Music---->Rich, Significant Gameplay Associations---->Emotional Response + Thoughtful Response

Naturally you'd probably be aiming for both unless you want the game to have its own, particularly weird set of rules (which might not be a bad idea if a unique feel is desired).

Instead of formulaicly generating the harmonies when a different monster enters the fray, you could possibly start with a basic melody, and then layer it into a symphonic piece. Attach layers to certain triggers which activate in game. That way there's really no way for it not to work.

That's possible, but I'd rather see things go further than simple layering. There's no reason you can't provide a set of basic themes / patterns / motifs / styles... to a procedural system, then let it adapt those based on context.
It'd make sense to have a system which allowed control to be as high/low level as desired, then mess around to see what worked best. Layering leaves you with rigid control over low level stuff, but no low level responsiveness/variety. A lower level solution would be more varied/responsive, but run the risk of sounding like shit.


Also, a small point on language: I don't think it's at all helpful to use the word "formulaic" for procedurally generated stuff, since it carries entirely the wrong associations. In a film/book something is "formulaic" when it's an off-the-peg construct which fails to adapt to the unique context within that particular film/book. In many ways this is entirely the opposite of a procedural system that generates content in-game: this adapts precisely to current context within the unique reality of that playthrough.
In many ways entirely pre-rendered music is the more "formulaic" (when used with procedural gameplay), since it follows a "We've reached a 'EmotionName' moment - cue the 'EmotionName' music" model. There's no subtlety/nuance there, since it's impossible to marry static music to a dynamic moment with subtlety/nuance.

 

[boynextdoor]

If there are no harmonic rules in a given musical culture, that means that the culture hasn't evolved to a harmonic structure yet.

I disagree because this is a very subjective topic. Some cultures 'evolved' much more complex relations of different voices very different from western harmony (like throat singing). Also I don't see why western concert music would be considered 'ultimate' or 'advanced' over other musics, since it is an almost dead genre (well maybe it's not very appropriate to call this a genre but...) and it's rules are now being followed by virtually no one except for conservatory students up to high school grade doing their harmony homeworks or film music compositors producing highly cheesy works... Even in jazz there were movements to prioritize melody over harmony, I think that alone disproves that music is evolving in a linear direction.