A plan hatched by members of Strangeloop, Sin Fronteras, on the back of a napkin. Much beer and meat was consumed:
In a cavernous and controversial basement nearby, Net Audio are planning their next Net Audio Festival for October. Our intention is to join forces with their festival and create an epic, possibly world first, multi-agent algorithmic improvisation orchestra.
Did you say an algorithmic improvisation orchestra?
Yes, that’s right, an algorithmic improvisation orchestra. We’re not exactly sure what it is yet (Rolf), and this page is serving as a space for collaboratively developing the idea. The main idea is that different people submit algorithmic music systems (that just means computer programs that generate music without anyone playing the music, but without the music just being pre-written already) which are required to improvise together in groups.
To facilitate the improvisation, we will build a master controller program (grandiosely referred to henceforth as an MCP, but possibly to earn a more affectionate name further down the line. I think “Deep Purple” would be appropriate), which will do various conductorly things, ranging from simple low level processes like providing a metronome, to higher level processes such as asking the various agents to be a bit more moody or gentle. In turn, the agents will be able to communicate with the MCP, and they will be able to tell it two types of things: firstly, what they are up to (if the MCP was really clever, it would be able to hear what the agents were up to. It will, to some extent, but there’s nothing like a bit of explicit clarification), and secondly, what they would like to happen next. The MCP will have to consolidate the various requests coming in and make an ultimate decision about what will happen next, or at least what it will ask the agents to do next.
We think that the MCP will provide quite a bit of other information, such as what’s going on in the room (via a motion tracking system), plus an overall analysis of the overall sound of the improvisation (things like harmonic content and noisiness), and in order to coordinate all of the what-will-happen-next events, it will have to run some kind of scheduling-messaging system which the agents can understand. Finally, the most important job that the MCP will have will be to switch on and off agents. Telling somebody to shut up is always a hard thing to do, but we don’t want the MCP to take an arbitrary approach to this, we want it to be opinionated, for example by booting out agents that are dominating the spectrum, or being dull.
What all of this means is that our immediate goal is to write a specification for all of this message data. We’re going to use the incredibly cool and not-used-enough Open Sound Control (OSC) protocol for sending these messages about. We’ll also be thinking about how the MCP will behave, since that’s integral to the messaging spec.
Once that’s done, we’ll need some agents. We’ll build some ourselves, and do a call for contributions. The agents need to be designed according to the spec, but there’s more to it than that. This is a collaborative composition and collaborators will need to work together as much musically as technically, listening, refining, working together towards the finest piece of generative music they can. We’ll need to discuss musical concepts, maybe even resulting in the modification of the spec as we go, to incorporate the best ideas.
But first up, the spec. Here’s a first draft:
- MCP_OSC_S - an OSC server which receives OSC messages from Agents. Possible OSC message:
- x, y position
- I am percussive, I am melodic, I am stockhausenic
- note data
- I am happy, I am drunk
- Something should happen soon
- MCP_OSC_C - an OSC client which sends OSC messages to Agents, e.g
- Notes recently played
- Times of upcoming events
- Clock
- IP addreses of nearby agents??
- MCP-Audio - an algorithmic audio mixer which receives 2 channels of audio from each client and mixes them to (4?) channels based on the positions of the agents. Possible mixer rules:
- Turn down consistently loud agents
- Filter spectrally noisy agents
- Power per cycle rule - agents can only output so much energy per cycle. Once energy used, agent is faded out until next cycle
- Agent - MCP Handshake. Agent connects to MCP_OSC_S and registers to receive OSC messages from MCP_OSC_C.
- Agents must also be OSC servers to receive messages from MCP_OSC_C
OSC message types (partially integrated with above):
Agent –> MCP :
/agent/statement : Statements made by agents about their state, collated by the MCP in order to broadcast general statements about the current state of the music and to make decisions.
/position : xy position of agent.
/energy : Some kind of mood tag. Could use a short vocabulary of descriptors, or just be an energy level. Could involve a statement about dynamics (i.e., energy_now, energy_target, time_to_get_there). Note regarding dynamics: perhaps we should borrow the ‘line’ format from MaxMSP so that whenever you see “a, b c” it means “going from a to b in c milliseconds. Then the MCP should be able to trigger clocks whenever it receives such messages in order to plan.
/key : Statement about key. Possibly the most general way to do this is to send 12 bits representing the inclusion of each of the twelve pitch classes.
/pattern : A melodic / rhythmic pattern expressed as a list. The MCP can forward these patterns and other agents can choose to use them or adapt them. Format should probably be loop_length, loop_offset, note1, time1, note2, time2, etc. Do we want to try to represent chords, percussive ‘notes’, non-metrical timings?
/frequencies : A description of the frequency bands occupied by the agent. Possibly formatted as a set of ranges, so that you could take the simple route of just specifying max and min, or the more complex route of pretty much defining whole timbres. Perhaps agents should only broadcast this once in a while, but then what if an agent is jumping all over the place?
/dynamics : A bit like energy, but perhaps using more standard common music notation (ppp, fff, and all that).
/activity : A measure of activity in events per second. Up to the agent to decide what an event is. e.g., individual pitch-bend adjustments may not be events, but perhaps the start and end points are.
/state : e.g., I am loading / not ready, I am ready (part of the handshake described by Matt above).
/agent/request : Requests by agents that will be logged with the MCP, pertaining to general direction of the music.
/tempo : Request for a new tempo
/dynamics : New dynamics
MCP –> Agent :
/mcp/statement : Statements broadcast about the general state of the music, mostly based on the forwarding and collating of agent/statement messages, or coming from the master clock of the MCP.
/beat : Beats will be count in ticks and will increment continually from zero to infinity. Beat will be in the form “current_beat, time_to_next_beat”, so that agents can keep track of the tempo at every beat. Does that work.
/activities : Activities of all agents.
/energies : Energies of all agents.
/frequencies : Frequencies of all agents. It would be good if agents can correlate frequencies, energies and activities, but on the other hand I imagine that agents still don’t communicate directly with each other. Instead the MCP forwards this stuff on. Any thoughts on that?
/pattern : Also forwarded by MCP from agents, but this time with a bit of discrimination / editorial judgement by MCP depending on what else is going on. Perhaps sometimes it is relevant to be forwarding patterns, sometimes not. Also, this suggests that we need a format for determining how long a pattern is relevant for? Perhaps the MCP can tag patterns, so that later the MCP can request that a particular pattern is dropped, by referring to its tag.
/mcp/instruction : More orderly instructions about what the agent should be doing. Can be to a specific agent, or to all agents (how?). Incorporates some of the above.
/key : Key should be more of an instruction than a statement. Obey! (Or not!)
/dynamics : Dynamic commands, which agents can interpret in terms of energy, activity, or just dynamics.
/pattern :
/energy :
/frequencies :
/tempo :
/meter :
/mcp/announcement : Announcements about what will happen next (always an instruction), and when. i.e., announcements are countdowns to instructions. These should also use the line format. i.e., “instruction, time_to_instruction”. Can we nest OSC messages like this?
/all_of_the_above : followed by a time tag.