Preparing to Capture The D Through Sound

Chantine Akiyama, MIT Media Lab

Hello friends, it’s been a while since our last correspondence, hasn’t it? In some ways, the Media Lab winds down a bit over the summer with many of the UROPs (undergrad researchers) abroad or elsewhere for internships. However, in other ways it really picks up as certain projects come to fruition. Recently, Tod and Co. have been quite busy with final preparations for another city symphony, A Symphony for Lucerne, to be debuted September 5th, even as we ramp up for Symphony in D, coming on November 20th.

It’s beautifully hectic, this city symphony making process that is. As such, it is quite fun, so I would like to give you a sneak peak into some of our experiences here at the Opera of the Future producing a Collaborative City Symphony like Symphony in D – coming from the perspective of one who assists and observes the collaborative process.

Phase one is listening. Listening to the city, its voice, its heartbeat.

One goal of this project is to get as much of a grasp on the city as possible through music, so Tod (and our team) spends many months, even years, getting to know the social and sonic fabric of a city like Detroit. What makes Detroit tick, what makes her sing, dance, “jit”? Is she understood? Misrepresented? Changing, and if so, into what? For a more detailed look at the technicalities of listening, Media Lab students Bryn Bliska and Charles Holbrow have shared their love and expertise for the act of listening once before on the Lucerne project blog.

Heavily overlapped in timing with phase one, phase two is recording.

The sounds of The D need to be preserved at least long enough for Tod to incorporate them into the final symphony, so the team tries to capture everything it hears. This phase is also when the hands-on collaboration begins between Tod, the Media Lab, and the citizens of Detroit. The research assistant team and Tod bring their recording equipment on listening trips to the city and capture teamwork moments with organizations like American HouseYouthVille, and ADULT. In addition to such collaborations, we also collect the sounds submitted by people from Detroit via the web and mobile apps for the next phase which is…

Sound editing.

Tod has trained a small army of graduate and undergraduate researchers to assist in pulling out the most special, provocative moments from the recordings, which can range from mere seconds to hours at times. We each do this in different ways, so I’ll share Akito’s process in detail as one example –

“First, I like to create  “listening machines” – programs that can listen to and automatically cut interesting segments from sound files. These programs analyze audio features such as pitch and loudness, determine onsets of sound events, and identify the unique timbres throughout the sound file. Some examples of tools I use for this are Sonic Annotator and SuperCollider.

Next is batch processing. Once the sounds are cut up, I add fade in/out effects and normalize sound files using SoX and bash scripts to automate this process.

Finally, I always listen to the samples myself. The ‘listening machines’ I make are not perfect and listening with my ears allows me to make final editorial decisions about each sound sample. If the sample is not interesting, I edit the sample further using Amadeus Pro, which has a built in sound file batch conversion that has yet to let me down.

I keep an archive of sound files as I edit them so I can return to them anytime to repeat these three steps and produce a variety of interesting samples.”

The final phase is composition time!

All the cut up samples and recordings are collected, along with the compositions submitted by the public via applications such as Constellation and Hyperscore, and compiled for Tod to listen to and do his thing. He’ll incorporate characteristic compositions and recordings directly into the piece, or compose instrumental music inspired by the sounds, or do something new altogether! After many creative storms and synergies, the symphony will be debuted for all to enjoy.

Stay tuned for more updates on symphony creation as Symphony in D begins to take shape!


Leave a Reply

Your email address will not be published. Required fields are marked *