The sound theory

You open the thick wooden door and step into a pool of liquid, primordial darkness — the sort impossible to experience in a city such as Mumbai with all its light pollution. It is so deep that you can’t make out the person standing beside you, let alone the dimensions of the room or the nearest wall. The only light comes from a tiny screen displaying a few multi-coloured sine waves at the opposite end of the room, which seems so far away it might as well be in another neighbourhood.

Other than that, the only sensory stimulus is sound — lush waves of sound slowly unfurling and lapping gently against your mind. In this state of near sensory deprivation, you can almost see the patterns swirl, coalesce and dissolve within the frequency soup. As your eyes adjust to the darkness, you can also make out tiny pinpricks of red. Light sensors. Stay in the room long enough, and your very presence becomes part of the music, triggering tiny changes in the sound-generating algorithm that result in entirely new patterns and shapes in the wall of sound. Not many stay, but for those who do, the effect is like a warm, comforting sound bath. Outside of a heavy dose of LSD, this is the closest most of us will come to experiencing synaesthesia.

“It’s all about follow-up action, how a small change in initial conditions can lead to massive changes in a dynamic system,” says Akash Sharma, one half of the duo who set up the installation in Bandra’s KCA Hall as part of the Reproduce Listening Room x RBMA experimental music showcase in February. He’s trying to explain the basics of American mathematician Edward Lorenz’s pioneering work in chaos theory — the inspiration for the installation. A tech geek with three degrees in sound design and music technology, Sharma quit a lucrative job as a music producer in 2014 to start sound research lab Sound.Codes with his friend and fellow sound geek Snehal Thomas Jacob. They set up shop in the living room of their one-bedroom flat in Kurla, which is where I meet them. Two large workstations dominate the room, while cables, circuits and other technological debris take up much of the remaining space.

“What we primarily do is the archiving of heritage spaces, we capture and record the acoustic signatures of these spaces,” says Jacob. In layman’s terms, they use an audio source and a microphone to map out how sound interacts with architecture, thus capturing the physical behaviour of sound in a space. This is important because many of our heritage structures have very curious acoustic properties despite being built centuries before acoustics became a field of scientific inquiry. For example, the Hoysaleswara temple in Karnataka has predetermined positions for the mridangam player, the vocalist and the dhol player, which ensure a balanced mix of sound for the audience. It’s almost as if the architect tuned the building, much like a sound engineer tunes a club PA system. “And most of the Buddhist viharas we’ve studied are tuned to resonate with human voice — the whole space vibrates with your voice so if you’re chanting or meditating you’re in that resonance,” adds Sharma.

The two started out doing this guerilla style, experimenting with different methods and technologies, depending on the element of surprise (“nobody knows what to make of the sine sweep sound”) to capture data and leave before anyone figures out what’s happening. Now, they have permission from the Architectural Survey of India and use a microphone and omni-speaker setup they have crafted themselves. Over the past three years, they have archived acoustic signatures of over 30 structures — from Ahom dynasty-era sports pavilions in Assam to temple cave complexes in Himachal Pradesh to Victorian-era theatres in Mumbai.The collected data is used to create reverb plugins — available on their website — that a producer can drop onto their tracks to see what it would sound like in that space. “Almost all music does have a sense of space which comes from reverb,” explains Jacob. “And we didn’t have any Indian reverb units. Now we do, so you can replicate what your music would sound like in a temple or a Buddhist vihara.”

Funding for the project comes from the sale of plugins and tools they have coded as part of their explorations into sound, and by putting up installations across the country. Their website has a long list of tools, but two of the most important ones are ATOM, which translates audio into MIDI data in real time, and Motomi, which uses computer vision to translate visual data into MIDI. With ATOM, you can hum a melody and hear it instantly reproduced in the tones of a sitar, a violin or even harsh static, depending on the virtual instruments you use. With Motomi, the camera frame becomes a MIDI controller of sorts, with your movement triggering sound — generating algorithms. MIDI is pure data, after all, so you can use it to trigger lights, actuate motors or think up a host of different ways to introduce interactivity into a work.

To get an idea of what you can do with the stuff Sound.Codes cooks up, let’s take a look at two of their projects. The first — Royalty on Motomi, is a collaboration with live electronica act The Liquid Bass Project for a gig at Bandra nightclub Royalty. Using Motomi, four ceiling-mounted cameras and complex data models, the two turned the dance floor into an interactive instrument. “They wanted the audience to jam with them,” says Sharma. “We set it up so that when you danced, you generated one layer of sound that blended with the layers generated by the band.”

Then there’s the work-in-progress collaboration with a French physicist who approached them to build a microphone based on the human ear. The idea is to record and analyse sound, and use a 3D printer to transform sound into an abstract but tangible physical object. “Here the idea was to print a particular type of sound — the testimonies of people who live in a deprived state, and all the things that they don’t dare to say in general conversation,” says Sharma. “Hence the title, ‘I See What You Say’.”

The two are also plugged into a wider network of sound art and technology aficionados who share their expertise, showcase their work and engage in esoteric sound theory discussions at Sound.Codes events such as Music Tech Sundays and Sunday Evening Experimentals. A sharper business mind would hoard these ideas instead of sharing them, and bring these products to the market. Though Sharma and Jacob are aware of the commercial potential of their work, they’re far too busy having fun to worry about marketing or profits. Ask them about their revenue model, and all I get is a chuckle and the comment that “it’s very weird”, before they shift my attention to their next big project — mapping the acoustic signatures of Maharashtra’s cave temple complexes. So if you’re a music producer who wants to know how your song would sound if it was recorded in Mumbai’s Mahakali Caves, you know who to call.

COMMENT NOW