Joint Jurisdictions…

Immersive Audio Experience…

w/c 6th February 2012…

Following a discussion with my student adviser - during which I was admittedly keiching myself - a few ideas regarding the capture and reproduction of an immersive sonic experience began to formulate in my mind.
The "Joint Jurisdiction" is based around the fact that these ideas for my dissertation caused me to have a "light bulb moment" regarding my Creative Media Applications module. I don’t just make up random blog titles.
Further to what I may or may not have said in my last entry, research is inevitable - and fun when you get into it.

Binaural is to stereo what stereo was to (dual) mono: a format which provides more depth and space to audio material.
Taking a binaural recording system into a gig (with permission of course - Grateful Dead etc…) may well capture a far more detailed and realistic recording than stereo, but how would it actually compare to the gig experience?

I’m not suggesting that I have (or will have) developed a binaural beat that gets you into the VIP area before getting drunk, smashing an ATM, totalling the PA tower, surfing through the overflow from a Porta John and generally running amok before assaulting a bouncer, passing out and waking up in hospital.
See Woodstock 1970 & This Is Spinal Tap if no-one’s gonna tell you what I’m on about.

What I am suggesting is that there should be some methodology that enables one to not only hear what’s going on, but to also feel what’s going on. Not unlike the movie Earthquake which employed a subsonic "pseudorandom" Low Frequency Effect known as SenSurround, that while inaudible, allowed the audience to feel the earthquake - and in several instances, partially demolished the movie theater.
However, as Alan Parsons observes, discrete surround sound has not been adopted by music as widely as it has been with film. This brings me to Haptic feedback.

Hapticus Feedbacticus

I am now drawn to Haptic - or kinaesthetic - feedback, which in theory, could improve the listening experience of a recorded live music event. AOL keyword: experience.

A haptic feedback mechanism is a transducer that can be used to provide a tactile force in response to an action or event - thus providing a human with the feeling of physical as well as visual or auditory stimulation.

A popular example of haptic feedback could be playing Gran Turismo on a Sony Dual Shock controller. The vibration motors in the human interface tells the user that their vehicle is driving over rough terrain or into an obstacle.
Another example is the vibration on a cell phone - this type of haptic feedback silently alerts the user that there is an incoming SMS or call.

Binaural, meet Haptics

A binaural recording may well capture the entire auditory atmosphere and indeed play this back to the user - assuming that a pair of stereo headphones are used. At this moment in time, I have no statistical data regarding the accuracy of a binaural recording played back on a stereo system.
Spatially realistic though this reproduction may be, it probably wouldn’t make one feel as if they were standing in front of Motörhead.

If a form of haptic feedback were to be applied in this context, it could be used to help the listener feel the music. Cum On Feel The Noize.
Another consideration is of course the playback transducer - and whether or not it could reproduce the required frequencies. From my own experience, this tends not to be the case.

One alternative would be to have a dedicated LFE playback channel, which would produce the bass vibrations similar to a SenSurround system that would be felt. In a way, this would be similar to a silent disco - where the feed from the DJs mixer is fed to a crossover. The lower bass content is sent to an array of sub-woofers, while everything else is sent to an FM radio transmitter. Thus, one can hear and feel the music as per a normal disco/rave, but taking ones headphones off allows one to have a conversation in an un-raised voice.

However, applying the above method to a field recording would stipulate that a third recording channel would have to be available - which would prove potentially problematic to most tapers.
It would not be possible to capture a dedicated LFE channel without using a 4 track tape or multitrack interface for iPod/iPad or two tape recorders with some type of sync. However, an alternative would be to carry a mini mixer (such as a Peavey PV6) with which to sum the stereo-binaural with the LFE into a stereo mix - leaving the low frequencies to be played back on any system that supported it.

How then to capture this low frequency “experience” content?
Using the example of a covert recording setup, it may be possible to attach a Shure PZM to my lower body, to embrace the bass that “hits you in the chest”. Also, while cell phones are in the back of my mind, it is important to note the significance of CMOS MEMS microphones. These could be embedded in the sole of my boot, so that the bass (which travels best through a solid - i.e. the ground) can also be recorded in this experiential manner, much in the same fashion as a seismometer detects an earthquake.

Creative Media Applications

A possible idea for my CMA module could be a Haptic-Binaural Sonic Experience for boy racers. Before you hit the "back" button on your browser, let me explain…

When I was in my final year at secondary school (those were the days… or not), the local driving test center and police came to my school to promote road safety, driving exams and stuff like that. With them, they had a computer driving simulator with a steering wheel and brake pedal connected. The purpose of this device was to teach you to anticipate an obstacle on the road and to take appropriate action. There was no sound and no haptics used here.

Having been run over by a boy racer myself, I thought they could have spent a little more time covering the dangers of speed and stuff like that - i.e. so every 17 year old wouldn’t buy a Vauxhall Nova and think they were Michael Schumacher or Jez Clarkson.

What I’m proposing is: that I work with a local authority to test and deploy a Binaural and Haptic crash simulator - that allows you to experience running over a pedestrian and then getting run over yourself.
Then you too can experience being propelled face first into a windshield and then head first onto a pavement.

I would achieve this by placing a test subject in a darkened mock-up of the front end of a car. The subject would then be exposed to a binaural recording of a driving session with movie playback so that they believed that they were driving a car for real. Haptics would be used to enforce the in-journey rumble of the road and of smashing into an obstacle. It might also be an idea to add a dummy steering wheel to give the experience of an out-of-control vehicle.

Further thought required…

References/Further Reading

Michael Getz & John Dwork, The Deadhead’s Taping Compendium, Volume 1: An In-Depth Guide to the Music of the Grateful Dead on Tape, 1959–1974, Holt Paperbacks, May 15, 1998, ISBN 0-8050-5847-8

Jonathon S. Epstein, Youth Culture: Identity in a Postmodern World, Blackwell Publishing, 1998 ISBN 1-55786-851-4, p. 127

Archive: Grateful Dead Live at Richmond Coliseum 1984

Alan Parsons on surround sound and ripping audiophiles:

SenSurround Horror Stories:

Kinesthetic feedback for virtual reality:

Force Feedback Devices:

What is a CMOS MEMS Mic?

Binaural time…