State of the Art: Part 1

State of the Art: Prior Works Research
Part 1

Thesis key words: Physical data visualization, data installation, data materiality, participatory data visualization

(1) Glue Society – “BT – Longterm Investor”

A series of (TV/digital media) spots using light sculpture to present estimates of investment data.

BT Financial ‘Superannuation’ from The Glue Society on Vimeo.

Materiality: Light

Architecture: digital, sloping, flat

Interaction: None, but animated

Pros: Metaphor equating light with idea and positivity and the future intact; narration clear; mood and graphics align with intended audience

Cons: Numbers not present until the end & their scale is small; light field is essentially flat, no conceivable reason for sloping plane; low resolution data

(2) Bryan Ku – “MB15 Minos”

An interactive installation for Moving Brands that visualized staff members as codified three-dimension, brightly patterned geometric solids based on office location, department, and other facts about the employees.

Materiality: none, digital

Architecture: Operating podium, projection

Interaction: Leap System, hand movements as a signal

Pros: Design of application provided approachability to party-goers; codified system of making able to be discovered (some hints found on side of operating podium); metaphors for socialization strong; integrated live stream of party goers tweets and instagrams

Cons: Lacks materiality; spatial presence brought about by utility; installation competes with experience of party

(3) Bryan Ku – “WIM•BLE•DON”

Flipbook data visualization that operates with a pair of users alternating page turns for the final game of a Wimbledon championship match.

Materiality: Paper, bound book

Architecture: none, mostly flat

Interaction: user-operated, chronological animation

Pros: metaphor in interaction between opponents; sleek visual design; excellent source of storytelling; user-operated creates controlled experience

Cons: unsure how unguided operation would begin; lack of relationship to body or space; experience heightened greatly by video track; assumes knowledge of rules of tennis to communicate story

(4) Doug McCune – “San Francisco Housing Prices

A 3D-printed data sculpture that abstractlyd displays average price per square foot for housing in the San Francisco area.


Physicality: 3D-printed plastic

Architecture: non, ~12″ tall

Interaction: None, static

Pros: Form takes on powerful metaphor of ripping apart; content well-researched and clearly discerned from sculpture; excellent craftsmanship; process well-documented

Cons: No sense of data scale; lack of relation to human body or architecture


Perspective Mapping: South Station

Tasked with returning to South Station, the following observations were made in light of an essay by urban media designer, Martijn de Waal (of The Mobile City) entitled The City as Interface: How New Media Are Changing the City.

In the essay, de Waal specifies some public space as part of the “urban public sphere,” namely any accessible place where people of various background can potentially meet. In certain aspects, successful public spaces are designed around the identities that weave though the space on a daily basis. A modern urban public are the inhabitants of this public sphere which share a common goal or action (such as transportation in a train station). American sociologist Lyn Lofland introduced a third classification of space (besides public and private): that of the “parochial” domain. A parochial sphere consists of a common group of people that share a sense of commonality despite having a publicly accessible location. Examples include a Turkish hooka bar in a Dutch neighborhood, a gay bar, a bench in a public park where teenagers commonly gather. The ubiquitous nature of personal cellular-run interfaces (phones, laptops, tablets) permits an interlocked analysis of public, parochial and private spaces. These observations try to take in consideration of these three classifications.

Patrons were observed in different zones of the South Station atrium/food court. For the course of three minutes, actions despite walking, were tallied. Included were (phone activity, talking, looking up at ads and way-finding, using the restroom, eating, etc). Actions were then classified into adding to the public domain or participating in a parochial/private domain (since often times it’s difficult to classify which one an individual is participating in).


These are the tallies/raw data from my 3 minute observations about the Atrium:


The concentric circle diagrams are the first steps in my algorithm to determine a network diagram at the perspective of an individual in South Station. My assumption: A patron of South Station will be drawn to areas in which other people are behaving similarly (headphones on, or looking for way-finding, or eating, etc.). My network diagram takes into account which zones are physically accessible to each other (clear pathways) and how much sonic or visual activity is perceivable from his/her vantage point.

The top image is the network diagram with the centers of each zone at their physical distances from each other. The circular chart on the left describes the journey of one individual through South Station in terms of talking on a mobile, looking up at way-finding, and having headphones on. The blue wedge and outlined zone #7 shows where in time and space (respetively) the individual’s perspective is currently. The size of the node represents how many people inhabit that zone on average. The saturation of the node represents how many people are conducting similar actions as the current user’s perspective. The length of the connection represents the potential of the user perceiving ambient information (sonic and visual) in his current perspective.


In this way, space is not a simple function of x-, y- and z-displacement; it is a dynamic system of goal-seeking, resource-exhausting, information-filtering agents which happen to be navigating four dimensions. And when these motivations act on a subject, it is not accurate to plot navigation of a public space in two dimensions.

Mapping Strategies: Final Project Proposal


Robert Shattuck, on Proust:

“Relativity tells us that no object by itself has either definable or measurable velocity. Two objects are required to yield a relative reading, and there is no universal grid like the ether to give an absolute figure. An object can be described as located somewhere and in a certain motion only in reference to what is about it. And so it is also with memories and experiences.

to see is to see against or beside something else

everything is perceived in clusters, each framed by another

proposal3 3


  • performance piece by Ray Lee initially performed in 2007
  • installation of large sound sculptures
    • metal tripods
    • rotating arms that emit electronic drones
  • “create a hypnotic composition”
    • audio visual spectacle / live experience
    • Wherever you stand in the space it sounds different.
    • As the arms rotate, the sound pulses past the listener with a Doppler-like effect
    • minimalist phasing of the rhythmic pulses
    • a constantly evolving polyrhythmic structure.
    • series of overtones that evoke the sense of an ethereal choir


proposal3 4


by FM3 in 2005

  • generative music maker
    • layering loops of ambient sound
    • named: popular Chinese device that intones repeating loops of Buddhist chanting
  • size of a pack of cigarettes
    • >3 editions exist now, with different loop settings
  • described as “poetic noise,” & “confrontationally tranquil,” & “engagingly intimate.”
  • “… a transistor radio sized device that plays endlessly changing sounds, chosen by the program, from a given set of notes and sounds. There is, as one would expect, no arc to these compositions—no beginning, middle and/or end. They are merely states of being, not substitutes for narrative. These indeterminate scores can be viewed a bit like the literature that emerges out of oral traditions—the great epics and sagas.” – David Byrne


proposal3 5


  • Futurist manifesto published in 1913
  • Sound is defined as the result of a succession of regular and periodic vibrations.
  • Noise is instead caused by motions that are irregular, as much in time as in intensity.
  • “Every manifestation of life is accompanied by noise. Noise is therefore familiar to our ears and has the power of immediately reminding us of life itself.” – Luigi Russolo


“meditative memory map”

  • Record dynamic noises (as defined by Russolo)
  • Focus on how things are framed in relation to each other
    • Inter-aural Level Differences
    • Inter-aural Intensity Differences
    • Start/Stop
  • Generative music dictated by patterns
    • transform sonic clutter into a meditative map
    • the tracks of mediative music buffer the clamorous sonic nature of live perceiving. A memory is softened and fluid compared the urban cacophony of transit systems.


How does sound and memory relate?
How does generative music relate to memory?
How could memory rehearsal and recall be translated into algorithms and rules for music?


While an innovative idea, generative programs are still very demanding of programmers and musicians. A specific instance of such a piece of music was then proposed.

Acoustic Scene Analysis was the subject of Memory Tests as shows the diagram below:


A proposed interface that links waveform, actual audio, temporal-shifted audio, and spatial location:g.png


A final sketch of the score of temporal-shifted audio:

odonnel_Acoustic_Memory 14.png

<p><a href=”″>SUBJECT 6 – 3D 2HR 32M AGO</a> from <a href=”″>Patrick J. O'Donnel</a> on <a href=””>Vimeo</a&gt;.</p>

Sonic Commons Project: Sound and Memory

For the Sonic Commons project, students should analyze a sonic phenomena from observations and experiences in the space at Ruggles Station.

“What does it mean to explore a phenomenon? An explanation is never the phenomenon itself, but only a refracted image of it, like looking at a scene through a prism.” — Barry Blesser and Linda-Ruth Salter, in Spaces speak, are you listening? : experiencing aural architecture

Ruggles Station – aerial view from the south

While visiting the site, we passed an escalator that connected the domed tunnel passageway and the below-ground bus bay (seen on the right side of the station in the picture above). This escalator, operating normally, emitted a distinct tone from some part of the motor, I’m assuming. While not a pure tone, it was an observable standing wave amongst the clamor of socialization and transit. After experiencing the area outside the station and coming back to the same place, I could hear the escalator’s unique standing wave from a greater distance and with more clarity, despite the dynamic sonic environment attempting to mask it. This lead to a fascination with a psychological understanding of sonic memory and it’s relation to architecture.

Psychology of Sonic Sensation, Perception, and Memory

Human Memory (2003), 2nd Ed.
Ian Neath & Aimée M. Surprenant

Chapter 4: Working Memory

Baddeley & Hitch (1974) Working Memory Model

Working Memory Model: A Central Executive function controls attention (voluntary processing of stimuli), learning, and retrieval functions.

  • Phonological Loop: Back and forth between Phonological Store and Articulatory Control Process.
  • Phonological Store: Short-term aural memory, said to only keep information for ~2 seconds without rehearsal
  • Articulatory Control Process: A repetition or refreshing process that rehearses/refreshes phonological store
  • Episodic Buffer: A theoretical addition to the Working Memory model; a system that may utilize multiple sensory inputs to support learning and recall.

theorized to help create hierarchy and meaningfulness in stimuli perception and memory recall

Chapter 5: Perspectives on Processing

Context-Dependent Memory: Recall of long-term memory occurs more accurately when context of rehearsal matches context of recall.

  • Context can be any environmental factor (light, volume, location) or additional stimuli.

State-/Mood-Dependent Memory: matching states (pharmological, stress) or matching moods both support accuracy in recall of long-term memory.

Chapter 12: Reconstructive Processes in Memory

Schema: an organized knowledge structure that reflects an individuals knowledge, experience and expectations about some aspect of the world. These are parts that define schemas:

  • Schemas represent experiential knowledge, and are dynamic with additional experiences.
  • Schemas can represent multiple knowledge levels, from concrete objects to complex social situations
  • Schemas can be nested and/or related, such as a schema about ice cream within a schema about commercial transactions (to buy ice cream)
  • Schemas information is very general, so they also have variables. A coffee schema temperature variable is usually hot, but it is not uncommon to experience iced coffee.

Sensation and Perception (2010), 8th ed.
E. Bruce Goldstein

Chapter 11: The Sound Stimulus

sound can have two definitions: a physical definition of sound is a pressure change in a medium; a perceptual definition of an experience when hearing stimuli

Sound as Pressure Changes

  • A movement of a speaker’s diaphragm causes rapid changes in volume. Outward motion pushes air together (higher pressure, or condensation); inward motion creates new space for air molecules to spread to (lower pressure, or rarefaction).
  • It’s the systems of pressure that move away from the speakers, not the air particles themselves

Perceiving Sound

  • A Decibel (dB) describes a physical measure of sound; Loudness describes a physiological sensation of sound. Loudness is a combination of both pressure changes and frequency.
  • Pitch (perceptual ranking of sound on a musical scale) is closely related to a wave’s frequency.
  • Timbre (or quality of sound) depends on relative strength of harmonics (various pure tones that overlap in a complex tone) as well as attack/decay of sound.
  • Aperiodic tones, those that do not repeat pressure changes (unlike musical tones), are even more complex yet.

Central Auditory Process in Brain

  • More areas of your brain are activated by aperiodic complex sounds than by pure tones, suggesting different areas of the brain aid in perception of different qualities of sound (decay rate, pitch, frequency, etc).
  • Different neuron pathways are activated depending on the perceptual usage of sound (what pathway identifying sounds is a separate neuron chain from where pathway spatially locating sound).

The Auditory Cortex is Shaped By Experience

  • Training of any kind (music, active listening, etc) can shape neuron connectivity within brain in two ways: formation of more connections, and more sensitive connections. Musicians use more of auditory cortex to perceive piano notes than non-musicians. Effects can be seen in as few as seven “learning” encounters with a stimuli of choice.

Chapter 12: Sound Localization and Auditory Scene

Auditory Localization: perceiving objects that emit sound at different spatial locations

Auditory Space: the perception of volume/space/architecture from sound information

Three dimensions of sound localization

  • azimuth: left/right sound dichotomy
  • elevation: up/down sound dichotomy
  • distance: near/far sound dichotomy

The cochlea (inner ear location of sound receptors) transmits information on pitch and timbre; localization requires location cues from sound interacting with the head and ears.

Binaural cues

  • Inter-aural Time Difference: time differs for sound to reach each ear (unless directly in front or behind listener). Helpful for low-frequency sound localization.
  • Inter-aural Level Difference: Head acts like a buffer and pressure perceived differently in each ear. Helpful for high-frequency sound localization. (See diagram)
An acoustic shadow is created by higher frequency sounds, not so much with lower frequency sounds.

Monaural cues

  • ITD and ILD still create a cone of confusion, or localization ambiguity, for the elevation dimension of sound. Monaural cues help establish elevation.
  • The shape of your pinna and ear canal (outer ear) reflect sound (and thus change frequency of stimuli) in unique ways depending on elevation.

Perceptual Organization of Sound

  • Auditory Scene Analysis: Separating perceptions of stimuli into being emitted by unique sources
  • Auditory Grouping: Peceptual organizing of sounds based on similarity or distinctiveness or dynamically of location, timbre, pitch, etc
  • Experience and memory can shape perception. If a melody is split between various octaves, those with a familiarity of the melody can still pick it out.

Sound and Architecture

Spaces speak, are you listening? : experiencing aural architecture
by Barry Blesser and Linda-Ruth Salter

1 Introduction to Aural Architecture

  • acoustical awareness is a skill everyone possesses, but often isn’t aware of it’s role in our lives
  • when we clap our hands (sonic event) a wall reflects an echo (passive acoustic object, the wall)
    • The distance to the wall determines the delay for the arrival of the echo, the area of the wall determines the intensity, and the material of the wall’s surface determines the frequency content. These physical facts relate only indirectly to perception. Our auditory cortex converts these physical attributes into perceptual cues, which we then use to synthesize an experience of the external world.
  • architecture can cue emotions, associations or social meaning
    • like dampening drapes for a reading room, or marble floors in a lobby for announcing arrival of guests
  • sensory anthropology studies how sense-usage results in meaning of perceptions across cultures and despite biology
  • aural architects focus on the space changing the physical properties of sound waves; acoustic architect focus on the way that listeners experience the space (subjective, dynamic).
    • it is for this reason that aural architects are often not people, but sociocultural forces influencing design and perception


  • Aural experiences are fleeting
  • Language for describing sound is weak and inadequate
  • Modern culture places little value on the importance of hearing/art of auditory awareness
  • Aural architecture is not generally recognized in intellectual inquiries, nor taught in academia

“ When fused together into a single concept, however, the marriage of aural architecture and auditory spatial awareness provides a way to explore our aural connection to the spaces built by humans and to those provided us by nature.

Within my research, I discovered that publications that connect sound and architecture are few and far in between. The two types of buildings that have information concerning their intersection are auditoriums for concerts and churches. Barry Blesser and Linda-Ruth Salter suggest that churches are constructed to permit sonic reflection and resonance to symbolically reference a higher power.

For the purposes of my research, I continued to treat the sonic phenomena from religious architecture as a explanation of the space, and to further study how sonic phenomena gives rise to the programming of the space. Below is a concept map connecting sound to religion.

Concept Map concerning Sound and Religion
Concept Map concerning Sound and Religion

From a variety of academic, philosophical, theological and fact-weary sources, I discovered many religions describe their higher power(s) in a sonic capacity, and some even constrict or ban its visual imagery. Similarly, science (if that is your religion) begins to have difficulty identifying the building blocks of the universe. Many phenomena are not physically observable, but it is changes from one state to another let us know things/matter exists. And after all, sound is by definition changes in pressure.

My research suggests the following be taken into account when developing a sonic understanding of architecture:

  • What makes an architectural space religious, sonically?
  • Is Ruggles Station a sonically religious space, and if so, what are the tenants of its religion?
  • Does architectural space dictate acoustic phenomena or does acoustic phenomena give rise to our perception of architectural space?

To gain a better understanding of sonic information processing within architectural spaces, my data collection methodology will begin with interviews. Subjects will be divided into two groups: (1) active stimuli experience, or those sonically near Ruggles Station and (2) reconstructed stimuli experience, or those sonically removed from Ruggles Station.

From there, the subject matter brought up by both groups will be explored with binaural microphone recordings of Ruggles Station. Discrepancies between sonic information and location of listening (or reconstructing sonic memory) will be the focus of my analyzation.

Sound Workshop: Ruggles Station

On October 8th, 2015, representatives from the Isabella Stewart Gardner Museum, the MBTA’s department of Architecture, and the MBTA’s department of System-Wide Accessibility were in conversation with Sam Auinger and Bruce Odland (O+A, in addition to students from Northeastern University’s Architecture Program, Northeaster University’s Information Design and Visualization Program, and Harvard University’s GSD.  The conversations between these groups will help O+A’s upcoming installation in the Ruggles Station as part of an exhibition sponsored by the Gardner museum and Northeastern Univeristy.

Notes from the Meeting:

The Office of System-Wide Accessibility

  • identifies issues of disability services on MBTA transit
    • trains front-line handling of situations where patrons need assistance
    • oversees maintenance of accessibility technology and tools (speakers, signs, elevators, etc)
  • publishes a quarterly report on accessibility functionality
  • works with both fixed-route and para-transit services
  • recent improvements:
    • wayfinding for the blind
    • audible countdowns for the trains arriving next
  • Information Equivalency
    • how can information many people take in visually be translated to sound and tactile sensations?
    • with non-consistent train station layouts, how can acoustic localization be upheld?
    • how can various sound information remain distinct? how can sound information be withheld in efforts to direct the flow of information release?
    • when is sound puddling effective communication in happenstance events, such as maintenance or repair of accessibility technology?
  • Accessibility needs call for consistency; they aim to clarify information with a lack of cacophony.
  • Sounds perceived as “ambient” or “noise” by most can actually be sonic information for the blind.
  • With the many competing sounds of transit, commerce, and socialization, sonic information competes with masking forces.

Department of Architecture

  • internal department of the MBTA
  • focuses on repairs and additions of current train stations as well as construction of new facilities
  • Art in train stations has always be in policy, but never has it been standardized
    • Temporary Art in train stations has recently had a policy clarified. Artists must suggest space and idea to be reviewed by architecture and accessibility departments. The cost to produce and install will be on the artist.
    • Integral Art is an enhancement to the underlying structure of the train station is partially funded by the US government. Artists/designers will work directly with architecture firms and their work will be incorporated into the construction bids. A fabrication company will then be hired to create the art and install it.
  • Policy is now in place to make sure both types of art in train stations is decided on fairly and with open procurement.
  • Art needs to consider the fundamental use of the space, how it affects accessibility of the space, and safety of materials.

Guest Speakers: Mapping and Sound

A collection of guest speakers, artists, and designers spoke with our Mapping Strategy class on their work with sound, maps and/or both.

Sven Anderson – Dublin, Ireland

Anderson is working with the city of Dublin, in effort to create a new position Urban Acoustic Planner. His efforts are best described as a “year-long public art commission for Dublin City Council, titled MAP or Manual for Acoustic Planning and Urban Sound Design”

Continuous Drift (2015)

Temple Bar’s Meeting House Square – Dublin, Ireland

The above is the site for an interactive, public sound installation (Continuous Drift). The square is populated by shops and bars and the four retractable canopies protected the open square from harsh elements. The architect originally built speakers into these canopies without any discernible use. Anderson commissioned 21 artists to compose/create sound to be controlled by the democratic nature of a singular remote accessible on the web. Users of the square can start, stop, change the volume, or change the track entirely at the press of a button on their smartphones.

“Generally we walk around the city and we have no say in what we hear, and a lot of people have different opinions about what’s noisy, what isn’t noisy, what should be there and shouldn’t be there.” – Sven Anderson, on Continuous Drift

Glass House (2015)

Smithfield Plaza + Torches – Dublin, Ireland

Pictured above is Smithfield Plaza of Dublin, Ireland, and the torches that line its perimeter. The plaza, an uncommon sight of open-space in the dense urban city of Dublin, was originally created to be an active neighborhood center, with open space to hold outdoor rock-concerts and markets. Noise complaints about various activities in the space has rendered it barren, and rarely-used.

Glass House, a sound installation, uses the torches (which no longer run because gas is expensive) as speakers to pipe ambient sound, a fragmented version of the audio from the theatre below the square, into the square. Named after composer John Cage’s analogy relating experimental music to glass architecture, the sonic environment transforms the above ground square from the already-present data from the theatre below.

Peter Cusack – London/Berlin

Favourite Sounds (1998-ongoing)

Peter Cusack, recording sounds at Chernobyl in 2007.

Cusack’s ongoing, crowd-sourced project began in his hometown of London, England by asking people not only what their favorite sound was, but also why. The sounds themselves weren’t the interesting part, it was the reasoning behind the sounds and how subjective pleasing sounds are because of differences in sonic perception and memory.

“It [Favourite Sounds project] was trying to get people talking about the way they hear everyday sounds and how they react to them, or what they think and feel about them, and how important (or not important) they are. And to a certain extent, that project has been successful in that. Because when you ask questions, then people will always talk about other things.” – Peter Cusack, for

Sarah Williams – Cambridge, MA

digitalMatatus (2014)

Mutatus (multi-passenger, fixed-route taxis) popular in Nairobi, Kenya and many other African urban centers

Sarah Williams, director of MIT’s Civic Data Design Lab, engaged with the favored transportation of Nairobi, Kenya: matatus. Matatu (or Matatus) are privately owned minibuses, often decorated vibrantly and play loud music, which offer the only wide-scale transportation needs of inner city Nairobi. Williams, in collaboration with the University of Nairobi, sent out GPS-equipped devices with students to track over 200 matatu routes in the greater area of Niarobi.

A digital map of the many matatu route in Nairobi, Kenya designed by Williams’ team

Upon creation of the digital map, city officials adopted this map as the official matatu map of Nairobi, began a large paper distribution, made it available for download, and have recognized the need for a more-organized public transit. When matatus drivers first saw the map, discussion quickly turned to the creation of new routes where gaps in the map exist. The data collected by Williams and her team has also been added to the Google Maps data base, and can proudly boast it’s inclusion as the first non-formal transit system to be searchable on Google Maps’ direction services.

Sam Auinger + Bruce Odland (O+A) – Berlin/New York

Hearing Perspective

O+A’s work has a foundation in discovering a hearing perspective, much like visual perspective, for a culture which has let its sonic awareness wither. With active listening and sonic thinking, this artist team creates large scale, public sound installations as well as gives lectures and workshops worldwide. With the rapid rise of technology/architecture and lack of sonic consideration, there often arises a disparity between what we see and what we hear. Auinger and Odland suggest “we will not understand ourselves until we understand our noise.” Our noise is our culture. Listening is also a perception of present/dynamic powers mediated by past powers, in that active stimuli are reflected and manipulated by our built environments and spaces.

Harmonic Bridge (1998-ongoing)

A sonic intervention near Mass MOCA in North Adams, MA, Harmonic Bridge connects the town and the museum despite the noise of a highway overpass. Resonating tubes placed under strategic parts of the highway bridge take data from the passing traffic and creates ambient chords that color the space that was once frequently unused.