Readings Beh/Exp W6

Readings Week 6
Design for Behavior & Experience
Patrick J. O’Donnel

Verbeek, Peter-Paul. “COVER STORY Beyond interaction: a short introduction to mediation theory.” interactions 22.3 (2015): 26-31.


    1. There are three approaches (or philosophies) to considering the human-technology duality: technology as extension, technology as dialectic, technology as hybridization.
    2. Don Ihde categorizes human-technology relationships as embodiment, hermeneutic, alterity, and background relations;  Verbeek proposes the existence of a three additional categories for relationship: cyborg and (embedded) interaction context and augmentation.
    3. If designing technologies is designing the mediation of the human-technology-world triad, then designing must be understood through an ethical lens. Verbeek recommends that designers should anticipate mediations and adopt libertarian paternalism philosophies while creating. Shying away from mediations won’t benefit society; literacy and responsibility of mediations is the only path forward.


    • “Still, interaction might not always be the most helpful concept for understanding the relations between humans and products, or for understanding technological artifacts in general. Recent insights from the philosophy of technology, specifically from the approach of ‘technological mediation,’ lead us to rethink the relations between humans and things, shedding new light on the field of interaction design.”
    • “[H]umans and technologies should not be seen as two ‘poles’ between which there is an interaction; rather, they are the result of this interaction.”
    • “Designing technology is designing human beings: robots, vacuum cleaners, smart watches—any technology creates specific relations between its users and their world, resulting in specific experiences and practices.”
    • “(Some) technologies do much more than merely function—they help to shape human existence.”
    • “Seeing technologies as more than neutral opens the door to arguments like “the machine made me do it” (Joe Pitt).
    • “Cognition, they claim, cannot be limited to the human mind, but rather is extended to the material artifacts people use, such as agendas, computers, and even brain implants: They help us to think, remember, and have experiences” (Andy Clark & David Chalmers).
    • “In Ernst Kapp’s philosophical-anthropological approach to technology, for instance, technologies are seen as projections of human organs. A hammer is a projection of the fist, a saw of teeth, and the telegraph network—the high-tech of his day—of the human nervous system.”
    • “Technologies and human beings help to shape each other. Technologies are an element of human nature: They are part of us.”
    • “(Adoption of the hybridization philosophy of human and technology relations) implies that designers, in fact, do not merely design products, but human practices and experiences as well. Products do not only have functional, interactive, and aesthetic qualities, but are in fact also mediators in the lives of human beings. Designing things is designing human existence.”
    • “In embodiment relations, technologies form a unity with a human being, and this unity is directed at the world: We speak with other people through the phone, rather than speaking to the phone itself, and we look through a microscope rather than at it. […] (human – technology) —> world.”
    • Hermeneutic relations, as Ihde calls them, are relations in which human beings read how technologies represent the world, such as an MRI scan that represents brain activity or the beeping of a metal detector that represents the presence of metal. Here, technologies form a unity with the world, rather than with the human being using them. […] human —> (technology-world).”
    • “In a third type of human-technology-world relations, which Ihde calls the alterity relation, human beings interact with technologies with the world in the background of this interaction. […] human —> technology (world).”
    • “Ihde distinguishes the background relation, in which technologies are the context for human experiences and actions. The sounds of air conditioners and fridges, the warm air from heating installations, the notification sounds from cellphones during a conversation—in all of these examples, technologies are a context for human existence, rather than being experienced themselves. […] human (technology/world).”
    • “A brain implant, for instance, that is used for deep brain stimulation to treat Parkinson’s disease or psychiatric disorders, is not merely embodied; rather, it merges with the human body into a new, hybrid being. I have proposed to call this a cyborg relation: human/technology —> world.”
    • “Other technologies merge with our environment, into ‘smart environments’ with ‘ambient intelligence’ and sometimes even ‘persuasive technologies’. Here, technologies are not just a background for our existence, but rather an interactive context: They detect if people are present or not, recognize faces, give feedback on behavior. […] human <—> technology/world.”
    • “Wearable technologies such as Google Glass give yet another human technology configuration. They result in a bifurcation of the human-world relation: On the one hand, smart glasses can be embodied to give an experience of the world, while, on the other hand, they give a representation of the world in a parallel screen. This relation could be called augmentation, combining an embodiment relation and a hermeneutic relation: (human – technology) —> world + human —> (technology – world).”
    • “Steven Dorrestijn has developed a framework to categorize these contact points, using the human body as a reference [13]. He distinguishes four types of contact, corresponding to four zones around the human body: ‘to the hand,’ ‘before the eye,’ ‘behind the back,’ and ‘above the head.’”
    • “Nynke Tromp et al. have distinguished two dimensions in the influence of technologies on human beings: its visibility and its force. The impact of technologies can be located somewhere on the continuum between ‘hidden’ and ‘apparent,’ on the one hand, and between ‘weak’ and ‘strong,’ on the other [14].”
    • “Strong, apparent influences can be called coercive: turnstiles that force you to buy a ticket before entering the subway, or cars that won’t start when you don’t wear a safety belt. Weak, apparent influences are persuasive. Technologies show their influence, without being overpowering: smart energy meters that give feedback on your energy consumption or e-coaching apps that help you lose weight.
      “The hidden types of influence are often seen as a little more creepy, but in fact they are very common. Hidden, weak influences can be called seductive. Their impact is non-cognitive and mild: placing a coffee machine in the hall of a company to stimulate social interaction, using material that ages beautifully to prevent people from discarding a product prematurely [10,15]. The final type of influence is both strong and hidden; it can be called decisive or implicative because it exerts influence without this influence being noticed. An example is an apartment building without an elevator, implicitly forcing people to use the stairs.”
    • “Technological mediation is part of the human condition—we cannot be human without technologies. This makes the design of technologies a highly responsible activity. Designing technology is designing humanity, in a sense.”
    • “Therefore, along with functionality, interaction, and aesthetics, mediation deserves a central place in the conceptual framework that implicitly and explicitly guides design activities.”
    • “Explicitly influencing people via design is a contested thing to do, though. It puts something at stake that has become one of the most sacred things in contemporary Western culture: human autonomy. For that reason, for instance, Thaler and Sunstein explicitly call their approach a form of ‘libertarian paternalism.’ It is inevitably paternalistic, in the sense that it exerts influence on human beings, but at the same time it explicitly aims to be libertarian, in the sense that it always gives people the possibility to opt out. Nudges should never be given invisibly or without the possibility of avoiding them.”
    • “Human freedom cannot be saved by shying away from technological mediations, but only by developing free relations to them, dealing in a responsible way with the inevitable mediating roles of technologies in our lives.”


Dourish, Paul. “Seeking a foundation for context-aware computing.” Human–Computer Interaction 16.2-4 (2001): 229-241.


    1. Wiesel proposed a vision of ubiquitous computing, where embedded processors are cheap enough to manufacture for specific contexts; Ishii & Ullmer suggested a program of Tangible Bits, that connects the materiality of the physical world (atoms) to the materiality digital world (“bits”) by utilizing physical controls of digital information. Both configurations attempt to tie the physical world to the digital world, as well as reduce the barrier between interface and action. Schuman’s “situated action perspective” suggests that humans, as improvisational actors, determine meaning from interaction based on context—cultural, organizational, physical, and temporal.
    2. Phenomenology, the study of phenomena, equates to the study of embodiment. Husserl sought to reconnect science to the real world by explicating a ‘natural attitude’ that makes sense of contexts with meaning. His student, Heidegger, furthered the radical idea that the world was filled with meaning inherently, and as explorers, our actions reveal the meanings already in place. Another student, Schutz, connected phenomenology to intersubjectivity with a common life-world, inside of which, two subjects gain access to a background that permits actions understood as rational.
    3. Embodiment of interactions places interactions inside of meaning; Meaning comes from actions and interactions, and therefore can only be suggested and not directly designed.


      • Why has context-aware technology developed? “ One spur to the emergence of context-aware computing has been the novel technical opportunities afforded by falling costs, sizes, and power requirements for a range of computational devices and associated advances in sensor technology, which jointly allow us to develop new forms of embedded interaction, augmenting physical environments with computation that can be responsive to the needs and activities of the people that occupy them. A second is the recognition of the mutual influence of the physical environment and the human activities that unfold within it, so that aspects of the setting can be used both to disambiguate and to provide specialized computational support for likely action. A third is an increasing understanding on the part of system developers that human activities, including those that we conduct with and through computation, are enmeshed in a variety of practices and relations that make them meaningful by setting a context within which they can be understood and evaluated. A fourth is the influence of design that draws attention to the symbolic as well as the instrumental use of technologies and the roles that each conception of technology need to play in their design and deployment.”
      • “ the idea of computation embedded into the everyday environment opened up the possibility of computer technology receding into the environment and became useful to us (the population) in completely new ways.”
      • “ Ishii and Ullmer observed that we operate in two different worlds— the world of computation (‘bits’) and the world of physical reality (‘atoms’). However, although the world of physical reality is one with which we are deeply and intimately familiar and one in which we are, as organisms, evolved to operate, most interactive systems make very little use of these natural skills and abilities in supporting interaction.”
      • Tangible Bits and UbiComp are similar in a few ways. “First, they both attempt to exploit our natural familiarity with the everyday environment and our highly developed spatial and physical skills to specialize and control how computation can be used in concert with naturalistic activities. Second, they both use spatial and temporal configurations of elements and activities in the real world to disambiguate actions and so make computational responses a better fit for the actions in which users are engaged. Third, they both look for opportunities to tie computational and physical activities together in such a way that the computer ‘withdraws’ into the activity, so that users engage directly with the tasks at hand and the distinction between ‘interface’ and ‘action’ is reduced.”
      • “ Critically, this means that, for ethnomethodology, social conduct is an improvised affair, carried on in real-time in the course of everyday activity. Social conduct is orderly not because it is governed by some overarching theoretical construction but because people make it orderly. Ethnomethodologists argue that people find, within the conduct of everyday affairs, the resources by which those affairs can be found to be meaningful and rational; so in turn, they recommend that the investigation of social order should not take the form of a search for theoretical principles, but rather should involve the careful examination of specific instances of organized action so as to be able to uncover the means by which people produced  the rationality that they exhibit.”
      • “This perspective, in which the sequential organization of conduct arises in response to the immediate circumstances in which it arises, Suchman termed the situated action  perspective, and it stands in contrast to the traditional planning model in which the sequential organization of action is predetermined by an algorithmic exploration of the ‘search space’ of goals and actions. Suchman did not reject the notion of ‘plans’; instead, she observed that plans, as prespecified formulations of future action, are merely one of a number of possible resources that people draw upon in answering the question, ‘what do I do next?’”
      • “ Context— the organizational and cultural context as much as the physical context— plays a critical role in shaping action, and also in providing people with the means to interpret and understand action. Similarly, because the meaning of action is interactionally determined, temporal context is also involved, as actions and utterances gain their meaning and intelligibility from the way in which they figure as part of a larger pattern of activity.”
      • “ Beyond this, we also need to take account of social, cultural, organizational, and interactional context, which are equally telling for the ways in which action will emerge.”
      • “[…] By embodiment I mean a presence and participation in the world, real-time and real-space, here and now. Embodiment denotes a participative status, the presence and occurrence of a phenomenon in the world.”
      • “[P]henomenology, which, loosely, is the philosophy of the phenomena of experience.”
      • Edmund Husserl, the earliest writer on phenomenology, sought to “reconnect science with the real world, and the means by which this was to be done was to develop the philosophy of human experience on a rigorous scientific footing. This philosophy of the phenomena of experience was phenomenology. Phenomenology set out to explore how people experience the world— how we progress from sense-impressions of the world to understandings and meanings. Fundamentally, it put primary emphasis on the everyday experience of people living and acting in the world, and the ‘natural attitude’ toward the world that lets them easily and unnoticeably make sense of their experience.”
      • “Heidegger rejected this idea. He argued that rather than assigning meaning to the world as we perceive it, we act in a world that is already filled with meaning. The world has meaning in how it is physically organized in relation to our physical abilities and in how it reflects a history of social practice. For Heidegger, the primary question is not ‘How do we assign meaning to our perceptions of the world?’ but rather, ‘How does the meaning of the world reveal itself to us through our actions within it?’”
      • “ [M]eaning, for us (humans), arises from the ways in which we engage with and act within the world. I believe that this is of central importance in trying to understand the notion of embodied interaction that lies at the heart of the two aspects of context-based computation discussed earlier and elsewhere in this issue.”
      • Alfred Schutz “proposed an approach to intersubjectivity rooted in our common experience of the world and on the way in which we can interpret and understand the actions and motivations of others by appeal to the assumption of a shared life-world (or lebenswelt) that, first, grounds our common experience and, second, gives me the necessary background to understand your actions as being rational.”
      • “ The design concern is not simply what kinds of physical skills, say, we might be able to capitalize upon in a tangible interface, or what sorts of contextual factors we can detect and encode into a UbiComp model. Instead, we need to be able to consider how those skills or factors contribute to the meaningfulness of actions.”
      • “Most important, the designer does not have absolute control, only influence. In turn, this suggests that if the meaning of the use of the technology is, first, in flux and, second, something that is worked out again and again in each setting, then the technology needs to be able to support this sort of repurposing, and needs to be able to support the communication of meaning through it, within a community of practice.”

Project:Thesis Readings 1

Embodiment in Data Sculpture: A Model of the Physical Visualization of Information
Jack Zhao and Andrew Vande Moere (2006)

“With human’s inherent proficiency in comprehending the physical affordances present in the real world, some researchers and designers are investigating how meaningful insights can be conveyed by way of sculpting data” (Zhao & Moere, 1).

  • Data sculpture is (1) created from data, (2) exists in space or is physical, (3) possesses both artistic and functional qualities, and (4) an attempt to make obvious the insights and relevance of the data.
  • How data is best presented to inform, educate non-expert audiences, capture attention, and maintain curiosity is largely subjective and contextual.
  • Interpretation of physical objects come from their affordances, something digital media and digital space does not inherently carry.
  • Data sculpture has the potential to communicate information to a mass, lay audience through touch, exploration, and possession. This externalization of data will now have functional and artistic qualities.


“Embodiment is based on the measurement of the distance between metaphor and data and between metaphor and reality” (Zhao & Moere, 2).

  • Qualities data sculpture can take on: physical property of depth and perspective, materiality, and nuance.
  • Data sculpture belongs to design subfields of information aesthetics, artistic visualization, or casual visualization.
  • A predecessor of data sculpture, ambient displays transform architectural space by implicating interfaces for stimulating audience’s attention where none was previously warranted.

“In data sculpture, embodiment describes the expression of abstract data in physical representation through the process of data mapping. In information visualization, and by extension, in data sculptures, data mapping describes the process of translating data values to representations using metaphors. In such processes, metaphors become manifested in representations and draw associations between the abstract data and the perceiver’s prior knowledge or experiences. Metaphor is defined as a concept that is regarded as representative or symbolic to another concept. The primary function of a metaphor is to help people conceive an unfamiliar domain in terms of another familiar domain through drawing connections of similarity between the two” (Zhao & Moere, 3).

  • In the field of tangible computing, the research into the use of metaphor has been based on the theory that users naturally relate what they are experiencing to what they already know. Stronger metaphors exist when they reference a specific mental image, afford the intended interaction and have a place in a mass audiences’ realm of familiarity.


“A more precise definition of data sculpture has emerged from the domain model: a data sculpture is a highly data-oriented physical form, possessing both artistic and functional qualities, to augment facilitates an audience’s understanding of the underlying data and issues” (Zhao & Moere, 4).

“Our model relies on following three axioms:

1. Data sculpture is a system of physical representation and abstract data coupled by a relationship called embodiment.

2. Metaphor is a contributing factor to embodiment and can be gauged by metaphorical distances from the data and reality.

3. Different modes of embodiment determined by different metaphorical distances in data sculpture can affect the informative value.”

Data Stories: Data Sculpture

Data Stores Podcast: Episode 17

Data Sculpture

State of the Art: Part 1

State of the Art: Prior Works Research
Part 1

Thesis key words: Physical data visualization, data installation, data materiality, participatory data visualization

(1) Glue Society – “BT – Longterm Investor”

A series of (TV/digital media) spots using light sculpture to present estimates of investment data.

BT Financial ‘Superannuation’ from The Glue Society on Vimeo.

Materiality: Light

Architecture: digital, sloping, flat

Interaction: None, but animated

Pros: Metaphor equating light with idea and positivity and the future intact; narration clear; mood and graphics align with intended audience

Cons: Numbers not present until the end & their scale is small; light field is essentially flat, no conceivable reason for sloping plane; low resolution data

(2) Bryan Ku – “MB15 Minos”

An interactive installation for Moving Brands that visualized staff members as codified three-dimension, brightly patterned geometric solids based on office location, department, and other facts about the employees.

Materiality: none, digital

Architecture: Operating podium, projection

Interaction: Leap System, hand movements as a signal

Pros: Design of application provided approachability to party-goers; codified system of making able to be discovered (some hints found on side of operating podium); metaphors for socialization strong; integrated live stream of party goers tweets and instagrams

Cons: Lacks materiality; spatial presence brought about by utility; installation competes with experience of party

(3) Bryan Ku – “WIM•BLE•DON”

Flipbook data visualization that operates with a pair of users alternating page turns for the final game of a Wimbledon championship match.

Materiality: Paper, bound book

Architecture: none, mostly flat

Interaction: user-operated, chronological animation

Pros: metaphor in interaction between opponents; sleek visual design; excellent source of storytelling; user-operated creates controlled experience

Cons: unsure how unguided operation would begin; lack of relationship to body or space; experience heightened greatly by video track; assumes knowledge of rules of tennis to communicate story

(4) Doug McCune – “San Francisco Housing Prices

A 3D-printed data sculpture that abstractlyd displays average price per square foot for housing in the San Francisco area.


Physicality: 3D-printed plastic

Architecture: non, ~12″ tall

Interaction: None, static

Pros: Form takes on powerful metaphor of ripping apart; content well-researched and clearly discerned from sculpture; excellent craftsmanship; process well-documented

Cons: No sense of data scale; lack of relation to human body or architecture

Rules for Animated Infographics


Information Animated Infographics
Editors T. Finke & S Manger (2012)

“Well-made animated information graphics are based on clear decisions about what matters and what should be left out.” – Stefan Fichtel (in Finke & Manger, 8)

“By comparing different data sets and visualizations of the data and embedding them in a context, the view sees the result in the form of a narration with the help of illustrations” (Finke & Manger, 21).

The Pros and Cons of “passive, animated information graphics that have a linear structure:”





In sculptural, interactive works…

Time Sequences
…if time is represented, interactivity may permit atemporality.

Directing the Viewer
…the viewer cannot experience events in a predetermined order.

Changing Perspective
…emphasis should not be encoded in perspective.

Viewing Time
…the viewer is permitted to spend as much time as needed (or desired) for perception of content.

…the information presented may be perceived in various orders, and will permit different outcomes depending on the viewer.

…the statistical content must balance between rich and generalized visualizations, understanding experience can cease upon exhaust of attention.

…the viewer is permitted to ask questions or examine the data in detail.


Combinations of Visualization Techniques in Animation:

Schematic Diagrams
A way to view structure of an object, connections (of objects), or abstract processes (temporal connections of objects). Drawings of the object should be only as detailed as necessary to communicate understanding. Cross sections should be carefully cut (and visualized) so that inside and outside distinctions can be made and the form as as whole is still readily perceived. In charts of process, if the process is abstract, consider employing visuals that offer interest-piquing metaphor.

When making process graphics, it is helpful to recognize an overall image, a sequence of images, and the feature graphic. The overall image should lead the eye through the whole, often helpful to read top-left to bottom-right. Any visualization of the process must be distinct from the elements or actors, as such the elements and actors must be perceptually grouped/organized to show relationships. Feature graphics or elements must be introduced early in sequences, so that the rest of the sequence can animate the process and relationships.

Cartographic information pertaining to event space should be placed in context, with appropriate amounts of labels for bordering lands or geographic features (like bodies of water). Thematic maps provide a spatial distribution of one or more phenomena; in these cases geographic data is linked to statistical data. Codifying the map with symbols or iconography is often necessary, but in the course of an animation the legend should not be present the entire time (labels or voice-over work can suffice), so clear metaphors for visual encoding must be established. Moving images of maps need not a title, because tracking and eye back and forth from title to map (or legend to map) costs time.

“A good information graphic makes its content, or its essence, as accessible as possible in a brief period of time” (Finke & Manger, 109).