Process & Pitfalls: Writing in InfoVis

Process and Pitfalls in Writing Information Visualization Research Papers
Tamara Munzner (2008)

Applied Reading 1

Patrick J. O’Donnel

Munzner begins her meta-research paper, or model paper, supported by her involvement as Posters and Papers Chair of the IEEE Symposium on Information Visualization, by recognizing common pitfalls witnessed in research writing for the information visualization community.

“A good way to begin a research project is consider where you want it to end” (Munzner, 2). This advice, as logical as it may sound, gives a false sense of applicability with its proverb-like brevity. It is my interpretation that Munzner wishes to espouses is one of a researcher’s awareness of a sound argument during the project’s conception. If taken too literally, one could bias any creative effort to eschew undesired form. Instead, she most likely supports her later categorizations of papers as having validation methods unique unto themselves. Breaking ground on a new research topic without these validation methods in mind could prove fruitless, despite richness of content and discovery.

Non-Exclusive Categories of Research Papers

(1) Technique Paper

The main contribution of a technique paper is a novel algorithm or implementation. The validation methods are beyond the scope of my thesis work at this time.

(2) Design Study

New visual representations in context of a problem are the contributions of Design Studies. In order to accurately justify the visual encodings utilized, one must include brief and relevant contextual history of the problem as well as any requirements obtained through task analysis, so that the appropriateness of the solution can be appraised. Furthermore, a researcher can also conduct and include case studies, scenarios of use, or evidence of adoption by a target audience to help support their solution’s approach. This style of paper is well within my personal technical abilities and theoretical scope for my thesis.

(3) Systems Paper

A Systems Paper evaluates the use of infrastructure, framework or toolkits in software or applications. These types of papers consider choices in structure rather than visual encodings. These types of papers are not within the scope of my abilities to author with my current thesis.

(4) Evaluation Paper

Information Visualization systems and techniques are examined in use by some target population in any Evaluation Paper. Both laboratory studies of abstracted tests, and real-world behavioral field studies fall under the umbrella of this category of research paper. The lines between Evaluation Papers, Design Studies, and Ethnography can be blurry and often co-exist. This style of research is within my capabilities, but does not exclusively match the creation-of-works approach of my thesis.

(5) Model (Meta-Research) Paper

A Model Paper is considered a Meta-Research Paper because it presents formalisms and abstractions about the nature of work, production, and process. Taxonomy models seek to detail the space of some topic (such as categorization of other works). A Formalism model provides new terminology and methods by which to analyze past (and future) works. Commentary models craft an argument for a position relating to the field, much like an opinion column or advice but supported by observation, reflection and prediction. Some parts of my thesis will likely lean towards a Formalism Model paper, as it will detail my conceptual model for working with material, space, and interaction simultaneously.

Pitfalls in writing research papers come in many forms during all stages of researching and writing. Munzner suggests that many researchers fail to connect their contributions to either technique (algorithmic) or design. In a well-drafted design study paper, a well-versed information visualization professional must know how to “clearly state the problem” that can be addressed through visualization techniques, know those very techniques, and justify the technique used against other techniques in existence. When writing a paper that exists in more than one of these categories, understand which category is guiding your writing structure most, and which categories are secondary—and how to properly embed them not to distract from the primary purpose.

Justifying visual encoding and interaction methods is a necessary consideration for design study papers; do not skip discussing task analysis. Similarly, any kind of technique proposed that does not discuss who or when it might be used is hardly useful. Specificity of use case is not a requirement, but at least abstractions of tasks in domains is advised to be included in research documentation.

Visualizations in three-dimensions are often necessary when the mental model of the content must be mapped less abstractly to afford quick understanding.  When working with 3D spatial data, consider occlusion and interactivity that permits navigation of perspective. But do not assume this to be solved, as human memory is limited to make judgments from a current viewpoint to a previous viewpoint.

Research papers should not read like a manual or a journal entry; they are not exhaustive of your process, rather they are tailored and designed to make an argument. The scope of your research (and thus paper) should be self-contained and not so dense as to cover too many topics. A proper research paper should present the amount of material necessary to make your point and be able to be reproduced. To avoid missing details or including unnecessary details, consider using a sentence such as “My contribution is…” near the end of the Introduction and ensure your writings address that contribution thoroughly.

“What can we do that wasn’t possible before? How can we do something better than before? What do we know that was unknown or unclear before?” (Munzner, 12).

Convince the reader of your paper that your contributions are unique by detailing how your work differs from the established work of the intellectual community past and present. Do not simply cite previous work, explain in what ways does it not solve the problem you’ve identified. Consider grouping previous works into categories to systematically carry out analysis of each work and it’s limitations. No assertion should go unattributed. If a fact is presented as justification and no source is cited (such as “general knowledge” or “conventional wisdom”) consider deleting it, making a different justification, or searching for research on that topic. Research papers that fail to disclose reflection on their own weaknesses, limitations, or implications are seen as unfinished.

When comparing your results to other work, compare with the most up-to-date work possible. Choosing data sets to test with use-cases should be indicative of the data sets actual users would come across. Tasks used to epitomize results should be justified, in that actual users would come across the need for this procedure. Cherry-picking tasks that showcase your solutions strengths (or worse, hides its weaknesses) dilutes your results with bias.

Writing a research paper requires a calculated style that aims to produce understanding in the audience. It is often helpful to present solution descriptions in the order of what it is, why you chose it, and then how it satisfies the problem. Captions should be written in full-sentence, paragraph form so that a chart, diagram, or image could justifiably stand alone, and flipping through the paper would allow an overview via the images only. When comparing visual techniques to others, it is helpful to do so side-by-side, rather than relying on the capacity of human memory.

Advertisements

Behavior/Experience Readings W1

Readings Week 1
Design for Behavior & Experience

“Chapter 7 Hurried Ethnography for the Harried Ethnographer” Ethnography: A way of Seeing. Wolcott, 2008.

  • What is the minimum or maximum time needed to be spent in the field to claim ethnographic validity? The researcher’s belief that enough time will produce guaranteed outcomes is at odd with the realist’s understanding that it’s impossible to exhaust any topic of inquiry.
  • “Ethnographic accommodation” is a researcher’s limitations of time that are put upon an study that employs ethnographic means.
  • How should a researcher handle time? As a scarce resource. Plan out time and set limits to fieldwork that take into account the other parts of the project.
    • When field work is not yielding any new or novel insights, it may be time to start writing.
    • Writing while still having time set aside for field work permits an initial analysis that a researcher can use to focus the remaining observation time.
    • Begin writing as soon as you think you might not be able to get it all. Do not put off writing by waiting to be a more astute observer.
    • Once you begin to review collected data in your head, start writing. Begin with the comfortable or well-synthesized ideas, and the writing should guide the remaining time you set aside for field work.
    • Start writing if half of your total time is remaining. Writing and organizing data often takes more time than observations themselves.
    • “People do writing this fashion [fieldwork first, then writing last] but this is one reason why so many monographs are uninspired.” Rosalie Wax
  • Writing about your thoughts before fieldwork can help to focus your observations on its ultimate purpose, and to organize your final account.
  • In subfields of anthropology, rapid fieldwork has become common but should not be dismissed. Common surveys, questionnaires, targeted interventions are not rushed as much as they are repetitive and formalized. Ethnographers are then considered data sources for teams conducting anthropological appraisals.
  • Anthropological appraisals may use common interviewing techniques, or even group interviews (focus groups), where an ethnographer would partake in a less structures interview/conversation and potentially identify a key informant.

“[…] Much stands to be gained for any researcher who pauses long enough to have a look around, with the intention of putting an inquiry into some broader perspective.”

Ethnographic Reconnaissance

  • –an ethnography precursor, permitting an initial survey or examination which is followed by a more detailed inquiry.
  • A windshield survey–or a survey of a community from a drive around town–is a rapid appraisal technique that can lead to valuable observations for further examination.
  • Ethnographic reconnaissance does not carry the notion that it must be done quickly, only that time must be set aside for getting one’s bearings in a new situation.
  • Do not disregard intuitive feelings about what is going on in a project, even if you’re part of a team performing pre-selected rapid assessments. Reconnaissance and getting one’s bearings should aware remain forefront while doing fieldwork.
  • Advice for initiating fieldwork to realize maximum return on ethnographic reconnaissance:
    • ethnographic reconnaissance develops organized common sense, free from inappropriate professionalism. don’t stick to overly scientific methods for observing the nuance complexity of human activity.
    • a researcher conducting ethnographic reconnaissance should try not assume a tourist or poll-taker or professional researcher identity, rather an interested human being. All of those identities could hinder natural conversation.
    • do not feel pressured to “sample” the population in any certain way. There will always be “gate keepers” of the community and they are worth your time as well.
    • write up notes quickly thereafter a reconnaissance effort. Initial opinions, anticipated problems, observations that go against your conceived notions are often fleeting once removed from the field.
    • keep in mind you are there to learn how those in this setting make sense of their world; if they can’t answer a question, it’s not part of their world. an idealistic goal for interviewing is to get them to talk with the fewest direct questions. ask questions as they come to you.
    • know what information is already out there (community history, maps) and do not feel obligated to reproduce it, but consider its biases and affordances carefully before inclusion in your reconnaissance or ethnography.

Systematic Research

  • Systematic research doesn’t always describe the existence of a community in ethnography. Care should be taken to understand when and how to count what needs to be counted, and measure what needs to be measured.
  • Problems and questions that are defined or redefined in terms of computational capabilities shift the researcher further away from orienting traditions of ethnography.
  • Photography, videography, and systematic research should not become ethnography–they are tools, not philosophies.
  • Stepwise research should not go against timing, but instead inform it. Dedicate chunks of times to different topics over the course of the fieldwork, and observe what changes and what doesn’t.
  • long-term fieldwork by ethnographers can provide “finer distinctions between change efforts and changed results, or between change and rhetoric of change.”

 

The Interpretation of Cultures “Chapter 1 Thick Description: Toward an Interprettive Theory of Culture” C. Geertz, 1973.

“To try to find the figure in the carpet of one’s writings can be as chilling as trying to find it in one’s life; to weave, post facto, a figure in —”this is what I meant to say”—is an intense temptation.

  • New ideas take hold of the intellectual community with vigor. It is applied to many problems and many situations. The community applied is where it is applicable and desists where it cannot be extended. What is does explain is now brought into focus, and the pseudoscience falls away.

“The concept of culture I espouse […] is essentially a semiotic one. [M]an is an animal suspended in webs of significance he himself has spun, I take culture to be those webs, and the analysis of it to be therefore not an experimental science in search of law but an interpretive one in search of meaning.”

  • If you want to understand science, look at what practicing scientists do. If you want to understand anthropology, look at ethnography.
  • Ethnography is more than just establishing rapport, selecting informations, transcribing texts, etc. It is an intellectual effort called “thick description” by Gilbert Ryle.
    • “Thick Description” is both thinking and reflecting, as well as thinking of thoughts.
    • A “thin” description of events merely records actions or direct observations, but a “thick” description finds a hierarchy of meaningful structures that leads to the actions. Where was this action learned? What was the intention behind the action? What is the context of the action? What is the action codifying?
  • Ethnography is a collection of data that is really a “construction of other people’s constructions of what they and their compatriots are up to. [I]t does lead to a view of anthropological research as rather more of a observational and rather less of an interpretive activity that it really is.” Analysis is sorting out the structures of signification.

“What the ethnographer is in fact faced with […] is a multiplicity of complex conceptual structures, many of them superimposed upon or knotted into one another, which are at once strange, irregular, and inexplicit, and which he must contrive somehow first to grasp and then to render.”

  • Human behavior as symbolic action has a significance. The communication by an action’s agency is of significance. With this componential analysis of culture, “culture is composed of psychological structures by means of which individuals or groups of individuals guide their behavior.”

“The cognitive fallacy—that culture consists of ‘mental phenomena which can [should] be analyzed by formal methods similar to those of mathematics and logic’—is as destructive of an effective use of the concept as are the behaviorist and idealist fallacies to which it is a misdrawn correction.”

  • Ethnographers seek to speak to and with individuals, not for them.

“Culture is not a power, something to which social events, behaviors, institutions, or processes can be causally attributed; it is a context, something within which they can be intelligibly—that is, thickly—described.”

  • Behaviors are the vehicle for culture. Behaviors’ relationships to each other is intrinsic, but it’s their role in patterns of life that give them meaning.
  • Inspecting events his how we interpret symbols and symbol systems, not by setting them up and organizing them into our own patterns.
  • Searching for sound, overarching order in cultural proceedings discredits analysis; do not divorce interpretations of events from the events themselves.

“If anthropological interpretation is constructing a reading of what happens, then to divorce it from what happens—from what, in this time or that place, specific people say, what they do, what is done to them, from the whole vast business of the world—is to divorce it from its applications and render it vacant. A good interpretation of anything—a poem, a person, a history, a ritual, an institution, a society—takes us into the heart of that of which it is the interpretation. When it does not do that, but leads us instead somewhere else—into an admiration of its own elegance, of its author’s cleverness, or of the beauties of Euclidean order—it may have its intrinsic charms; but it is something else than what the task at hand—figuring out what all that rigamarole with the sheep is about—calls for.”

  • There is a distinction being speaking and writing. An informant speaks; speaking is the event.  An ethnographer writes (inscribes); writing is the thought of speaking… or the meaning of the speaking.
  • Cultural analysis is guessing at meanings and assessments of such guesses; it is not all-encompassing or robust, rather its nuanced and situational.
  • Ethnographic descriptions are…
    1. …interpretive.
    2. …interpretive of the flow of social discourse.
    3. …attempts to rescue the “said” of discourse from ephemerality.
    4. …microscopic by nature.
  • Any larger, broader implications or abstractions of culture are often from extended acquaintances with extremely small matters, not a wide variety.
  • “To regard [ethnographic findings] as anything more (or anything less) than [particular] distorts both them and their implications, which are far profounder than mere primitivity, for social theory.”
  • This methodological critique of the microscopic nature of ethnography is valid, yet not to be resolved by considering the events as an actual microcosm, but resolved by “realizing social actions are comments” on society, with no bounds nor guarantee where the interpretation can go.
  • Generality of cultural theory grows from the “delicacy of its distinctions, not the sweep of its abstractions.”
  • Cultural analysis does not build on other analysis; it “plunges deeper into the same things” and enriches the understanding of both analyses.
  • Cultural theory is not predictive. It guides the lens to which we can view past events, and potentially anticipate future occurrences of the event.

“In ethnography, the office of theory is to provide a vocabulary in which what symbolic action has to say about itself—that is, about the role of culture in human life—can be expressed.”

  • Cultural analysis is, by definition, incomplete. They are sustained by continued debate and discussion. Yet, the more narrow the details, the less complete they become.

“The danger that cultural analysis, in search of all-too-deep-lying turtles*, will lose touch with the hard surfaces of life-with the political, economic, stratificatory realities within which men are everywhere contained-and with the biological and physical necessities on which those surfaces rest, is an ever-present one. The only defense against it, and against, thus, turning cultural analysis into a kind of sociological aestheticism, is to train such analysis on such realities and such necessities in the first place.”

*An old Indian story goes: The world rests upon a platter that is on the back of an elephant and that elephant stands on the shell of a turtle. Each turtle, in turn, rests on another turtle.

“Ethnography” in The Encyclopedia of Human-Computer Interaction, 2nd Ed. D. Randall and M. Rouncefield.

What is Ethnography?

  • “Ethnography is a qualitative orientation to research that emphasizes the detailed observation of people in naturally occurring settings.”

Why use Ethnography?

  • “Perhaps the main virtue of ethnography is its ability to make visible the ‘real world’ sociality of a setting through detailed descriptions of the ‘workaday’ activities of social actors within specific contexts.”
  • Ethnography seeks to observe and record activies as social actions embedded in a socially organized domain.

Doing Ethnography – Relying on the Kindness of Strangers

  • The social world ethnographers plunge into is often not organized in the way researchers expect to find it.
  • Recording the richness of everyday activities is often difficult because of its commonplaceness.
  • Ethnography does not take immense amounts of training, nor is its goal to search for hard-to-find things; nor is it simply hanging around or experiencing another community. It is listening and watching to the guiding principles that structure others’ activities.

What does an Ethnographer do?

  • An ethnographer does not need to go look for data; they look for communities and actions within these contexts to observe, the data is then presented to them in that context.

“The point of fieldwork is to understand the social organization of activities within the setting.”

Collecting Data

  • “In terms of what the fieldworker collects by way of data, […] it will be dictated not by strategic methodological considerations, but by the flow of activity within the social setting.”
  • Everything an ethnographer experiences or witnesses or observes is data. There is no sense in having all the data, but nevertheless your record of it should far exceed your use of it.
  • Ethnography is “the deliberate attempt to generate more data than the investigator is aware of at the time of collection” (quoted in Dourish (n.d.: 2). Mass amounts of data collection tends to happen as a consequence of the ethnographer’s immersion in a setting. Recording things often comes naturally as a foreigner in a new country wants to document everything they don’t understand.

Perspective Mapping: South Station

Tasked with returning to South Station, the following observations were made in light of an essay by urban media designer, Martijn de Waal (of The Mobile City) entitled The City as Interface: How New Media Are Changing the City.

In the essay, de Waal specifies some public space as part of the “urban public sphere,” namely any accessible place where people of various background can potentially meet. In certain aspects, successful public spaces are designed around the identities that weave though the space on a daily basis. A modern urban public are the inhabitants of this public sphere which share a common goal or action (such as transportation in a train station). American sociologist Lyn Lofland introduced a third classification of space (besides public and private): that of the “parochial” domain. A parochial sphere consists of a common group of people that share a sense of commonality despite having a publicly accessible location. Examples include a Turkish hooka bar in a Dutch neighborhood, a gay bar, a bench in a public park where teenagers commonly gather. The ubiquitous nature of personal cellular-run interfaces (phones, laptops, tablets) permits an interlocked analysis of public, parochial and private spaces. These observations try to take in consideration of these three classifications.

Patrons were observed in different zones of the South Station atrium/food court. For the course of three minutes, actions despite walking, were tallied. Included were (phone activity, talking, looking up at ads and way-finding, using the restroom, eating, etc). Actions were then classified into adding to the public domain or participating in a parochial/private domain (since often times it’s difficult to classify which one an individual is participating in).

observatations_ss_2b

These are the tallies/raw data from my 3 minute observations about the Atrium:

observations_ss_2

The concentric circle diagrams are the first steps in my algorithm to determine a network diagram at the perspective of an individual in South Station. My assumption: A patron of South Station will be drawn to areas in which other people are behaving similarly (headphones on, or looking for way-finding, or eating, etc.). My network diagram takes into account which zones are physically accessible to each other (clear pathways) and how much sonic or visual activity is perceivable from his/her vantage point.

The top image is the network diagram with the centers of each zone at their physical distances from each other. The circular chart on the left describes the journey of one individual through South Station in terms of talking on a mobile, looking up at way-finding, and having headphones on. The blue wedge and outlined zone #7 shows where in time and space (respetively) the individual’s perspective is currently. The size of the node represents how many people inhabit that zone on average. The saturation of the node represents how many people are conducting similar actions as the current user’s perspective. The length of the connection represents the potential of the user perceiving ambient information (sonic and visual) in his current perspective.

network_south_station

In this way, space is not a simple function of x-, y- and z-displacement; it is a dynamic system of goal-seeking, resource-exhausting, information-filtering agents which happen to be navigating four dimensions. And when these motivations act on a subject, it is not accurate to plot navigation of a public space in two dimensions.

Plot and ggplot2 with R-Studio

Data from Vision Problems in the U.S. provides estimates of the prevalence of eye-disorders in the US by state in adults 40 and older.

Here is the initial outputs of my work with R-Studio statistical software:

  1. When plot() is envoked on R-Studio, a matrix of plots are established by header columns. This particular function ran very slow, so I limited the data plotted to only entries from state: “OHIO”. Most noticeably, my values for vp (vision problem), age, race and sex are all categorical.
  2. I decided to further install the ggplot2 library for some more customizable plots. The color graph plots age against rate, with color determined by vp (vision problem). Each of my categorical variables has a “total/all” section that is not separated out from the individual state, race, gender or age data.
  3. I then attempted to use the box plot feature, which did not yield any additional insights.

Further steps to clearly visualize this data will address these problems:

  • What strategies are best to see averages for “all” categorical data alongside individual categorical data (e.g. female, or white, or 55-64 yrs)?
  • How can multiple categorical variables be presented at the same time (40-50yr Hispanic male vs. 51-60yr Black female vs. etc.)? Shapes, colors, other?
  • Should the data be presented as exploratory or tailored to meet a specific idea or perspective?
  • Which of these diseases/disorders result in a prescription for eyeglasses or vision correction?