Process and Pitfalls in Writing Information Visualization Research Papers
Tamara Munzner (2008)
Applied Reading 1
Patrick J. O’Donnel
Munzner begins her meta-research paper, or model paper, supported by her involvement as Posters and Papers Chair of the IEEE Symposium on Information Visualization, by recognizing common pitfalls witnessed in research writing for the information visualization community.
“A good way to begin a research project is consider where you want it to end” (Munzner, 2). This advice, as logical as it may sound, gives a false sense of applicability with its proverb-like brevity. It is my interpretation that Munzner wishes to espouses is one of a researcher’s awareness of a sound argument during the project’s conception. If taken too literally, one could bias any creative effort to eschew undesired form. Instead, she most likely supports her later categorizations of papers as having validation methods unique unto themselves. Breaking ground on a new research topic without these validation methods in mind could prove fruitless, despite richness of content and discovery.
Non-Exclusive Categories of Research Papers
(1) Technique Paper
The main contribution of a technique paper is a novel algorithm or implementation. The validation methods are beyond the scope of my thesis work at this time.
(2) Design Study
New visual representations in context of a problem are the contributions of Design Studies. In order to accurately justify the visual encodings utilized, one must include brief and relevant contextual history of the problem as well as any requirements obtained through task analysis, so that the appropriateness of the solution can be appraised. Furthermore, a researcher can also conduct and include case studies, scenarios of use, or evidence of adoption by a target audience to help support their solution’s approach. This style of paper is well within my personal technical abilities and theoretical scope for my thesis.
(3) Systems Paper
A Systems Paper evaluates the use of infrastructure, framework or toolkits in software or applications. These types of papers consider choices in structure rather than visual encodings. These types of papers are not within the scope of my abilities to author with my current thesis.
(4) Evaluation Paper
Information Visualization systems and techniques are examined in use by some target population in any Evaluation Paper. Both laboratory studies of abstracted tests, and real-world behavioral field studies fall under the umbrella of this category of research paper. The lines between Evaluation Papers, Design Studies, and Ethnography can be blurry and often co-exist. This style of research is within my capabilities, but does not exclusively match the creation-of-works approach of my thesis.
(5) Model (Meta-Research) Paper
A Model Paper is considered a Meta-Research Paper because it presents formalisms and abstractions about the nature of work, production, and process. Taxonomy models seek to detail the space of some topic (such as categorization of other works). A Formalism model provides new terminology and methods by which to analyze past (and future) works. Commentary models craft an argument for a position relating to the field, much like an opinion column or advice but supported by observation, reflection and prediction. Some parts of my thesis will likely lean towards a Formalism Model paper, as it will detail my conceptual model for working with material, space, and interaction simultaneously.
Pitfalls in writing research papers come in many forms during all stages of researching and writing. Munzner suggests that many researchers fail to connect their contributions to either technique (algorithmic) or design. In a well-drafted design study paper, a well-versed information visualization professional must know how to “clearly state the problem” that can be addressed through visualization techniques, know those very techniques, and justify the technique used against other techniques in existence. When writing a paper that exists in more than one of these categories, understand which category is guiding your writing structure most, and which categories are secondary—and how to properly embed them not to distract from the primary purpose.
Justifying visual encoding and interaction methods is a necessary consideration for design study papers; do not skip discussing task analysis. Similarly, any kind of technique proposed that does not discuss who or when it might be used is hardly useful. Specificity of use case is not a requirement, but at least abstractions of tasks in domains is advised to be included in research documentation.
Visualizations in three-dimensions are often necessary when the mental model of the content must be mapped less abstractly to afford quick understanding. When working with 3D spatial data, consider occlusion and interactivity that permits navigation of perspective. But do not assume this to be solved, as human memory is limited to make judgments from a current viewpoint to a previous viewpoint.
Research papers should not read like a manual or a journal entry; they are not exhaustive of your process, rather they are tailored and designed to make an argument. The scope of your research (and thus paper) should be self-contained and not so dense as to cover too many topics. A proper research paper should present the amount of material necessary to make your point and be able to be reproduced. To avoid missing details or including unnecessary details, consider using a sentence such as “My contribution is…” near the end of the Introduction and ensure your writings address that contribution thoroughly.
“What can we do that wasn’t possible before? How can we do something better than before? What do we know that was unknown or unclear before?” (Munzner, 12).
Convince the reader of your paper that your contributions are unique by detailing how your work differs from the established work of the intellectual community past and present. Do not simply cite previous work, explain in what ways does it not solve the problem you’ve identified. Consider grouping previous works into categories to systematically carry out analysis of each work and it’s limitations. No assertion should go unattributed. If a fact is presented as justification and no source is cited (such as “general knowledge” or “conventional wisdom”) consider deleting it, making a different justification, or searching for research on that topic. Research papers that fail to disclose reflection on their own weaknesses, limitations, or implications are seen as unfinished.
When comparing your results to other work, compare with the most up-to-date work possible. Choosing data sets to test with use-cases should be indicative of the data sets actual users would come across. Tasks used to epitomize results should be justified, in that actual users would come across the need for this procedure. Cherry-picking tasks that showcase your solutions strengths (or worse, hides its weaknesses) dilutes your results with bias.
Writing a research paper requires a calculated style that aims to produce understanding in the audience. It is often helpful to present solution descriptions in the order of what it is, why you chose it, and then how it satisfies the problem. Captions should be written in full-sentence, paragraph form so that a chart, diagram, or image could justifiably stand alone, and flipping through the paper would allow an overview via the images only. When comparing visual techniques to others, it is helpful to do so side-by-side, rather than relying on the capacity of human memory.