This two-part article offers a brief history of information visualization, mostly by presenting outstanding examples. Part I of the article takes a look at the pre-computer era, while Part II looks at the more recent history and discusses the impact of the computer and the Internet on information visualization. It also presents a list of examples from the 1990s that were created with computers; some of these examples can even be used interactively.
The most extensive retrospect on information visualization can probably be found in Bill Ferster’s book Interactive Visualization published in 2012 (see references). I therefore refer to this book throughout this two-part article.
The Impact of Computers and of the Internet
In his paper History of Visualization from 1999, G. Scott Owen states that “visualization, in the presentation sense, is not a new phenomenon. It has been used in maps, scientific drawings, and data plots for over a thousand years. Examples of this are the map of China (1137 a.d.) and the famous map of Napoleon’s invasion of Russia in 1812, by Charles* Minard [which is presented in the first part of this article and in Figure 1 below]. Most of the concepts learned in devising these images carry over in a straight forward manner to computer visualization and can be incorporated in courses in visualization.”
Figure 1: Charles Minard’s Map of Napoleon’s March to Moscow 1812-1813 (1861) (from Wikipedia)
*) Owen incorrectly refers to Minard as “Jacque”)
The Impact of Computers
Owen also writes that “Computer Graphics has from its beginning been used to study scientific problems. However, in its early days the lack of graphics power often limited its usefulness. The recent emphasis on visualization started in 1987 with the special issue of Computer Graphics on Visualization in Scientific Computing. Since then there have been several conferences and workshops, co-sponsored by the IEEE and ACM SIGGRAPH,” devoted to the general topics data visualization, information visualization and scientific visualization, and special areas in the field. “There have also been numerous books and research articles on visualization in the past several years,” he adds.
In the same year, Stuart Card, Jock Mackinlay, and Ben Shneiderman (1999) offered their “classic” definition of information visualization. Not only can this definition still be regarded as “prototypical,” it also expressly includes the computer:
- Information visualization is “the use of computer-supported, interactive, visual representations of abstract data to amplify cognition.”
Since the late 1980s, the field of visualization has become a flourishing area of research and practice, including the spread of commercially available products, spanning a wide range from spreadsheet applications for creating charts to complex CAD applications – not to mention, the huge gaming industry, which has been a key driver in developing software and hardware with ever-increasing graphics power and capabilities.
Computers, Art, Animation, Games, and Simulations
Ferster (2012) notes that “the advent of computers that are capable of drawing sophisticated graphics provided a new opportunity for both designers and statistics to experiment with a new medium of expression, one that could render complex graphics based on large sets of data to create data-rich graphical visualizations beyond the static drawings of the previous centuries.” In the following, I take a small side-step to computer arts and graphics to drop a few names of pioneers in this era.
Melvin L. Prueitt, a theoretical physicist at the Los Alamos National Laboratory, was one such pioneer. He published two books full of computer graphics created by him and others, namely Computer Graphics: 118 Computer-Generated Designs in 1976, and Art and the Computer in 1985. I received the second book (German version) as a Christmas gift from my parents, but was slightly disappointed with it – I had expected more “state of the art” 3D raytracing images. But I still own the book and won’t, of course, give it away. In this book, Prueitt included two images from Benoit Mandelbrot (1924-2010), a mathematician born in Poland and the “father” of fractal geometry,” who pioneered fractal-based computer graphics. Back when I calculated my first Mandelbrot sets on my home computer using BASIC, it took me a whole day. Today, Mandelbrot images are ready in seconds or fractions thereof (see Figures 2 and 3).
Figures 2-3: Some of my own Mandelbrot set experiments
Vera Molnár (*1924), born in Hungary and living in France, is another pioneer in computer arts. Since the late 1960s she programs computers to create formal systems and random generators. These make up the syntax for drawing forms, lines, and colors. Typically, Molnár produces series of drawings, using few shapes reduced color palettes, which are varied systematically (or randomly) – there is a lot of mathematics involved in her works. She displayed some of her works in an exhibition at SAP in 2011, but regrettably she was not able to attend the opening in person. Below are “reproductions” of two of her works using the programming language Processing (see Figures 4 and 5).
Figures 4-5: “Recreations” of two of Vera Molnár’s works using Processing
John Whitney (1917-1995) used computer algorithms to experiment with creating and animating abstract graphics. In the beginning, he used analog computers that he adapted from war equipment. In the 1970s, he moved to the digital computer “to produce a mesmerizing relationship between music and visual design,” as Ferster (2012) puts it. Whitney became IBM’s first artist-in-residence in 1966.
Computing power has increased dramatically over the last few decades, and computer graphics profited immensely from this progress. Computer-animated movies are excellent proof of this – as are 3D games and simulations. Having started as text adventures with simple text commands in the 1970s, today’s video games provide gamers with nearly perfect 3D environments in which they can move around freely. The same is true for visual simulations such as flight simulators, urban planning tools, or virtual words on the Internet like Second Life. All in all, we have come close to photorealistic animations and simulations, but it may still take a decade or so until we really get there.
The Impact of the Internet
The Internet also gave information visualization a strong boost, this time in terms of opening its potential to a wide audience. Ferster (2012) notes that “the Internet and the open-source ethos of free accessibility have made data more widely available for instantaneous download. … These data are increasingly used by interactive visualization producers to provide powerful tools for information and inquiry.”
Ferster (2012) adds that “recently, a new genre of Websites has emerged that makes it easy for people who do not know who to create Websites or programs to produce sophisticated visualizations using their own data. These Web-apps make it easy to upload datasets and make a variety of compelling visualizations that can be shared worldwide.” Examples of this trend are IBM’s ManyEyes Website and Ferster’s own VisualEyes tool.
Figure 6: Homepage of the VisualEyes Website
Researchers and Locations
The majority of newer information visualization techniques was developed in a small number of “special” locations. I have listed these locations and their most prominent researchers in the following table:
|Xerox PARC||Stuart Card, Jock Mackinlay, George Robertson, Ramana Rao, Peter Pirolli||The location refers to the time when the research listed below was done|
|HCIL, University of Maryland||Ben Shneiderman, Ben Bederson, Catherine Plaisant||Founded in 1993 by Ben Shneiderman, now led by Ben Bederson|
|Imperial College of Science, Technology and Medicine, London||Robert Spence||Spence is professor emeritus|
Examples of Computer-Based Visualization Techniques from the 1990s
In the following, I present a selection of seminal examples of visualization techniques, most of them were proposed in the 1990s.
The Attribute Explorer (Figure 7) was developed between 1994 and 1998 by Robert Spence and Lisa Tweedy at the Imperial College of Science, Technology and Medicine, London, to explore the relations between attributes of multivariate data to gain insight. The tool consists of linked histograms and includes a technique called brushing for providing immediate feedback.
Figure 7: Screenshot from a Java standalone prototype of the Attribute Explorer in Java by Andy Smith et al. (IBM, UK) (by the author); note the highlighted house that is being brushed to linked histograms
This technique was applied in a commercial product called EZChooser, in which users could explore a selection of cars and choose one based on attributes such as make, rating, price, fuel efficiency, and do on.
Dynamic Queries and Starfield Displays: HomeFinder and FilmFinder
Like the AttributeExplorer, the techniques described here, emphasize direct manipulation (“the user is in control”) and immediate feedback (also called “tight coupling”). Dynamic queries and starfield displays were developed in the 1990s at the HCIL, University of Maryland, under the leadership of by Ben Shneiderman who extended the principles of “direct manipulation,” a term that he himself had coined, to the interaction with databases:
- Dynamic queries continuously update the data that is filtered from a database and visualized. They work instantly within a few milliseconds as users adjust sliders or select buttons to form simple queries or to find patterns or exceptions; the dynamic-query approach thus applies the principles of direct manipulation to the database (Shneiderman, 1994, adapted)
- Starfield displays are two-dimensional scatter diagrams with ordinal attributes of database items that are displayed as small selectable glyphs and laid out along the axes, and are typically used for dynamic queries: The query parameters are controlled by widgets such as sliders and buttons and the query can be regarded as a filter on the database. (After Jog and Shneiderman, 1994, adapted)
Shneiderman highlights the importance of tight coupling by pointing out that “unlike traditional applications such as image browsers that do zooming in large fixed stages, zooming a starfield display should be incremental and flicker-free so that users can track the motion of each rectangle. This gives users a feeling of flying through the data instead of getting disoriented by sudden large changes in view.”
Shneiderman and co-workers applied this principle to two applications, the HomeFinder (Shneiderman, Williamson 1992), and the FilmFinder (Shneiderman, Ahlberg, 1994). In the HomeFinder, the starfield display represented latitude and longitude along the axes – thereby making it a map of locations (see Figure 8). The FilmFinder was a commercial application for home use that was marketed by Ahlberg’s company Spotfire. It helped people find films and videos that fit their personal interests and taste by manipulating query devices such as rangesliders, alphasliders, and toggles. The system presented films in a starfield (interactive scatter plot) with the year of production and the popularity of the films as axes (see Figure 9). The FilmFinder comprised 1,838 film titles, 5,468 actors and 1,463 directors.
Figure 8-9: HomeFinder (left) and FilmFinder (right)
Table Lens: Focus+Context
The Table Lens (Figure 11) was developed by Ramana Rao, Stuart Card, and Peter Pirolli between 1994 and 1996 at Xerox PARC to simplify the interaction with large data tables. The tool was later marketed by InXight, a Xerox offspring (later acquired by BusinessObjects). The InXight Website (no longer available) stated that with this tool, “users can quickly get an overview of large data sets, looking for patterns and relationships, as well as focusing on specific items in their full context. The interactivity of Table Lens allows users to make sense of data even when it is published to the Web.”
Figure 11: Example of a Table Lens
The Table Lens was based on the focus+context concept, in which the focal information is displayed together with its context in one coherent presentation: The focus is shown with more detail, whereas the context exhibits decreasing detail the farther it is from the focus area. Distortion, suppression, or a combination of both, can be used to achieve this effect. According to Spence (2007), the TableLens uses distortion, but I would reserve this term for tools like the BifocalDisplay and the Hyperbolic Browser, which use geometric transformations for achieving distortions and are presented below.
Bifocal Display and the Perspective Wall: Applying Distortion to Focus+Context
Like the table lens, the Bifocal Display (Figures 12 to 14) addressed the problem of large data spaces by adopting the focus+context approach. It was invented in 1980 by Robert Spence (Imperial College of Science, Technology and Medicine, London) and Mark Apperley (University of Waikato, New Zealand).
The authors demonstrate the concept of the Bifocal Display using a physical analogy. Figure 12 shows an information space containing many items, such as documents, sketches, e-mails and manuscripts. Since this information space is too large to be viewed in its entirety in a window, horizontal scrolling would be needed to see all the items. Wrapping the space around two uprights (Figure 13) would allow a user to see part of the information space in its original detail (focus) and also a “squashed” view of the remainder of the information space (context). The information space can be scrolled to bring items into the focus region to inspect details. Thus, according to Spence and Apperley (2013) “stretching” or “distorting” of information space is central to the concept of the Bifocal Display.
Figure 12: A large information space containing documents, e-mails, and so on (from interaction-design.org)
Figures 13-14: The same space wrapped around two uprights (left); appearance of the information space when viewed from an appropriate direction (right) (from interaction-design.org)
The Perspective Wall (Figure 15), developed by Jock Mackinlay, George Robertson, and Stuart Card at Xerox PARC in 1991 is a 3D variant of the Bifocal Display with features such as the stretching of the focal region (Spence, 2007) and is thus also a potential solution for the focus+context problem. Robertson et al. (1991) note that “tasks that involve large information spaces overwhelm workspaces that do not support efficient use of space and time. The Perspective Wall visualizes linear information by smoothly integrating detailed and contextual views. It folds a 2D layout into a 3D wall that smoothly integrates a region for viewing details with perspective regions for viewing context. The resulting visualization supports, according to the authors, efficient use of space and time.” (from Robertson et al. , 1991, adapted)
Figure 15: Perspective Wall
Like many of the visualization tools, the Perspective Wall was marketed by InXight (see Figure 16 for a commercial version) before the company was taken over by BusinessObjects.
Figure 16: Perspective Wall, commercial version by Inxight
Tree Browsers: Hyperbolic Browser (Star Tree) and TreeMap
Similar to the presentation of huge tabular data sets, the efficient presentation of hierarchical data, or trees, attracted a great deal of attention in the research community. In the following, I present two very different approaches to this task.
The Hyperbolic Browser, developed by J. Lamping and Ramana Rao between 1994 and 1996 at Xerox PARC, is a tree browser that uses hyperbolic geometry to distort the tree and emphasize the data in focus. The tree can be moved around in real time.
Figure 17: An older version of the Hyperbolic Browser
Figure 18: An implementation of the Hyperbolic Browser , marketed by InXight (called Star Tree)
Like many of the visualization tools, the Hyperbolic Browser alias Star Tree, was marketed by InXight before the company was taken over by BusinessObjects.
TreeMap is a tree browser developed by Ben Shneiderman and co-workers from 1991 on at the HCIL, University of Maryland. It supports the space-constrained visualization of hierarchical structures. A TreeMap can display up to thousands of elements at once showing multiple attributes. By recursively partitioning an element’s rectangle among its children, a TreeMap presents the “big picture” of a hierarchy giving users X-ray vision into their mountains of data. Zooming, enter/leave notification, select actions, and much more are supported.
Figures 19-20: Two variants of the TreeMap
The authors regard TreeMap as very effective in showing attributes of leaf nodes using size and color coding. Users can compare nodes and sub-trees even at varying depth in the tree, and the tool helps them spot patterns and exceptions. TreeMap was and is being used in a number of commercial applications such as the display of stock exchange data.
Finally, the Visible Human Project conducted in 1996 by Ben Shneiderman and Catherine Plaisant from the HCIL, and Chris North from the Department of Computer Science, University of Maryland, is a different kind of visualization in that it does not display abstract or geographical data. Instead, it displays physical data, presenting various anatomical views of the human body that can be walked through via remote access.
Figure 21: Visible Human interface
Where to Find Out More…
Most books about [information] visualization take a brief look at the history of the field. The most extended retrospect can probably be found in Bill Ferster’s book Interactive Visualization from 2012. Robert Spence discusses four historic examples (Minard, Nightingale, Snow, Beck) in the first chapter of his book, Information Visualization, 2nd edition, from 2007, all of which are included in the retrospect in Part 1 of this article.
The book The Craft of Information Visualization: Readings and Reflections, edited by Ben Bederson and Ben Shneiderman, is a collection of nearly 40 of the key papers on information visualization from the University of Maryland’s Human-Computer Interaction Lab (HCIL). The book was published on the HCIL’s 20th anniversary in 2003 – and just recently the HCIL celebrated its 30th anniversary. Another “classic” is the book Readings in Information Visualization – Using Vision to Think, edited by Card, Mackinlay, and Shneiderman, from 1999.
Last but not least, the references below include links to Wikipedia articles about some of the people and examples presented above. They provide more detailed information and point to further examples.
- Stuart Card, Jock Mackinlay, and Ben Shneiderman (1999). Readings in Information Visualization – Using Vision to Think. Morgan Kaufmann • ISBN: 1558605339
- Robert Spence(2007). Information Visualization (2nd edition). Prentice-Hall (Pearson) • ISBN: 0132065509 (Hardcover) (Review • Short presentation)
- Bill Ferster (2012). Interactive Visualization: Insight through Inquiry. The MIT Press • ISBN-10: 0262018152, ISBN-13: 978-0262018159 (Review • Short presentation)
- Benjamin B. Bederson & Ben Shneiderman (2003). The Craft of Information Visualization – Readings and Reflections. Morgan Kaufmann. ISBN: 1558609156 (Paperback) (Review • Short presentation)
- Infovis Wiki: www.infovis-wiki.net/index.php?title=Main_Page
- G. Scott Owen (1999): History of Visualization (page in Hyper Vis)
- G. Scott Owen (1999): HyperVis – Teaching Scientific Visualization Using Hypermedia
- Spence, Robert and Apperley, Mark (2013): Bifocal Display. In: Soegaard, Mads and Dam, Rikke Friis (eds.). “The Encyclopedia of Human-Computer Interaction, 2nd Ed.”. Aarhus, Denmark: The Interaction Design Foundation. Available online at www.interaction-design.org/encyclopedia/bifocal_display.html
- Shneiderman, Ben (1994). Dynamic Queries for Visual Information Seeking, IEEE Software, 11(6), p. 70-77.
- Christopher Ahlberg & Staffan Truvé (1995). Tight Coupling: Guiding User Actionsin a Direct Manipulation Retrieval System, Chalmers University of Technology (Artikel)
- Williamson, Christopher, and Shneiderman, Ben, 1992. The Dynamic HomeFinder: Evaluating dynamic queries in a real-estate information exploration system, Proc. ACM SIGIR’92 Conference, Copenhagen, Denmark, (June 1992), 338-346. Reprinted in Shneiderman, B. (Editor), Sparks of Innovation in Human-Computer Interaction, Ablex Publishers, Norwood, NJ, (1993), 295-307.
- Jock D. Mackinlay, George Robertson, and Stuart Card (1991). The perspective wall: Detail and context smoothly integrated. Proceedings of CHI ’91 Conference, p. 173-179.
- Lamping, J., Rao R. (1996). Visualizing Large Trees Using the Hyperbolic Browser. Proc. ACM CHI ’96 Conference (www.acm.org/sigchi/chi96/proceedings/videos/Lamping/hb-video.html)
- Lamping, J., Rao R. & Pirolli, P. (1995). A focus+context technique based on hyperbolic geometry for visualizing large hierarchies. Proc. ACM CHI ’95 Conference, 401-408.
- Many Eyes Website (IBM):www-958.ibm.com/software/analytics/manyeyes (allows to experiment with data visualizations)
- VisualEyes Website: www.viseyes.org • VisEdit: VisualEyes Project Tool
For documentation and sample projects, see the VisualEyes Website
- Information Visualization: en.wikipedia.org/wiki/Information_visualization
- Benoit Mandelbrot: en.wikipedia.org/wiki/Benoit_Mandelbrot
- Vera Molnar: de.wikipedia.org/wiki/Vera_Molnár (German)
- John Whitney: en.wikipedia.org/wiki/John_Whitney_(animator)