Data visualization

Data visualization or data visualisation is viewed by many disciplines as a modern equivalent of visual communication. It involves the creation and study of the visual representation of data, meaning "information that has been abstracted in some schematic form, including attributes or variables for the units of information".^[1]

A primary goal of data visualization is to communicate information clearly and efficiently via statistical graphics, plots and information graphics. Numerical data may be encoded using dots, lines, or bars, to visually communicate a quantitative message.^[2] Effective visualization helps users analyze and reason about data and evidence. It makes complex data more accessible, understandable and usable. Users may have particular analytical tasks, such as making comparisons or understanding causality, and the design principle of the graphic (i.e., showing comparisons or showing causality) follows the task. Tables are generally used where users will look up a specific measurement, while charts of various types are used to show patterns or relationships in the data for one or more variables.

Data visualization is both an art and a science. It is viewed as a branch of descriptive statistics by some, but also as a grounded theory development tool by others. The rate at which data is generated has increased. Data created by internet activity and an expanding number of sensors in the environment, such as satellites, are referred to as "Big Data". Processing, analyzing and communicating this data present a variety of ethical and analytical challenges for data visualization. The field of data science and practitioners called data scientists have emerged to help address this challenge.^[3]

Overview

Data visualization is one of the steps in analyzing data and presenting it to users.

Data visualization refers to the techniques used to communicate data or information by encoding it as visual objects (e.g., points, lines or bars) contained in graphics. The goal is to communicate information clearly and efficiently to users. It is one of the steps in data analysis or data science. According to Friedman (2008) the "main goal of data visualization is to communicate information clearly and effectively through graphical means. It doesn't mean that data visualization needs to look boring to be functional or extremely sophisticated to look beautiful. To convey ideas effectively, both aesthetic form and functionality need to go hand in hand, providing insights into a rather sparse and complex data set by communicating its key-aspects in a more intuitive way. Yet designers often fail to achieve a balance between form and function, creating gorgeous data visualizations which fail to serve their main purpose — to communicate information".^[4]

Indeed, Fernanda Viegas and Martin M. Wattenberg have suggested that an ideal visualization should not only communicate clearly, but stimulate viewer engagement and attention.^[5]

Not limited to the communication of an information, a well-crafted data visualization is also a way to a better understanding of the data (in a data-driven research perspective),^[6] as it helps uncover trends, realize insights, explore sources, and tell stories.^[7]

Data visualization is closely related to information graphics, information visualization, scientific visualization, exploratory data analysis and statistical graphics. In the new millennium, data visualization has become an active area of research, teaching and development. According to Post et al. (2002), it has united scientific and information visualization.^[8]

Characteristics of effective graphical displays

Charles Joseph Minard's 1869 diagram of Napoleon's March - an early example of an information graphic.

The greatest value of a picture is when it forces us to notice what we never expected to see.

John Tukey^[9]

Professor Edward Tufte explained that users of information displays are executing particular analytical tasks such as making comparisons or determining causality. The design principle of the information graphic should support the analytical task, showing the comparison or causality.^[10]

In his 1983 book The Visual Display of Quantitative Information, Edward Tufte defines 'graphical displays' and principles for effective graphical display in the following passage: "Excellence in statistical graphics consists of complex ideas communicated with clarity, precision and efficiency. Graphical displays should:

• show the data
• induce the viewer to think about the substance rather than about methodology, graphic design, the technology of graphic production or something else
• avoid distorting what the data has to say
• present many numbers in a small space
• make large data sets coherent
• encourage the eye to compare different pieces of data
• reveal the data at several levels of detail, from a broad overview to the fine structure
• serve a reasonably clear purpose: description, exploration, tabulation or decoration
• be closely integrated with the statistical and verbal descriptions of a data set.

Graphics reveal data. Indeed graphics can be more precise and revealing than conventional statistical computations."^[11]

For example, the Minard diagram shows the losses suffered by Napoleon's army in the 1812–1813 period. Six variables are plotted: the size of the army, its location on a two-dimensional surface (x and y), time, direction of movement, and temperature. The line width illustrates a comparison (size of the army at points in time) while the temperature axis suggests a cause of the change in army size. This multivariate display on a two dimensional surface tells a story that can be grasped immediately while identifying the source data to build credibility. Tufte wrote in 1983 that: "It may well be the best statistical graphic ever drawn."^[11]

Not applying these principles may result in misleading graphs, which distort the message or support an erroneous conclusion. According to Tufte, chartjunk refers to extraneous interior decoration of the graphic that does not enhance the message, or gratuitous three dimensional or perspective effects. Needlessly separating the explanatory key from the image itself, requiring the eye to travel back and forth from the image to the key, is a form of "administrative debris." The ratio of "data to ink" should be maximized, erasing non-data ink where feasible.^[11]

The Congressional Budget Office summarized several best practices for graphical displays in a June 2014 presentation. These included: a) Knowing your audience; b) Designing graphics that can stand alone outside the context of the report; and c) Designing graphics that communicate the key messages in the report.^[12]

Quantitative messages

A time series illustrated with a line chart demonstrating trends in U.S. federal spending and revenue over time.

A scatterplot illustrating negative correlation between two variables (inflation and unemployment) measured at points in time.

Author Stephen Few described eight types of quantitative messages that users may attempt to understand or communicate from a set of data and the associated graphs used to help communicate the message:

1. Time-series: A single variable is captured over a period of time, such as the unemployment rate over a 10-year period. A line chart may be used to demonstrate the trend.
2. Ranking: Categorical subdivisions are ranked in ascending or descending order, such as a ranking of sales performance (the measure) by sales persons (the category, with each sales person a categorical subdivision) during a single period. A bar chart may be used to show the comparison across the sales persons.
3. Part-to-whole: Categorical subdivisions are measured as a ratio to the whole (i.e., a percentage out of 100%). A pie chart or bar chart can show the comparison of ratios, such as the market share represented by competitors in a market.
4. Deviation: Categorical subdivisions are compared against a reference, such as a comparison of actual vs. budget expenses for several departments of a business for a given time period. A bar chart can show comparison of the actual versus the reference amount.
5. Frequency distribution: Shows the number of observations of a particular variable for given interval, such as the number of years in which the stock market return is between intervals such as 0-10%, 11-20%, etc. A histogram, a type of bar chart, may be used for this analysis. A boxplot helps visualize key statistics about the distribution, such as median, quartiles, outliers, etc.
6. Correlation: Comparison between observations represented by two variables (X,Y) to determine if they tend to move in the same or opposite directions. For example, plotting unemployment (X) and inflation (Y) for a sample of months. A scatter plot is typically used for this message.
7. Nominal comparison: Comparing categorical subdivisions in no particular order, such as the sales volume by product code. A bar chart may be used for this comparison.
8. Geographic or geospatial: Comparison of a variable across a map or layout, such as the unemployment rate by state or the number of persons on the various floors of a building. A cartogram is a typical graphic used.^[2][13]

Analysts reviewing a set of data may consider whether some or all of the messages and graphic types above are applicable to their task and audience. The process of trial and error to identify meaningful relationships and messages in the data is part of exploratory data analysis.

Visual perception and data visualization

A human can distinguish differences in line length, shape orientation, and color (hue) readily without significant processing effort; these are referred to as "pre-attentive attributes." For example, it may require significant time and effort ("attentive processing") to identify the number of times the digit "5" appears in a series of numbers; but if that digit is different in size, orientation, or color, instances of the digit can be noted quickly through pre-attentive processing.^[14]

Effective graphics take advantage of pre-attentive processing and attributes and the relative strength of these attributes. For example, since humans can more easily process differences in line length than surface area, it may be more effective to use a bar chart (which takes advantage of line length to show comparison) rather than pie charts (which use surface area to show comparison).^[14]

Human perception/cognition and data visualization

There is a human side to data visualization. With the "studying [of] human perception and cognition ..." we are better able to understand the target of the data which we display.^[15] Cognition refers to processes in human beings like perception, attention, learning, memory, thought, concept formation, reading, and problem solving.^[16] The basis of data visualization evolved because as a picture is worth a thousand words, data displayed graphically allows for an easier comprehension of the information. Proper visualization provides a different approach to show potential connections, relationships, etc. which are not as obvious in non-visualized quantitative data. Visualization becomes a means of data exploration. Human brain neurons involve multiple functions but 2/3 of the brain's neurons are dedicated to vision.^[17] With a well-developed sense of sight, analysis of data can be made on data, whether that data is quantitative or qualitative. Effective visualization follows from understanding the processes of human perception and being able to apply this to intuitive visualizations is important. Understanding how humans see and organize the world is critical to effectively communicating data to the reader. This leads to more intuitive designs.

History of Data Visualization

There is a history of data visualization: beginning in the 2nd century C.E. with data arrangement into columns and rows and evolving to the initial quantitative representations in the 17th century.^[15] According to the Interaction Design Foundation, French philosopher and mathematician René Descartes laid the ground work for Scotsman William Playfair. Descartes developed a two-dimensional coordinate system for displaying values, which in the late 18th century Playfair saw potential for graphical communication of quantitative data.^[15] In the second half of the 20th century, Jacques Bertin used quantitative graphs to represent information "intuitively, clearly, accurately, and efficiently".^[15] John Tukey and more notably Edward Tufte pushed the bounds of data visualization. Tukey with his new statistical approach: exploratory data analysis and Tufte with his book "The Visual Display of Quantitative Information", the path was paved for refining data visualization techniques for more than statisticians. With the progression of technology came the progression of data visualization; starting with hand drawn visualizations and evolving into more technical applications – including interactive designs leading to software visualization.^[18] Programs like SAS, SOFA, R, Minitab, and more allow for data visualization in the field of statistics. Other data visualization applications, more focused and unique to individuals, programming languages such as D3, Python and JavaScript help to make the visualization of quantitative data a possibility.

Terminology

Data visualization involves specific terminology, some of which is derived from statistics. For example, author Stephen Few defines two types of data, which are used in combination to support a meaningful analysis or visualization:

• Categorical: Text labels describing the nature of the data, such as "Name" or "Age". This term also covers qualitative (non-numerical) data.
• Quantitative: Numerical measures, such as "25" to represent the age in years.

Two primary types of information displays are tables and graphs.

• A table contains quantitative data organized into rows and columns with categorical labels. It is primarily used to look up specific values. In the example above, the table might have categorical column labels representing the name (a qualitative variable) and age (a quantitative variable), with each row of data representing one person (the sampled experimental unit or category subdivision).
• A graph is primarily used to show relationships among data and portrays values encoded as visual objects (e.g., lines, bars, or points). Numerical values are displayed within an area delineated by one or more axes. These axes provide scales (quantitative and categorical) used to label and assign values to the visual objects. Many graphs are also referred to as charts.^[19]

KPI Library has developed the "Periodic Table of Visualization Methods," an interactive chart displaying various data visualization methods. It includes six types of data visualization methods: data, information, concept, strategy, metaphor and compound.^[20]

Examples of diagrams used for data visualization

	Name	Visual Dimensions	Example Usages
Bar chart of tips by day of week	Bar Chart	length/count category (color)	Comparison of values, such as sales performance for several persons or businesses in a single time period. For a single variable measured over time (trend) a line chart is preferable.
Histogram of housing prices	Histogram	bin limits count/length (color)	Determining frequency of annual stock market percentage returns within particular ranges (bins) such as 0-10%, 11-20%, etc. The height of the bar represents the number of observations (years) with a return % in the range represented by the bin.
Basic scatterplot of two variables	Scatter plot	x position y position (symbol/glyph) (color) (size)	Determining the relationship (e.g., correlation) between unemployment (x) and inflation (y) for multiple time periods.
Scatter Plot	Scatter plot (3D)	position x position y position z color
Network Analysis	Network	nodes size nodes color ties thickness ties color spatialization	Finding clusters in the network (e.g. grouping Facebook friends into different clusters). Determining the most influential nodes in the network (e.g. A company wants to target a small group of people on Twitter for a marketing campaign).
Streamgraph	Streamgraph	width color time (flow)
Treemap	Treemap	size color	disk space by location / file type
Gantt Chart	Gantt chart	color time (flow)	schedule / progress, e.g. in project planning
Heat Map	Heat Map	row column cluster color	Analyzing risk, with green, yellow and red representing low, medium, and high risk, respectively.

Other perspectives

There are different approaches on the scope of data visualization. One common focus is on information presentation, such as Friedman (2008) presented it. In this way Friendly (2008) presumes two main parts of data visualization: statistical graphics, and thematic cartography.^[1] In this line the "Data Visualization: Modern Approaches" (2007) article gives an overview of seven subjects of data visualization:^[21]

• Articles & resources
• Displaying connections
• Displaying data
• Displaying news
• Displaying websites
• Mindmaps
• Tools and services

All these subjects are closely related to graphic design and information representation.

On the other hand, from a computer science perspective, Frits H. Post (2002) categorized the field into a number of sub-fields:^[8]

^[22]

• Information visualization
• Interaction techniques and architectures
• Modelling techniques
• Multiresolution methods
• Visualization algorithms and techniques
• Volume visualization

Data presentation architecture

A data visualization from social media

Data presentation architecture (DPA) is a skill-set that seeks to identify, locate, manipulate, format and present data in such a way as to optimally communicate meaning and proffer knowledge.

Historically, the term data presentation architecture is attributed to Kelly Lautt:^[23] "Data Presentation Architecture (DPA) is a rarely applied skill set critical for the success and value of Business Intelligence. Data presentation architecture weds the science of numbers, data and statistics in discovering valuable information from data and making it usable, relevant and actionable with the arts of data visualization, communications, organizational psychology and change management in order to provide business intelligence solutions with the data scope, delivery timing, format and visualizations that will most effectively support and drive operational, tactical and strategic behaviour toward understood business (or organizational) goals. DPA is neither an IT nor a business skill set but exists as a separate field of expertise. Often confused with data visualization, data presentation architecture is a much broader skill set that includes determining what data on what schedule and in what exact format is to be presented, not just the best way to present data that has already been chosen (which is data visualization). Data visualization skills are one element of DPA."

Objectives

DPA has two main objectives:

• To use data to provide knowledge in the most efficient manner possible (minimize noise, complexity, and unnecessary data or detail given each audience's needs and roles)
• To use data to provide knowledge in the most effective manner possible (provide relevant, timely and complete data to each audience member in a clear and understandable manner that conveys important meaning, is actionable and can affect understanding, behavior and decisions)

Scope

With the above objectives in mind, the actual work of data presentation architecture consists of:

• Creating effective delivery mechanisms for each audience member depending on their role, tasks, locations and access to technology
• Defining important meaning (relevant knowledge) that is needed by each audience member in each context
• Determining the required periodicity of data updates (the currency of the data)
• Determining the right timing for data presentation (when and how often the user needs to see the data)
• Finding the right data (subject area, historical reach, breadth, level of detail, etc.)
• Utilizing appropriate analysis, grouping, visualization, and other presentation formats

Related fields

DPA work shares commonalities with several other fields, including:

• Business analysis in determining business goals, collecting requirements, mapping processes.
• Business process improvement in that its goal is to improve and streamline actions and decisions in furtherance of business goals
• Data visualization in that it uses well-established theories of visualization to add or highlight meaning or importance in data presentation.
• Graphic or user design: As the term DPA is used, it falls just short of design in that it does not consider such detail as colour palates, styling, branding and other aesthetic concerns, unless these design elements are specifically required or beneficial for communication of meaning, impact, severity or other information of business value. For example:
  • choosing locations for various data presentation elements on a presentation page (such as in a company portal, in a report or on a web page) in order to convey hierarchy, priority, importance or a rational progression for the user is part of the DPA skill-set.
  • choosing to provide a specific colour in graphical elements that represent data of specific meaning or concern is part of the DPA skill-set
• Information architecture, but information architecture's focus is on unstructured data and therefore excludes both analysis (in the statistical/data sense) and direct transformation of the actual content (data, for DPA) into new entities and combinations.
• Solution architecture in determining the optimal detailed solution, including the scope of data to include, given the business goals
• Statistical analysis or data analysis in that it creates information and knowledge out of data

See also

• Analytics
• Balanced scorecard
• Business analysis
• Business intelligence
• Data analysis
• Data profiling
• Data warehouse
• Exploratory data analysis
• Infographic
• Information architecture
• Information design
• Information visualization
• Interaction design
• Interaction techniques
• Scientific visualization
• Software visualization
• Statistical analysis
• Statistical graphics
• Visual analytics

People (Historical)

• Charles Joseph Minard
• John Tukey
• John Snow
• Otto Neurath
• Florence Nightingale
• William Playfair

People (active today)

• Albero Cairo
• Edward Tufte
• Ola Rosling - Rosling developed the scatter-plot graphing tool used on Gapminder.org.
• Hans Rosling
• Aaron Koblin
• Manuel Lima
• Max Roser - Roser is an economist at the University of Oxford and author of the online data visualisation publication Our World In Data.
• Moritz Stefaner
• Ben Shneiderman
• Fernanda Viégas
• Martin M. Wattenberg
• Mona Chalabi - Data journalist at FiveThirtyEight. Previously at the Guardian, the Bank of England, and the Economist Intelligence Unit.
• George Furnas
• Branko Milanovic
• Mike Bostock - Bostock is one of the key developers of the Javascript library d3.js.
• Adrien Segal - Oakland, CA based artist known for her sculptures based on tidal and snow data.

References

1. Michael Friendly (2008). "Milestones in the history of thematic cartography, statistical graphics, and data visualization".
2. Stephen Few-Perceptual Edge-Selecting the Right Graph for Your Message-2004
3. Forbes-Gil Press-A Very Short History of Data Science-May 2013
4. Vitaly Friedman (2008) "Data Visualization and Infographics" in: Graphics, Monday Inspiration, January 14th, 2008.
5. Fernanda Viegas and Martin Wattenberg, "How To Make Data Look Sexy", CNN.com, April 19, 2011. http://articles.cnn.com/2011-04-19/opinion/sexy.data_1_visualization-21st-century-engagement?_s=PM:OPINION
6. Grandjean, Martin (2015) "Introduction à la visualisation de données, l'analyse de réseau en histoire", Geschichte und Informatik, 18-19, 109-128.
7. A Beginner's Guide to Creating Data Visualizations. http://knowledgent.com/infographics/data-viz-101/
8. Frits H. Post, Gregory M. Nielson and Georges-Pierre Bonneau (2002). Data Visualization: The State of the Art. Research paper TU delft, 2002..
9. Tukey, John (1977). Exploratory Data Analysis. Addison-Wesley. ISBN 0-201-07616-0.
10. Edward Tufte-Presentation-August 2013
11. Tufte, Edward (1983). The Visual Display of Quantitative Information. Cheshire, Connecticut: Graphics Press. ISBN 0-9613921-4-2.
12. CBO-Telling Visual Stories About Data-June 2014
13. Stephen Few-Perceptual Edge-Graph Selection Matrix
14. Steven Few-Tapping the Power of Visual Perception-September 2004
15. "Data Visualization for Human Perception". The Interaction Design Foundation. Retrieved 2015-11-23.
16. "Visualization" (PDF). SFU. SFU lecture. Retrieved 2015-11-22.
17. "How much of the brain is involved with vision? - Quora". www.quora.com. Retrieved 2015-11-23.
18. Friendly, Michael (2006). "A Brief History of Data Visualization" (PDF). York University. Springer-Verlag. Retrieved 2015-11-22.
19. Steven Few-Selecting the Right Graph for Your Message-September 2004
20. Lengler, Ralph; Eppler, Martin. J. "Periodic Table of Visualization Methods". www.visual-literacy.org. Retrieved 15 March 2013.
21. "Data Visualization: Modern Approaches". in: Graphics, August 2nd, 2007
22. Frits H. Post, Gregory M. Nielson and Georges-Pierre Bonneau (2002). Data Visualization: The State of the Art.
23. The first formal, recorded, public usages of the term data presentation architecture were at the three formal Microsoft Office 2007 Launch events in Dec, Jan and Feb of 2007–08 in Edmonton, Calgary and Vancouver (Canada) in a presentation by Kelly Lautt describing a business intelligence system designed to improve service quality in a pulp and paper company. The term was further used and recorded in public usage on December 16, 2009 in a Microsoft Canada presentation on the value of merging Business Intelligence with corporate collaboration processes.

Further reading

• Chandrajit Bajaj, Bala Krishnamurthy (1999). Data Visualization Techniques.
• William S. Cleveland (1993). Visualizing Data. Hobart Press.
• William S. Cleveland (1994). The Elements of Graphing Data. Hobart Press.
• Alexander N. Gorban, Balázs Kégl, Donald Wunsch, and Andrei Zinovyev (2008). Principal Manifolds for Data Visualization and Dimension Reduction. LNCSE 58. Springer.
• John P. Lee and Georges G. Grinstein (eds.) (1994). Database Issues for Data Visualization: IEEE Visualization '93 Workshop, San Diego.
• Peter R. Keller and Mary Keller (1993). Visual Cues: Practical Data Visualization.
• Frits H. Post, Gregory M. Nielson and Georges-Pierre Bonneau (2002). Data Visualization: The State of the Art.
• Stewart Liff and Pamela A. Posey, Seeing is Believing: How the New Art of Visual Management Can Boost Performance Throughout Your Organization, AMACOM, New York (2007), ISBN 978-0-8144-0035-7
• Stephen Few (2009) Fundamental Differences in Analytical Tools - Exploratory, Custom, or Customizable.

External links

• Milestones in the History of Thematic Cartography, Statistical Graphics, and Data Visualization, An illustrated chronology of innovations by Michael Friendly and Daniel J. Denis.
• Duke University-Christa Kelleher Presentation-Communicating through infographics-visualizing scientific & engineering information-March 6, 2015
• VisualComplexity
• Big Data Visualization: Review of the 20 Best Tools