PSCI 3744 Visualizing a Policy Problem Memorandum

For David Kane
EDWA RD R . TUFTE
VISUAL AND STATISTIC AL THINKING :
DISPLAYS OF EVIDENCE FOR MAKING DECISIONS
O-ring damage
index, each launch
12
12
srm 15
8
8
srm 22
4
4
26?-29? range of forecasted temperatures
(as of January 27, 1986) for the launch
of space shuttle Challenger on January 28
0
0
25³0³5´0´5µ0µ5¶0¶5”emperature (?F) of field joints at time of launch
JOHN SNOW AND THE CHOLERA EPIDEMIC
THE DECISION TO LAUNCH THE SPACE SHUTTLE CHALLENGER
ISBN 978 -1-930824 -18 – 8
70·5¸0¸5…dward Tufte has written eight books, including Beautiful Evidence, Visual Explanations, Envisioning
Information, The Visual Display of Quantitative Information, Data Analysis for Politics and Policy, Political
Control of the Economy, and Size and Democracy (with Robert A. Dahl). He writes, designs, and self-publishes
his books on information design, which have received more than 30 awards for content and design. He is
Professor Emeritus at Yale University, where he taught courses in statistical evidence, information design,
and interface design.
Copyright 1997 by Edward Rolf Tufte
P UBLISHED BY G RAPHICS P RESS LLC
P OST O FFICE B OX 430, C HESHIRE , C ONNECTICUT 06410
WWW. TUFTE . COM
All rights to illustrations and text reserved by Edward Rolf Tufte. This work may not be copied, reproduced, or translated in whole or in
part without written permission of the publisher, except for brief excerpts in connection with reviews or scholarly analysis. Use in any form
of information storage and retrieval, electronic adaptation or whatever, computer software, or by similar or dissimilar methods now known
or developed in the future is also strictly forbidden without written permission of the publisher.
For David Kane
Introduction
This booklet, meant for students of quantitative thinking, reproduces
chapter 2 of my book Visual Explanations: Images and Quantities, Evidence
and Narrative.
The general argument is straightforward:
An essential analytic task in making decisions based on evidence
is to understand how things workechanism, trade-oVs, process
and dynamics, cause and eVect. That is, intervention-thinking and
policy-thinking demand causality-thinking.
Making decisions based on evidence requires the appropriate display
of that evidence. Good displays of data help to reveal knowledge
relevant to understanding mechanism, process and dynamics, cause
and eVect. That is, displays of statistical data should directly serve the
analytic task at hand.
What is reasonable and obvious in theory may not be implemented
in the actual practice of assessing data and making decisions. Here we
will see two complex cases of the analysis and display of evidence(e
celebrated investigation of a cholera epidemic by Dr. John Snow and
the unfortunate decision to launch the space shuttle Challenger.
Edward Tufte
Although we often hear that data speak for themselves, their voices can be
soft and sly.
Frederick Mosteller, Stephen E. Fienberg, and Robert
E. K. Rourke, Beginning Statistics with Data Analysis
(Reading, Massachusetts, 1983), 234.
Negligent speech doth not only discredit the person of the Speaker, but it
discrediteth the opinion of his reason and judgment; it discrediteth the force
and uniformity of the matter, and substance.
Ben Jonson, Timber: or, Discoveries (London, 1641), first
printed in the Folio of 1640, The Workes . . . , p. 122
of the section beginning with Horace his Art of Poetry.
The final approval and rationale for the launch of the
space shuttle Challenger, faxed by the rocket-maker to
nasa the night before the launch. The rocket blew up
12 hours later as a result of cold temperatures.
For David Kane
Visual and Statistical Thinking:
Displays of Evidence for Making Decisions
5
When we reason about quantitative evidence, certain methods for
displaying and analyzing data are better than others. Superior methods
are more likely to produce truthful, credible, and precise findings. The
diVerence between an excellent analysis and a faulty one can sometimes
have momentous consequences.
This chapter examines the statistical and graphical reasoning used in
making two life-and-death decisions: how to stop a cholera epidemic
in London during September 1854; and whether to launch the space
shuttle Challenger on January 28, 1986. By creating statistical graphics
that revealed the data, Dr. John Snow was able to discover the cause
of the epidemic and bring it to an end. In contrast, by fooling around
with displays that obscured the data, those who decided to launch the
space shuttle got it wrong, terribly wrong. For both cases, the consequences resulted directly from the quality of methods used in displaying
and assessing quantitative evidence.
? John Snow, On the Mode of Communi-
The Cholera Epidemic in London, 1854
In a classic of medical detective work, On the Mode of Communication of
Cholera,? John Snow describedéth an eloquent and precise language
of evidence, number, comparison(e severe epidemic:
The most terrible outbreak of cholera which ever occurred in this kingdom, is
probably that which took place in Broad Street, Golden Square, and adjoining
streets, a few weeks ago. Within two hundred and fifty yards of the spot where
Cambridge Street joins Broad Street, there were upwards of five hundred fatal attacks of cholera in ten days. The mortality in this limited area probably equals any
that was ever caused in this country, even by the plague; and it was much more
sudden, as the greater number of cases terminated in a few hours. The mortality
would undoubtedly have been much greater had it not been for the flight of the
population. Persons in furnished lodgings left first, then other lodgers went away,
leaving their furniture to be sent for. . . . Many houses were closed altogether
owing to the death of the proprietors; and, in a great number of instances, the
tradesmen who remained had sent away their families; so that in less than six days
from the commencement of the outbreak, the most aZicted streets were deserted
by more than three-quarters of their inhabitants.£ation of Cholera (London, 1855). An
acute disease of the small intestine, with
severe watery diarrhea, vomiting, and
rapid dehydration, cholera has a fatality
rate of 50 percent or more when untreated. With the rehydration therapy
developed in the 1960s, mortality can be
reduced to less than one percent. Epidemics still occur in poor countries, as
the bacterium Vibrio cholerae is distributed
mainly by water and food contaminated
with sewage. See Dhiman Barua and
William B. Greenough iii, eds., Cholera
(New York, 1992); and S. N. De,
Cholera: Its Pathology and Pathogenesis
(Edinburgh, 1961).
now, Cholera, 38. See also Report on
the Cholera Outbreak in the Parish of St.
Jamesì Westminster, during the Autumn
of 1854, presented to the Vestry by The
Cholera Inquiry Committee (London,
1855); and H. Harold Scott, Some Notable
Epidemics (London, 1934).
v i s ua l an d s tat i s t i c a l t h i n k i n g
6
Cholera broke out in the Broad Street area of central London on
the evening of August 31, 1854. John Snow, who had investigated
earlier epidemics, suspected that the water from a community pumpwell at Broad and Cambridge Streets was contaminated. Testing the
water from the well on the evening of September 3, Snow saw no
suspicious impurities, and thus he hesitated to come to a conclusion.
This absence of evidence, however, was not evidence of absence:
Further inquiry . . . showed me that there was no other circumstance or agent
common to the circumscribed locality in which this sudden increase of cholera
occurred, and not extending beyond it, except the water of the above mentioned
pump. I found, moreover, that the water varied, during the next two days, in the
amount of organic impurity, visible to the naked eye, on close inspection, in the
form of small white, flocculent [loosely clustered] particles. . . .†rom the General Register OYce, Snow obtained a list of 83 deaths
from cholera. When plotted on a map, these data showed a close link
between cholera and the Broad Street pump. Persistent house-by-house,
case-by-case detective work had yielded quite detailed evidence about
a possible cause-eVect relationship, as Snow made a kind of streetcorner
correlation:
On proceeding to the spot, I found that nearly all of the deaths had taken place
within a short distance of the pump. There were only ten deaths in houses situated
decidedly nearer to another street pump. In five of these cases the families of the
deceased persons informed me that they always sent to the pump in Broad Street,
as they preferred the water to that of the pump which was nearer. In three other
cases, the deceased were children who went to school near the pump in Broad
Street. Two of them were known to drink the water; and the parents of the third
think it probable that it did so. The other two deaths, beyond the district which
this pump supplies, represent only the amount of mortality from cholera that was
occurring before the irruption took place.
With regard to the deaths occurring in the locality belonging to the pump, there
were sixty-one instances in which I was informed that the deceased persons used to
drink the pump-water from Broad Street, either constantly or occasionally. In six
instances I could get no information, owing to the death or departure of every
one connected with the deceased individuals; and in six cases I was informed that
the deceased persons did not drink the pump-water before their illness.”hus the theory implicating the particular pump was confirmed by
the observed covariation: in this area of London, there were few
occurrences of cholera exceeding the normal low level, except among
those people who drank water from the Broad Street pump. It was
now time to act; after all, the reason we seek causal explanations is
in order to intervene, to govern the cause so as to govern the eVect:
/licy-thinking is and must be causality-thinking.àSnow described
his findings to the authorities responsible for the community water
supply, the Board of Guardians of St. JamesàParish, on the evening
of September 7, 1854. The Board ordered that the pump-handle on the
Broad Street well be removed immediately. The epidemic soon ended.
now, Cholera, 39. A few weeks after
the epidemic, Snow reported his results
in a first-person narrative, more like
a laboratory notebook or a personal
journal than a modern research paper
with its pristine, reconstructed science.
Postmodern research claims to have
added some complexities to the story
of John Snow; see Howard Brody, et al.,
ap-Making and Myth-Making in
Broad Street: The London Cholera
Epidemic, 1854, he Lancet 356 (July 1,
2000), 64-68.
now, Cholera, 39-40.
? Robert A. Dahl, áuse and EVect in
the Study of Politics,)n Daniel Lerner,
ed., Cause and Effect (New York, 1965),
88. Wold writes `frequent situation is
that description serves to maintain some
modus vivendi (the control of an established production process, the tolerance
of a limited number of epidemic cases),
whereas explanation serves the purpose
of reform (raising the agricultural yield,
reducing the mortality rates, improving
a production process). In other words,
description is employed as an aid in the
human adjustment to conditions, while
explanation is a vehicle for ascendancy
over the environment. erman Wold,
áusal Inference from Observational
Data,
ournal of the Royal Statistical
Society, A, 119 (1956), 29.
For David Kane
v i s ua l an d s tat i s t i c a l t h i n k i n g
7
Moreover, the result of this intervention (a before/after experiment
of sorts) was consistent with the idea that cholera was transmitted by
impure water. Snowàexplanation replaced previously held beliefs
that cholera spread through the air or by some other means. In those
times many years before the discovery of bacteria, one fantastic theory
speculated that cholera vaporously rose out of the burying grounds of
plague victims from two centuries earlier.? In 1886 the discovery of the
bacterium Vibrio cholerae confirmed Snowàtheory. He is still celebrated
for establishing the mode of cholera transmission and consequently the
method of prevention: keep drinking water, food, and hands clear of
infected sewage. Today at the old site of the Broad Street pump there
stands a public house (a bar) named after John Snow, where one can
presumably drink more safely than 140 years ago.
? H. Harold Scott, Some Notable Epidemics
(London, 1934), 3-4.
Why was the centuries-old mystery of cholera finally solved? Most
importantly, Snow had a good idea`causal theory about how the
disease spread(at guided the gathering and assessment of evidence.
This theory developed from medical analysis and empirical observation;
by mapping earlier epidemics, Snow detected a link between diVerent
water supplies and varying rates of cholera (to the consternation of
private water companies who anonymously denounced Snowàwork).
By the 1854 epidemic, then, the intellectual framework was in place,
and the problem of how cholera spread was ripe for solution.long with a good idea and a timely problem, there was a good
method. Snowàscientific detective work exhibits a shrewd intelligence
about evidence, a clear logic of data display and analysis:
cientists are not dmired for failing
in the attempt to solve problems that
lie beyond [their] competence. . . . If
politics is the art of the possible, research is surely the art of the soluble.
Both are immensely practical-minded
aVairs. . . . The art of research [is] the
art of making diYcult problems soluble
by devising means of getting at them.
Certainly good scientists study the most
important problems they think they can
solve. It is, after all, their professional
business to solve problems, not merely
to grapple with them. The spectacle of
a scientist locked in combat with the
forces of ignorance is not an inspiring
one if, in the outcome, the scientist is
routed. That is why so many of the
most important biological problems
have not yet appeared on the agenda
of practical research. eter Medawar,
PlutoàRepublic (New York, 1984),
253-254; 2-3.
1. Placing the data in an appropriate context for assessing cause and effect.
The original data listed the victims.ames and described their circumstances, all in order by date of death. Such a stack of death certificates
naturally lends itself to time-series displays, chronologies of the epidemic as shown below. But descriptive narration is not causal explanation;
the passage of time is a poor explanatory variable, practically useless in
discovering a strategy of how to intervene and stop the epidemic.
140
120
Deaths from
cholera, each
day during
the epidemic
700
600
100
500
80
400
60
300
40
200
20
100
20 22 24 26 28 30
1
August
September
3
5
7
9
11 13 15 17 19 21 23 25 27 29
Cumulative deaths from cholera,
beginning August 19, 1854; ?nal
total 616 deaths
20 22 24 26 28 30
1
August
September
3
5
7
9
11 13 15 17 19 21 23 25 27 29
v i s ua l an d s tat i s t i c a l t h i n k i n g
Instead of plotting a time-series, which would simply report each dayàbad news,
Snow constructed a graphical display that provided direct and powerful testimony about
a possible cause-eVect relationship. Recasting the original data from their one-dimensional
temporal ordering into a two-dimensional spatial comparison, Snow marked deaths from
cholera (
) on this map, along with locations of the areaà13 community water pumpwells ( ). The notorious well is located amid an intense cluster of deaths, near the d in
broad street. This map reveals a strong association between cholera and proximity to
the Broad Street pump, in a context of simultaneous comparison with other local water
sources and the surrounding neighborhoods without cholera.
8
For David Kane
v i s ua l an d s tat i s t i c a l t h i n k i n g
2. Making quantitative comparisons. The deep, fundamental question in statistical
analysis is Compared with what? Therefore, investigating the experiences of the victims
of cholera is only part of the search for credible evidence; to understand fully the cause
of the epidemic also requires an analysis of those who escaped the disease. With great
clarity, the map presented several intriguing clues for comparisons between the living
and the dead, clues strikingly visible at a brewery and a workhouse (tinted yellow
here). Snow wrote in his report:
There is a brewery in Broad Street, near to the pump, and on perceiving that no breweràmen
were registered as having died of cholera, I called on Mr. Huggins, the proprietor. He informed
me that there were above seventy workmen employed in the brewery, and that none of them
had suVered from cholerat least in severe formîly two having been indisposed, and that
not seriously, at the time the disease prevailed. The men are allowed a certain quantity of malt
liquor, and Mr. Huggins believes they do not drink water at all; and he is quite certain that the
workmen never obtained water from the pump in the street. There is a deep well in the brewery,
in addition to the New River water. (p. 42)
Saved by the beer! And at a nearby workhouse, the circumstances of non-victims of
the epidemic provided important and credible evidence about the cause of the disease,
as well as a quantitative calculation of an expected rate of cholera compared with the
actual observed rate:
The Workhouse in Poland Street is more than three-fourths surrounded by houses in which
deaths from cholera occurred, yet out of five-hundred-thirty-five inmates only five died of
cholera, the other deaths which took place being those of persons admitted after they were
attacked. The workhouse has a pump-well on the premises, in addition to the supply from the
Grand Junction Water Works, and the inmates never sent to Broad Street for water. If the
mortality in the workhouse had been equal to that in the streets immediately surrounding it
on three sides, upwards of one hundred persons would have died. (p. 42)
Such clear, lucid reasoning may seem commonsensical, obvious, insuYciently
technical. Yet we will soon see a tragic instance, the decision to launch the space
shuttle, when this straightforward logic of statistical (and visual) comparison was
abandoned by many engineers, managers, and government oYcials.
9
v i s ua l an d s tat i s t i c a l t h i n k i n g
3. Considering alternative explanations and contrary cases. Sometimes it
can be diYcult for researchersèo both report and advocate their
findings/ face up to threats to their conclusions, such as alternative
explanations and contrary cases. Nonetheless, the credibility of a report
is enhanced by a careful assessment of all relevant evidence, not just the
evidence overtly consistent with explanations advanced by the report.
The point is to get it right, not to win the case, not to sweep under the
rug all the assorted puzzles and inconsistencies that frequently occur in
collections of data.‚oth Snowàmap and the time-sequence of deaths show several
apparently contradictory instances, a number of deaths from cholera
with no obvious link to the Broad Street pump. And yet . . .
In some of the instances, where the deaths are scattered a little further from
the rest on the map, the malady was probably contracted at a nearer point to
the pump. A cabinet-maker who resided on Noel Street [some distance from
Broad Street] worked in Broad Street. . . . A little girl, who died in Ham
Yard, and another who died in Angel Court, Great Windmill Street, went to
the school in DufouràPlace, Broad Street, and were in the habit of drinking
the pump-water. . . .° The distinction between science and
advocacy is poignantly posed when
statisticians serve as consultants and
witnesses for lawyers. See Paul Meier,
!mned Liars and Expert Witnesses,¡nd Franklin M. Fisher, ôatisticians,
Econometricians, and Adversary Proceedings,
ournal of the American
Statistical Association, 81 (1986), 269276 and 277-286.
now, Cholera, 47.
In a particularly unfortunate episode, one London resident made
a special eVort to obtain Broad Street well water, a delicacy of taste
with a side eVect that unwittingly cost two lives. Snowàreport is
one of careful description and precise logic:
Dr. Fraser also first called my attention to the following circumstances, which
are perhaps the most conclusive of all in proving the connexion between the
Broad Street pump and the outbreak of cholera. In the åekly Return of
Births and Deaths/f September 9th, the following death is recorded: t West
End, on 2nd September, the widow of a percussion-cap maker, aged 59 years,
diarrhea two hours, cholera epidemica sixteen hours. was informed by this ladyÊson that she had not been in the neighbourhood of Broad Street for many
months. A cart went from Broad Street to West End every day, and it was the
custom to take out a large bottle of the water from the pump in Broad Street,
as she preferred it. The water was taken on Thursday, 31st August, and she drank
of it in the evening, and also on Friday. She was seized with cholera on the
evening of the latter day, and died on Saturday. . . . A niece, who was on a
visit to this lady, also drank of the water; she returned to her residence, in a high
and healthy part of Islington, was attacked with cholera, and died also. There
was no cholera at the time, either at West End or in the neighbourhood where
the niece died.?lthough at first glance these deaths appear unrelated to the Broad
Street pump, they are, upon examination, strong evidence pointing to
that well. There is here a clarity and undeniability to the link between
cholera and the Broad Street pump; only such a link can account for
what would otherwise be a mystery, this seemingly random and unusual
occurrence of cholera. And the saintly Snow, unlike some researchers,
gives full credit to the person, Dr. Fraser, who actually found this
crucial case.
10
? now, Cholera, 44-45.
For David Kane
v i s ua l an d s tat i s t i c a l t h i n k i n g
140
120
11
Deaths from
cholera, each
day during
the epidemic
100
80
Handle removed from
Broad Street pump,
September 8, 1854
60
40
20
Data source: plotted from the table in
Snow, Cholera, 49.
20 22 24 26 28 30
1
August
September
3
5
7
9
11 13 15 17 19 21 23 25 27 29
Ironically, the most famous aspect of Snowàwork is also the most
uncertain part of his evidence: it is not at all clear that the removal
of the handle of the Broad Street pump had much to do with ending
the epidemic. As shown by this time-series above, the epidemic was
already in rapid decline by the time the handle was removed. Yet, in
many retellings of the story of the epidemic, the pump-handle removal
is the decisive event, the unmistakable symbol of Snowàcontribution.
Here is the dramatic account of Benjamin Ward Richardson:
On the evening of Thursday, September 7th, the vestrymen of St. Jamesàwere
sitting in solemn consultation on the causes of the [cholera epidemic]. They might
well be solemn, for such a panic possibly never existed in London since the days
of the great plague. People fled from their homes as from instant death, leaving
behind them, in their haste, all the mere matter which before they valued most.
While, then, the vestrymen were in solemn deliberation, they were called to consider a new suggestion. A stranger had asked, in modest speech, for a brief hearing.
Dr. Snow, the stranger in question, was admitted and in few words explained his
view of the %ad and front of the oVending. e had fixed his attention on the
Broad Street pump as the source and centre of the calamity. He advised removal
of the pump-handle as the grand prescription. The vestry was incredulous, but
had the good sense to carry out the advice. The pump-handle was removed, and
the plague was stayed.??
Note the final sentence, a declaration of cause and eVect.?
odern
epidemiologists, however, are somewhat skeptical about the evidence
that links the removal of the pump-handle directly to the epidemicÊend. Nonetheless, the decisive point is that ultimately John Snow got it
exactly right:
John Snow, in the seminal act of modern public health epidemiology, performed
an intervention that was non-randomized, that was appraised with historical controls, and that had major ambiguities in the equivocal time relationship between
his removal of the handle of the Broad Street pump and the end of the associated
epidemic of choleraµt he correctly demonstrated that the disease was transmitted
through water, not air.?¿? Benjamin W. Richardson, (e Life
of John Snow, M.D.,&oreword to John
Snow, On Chloroform and Other Anaesthetics: Their Action and Administration
(London, 1858), xx-xxi.
? nother example of the causal claim:
î September 8, at Snowàurgent
request, the handle of the Broad Street
pump was removed and the incidence of
new cases ceased almost at once, . W.
Gilbert, )oneer Maps of Health and
Disease in England, he Geographical
Journal, 124 (1958), 174. Gilbertàassertion
was repeated in Edward R. Tufte, The
Visual Display of Quantitative Information
(Cheshire, Connecticut, 1983), 24.
? lvan R. Feinstein, Clinical Epidemiology: The Architecture of Clinical Research
(Philadelphia, 1985), 409-410. And A.
Bradford Hill [îown Appreciation, roceedings of the Royal Society
of Medicine, 48 (1955), 1010] writes:
(ough conceivably there might have
been a second peak in the curve, and
though almost certainly some more
deaths would have occurred if the pump
handle had remained in situ, it is clear
that the end of the epidemic was not
dramatically determined by its removal.¶ i s ua l an d s tat i s t i c a l t h i n k i n g
At a minimum, removing the pump-handle prevented a recurrence
of cholera. Snow recognized several diYculties in evaluating the eVect
of his intervention; since most people living in central London had fled,
the disease ran out of possible victimsèich happened simultaneously
with shutting down the infected water supply.? he case against the
Broad Street pump, however, was based on a diversity of additional
evidence: the cholera map, studies of unusual instances, comparisons of
the living and dead with their consumption of well water, and an idea
about a mechanism of contamination (a nearby underground sewer had
probably leaked into the infected well). Also, the finding that cholera
was carried by water`life-saving scientific discovery that showed how
to intervene and prevent the spread of cholera%rived not only from
study of the Broad Street epidemic but also from Snowàmappings of
several other cholera outbreaks in relation to the purity of community
water supplies.
? There is no doubt that the mortality
was much diminished, as I said before,
by the flight of the population, which
commenced soon after the outbreak; but
the attacks had so far diminished before
the use of the water was stopped, that it
is impossible to decide whether the well
still contained the cholera poison in an
active state, or whether, from some
cause, the water had become free from
it. now, Cholera, 51-52.
4. Assessment of possible errors in the numbers reported in graphics. SnowÊanalysis attends to the sources and consequences of errors in gathering
the data. In particular, the credibility of the cholera map grows out of
supplemental details in the texts image, word, and number combine
to present the evidence and make the argument. Detailed comments on
possible errors annotate both the map and the table, reassuring readers
about the care and integrity of the statistical detective work that produced the data graphics:
The deaths which occurred during this fatal outbreak of cholera are indicated
in the accompanying map, as far as I could ascertain them. There are necessarily
some deficiencies, for in a few of the instances of persons who died in the hospitals after their removal from the neighbourhood of Broad Street, the number of the house from which they had been removed was not registered. The
address of those who died after their removal to St. JamesàWorkhouse was not
registered; and I was only able to obtain it, in a part of the cases, on application
at the MasteràOYce, for many of the persons were too ill, when admitted, to
give any account of themselves. In the case also of some of the workpeople and
others who contracted the cholera in this neighbourhood, and died in diVerent
parts of London, the precise house from which they had removed is not stated
in the return of deaths. I have heard of some persons who died in the country
shortly after removing from the neighbourhood of Broad Street; and there must,
no doubt, be several cases of this kind that I have not heard of. Indeed, the full
extent of the calamity will probably never be known. The deficiencies I have
mentioned, however, probably do not detract from the correctness of the map
as a diagram of the topography of th