Modules

Statistics & Pandemics

This module will explore a method for reconstructing and evaluating standard statistical arguments.  Simply put, predictive statistical arguments use premises describing observed phenomena to draw conclusions about phenomena that haven’t been observed.  This module will focus on two standard patterns of argument and common objections to each.  In Session 1, we will focus on statistical survey arguments.  In Session 2, we will focus on correlation arguments.

Welcome to Statistics and Pandemics. This module will explore a method for reconstructing and evaluating standard statistical arguments.  This module will focus on two standard patterns of argument and common objections to each.  In Session 1, we will focus on statistical survey arguments.  In Session 2, we will focus on correlation arguments.

Although statistics courses may have a bad reputation, we avoid them to our detriment.  Since March 2020, we’ve surely learned there’s no escape from statistical arguments, both good and bad.  As an introduction to this module, consider the examples of statistical reasoning on display in this article: A cognitive scientist explains Covid-19 death statistics, faulty causal reasoning, and the CDC’s 6%.

By the end of this module, you will be able to:

  • Reconstruct and evaluate Statistical and Correlation Standard Form Arguments (following the patterns explained below)
  • Apply this skill to the arguments we have encountered during the Covid-19 Pandemic.
  • Beyond the Pandemic — Broader Applications: Statistical arguments are everywhere.  For example, you encounter this type of argument in attempts to predict the results of sports games, elections, or the future state of the economy.  Learning how to evaluate sample size, margin of error, and how to understand what it means to say that “two factors are positively correlated” is essential for engaging in a wide range of important live debates in our world.  Furthermore, these skills will help you navigate many personal decisions requiring interpretation of statistical data (i.e. making treatment decisions after diagnostic testing).  So, this is a module for the COVID-19 pandemic, but it has significant applications beyond evaluating pandemic arguments.

By the end of this module, you will become familiar with reconstructing and evaluating arguments in the following two forms:

Standard Statistical Argument:

Background information: Description of sample: who or what was sampled, when, where, what they were asked or how they were observed, etc.

  1. Result of Sample: x percent of the sample population has the measured property.
  2. Accuracy Premise: if x percent of the sample population has the measured property, then x percent of the sample population has the target property.
  3. Conclusion about Sample: x percent of the sample population has the target property.
  4. Representativeness Premise: if x percent of the sample population has the target property, then approximately x percent of the target population has the target property.
  5. Final Conclusion: approximately x percent of the target population has the target property.

Standard Correlation Argument:

Background information: Description of survey, including who or what was sampled, when, where, and how.

  1. Results Obtained: results can be expressed as several statistical statements reporting the rate at which the measured property was found in the two groups studied in the sample.
  2. Measured Correlation in Sample: a statement of the correlation between the measured factors in the sample population.
  3. Accuracy Premise: this says that the factors measured accurately measure the target properties.
  4. Target Correlation in Sample: This says that the target factors are correlated in the sample.
  5. Representativeness Premise: this says that the sample populations are representative of the target populations.
  6. Final Conclusion: the target factors are (positively or negatively) correlated in the overall target population.

 

  1. Reading: Richard Feldman, Reason and Argument, Pearson New International Edition Page 260-286
  2. Reading: Jonathan Fuller, “What’s Missing in Pandemic Models,” Nautilus
  3. Discussion: Reconstruction + Evaluation Practice

This discussion is best done in class in teams for 3-4.  I recommend giving the two short essays to the students in advance (along with these instructions) so that class time is spent reconstructing the argument from Essay 1 and evaluating the objections from Essay 2.  If teaching online, you can create this discussion as a team discussion assignment.

Step 1: Read “Essay 1” below.  Reconstruct the statistical argument based on the survey described in Essay 1.

Step 2: Read “Essay 2” below.  Formulate precisely the objections that are raised to the statistical argument from Essay 1.  Evaluate the strength of the objections.

Essay 1: One in Five Polled Voices Doubt on Holocaust

A poll released today found that 22 percent of adults and 20 percent of high school students who were surveyed said they thought it was possible that the Holocaust, Nazi Germany’s extermination of six million Jews, never happened.  In addition to the 22 percent of adult respondents to the survey by the Roper Organization who said it seemed possible that the Holocaust never happened, 12 percent more said they did not know if it was possible or impossible, according to the survey’s sponsor, the American Jewish Committee.  The findings shocked Holocaust survivors, some of whom had devoted much of their lives to keeping alive the memory of the systematic extermination of Jews in World War II.  Roper interviewed 592 adults from November 14 to November 21 and 506 high school students from October 19 to October 30. All were asked, “Does it seem possible, or does it seem impossible to you, that the Nazi extermination of the Jews never happened?”  Sixty-five percent of adults and 63 percent of high school students said it was impossible to believe that the Holocaust never happened. Twelve percent of the adults and 17 percent of the high school students said they did not know. The margin of sampling error was plus or minus four percentage points for the adult survey and plus or minus five percentage points for the survey of students.  “What have we done?” asked a stunned Elie Wiesel, the Nobel laureate who chronicled his experiences at the Auschwitz and Buchenwald concentration camps. “We have been working for years and years. I am shocked that 22 percent—oh, my God.”. . .  The survey also found that 72 percent of adults and 64 percent of high school students said it was essential or very important for all Americans to know about and understand what was done to the Jews by the Nazis.  In addition, 63 percent of adult respondents and 54 percent of high school respondents rejected the idea that 50 years had erased the relevance of the Holocaust.

Source: “One in Five Polled Voices Doubt on Holocaust,” New York Times, April 20, 1993, p. A12. © Associated Press.

Essay 2: Putting a Ready Check on “Holocaust Denial”

Do Americans really doubt the Holocaust occurred?  A now-famous Roper poll shocked almost everyone last year with the answer it got to the question, “Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened?”  Twenty-two percent said “it seemed possible.” Another 12% “didn’t know.” Thus, nearly a third of the country either didn’t know if it happened or believed it was possible it didn’t. Only 65 percent believe it was “impossible it never happened.”  The poll was cited by major newspapers, including the New York Times and the Washington Post as evidence of growing anti-Semitism and Holocaust denial. “One in five people in the United States is open to the idea that the Holocaust is a myth,” declared USA Today.  “Figure one out of five Americans could be willfully stupid,” opined the San Francisco Examiner.  Not quite.  The poll didn’t say either of those things. And the truth is, Americans know the Holocaust was no myth.  Gallup decided to test the Roper question by asking it differently.  They simplified the convoluted language of the original question by eliminating the double negative and the forced choice of two extremes.  Their question: “The term Holocaust usually refers to the killing of millions of Jews in Nazi death camps during World War II. In your opinion, did the Holocaust definitely happen, probably happen, probably not happen, or definitely not happen?”  The results: 83% said the Holocaust definitely happened, 13% said it probably happened, 2% said probably did not, 1% said definitely not, 1% had no opinion.  What that means is 96% believe the Holocaust happened; 3% do not.  In a separate sample, Gallup asked the Roper question again. This time, 37% said it was possible it never happened—15 points higher than the first time it was asked. Why does Roper’s question seem to find more doubters?  Beyond simple confusion caused by the awkward sentence, the problem may lie in the words “impossible” and “never.” Despite what people believe, they are reluctant to use such absolute terms. There is a weary skepticism that seems to tell us anything is possible—whether or not we strongly doubt it. Or, as the adage states, “The older I get, the less I say ‘always’ and ‘never.’ ”  Finally, notice that the word “Holocaust” never appears in Roper’s question. Instead they use “extermination of the Jews.” Extermination implies “complete extinction.” Jews are not extinct. Maybe Americans are smarter than the media seem to believe. Maybe they simply understand the subtleties of language better than the pollsters and journalists who misuse it daily.  Whatever the case, Americans are not “willfully stupid.” Virtually all of them realize the Holocaust is historical fact.

Source: Joe Urschel, “Putting a Reality Check on ‘Holocaust Denial’,” USA Today, January 13, 1994. Copyright© 1994, USA Today.

 

 

  1. Reading: Richard Feldman, Reason and Argument, Pearson New International Edition Page 286-300
  2. Discussion: Correlation / Causation Confusion

Consider the following article discussing an example of correlation/causation confusion.  Briefly summarize the alleged error and give your evaluation.

Link to article: https://www.propublica.org/article/state-coronavirus-data-doesnt-support-trumps-misleading-testing-claims

 

Consider a variety of publications about how statistical arguments have been (mis)used during the pandemic.

We are suggesting a few options below (but you are welcome to investigate others):

Instructions:

  1. Give two examples from the articles above (or discovered by your team) that illustrate a concept you learned in textbook Chapter 9 (Statistical Arguments and Predictions). 1-2 paragraphs is sufficient to present each example.
  2. Reconstruct in standard form one statistical argument from one article.  Briefly comment on whether you think the argument is strong or not.  As our textbook author says, it is acceptable to be uncertain.  If you don’t think you have enough information to evaluate the argument as strong or weak (and obviously there’s a spectrum here) say so and explain why.  State what further facts your final evaluation would depend on.

Instructor