Surprise vs. Probability as a Metric for Proof
Cheng, Edward K.
In this Symposium issue celebrating his career, Professor Michael Risinger in Leveraging Surprise proposes using "the fundamental emotion of surprise" as a way of measuring belief for purposes of legal proof. More specifically, Professor Risinger argues that we should not conceive of the burden of proof in terms of probabilities such as 51%, 95%, or even "beyond a reasonable doubt." Rather, the legal system should reference the threshold using "words of estimative surprise" -asking jurors how surprised they would be if the fact in question were not true. Toward this goal (and being averse to cardinality), he suggests categories such as "mildly surprised, surprised, quite surprised, greatly surprised, astonished, shocked, etc." We find Professor Risinger's proposal intriguing. After all, one can imagine important theoretical reasons why surprise might generate different results from probability. To the extent that the surprise formulation is unfamiliar, it might cause jurors to think holistically ("System 1") as opposed to attempting to use rules (often misremembered or misapplied) about probability ("System 2"). Surprise might be easier to approach qualitatively, unlike probability, which cries out for quantitative calculation and invokes the fear of numbers for some. Surprise is also notably framed in the negative ("How surprised would you be if the fact were not true?") compared to its probability counterpart ("What is the probability that the fact is true?"). Being empiricists, we thus could not help but put Professor Risinger's worthy proposal to the test, if only in a preliminary way. Just what might conceptualizing evidence under a framework of "surprise" look like and get us in practice? Here, we report on a simple experiment where potential jurors were recruited to evaluate evidence using both a surprise framework and a framework using probabilistic language. Using the experiment, we set out to answer two questions: First, does using degrees of "surprise" actually produce intelligible and consistent results among laypersons? And second, does using degrees of "surprise" to evaluate evidence produce results that provide a material benefit, either empirically or normatively, when compared to frameworks based on probability?