Bias & Base Rates

Bias in Decision Making

The Scientific Method begins with systematic observation. Knowing this, scientists focus attention on methods of data collection, and types of data. Across methodologies, from forensic analysis of fingerprinting to applied psychology, examination of methodologies is the basis of scientific peer review.

Despite this reality, disputes about biased interpretation of data drive reviews of scientific works. However, bias is not restricted to interpretation, but impacts every aspect of scientific observation and methodology.

Bias inherent in the study can impact the data, reference materials, and context of the investigation; for example, selection bias. Bias in the specific person, education, and training of the scientist can guide observations and data collection. These individual biases are in addition to general human biases, such as Confirmation Bias, which challenge the application of scientific methodologies. Finally, biases related to interpretation, such as unreliable base rates, can influence data interpretation.

add description

Base Rates

Debiasing methods are obvious once the scope of the challenges is evident. Simply by pre-determining what methods will be used and which data will be collected, scientists can reduce the impact of bias. This is one reason why scientists have adopted the standard of transparency in identifying their methods related to the purpose of the research. This is true even when the focus of the investigation is observational, rather than a design for testing competing hypotheses. Standards for observation methods are routinely identified in advance and described in detail to provide a foundation for the investigation as a whole. Not every scientific study will be about testing hypotheses, but every scientific study will be about identifying and categorizing observations.

Base Rates are the naturally occurring frequency of a phenomenon in a population. Before observations become categories, scientists must consider how common their categories are, or they risk assuming they have found rubies when they have ruble. Ignoring base rates gives rise to the base-rate fallacy, and this is at the heart of biases people encounter every day about age, ethnicity, gender, and socioeconomic status. If assuming makes a fool of you and me, it begins when we think we can make up probabilities as we go along in life. Studying base rates is equivalent to checking your bank balance before a spending spree. Without recognition of base rates, personal prejudices may ruin the credit of the most experienced scientist.

Dror, I. (2020). Cognitive and human factors in expert decision making: Six fallacies and eight sources of bias. Analytical Chemistry, 92, 7998-8004.

APA (2013, 2015). APA Dictionaries of Clinical Psychology & Psychology (G.R. VanderBos, ed)(2nd eds.). APA: Washington, D.C.