2002 BiasandCausalAssociationsinObse
- (Grimes & Schulz, 2002) ⇒ David A Grimes, and Kenneth F Schulz. (2002). “Bias and Causal Associations in Observational Research." The Lancet, The Lancet, 359(9302). doi:10.1016/S0140-6736(02)07451-2
Subject Headings: Observational Study; Case-Control Study
Notes
Cited By
Quotes
Abstract
Readers of medical literature need to consider two types of validity, internal and external. Internal validity means that the study measured what it set out to; external validity is the ability to generalise from the study to the reader's patients. With respect to internal validity, selection bias, information bias, and confounding are present to some degree in all observational research. Selection bias stems from an absence of comparability between groups being studied. Information bias results from incorrect determination of exposure, outcome, or both. The effect of information bias depends on its type. If information is gathered differently for one group than for another, bias results. By contrast, non-differential misclassification tends to obscure real differences. Confounding is a mixing or blurring of effects: a researcher attempts to relate an exposure to an outcome but actually measures the effect of a third factor (the confounding variable). Confounding can be controlled in several ways: restriction, matching, stratification, and more sophisticated multivariate techniques. If a reader cannot explain away study results on the basis of selection, information, or confounding bias, then chance might be another explanation. Chance should be examined last, however, since these biases can account for highly significant, though bogus results. Differentiation between spurious, indirect, and causal associations can be difficult. Criteria such as temporal sequence, strength and consistency of an association, and evidence of a dose-response effect lend support to a causal link.
Introduction
Clinicians face two important questions as they read medical research: is the report believable, and, if so, is it relevant to my practice? Uncritical acceptance of published research has led to serious errors and squandered resources.1 Here, we will frame these two questions in terms of study validity, describe a simple checklist for readers, and offer some criteria by which to judge reported associations.
Internal and external validity
Analogous to a laboratory test, a study should have internal validity — ie, the ability to measure what it sets out to measure.2 The inference from participants in a study should be accurate. In other words, a research study should avoid bias or systematic error.3 Internal validity is the sine qua non of clinical research; extrapolation of invalid results to the broader population is not only worthless but potentially dangerous.
…
Panel 1: What to look for in observational studies
- Is selection bias present?
In a cohort study, are participants in the exposed and unexposed groups similar in all important respects except for the exposure?
In a case-control study, are cases and controls similar in all important respects except for the disease in question?
- Is information bias present?
In a cohort study, is information about outcome obtained in the same way for those exposed and unexposed?
In a case-control study, is information about exposure gathered in the same way for cases and controls?
- Is confounding present?
Could the results be accounted for by the presence of a factor — eg, age, smoking, sexual behaviour, diet — associated with both the exposure and the outcome but not directly involved in the causal pathway?
- If the results cannot be explained by these three biases, could they be the result of chance?
What are the relative risk or odds ratio and 95% CI?11,12
Is the difference statistically significant, and, if not, did the study have adequate power to find a clinically important difference?13,14
- If the results still cannot be explained away, then (and only then) might the findings be real and worthy of note.
…
In case-control studies, selection bias implies that cases and controls differ importantly aside from the disease in question. Two types of selection bias have earned eponyms: Berkson and Neyman bias. Also known as an admission-rate bias, Berkson bias (or paradox) results from differential rates of hospital admission for cases and controls. Berkson initially thought that this phenomenon was due to presence of a simultaneous disease.5 Alternatively, knowledge of the exposure of interest might lead to an increased rate of admission to hospital. For example, doctors who care for women with salpingitis were more likely to recommend hospital admission for those using an intrauterine device (IUD) than for those using a hormonal method of contraception.18,19 In a hospital-based case-control study, this would stack the deck (or gynaecology ward) with a high proportion of IUD-exposed cases, spuriously increasing the odds ratio.
…
Information bias
- Has information been gathered in the same way?
Information bias, also known as observation, classification, or measurement bias, results from incorrect determination of exposure or outcome, or both. In a cohort study or randomised controlled trial, information about outcomes should be obtained the same way for those exposed and unexposed. In a case-control study, information about exposure should be gathered in the same way for cases and controls.
…
Confounding
…
Control for confounding
…
- Matching
Another way to control for confounding is pairwise matching. In a case-control study in which smoking is deemed a confounding factor, cases and controls can be matched by smoking status. For each case who smokes, a control who smokes is found. This approach, although often used by investigators, has two drawbacks. If matching is done on several potential confounding factors, the recruitment process can be cumbersome, and, by definition, one cannot examine the effect of a matched variable.28
- Stratification
Investigators can also control for confounding after a study has been completed. One approach is stratification. Stratification can be considered a form of post hoc restriction, done during the analysis rather than during the accrual phase of a study. For example, results can be stratified by levels of the confounding factor. In the smoking example, results are calculated separately for smokers and non-smokers to see if the same effect arises independent of smoking. The Mantel-Haenszel procedure38 combines the various strata into a summary statistic that describes the effect. The strata are weighted inversely to their variance — ie, strata with larger numbers count more than those with smaller numbers. If the Mantel-Haenszel adjusted effect differs substantially from the crude effect, then confounding is deemed present. In this instance, the adjusted estimate of effect is considered the better estimate to use.
…
…
Conclusion
Studies need to have both internal and external validity: the results should be both correct and capable of extrapolation to the population. A simple checklist for bias (selection, information, and confounding) then chance can help readers decipher research reports. When a statistical association appears in research, guidelines for judgment of associations can help a reader decide whether the association is bogus, indirect, or real.
References
,
| Author | volume | Date Value | title | type | journal | titleUrl | doi | note | year | |
|---|---|---|---|---|---|---|---|---|---|---|
| 2002 BiasandCausalAssociationsinObse | David A Grimes Kenneth F Schulz | Bias and Causal Associations in Observational Research | 10.1016/S0140-6736(02)07451-2 | 2002 |