The source for information about the future of any particular patient is the collective experience of previous patients with the same condition. The accuracy of the information so derived depends on the manner in which that experience is collected and recorded and on the degree of similarity between the patient at hand and past patients who have been followed over time.

Theoretically, the best mechanism for studying prognosis would be to characterize patients carefully at the time of diagnosis (or when a risk factor is identified) with regard to disease stage and severity, presence or absence of any comorbid conditions, and other factors that could be expected to have an impact on outcome. Because such factors are often systematically influenced by the pattern of patient referral, the setting in which patients are seen and the manner in which they happened to be there would be described. All patients would be examined with the same level of scrutiny at the time they entered the cohort and during subsequent follow-up examinations. Relevant outcomes, and criteria by which they would be measured, would be specified in advance. Those conducting follow-up examinations would be unaware of baseline differences among patients so as not to be influenced by expected associations between these variables and outcomes. All patients would be followed, and their status with regard to outcomes would be known at the time the experience was analyzed. The impact of different baseline characteristics on relevant outcomes would be examined by reporting experience for different subgroups or developing statistical models. The predictive validity of these models would be tested in separate samples of patients.

Rarely, if ever, is it possible to meet all of these methodologic objectives. As a result, much of the research that clinicians rely on for information about risk and prognosis is potentially misleading. To avoid being misled and misleading patients, clinicians must
understand the biases that can be introduced when suboptimal methods are used to gather information to help predict the future.

When the outcomes of interest are rare events, it may not be feasible to assemble a cohort large enough, and follow it long enough, to accumulate sufficient experience to provide useful estimates of prognosis or risk. Alternatively, patients who have already experienced the outcome can be identified as cases, and their past histories can be examined to identify events or exposures that may have conferred risk and may be of prognostic value. Control patients without the outcome of interest can also be questioned about the same exposures. Comparison of the rates of exposure among cases and controls can produce an estimate of the degree of risk associated with the exposure. The odds ratio is an estimate of the relative risk that is very accurate when the disease or outcome in question is rare. This retrospective case-control approach places a heavy burden on the investigator and sources of information to ensure that similar degrees of scrutiny are applied to the histories of cases and controls. Selective recollection of the past, often with greater vigilance stimulated by the outcome of interest, can produce misleading estimates of risk and prognosis.

Even if patients with a particular risk factor or diagnosis are identified prospectively and followed forward in time, biases that can lead to faulty conclusions may be introduced. Perhaps the most important bias for primary care physicians to recognize has been termed referral filter bias. It occurs frequently as a result of the fact that patients in many published reports have been described because they have been referred to academic centers. Such patients often have complicating characteristics and exhibit a worse prognosis than do patients who are drawn from an entire population or a representative sample of a population. Similar problems arise when patients are selected for study based on particular test results. Patients with more-worrisome signs and symptoms may be more likely to be tested and more likely to fare poorly over time. Alternatively, patients who are tested may have better access to medical care and fare better than average as a result.

Differences in the ways patients are followed over time can also introduce important biases. Patients lost to follow-up may be different from those who remain in the cohort. A conservative approach to estimating prognosis when some patients are lost to follow-up before the relevant outcome has occurred is to assume that all lost patients experienced the outcome and then, in contrast, to assume that no lost patients experienced it. The first assumption produces an upper-bound estimate, with lost patients included in both numerator and denominator; the second produces a lowerbound estimate, with lost patients included only in the denominator.

Even when patients are successfully followed over time, biases can be introduced by the selective use of tests and other outcome measures or by the expectations of clinicians who are aware of patient characteristics that may or may not have real prognostic significance. Statistical models that have not been validated on independent samples may mistake random variations among characteristics and outcomes for important prognostic associations.