Meta-analysis of clinical trials promises the ability to combine the results from several small studies to provide findings comparable to those of large, expensive clinical trial. While this can be a powerful technique, it can also provide misleading information when bias is present. Recognition of the sources of bias and ways to avoid it can reduce the chance for error.
Analogy: An array of separate telescopes, the output from which can be combined to give an image comparable to an extremely large telescope. This requires that all of the telescopes be working properly and that there is a method to combine the output reliably.
Types of bias (after Egger, 1997):
(1) selection of studies for inclusion in the meta-analysis
(2) true heterogeneity between the studies
(3) data-handling bias
(4) artifactual = choice of effect measure
|
Selection Bias |
Descriptions |
|
publication bias |
studies with positive results are more likely to be published than those with negative results |
|
journal prestige bias |
studies in a prestigious journal tend to be assigned greater weight than a study from a nonprestigious journal |
|
language bias |
English-only or failure to include some languages or tendency to exclude studies from certain counties |
|
citation bias |
studies with positive results are more likely to be cited than those with negative results |
|
multiple publication bias |
studies with positive results are more likely to be noticed and published repeatedly than those with negative results |
|
investigator bias |
studies known to support a hypothesis are included while studies that do not support it are excluded |
|
database bias |
studies not listed in Medline or other citation database may not be retrieved |
|
indexing bias |
how a study is indexed can help determine if a search will retrieve it |
|
inconsistent application of criteria bias |
inconsistent application of selection criteria resulting in similar studies being handled differently |
|
True Heterogeneity Between Studies |
Descriptions |
|
severity of disease bias |
differences in disease severity in the study patients |
|
performance bias |
differences in type and degree of therapeutic interventions |
|
associated risk bias |
number of additional risk factors present |
|
Data Handling Bias |
Descriptions |
|
allocation bias |
bias in assignment of patients to the different groups |
|
inclusion and exclusion |
bias in choice of inclusion and exclusion criteria |
|
attrition bias |
biases in occurrence and handling of deviations from the protocol and loss to followup |
|
analysis bias |
improper statistical method chosen or method poorly done |
|
detection bias |
biased assessment of outcome |
|
fraud |
trying to make results more positive or negative for personal gain |
|
provision of data (data availability) bias |
access to study findings needed for the meta-analysis may be limited |
Ways suggested to reduce bias:
(1) funnel plots to detect asymmetry caused by bias (Egger, 1997)
(2) removal of one study at the time with re-analysis, looking for major shifts in results
(3) registering studies in a study register before results are known (this reduces the tendency to under-report negative studies)
Ways that have proved suboptimal in reducing bias:
(1) quality scores: These may provide a false sense of confidence in the included studies. Analysis of the score components may be more useful than an analysis of the composite score.
