HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Figures Only Full Text (PDF) Services Related articles in JDR Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (1) Citing Articles via Google Scholar Google Scholar Articles by Moles, D.R. Articles by Lau, J. Search for Related Content PubMed PubMed Citation Articles by Moles, D.R. Articles by Lau, J.

Introduction to Cumulative Meta-analysis in Dentistry: Lessons Learned from Undertaking a Cumulative Meta-analysis in Periodontology

¹ Eastman Dental Institute for Oral Health Care Science, University College London, 256 Gray’s Inn Road, London, WC1X 8LD, UK;
² dsm-Forsyth Center for Evidence-Based Dentistry, The Forsyth Institute, Boston, MA, USA, and Department of Health Policy and Health Services Research, Goldman School of Dental Medicine, Boston University, Boston, MA, USA; and
³ Center for Clinical Evidence Synthesis, Tufts-New England Medical Center, Boston, MA, USA;

^* corresponding author, d.moles{at}eastman.ucl.ac.uk

	ABSTRACT

TOP ABSTRACT INTRODUCTION MATERIALS & METHODS RESULTS DISCUSSION REFERENCES

 
Improving health and well-being from the consideration of isolated studies is problematic. Systematic reviews have been developed to address this problem and may include a quantitative data synthesis in the form of a meta-analysis, or a cumulative meta-analysis. The value of systematic reviews depends greatly on the availability and quality of the results of primary research. The objective of the current project was to demonstrate the technique of cumulative meta-analysis in dentistry using data from a previously published systematic review. The process highlights an issue that some trials could not be synthesized due to the lack of reporting of measures of variation. This represents a potential source of bias. Investigators are encouraged to consider their trials as part of an information continuum and to report sufficient detail to permit the trials’ incorporation into subsequent syntheses.

KEY WORDS: systematic review • clinical trials • periodontal diseases • meta-analysis

	INTRODUCTION

TOP ABSTRACT INTRODUCTION MATERIALS & METHODS RESULTS DISCUSSION REFERENCES

 
Improving health and well-being from the consideration of studies isolated from each other is problematic. Systematic reviews can address this problem and aid decision-making. A systematic review is a review of a clearly formulated question that attempts to minimize bias using systematic and explicit methods to identify, select, critically appraise, and summarize relevant research. Once the available data about a particular topic have been systematically collected and quality-appraised, the information is pooled for analysis. Most commonly, this will take the form of a series of ‘evidence tables’ that summarize the findings from the studies included in the review. Systematic reviews may also include meta-analyses, the statistical technique of quantitatively synthesizing the review data to give an overall estimate of the effect size and its precision (Egger et al., 2001). The standard approach for undertaking a meta-analysis is to pool all available suitable data simultaneously at a single point in time. This provides a single estimate of the effect size of interest, and an associated confidence interval as a measure of the imprecision of the point estimate. Complementary to this is the technique of cumulative meta-analysis (Lau et al., 1992). Cumulative meta-analysis is the product of performing a new meta-analysis every time a new piece of evidence emerges (Lau et al., 1995). This permits evaluation of the additional contributions made by individual studies to the cumulatively pooled results of the preceding studies.

Examples of the value of these techniques can be found in the medical literature. One such example consists of 33 sequential trials, between 1959 and 1988, examining the use of intravenous streptokinase for myocardial infarction, involving the enrollment of 36,974 patients. Had a cumulative meta-analysis been performed, a consistent statistically significant benefit could have been demonstrated as early as 1973 after only 2432 patients were enrolled (Lau et al., 1992). Cumulative meta-analyses have also been used to monitor the discrepancy between the emergence of sufficient evidence of the effectiveness of interventions and the delay in the recommendation of their adoption by clinical experts. Similarly, the delay before recommending the cessation of harmful or ineffective interventions has also been examined by this approach (Antman et al., 1992).

For meta-analysis to be carried out, primary research must be reported in sufficient detail for subsequent analysis. We have reviewed the reporting of randomized control trials in periodontology, and found that it was not always adequate (Montenegro et al., 2002) and does not always follow the CONSORT guidelines (Begg et al., 1996). Our previous research focused solely on the quality of the reporting of methodological aspects of randomized controlled trials and refrained from a consideration of the quality of reporting of the actual results of the trials.

The aim of the current research was to illustrate the potential value of the cumulative meta-analysis approach in dentistry. However, the topic chosen as the subject for the cumulative meta-analysis also illustrates some of the difficulties and potential biases that may arise when results of trials are not reported in sufficient detail.

	MATERIALS & METHODS

TOP ABSTRACT INTRODUCTION MATERIALS & METHODS RESULTS DISCUSSION REFERENCES

 
The starting point for the current investigation was the systematic review of the effectiveness of adjunctive systemic antimicrobials for the treatment of periodontitis undertaken by the European Federation of Periodontology (Herrera et al., 2002). The focused question of that systematic review was, "In patients with chronic or aggressive periodontitis, what is the effect of systemic antimicrobials as an adjunct to scaling and root planing (SRP+AB) compared with SRP alone, in terms of clinical outcomes?" The review was restricted to randomized-controlled trials (RCT) or controlled clinical trials (CCT) of at least 6 months’ duration, and the reviewers identified 25 articles that fulfilled their inclusion criteria. The primary outcome measures reported in the review were differences in mean change in clinical attachment level (CAL) between test and control groups, and differences in mean change in probing pocket depths (PPD).

For the purpose of the current exercise, the analyses are restricted to changes in CAL, and further restricted to exclude refractory/aggressive/rapidly progressive periodontal diseases or studies in which the antimicrobial was used at sub-therapeutic levels. Thus, the outcome of interest is the difference in mean CAL change between control (SRP) and test (SRP+AB) groups, such that a positive difference favors the antimicrobial therapy while a negative result favors the control.

The authors of the original systematic review supplied copies of the articles that fulfilled their inclusion criteria. They marked these with highlighter pen to indicate the data that they abstracted. They also provided access to their original computer spreadsheets containing the abstracted data. These were verified and supplemented by the addition of standard deviations (SD) for change in CAL, where these had not been reported in the original articles but where there was sufficient information to calculate them directly from standard errors (SE) and sample sizes, or where it was possible to impute them.

Conventional and cumulative meta-analyses were undertaken with the use of both fixed- and random-effects models, with the ‘Meta-View’ software incorporated within the Cochrane ‘Review Manager’ program (Cochrane Collaboration, 2003). This was achieved by use of an iterative approach, in which a series of separate meta-analyses was undertaken to include all the studies available at each information step. Thus, for each occasion that information became available, the new information was incorporated into the existing dataset, and the meta-analysis was repeated to update the estimates. This was undertaken separately for patient-level means of difference in CAL change in:

1. all/moderate pockets at baseline (where moderate is defined as having a PPD 4–6 mm); and
   
2. deep pockets at baseline (PPD > 6 mm).
   

For trials in which a single control group was compared with multiple test arms (e g., several different antibiotic regimes), the test arms were combined into a single intervention group with summary weighted means and SDs. This was undertaken to avoid double-counting the control patients, which would have artificially inflated the sample size.

Studies cannot be incorporated into the meta-analyses if there is no information on the study variability (SD or SE). This lack of information represents a possible source of bias. We investigated the potential for bias graphically by plotting the point estimates from these studies along with the means and 95% confidence intervals from those studies in which the information was available, to see if the point estimates were consistent with the overall pattern.

	RESULTS

TOP ABSTRACT INTRODUCTION MATERIALS & METHODS RESULTS DISCUSSION REFERENCES

 
All/Moderate Sites
Only eight of 25 trials contained sufficient information to be incorporated into meta-analyses of the responses of all/moderate sites to adjunctive systemic antibiotics (Lindhe et al., 1983b; Magnusson et al., 1994; Berglundh et al., 1998; Flemmig et al., 1998; Palmer et al., 1998; Winkel et al., 1999; Caton et al., 2000; Sigusch et al., 2001). However, of these, two trials were based on patients with rapidly progressive disease (Magnusson et al., 1994; Sigusch et al., 2001), and one used an antimicrobial at sub-therapeutic doses (Caton et al., 2000). These three trials were therefore excluded from the meta-analyses. The remaining five trials investigated the effectiveness of a variety of antibiotic systems: long-term tetracycline (Lindhe et al., 1983b); Augmentin^® (Magnusson et al., 1994; Winkel et al., 1999); metronidazole (Palmer et al., 1998); and metronidazole and amoxicillin in combination (Berglundh et al., 1998; Flemmig et al., 1998). The forest plots for the conventional and cumulative fixed-effects meta-analyses are shown in Fig. 1.

View larger version (13K): [in this window] [in a new window]   Figure 1. Conventional and Cumulative Meta-analyses of 5 Trials of Systemic Antimicrobials as an Adjunct to Scaling and Root Planing for All/Moderate Periodontal Pockets: Difference in Mean Gain in Clinical Attachment Level and 95% CI (SRP + antimicrobial vs. SRP alone) in mm. Key to Figs.: Aug, Augmentin®; Clin, Clindamycin; Dox, Doxycyline; Met, Metronidazole; Met+Amo, Metronidazole and Amoxicillin in combination; Rod, Rodogyl® (Metronidazole and Spiramycin in combination); Spi, Spiramycin; Tet, Tetracycline; and Tet/Spi, Tetracycline or Spiramycin.

Deep Sites
Seven trials contained sufficient information to be included in meta-analyses for deep sites (Lindhe et al., 1983b; Al Joburi et al., 1989; Bain et al., 1994; Berglundh et al., 1998; Flemmig et al., 1998; Caton et al., 2000; Sigusch et al., 2001). One trial in isolation considered the effect of therapy on patients with rapidly progressive disease (Sigusch et al., 2001), and one utilized sub-therapeutic doses (Caton et al., 2000). These were not combined with the other trials. The forest plots for the conventional and cumulative meta-analyses are shown in Fig. 2.

View larger version (14K): [in this window] [in a new window]   Figure 2. Conventional and Cumulative Meta-analyses of 5 Trials of Systemic Antimicrobials as an Adjunct to Scaling and Root Planing for Deep Periodontal Pockets (> = 6 mm at baseline): Difference in Mean Gain in Clinical Attachment Level and 95% CI (SRP + antimicrobial vs. SRP alone) in mm (key as for Fig. 1).

Potential Bias from the Inability to Include Studies that Reported Only Point Estimates of Effect Size
Fig. 3 shows all the available effect size estimates from the trials, with 95% confidence intervals displayed where they could be calculated. In their absence, a single point estimate is indicated. For the studies on all/moderate sites, there is 1 point estimate indicating a result favoring the control group [Al Joburi (Spi)], and 2 point estimates indicating virtually no difference between test and control [Chin Quee (Rod) and Al Joburi (Tet)], while the remaining 3 all favor the antimicrobial groups. For the trials reporting results from deep sites, 1 point estimate favors the control group, while the other favors the antimicrobial group. However, the magnitude of the point estimate favoring antimicrobial adjunctive therapy is more than double that of the estimate favoring the control (Fig. 3).

View larger version (13K): [in this window] [in a new window]   Figure 3. Results of Trials of Systemic Antimicrobials as an Adjunct to Scaling and Root Planing: Point Estimates of Differences in Mean Gain in Clinical Attachment Level and 95% CI Where Available (SRP + antimicrobial vs. SRP alone) in mm (key as for Fig. 1). Note: *Different antimicrobial sub-studies that share control groups. In each case, the total number of patients for the study is listed once only, to avoid double-counting the controls. ?Not reported or unclear.

	DISCUSSION

TOP ABSTRACT INTRODUCTION MATERIALS & METHODS RESULTS DISCUSSION REFERENCES

In this demonstration exercise, pooling of the available information, whether by cumulative or conventional meta-analyses, indicates a small statistically significant benefit in terms of clinical attachment gain for scaling and root planing with adjunctive systemic antibiotics over and above the gain obtained from scaling and root planing alone. However, it is not appropriate to use the limited results of the current investigation as a basis for making recommendations about the adjunctive use of systemic antimicrobials in periodontal therapy. Not all the relevant trials provided sufficient information for incorporation within the meta-analyses, and the results may be subject to bias. The magnitude of the benefit is small and difficult to place in clinical context. Further, this research made no attempt to assess the negative consequences of such therapies (e g., the potential development of antibiotic-resistant strains of bacteria).

When interpreting meta-analyses, whether conventional or cumulative, it is important that one bear in mind the potential biases that may affect the results. The phenomenon of publication bias is well-documented in the literature, and this occurs when trials are published selectively, such that the published studies are not representative of all the studies that have been undertaken. For example, results that are negative or of small magnitude may be less likely to be submitted and/or accepted for publication. This has been the subject of considerable interest and debate in the medical literature, and techniques have been developed that aim to detect when this may have occurred (Egger et al., 1997; Sutton et al., 2000). However, in the absence of a true reference standard, proposed methods to detect or correct for publication bias remain controversial (Thornton and Lee, 2000).

We previously found that many authors of periodontal trials did not publish sufficient details to allow for a determination of whether their trials were randomized or not, or whether the randomization that was performed was adequate (Montenegro et al., 2002). The current exercise highlights another potential source of bias, in that not all of the trials identified in the review of Herrera et al.(2002) reported measures of the variation in the data (standard deviations, standard errors, or confidence intervals). As such, these trials could not be included in meta-analyses (Fig. 3), and the information is effectively lost to any attempt at quantitative synthesis.

These foregoing considerations have important consequences for the way that clinical trials are designed and reported in dentistry. It is common for there to be multiple randomized controlled trials over time, all addressing the same or similar clinical questions. Thus, each individual trial should be viewed as forming part of a continuum and should not be designed, conducted, analyzed, and interpreted in isolation. The lessons from previous trials are enormously valuable in the design of subsequent trials, in addition to contributing to the overall cumulative knowledge base. However, a review of 26 reports of randomized trials in 5 prestigious medical journals found that, in 19 articles, the authors made no systematic attempt to set their trials’ results within the context of previous trials (Clarke and Chalmers, 1998).

We would recommend the following for the design and reporting of clinical trials in dentistry:

1. Trials should use the results of earlier trials to ensure that they are adequately powerful to address the proposed question. Results should be viewed and reported as a continuum and be pooled as part of an ongoing process.
   
2. Trials should report the extent of sampling variation along with point estimates. This will allow their results to be incorporated into future meta-analyses.
   

	ACKNOWLEDGMENTS

 
The authors are grateful to David Herrera for providing access to the material used in the EFP systematic review. Salary support was provided by University College London (DM and IN), DE13850 and dsm (RN). DM was the recipient of a dsm-Forsyth Center for Evidence-Based Dentistry fellowship, Dental Service of Massachusetts, Inc.

Received April 16, 2004; Last revision December 23, 2004; Accepted January 13, 2005

	REFERENCES

TOP ABSTRACT INTRODUCTION MATERIALS & METHODS RESULTS DISCUSSION REFERENCES

 
Al Joburi W, Quee TC, Lautar C, Iugovaz I, Bourgouin J, Delorme F, et al. (1989). Effects of adjunctive treatment of periodontitis with tetracycline and spiramycin. J Periodontol 60:533–539.[Medline]

Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC (1992). A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. J Am Med Assoc 268:240–248.[Abstract]

Bain CA, Beagrie GS, Bourgoin J, Delorme F, Holthuis A, Landry RG, et al. (1994). The effects of spiramycin and/or scaling on advanced periodontitis in humans. J Can Dent Assoc 60:209, 212–217.

Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. (1996). Improving the quality of reporting of randomized controlled trials: the CONSORT statement. J Am Med Assoc 276:637–639.[ISI][Medline]

Berglundh T, Krok L, Liljenberg B, Westfelt E, Serino G, Lindhe J (1998). The use of metronidazole and amoxicillin in the treatment of advanced periodontal disease. A prospective, controlled clinical trial. J Clin Periodontol 25:354–362.[Medline]

Caton JG, Ciancio SG, Blieden TM, Bradshaw M, Crout RJ, Hefti AF, et al. (2000). Treatment with subantimicrobial dose doxycycline improves the efficacy of scaling and root planing in patients with adult periodontitis. J Periodontol 71:521–532.[ISI][Medline]

Clarke M, Chalmers I (1998). Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? J Am Med Assoc 280:280–282.[Abstract/Free Full Text]

Cochrane Collaboration, The (2003). Review manager [4.2.2]. Available online http://www.cochrane.org/resources/handbook/ index.htm. Accessed November 15, 2004.

Egger M, Smith GD, Schneider M, Minder C (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ 315:629–634.[Abstract/Free Full Text]

Egger M, Davey-Smith G, O’Rourke K (2001). Rationale, potentials, and promise of systematic reviews. In: Systematic reviews in health care: meta-analysis in context. 2nd ed. Egger M, Davey-Smith G, Altman DG, editors. London: BMJ Publishing Group, pp. 3–19.

Flemmig TF, Millián E, Karch H, Klaiber B (1998). Differential clinical treatment outcome after systemic metronidazole and amoxicillin in patients harboring Actinobacillus actinomycetemcomitans and/or Porphyromonas gingivalis. J Clin Periodontol 25:380–387.[Medline]

Herrera D, Sanz M, Jepsen S, Needleman I, Roldán S (2002). A systematic review on the effect of systemic antimicrobials as an adjunct to scaling and root planing in periodontitis patients. J Clin Periodontol 29(Suppl 3):136–159; discussion, 160–162.

Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC (1992). Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 327:248–254.[Abstract]

Lau J, Schmid CH, Chalmers TC (1995). Cumulative meta-analysis of clinical trials builds evidence for exemplary medical care. J Clin Epidemiol 48:45–57; discussion, 59–60.[ISI][Medline]

Lindhe J, Liljenberg B, Adielson B, Börjesson I (1983a). Use of metronidazole as a probe in the study of human periodontal disease. J Clin Periodontol 10:100–112.[Medline]

Lindhe J, Liljenberg B, Adielsson B (1983b). Effect of long-term tetracycline therapy on human periodontal disease. J Clin Periodontol 10:590–601.[Medline]

Magnusson I, Low SB, McArthur WP, Marks RG, Walker CB, Maruniak J, et al. (1994). Treatment of subjects with refractory periodontal disease. J Clin Periodontol 21:628–637.[Medline]

Montenegro R, Needleman I, Moles D, Tonetti M (2002). Quality of RCTs in periodontology—a systematic review. J Dent Res 81:866–870.[Abstract/Free Full Text]

Palmer RM, Matthews JP, Wilson RF (1998). Adjunctive systemic and locally delivered metronidazole in the treatment of periodontitis: a controlled clinical study. Br Dent J 184:548–552.[Medline]

Sigusch B, Beier M, Klinger G, Pfister W, Glockmann E (2001). A 2-step non-surgical procedure and systemic antibiotics in the treatment of rapidly progressive periodontitis. J Periodontol 72:275–283.[Medline]

Sutton AJ, Duval SJ, Tweedie RL, Abrams KR, Jones DR (2000). Empirical assessment of effect of publication bias on meta-analyses. BMJ 320:1574–1577.[Abstract/Free Full Text]

Thornton A, Lee P (2000). Publication bias in meta-analysis: its causes and consequences. J Clin Epidemiol 53:207–216.[ISI][Medline]

Winkel EG, van Winkelhoff AJ, Barendregt DS, van der Weijden GA, Timmerman MF, van der Velden U (1999). Clinical and microbiological effects of initial periodontal therapy in conjunction with amoxicillin and clavulanic acid in patients with adult periodontitis. A randomised double-blind, placebo-controlled study. J Clin Periodontol 26:461–468.[Medline]

Related articles in JDR:

To the Editor: Caveat for a Cumulative Meta-analysis

Sok-Ja Janket, David R Moles, Joseph Lau, Ian Needleman, and Richard Niederman
JDR 2005 84: 487. [Full Text]  

This article has been cited by other articles:

				S.-J. Janket, D. R Moles, J. Lau, I. Needleman, and R. Niederman To the Editor: Caveat for a Cumulative Meta-analysis J. Dent. Res., June 1, 2005; 84(6): 487 - 487. [Full Text] [PDF]

This Article Abstract Figures Only Full Text (PDF) Services Related articles in JDR Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (1) Citing Articles via Google Scholar Google Scholar Articles by Moles, D.R. Articles by Lau, J. Search for Related Content PubMed PubMed Citation Articles by Moles, D.R. Articles by Lau, J.