JDR JDR Most Cited Articles
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Data Supplement
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (21)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Tu, Y.-K.
Right arrow Articles by Griffiths, G.S.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Tu, Y.-K.
Right arrow Articles by Griffiths, G.S.
J Dent Res 81(10): 722-726, 2002
© 2002 International and American Associations for Dental Research

RESEARCH REPORT
Clinical

Is Reduction of Pocket Probing Depth Correlated with the Baseline Value or is it "Mathematical Coupling"?

Y.-K. Tu1, M.S. Gilthorpe1,*, and G.S. Griffiths2

1 Biostatistics Unit, Academic Unit of Epidemiology and Health Services Research, Medical School, University of Leeds, 24 Hyde Terrace, Leeds, LS2 9LN, UK; and
2 Department of Periodontology, Eastman Dental Institute, University College London, 256 Gray’s Inn Road, London, WC1X 8LD, UK;

* corresponding author, m.s.gilthorpe{at}leeds.ac.uk


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS & METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Previous studies using correlation or regression analysis have showed that treatment effects measured by the change in clinical parameters are often associated with baseline values of the same parameters. These studies, however, have a methodological weakness. Correlation/regression between baseline measures and the derived change variable invalidates the statistical procedures of testing the null hypothesis: that the coefficient of correlation/regression is zero. This is due to the phenomenon of mathematical coupling. To investigate the impact that this has on the observed correlation/regression coefficient when in reality this is zero, we used random simulations of hypothetical data to model the treatment of periodontal pockets. Results showed a strong probability of obtaining statistically significant correlation/regression coefficients. To separate this artificial effect of mathematical coupling from the true underlying biological relationship, one must apply appropriate analytical strategies to re-evaluate previous evidence within the periodontal literature.

KEY WORDS: correlation coefficient • regression coefficient • regression to the mean • periodontal research


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS & METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
In the periodontal literature, many studies have used correlation or regression to demonstrate that the treatment effect measured by the change of a clinical parameter—such as pocket probing depth, clinical attachment level, or gingival recession—is strongly associated with baseline values of the same parameter. There are many studies showing that the baseline value is a good indicator of the change following treatment (Newcomb and Nixon, 1990; Machtei et al., 1993, 1994; Selvig et al., 1993; Mora et al., 1996; Pini Prato et al., 1996; Falk et al., 1997; Mombelli et al., 1997). These studies, however, all have a common methodological weakness: They examine the correlation or the regression slope between a baseline variable prior to treatment and a derived variable based on the change between the respective pre-treatment and post-treatment values. In his classic book, Andersen (1990) warned against this phenomenon, known as mathematical coupling, since it distorts the relationship between variables and violates basic assumptions underpinning correlation and regression. Classic statistical procedures of testing the null hypothesis, i.e., that the coefficient of correlation or the slope of regression is zero, become erroneous. Andersen devoted a whole chapter in his book to illustrating this statistical consequence by various examples, and many people have appealed for discontinuing this malpractice of correlation/regression analysis (Archie, 1981; Altman, 1982, 1991).

For this study, we simulated and assessed clinical research data on the treatment of periodontal pockets to discover the impact of MC in artificially explaining the "observed" relationship between baseline pocket depth and change in pocketing following surgery when there is no "true" underlying effect.


   MATERIALS & METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS & METHODS
RESULTS
 DISCUSSION
 REFERENCES
 
We simulated hypothetical studies to evaluate the efficacy of a new periodontal therapy. The chosen outcome was the change of pocket probing depth (PPD), as derived from pre- and post-treatment pocket depth measurements. The relationship linking baseline PPD (X), post-treatment PPD (Y), and the corresponding change variable (Z), is:

(Eq. 1)
where the X and Y values denote the unobserved "true" PPD values (i.e., free of any measurement error), the intercept (A) is associated with the overall "true" reduction in PPD, and the regression slope (S) represents the "true" underlying relationship between change in PPD (Z = X-Y) and baseline value (X). From Eq. 1, the relationship between pre- and post-treatment values is:

(Eq. 2)
where 1 - S is the slope of post-treatment PPD regressed on pre-treatment values.

Simulations were undertaken where the aim was to test if the baseline PPD appeared to be a good predictor for the treatment outcome, i.e., the change in PPD, when there was no such relationship (S = 0). In practice, it is assumed that the "observed" pre- (x) and post-treatment (y) measurements contain some measurement error (ex and ey), such that x = X + ex and y = Y + ey, where both error distributions have zero mean and equal variances (square of the standard deviation). For the purposes of this study, it was further assumed that these errors are uncorrelated and independent of the unobserved values (X and Y).

In a typical treatment study, sites would be selected according to treatment needs, not randomly. Thus, for any site to be included in a hypothetical study, its pocket depth must be greater than a pre-defined minimum, e.g., at least 4 mm. To model this, we simulated a population of 1000 error-free X values, generated from normally distributed random numbers, with a baseline mean value of 9 mm PPD, based on typical values observed in the periodontal literature (Laurell et al., 1998). Observed pre-treatment PPD measurements were then derived (x = X + ex). Values for the standard deviation (SD) of the population (PSD) were taken to be 2 mm and 3 mm, yielding reference ranges (i.e., the range in which 95% of the observations lie) of 5 to 13 mm and 3 to 15 mm initial PPD, respectively. A key factor that determines the effect of MC is the ratio (R) of the population SD to that of the measurement error SD (Hayes, 1988). Ratios of 1, 2, and 3 were considered, generating hypothetical population data with error SDs ranging from 0.7 mm to 3.0 mm.

The simulated population data were sampled (without replacement) to yield a hypothetical study of N sites. In the event that a sampled pre-treatment PPD was less than 4 mm, the site was excluded and sampling continued until the required study size was attained. In this manner, no hypothetical study contained sites with observed pre-treatment PPD values of less than 4 mm. We considered various study sizes, with N = 10, 30, 50, and 500, to assess the effect this had on statistical power (i.e., the probability of correctly finding that a coefficient differs significantly from zero).

We calculated post-treatment error-free PPD values (Y) from the pre-treatment error-free values (X), according to Eq. 2, thereby assuming no change between measurement occasions apart from that induced by the new therapy. The observed post-treatment PPD measurements were also derived (y = Y + ey), and if any observation was found to be negative, it was set to zero. The observed changes in PPD (z) were calculated based on a relationship similar to that in Eq. 1 applied to the observed values (x, y) instead of the unobserved values (X, Y). It can be shown that the correlation/regression coefficient is independent of the baseline mean PPD and the overall treatment effect (A) (Moreno et al., 1986), provided that simulated values are not truncated, as was the case for initially negative post-treatment PPD values. To investigate the effect of truncation, we explored two options for the overall reduction of PPD following treatment: no overall mean reduction (i.e., A = 0) and 4 mm overall mean reduction (i.e., A = 4).

Each hypothetical population and its associated study sample were simulated 10,000 times by means of the modeling and simulation software MLwiN (Rasbash et al., 2000), for all possible scenarios being considered. For each simulation, the Pearson (parametric) correlation and the Spearman (non-parametric) rank correlation were assessed by means of the two-tailed t test (Kirkwood, 1992); significance was assumed at the 5% level. According to the t test, the test statistic for the Pearson correlation and the regression slope are equivalent under the corresponding null hypothesis; hence the results for the latter are not presented. The empirical median values for all statistics, along with 95% confidence intervals, were derived for each set of simulations.


   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS & METHODS
 RESULTS
DISCUSSION
 REFERENCES
 
The results of computer simulations are summarized for no overall mean treatment effect (Table 1Go) and for an overall mean PPD reduction of 4 mm following treatment (Table 2Go). In both instances, the null hypothesis of no relationship between change and baseline values is true, yet significant correlation/regression coefficients were frequently observed. The likelihood of significance varied considerably across the range of hypothetical scenarios investigated. The proportion of coefficients that tested significant ranged from around 8.5% (Table 1Go: Spearman correlation, N = 10, PSD = 3 mm, R = 3) to 100% (Tables 1, 2GoGo: N = 500, PSD = 2 mm and 3 mm, R = 1 and 2). In general, there was little difference between the Pearson and Spearman correlation tests.


View this table:
[in this window]
[in a new window]
  		Table 1. Correlation Coefficients for Hypothetical Research Data with Zero Baseline Effect and No Overall Mean Treatment Effect (i.e., A = 0)
 

View this table:
[in this window]
[in a new window]
  		Table 2. Correlation Coefficients for Hypothetical Research Data with Zero Baseline Effect and an Overall Mean Treatment Effect (i.e., A = 4)
 
As the ratio of population to measurement errors SD, R = PSD/ESD, increased, the proportion of coefficients that spuriously tested as significant fell. The larger the study size, the more likely a significant coefficient was observed. An overall treatment effect of 4 mm reduction in PPD, following therapy, increased the number of truncated simulated negative PPD values and gave rise to an elevated likelihood of spuriously significant results (Table 2Go) compared with the scenario of no overall treatment effect (Table 1Go). However, the effect of truncation was modest compared with all other factors being considered. Increasing the population SD while the error SD remained constant yielded a reduction in spuriously significant results when minimal truncation was observed (Table 1Go: Pearson and Spearman correlations, N = 10, 30 and 50, R = 1, PSD = 2 mm cf. 3 mm). In contrast, this reduction diminished (or even reversed) when more truncation occurred (Table 2Go: Spearman correlation, N = 30, R = 2, PSD = 2 mm cf. 3 mm).


   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS & METHODS
 RESULTS
 DISCUSSION
REFERENCES
 
Archie (1981) has proposed four types of mathematical coupling, and the statistical results based on any one should be carefully interpreted. The common problem in each type is that one variable either directly or indirectly contains the whole or components of another variable. In this hypothetical study, as within the previous periodontal literature, the correlation/regression coefficient between the change of a clinical parameter and its baseline value is known as Type III mathematical coupling. This problem arises through addition, subtraction, multiplication, or division of one variable by another and is the most common type in clinical research (Archie, 1981). The MC phenomenon casts doubt upon the validity of conclusions reported in many studies, because any inference on the relationship between change and baseline values that has been subjected to this type of analysis could be highly erroneous, yielding misleading results and consequently incorrect conclusions.

Within medicine, studies on the use of calcium channel blockers to reduce blood pressure in patients with hypertension (Gill et al., 1985), surgical treatment of obesity (Halverson and Koehler, 1981), protein clearance of patients under dialysis (Lowrie, 1996), and oxygen consumption in relation to oxygen delivery (Yu et al., 1996; Granton et al., 1998) have all cited MC. In contrast, we are not aware that this issue has been discussed in the periodontal literature, yet the problem of assessing an outcome in relation to the initial disease severity is common in periodontal research.

There has been some discussion in the periodontal literature on the issue of regression to the mean (RTM) (Blomqvist, 1987; Egelberg, 1989; Gunsolley et al., 2001), which is where, due to measurement error or within-subject variation, initially high/low values are subsequently recorded to be lower/higher and vice versa (Yudkin and Stratton, 1996). RTM is a special case of MC, where coupling occurs through the "observed" variables as a consequence of the measurement error. However, one can experience MC without RTM. The concepts of MC without RTM and MC that is entirely RTM are illustrated (Figs. 1, 2GoGo) and described mathematically in the Appendix (www.dentalresearch.org). Consider the situation where there is no underlying relationship between the error-free X and Y values (Fig. 1aGo). The null hypothesis for the correlation and regression coefficients between Z (= X - Y) and X is that they equal 1/ {approx} 0.707 and 1.0, respectively, i.e., not zero (Fig. 1bGo). This shifting of the null hypothesis is a direct consequence of the coupling, and the spurious association between Z (= X - Y) and X (Fig. 1bGo) is a consequence of MC without any RTM. For the situation where the slope of error-free Y values regressed on error-free X values is exactly 1, there is no underlying relationship between change and baseline value, and the correlation/regression coefficient is zero. However, in the presence of measurement error, "observed" values no longer fall on the line Y = X (Fig. 2aGo), giving rise to a spurious non-zero correlation/regression coefficient between z (= x - y) and x, where x = X + ex and y = X + ey (Fig. 2bGo). This is MC that is entirely RTM and is a direct consequence of the coupling of "observed" values through measurement error. In reality, one does not know the relationship between X and Y, nor does one know the degree of measurement error (although the latter may be estimated from duplicate measurements made during the measurement of x and y). In practice, therefore, one is faced with a combination of "true" and "observed" effects. Although some studies take account of RTM, many overlook the general problem of MC.




View larger version (13K):
[in this window]
[in a new window]
  		Figure 1. An illustration of mathematical coupling (MC) without any effect of regression to the mean (RTM). (A) True Y values plotted against true X values, with no underlying relationship between them. (B) A spurious correlation between true Z (= X - Y) and true X is shown; the solid line is the regression of Z on X; this represents the hypothetical situation of MC without RTM.

 



View larger version (15K):
[in this window]
[in a new window]
  		Figure 2. An illustration of mathematical coupling (MC) due entirely to the effect of regression to the mean (RTM). (A) Observed y values plotted against observed x values, with the underlying relationship between them represented by the solid line. (B) A spurious correlation between z (= x - y) and x is shown; the solid line is the regression of z on x; this represents the hypothetical situation of MC as entirely RTM.

 
A recent meta-analysis showed that most periodontal studies in the literature have used 15 or more patients in either test or control groups (Laurell et al., 1998). Within this study, simulation sample size was increased from 10, through 30 and 50, to 500. When there was no underlying relationship between baseline and change, MC was more likely to lead to a significant finding for larger studies. The likelihood of finding an incorrect significant result, due to MC, approached 100% asymptotically as the study size increased. This is because the standard error of a correlation/regression coefficient reduces with increasing study size, and the likelihood of obtaining significance for the coefficient increases. However, it should not be inferred that small studies are to be preferred simply because they experience reduced likelihood of false-positive results; all statistical tests have less power within smaller studies, and the likelihood of finding a genuine underlying biological relationship between change and baseline would also reduce.

For all study sizes, the simulations demonstrated beyond reasonable doubt that there exists a non-zero probability of acquiring a fraudulent significance in correlation/regression due to MC when in fact there is no underlying relationship between baseline outcome and change following treatment. Inclusion of pre-treatment sites with at least 4 mm pocketing and the truncation to zero of simulated post-treatment negative PPD values replicate real-life scenarios of patient recruitment and physical limitations of treatment outcome following surgery. However, these restrictions had minimal impact on the role of MC. There was also minimal difference between the two correlation methods adopted, which was perhaps to be expected given the underlying assumption of a linear relationship between pre- and post-treatment PPD values.

Based on our study, and other medical literature, it is evident that the effect of MC cannot be neglected in periodontal research. One solution, proposed and recommended by many authors (Oldham, 1962; Altman, 1982, 1991), is to correlate the average of pre- and post-treatment values with the change variable. A more sophisticated regression approach is multilevel modeling (Gilthorpe et al., 2000). By construction of a random coefficient model (Gilthorpe et al., 2001), with outcome modeled over time and the pre- and post-treatment measures nested within subjects, the required correlation would be that between the random intercept and random slope (Bryk and Raudenbush, 1992).

In the absence of such strategies, interpretation of the results and the conclusions reached in previous studies that fail to address the MC phenomenon are questionable. Moreover, the biological and functional mechanisms put forward to explain the strong connection between baseline values and treatment effects are suspect. It is therefore strongly suggested that the results and conclusions of previous periodontal literature, whose statistical evidence is tainted by MC, should be critically reviewed and re-analyzed. While the biological mechanisms and the clinical association between different parameters and measurements proposed might be genuine, initial conclusions need to be clarified in view of the highlighted problem. Until the artificial effect of MC has been separated from the "true" biological relationship, evidence of a relationship between baseline value and change following surgery remains questionable and could be potentially misleading.

Careful formulation of the study question and knowledge of the measurements (in particular, the estimation of measurement errors) are crucial for the adoption of appropriate strategies whereby such problems can be avoided in future periodontal research.


   ACKNOWLEDGMENTS
 
The first author was self-funded, while the remaining two authors were funded by the UK government’s Higher Education Funding Council for England (HEFCE).


   FOOTNOTES
 
A supplemental appendix to this article is published electronically only at http://www.dentalresearch.org.

Received June 27, 2001; Last revision July 8, 2002; Accepted July 24, 2002


   REFERENCES
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS & METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Altman DG (1982). Statistics in medical journals. Stat Med 1:59–71.[Medline]

Altman DG (1991). Practical statistics for medical research. London: Chapman and Hall/CRC.

Andersen B (1990). Methodological errors in medical research. London: Blackwell, pp. 157-168.

Archie JP (1981). Mathematical coupling of data: a common source of error. Ann Surg 193:296–303.[Medline]

Blomqvist N (1987). On the bias caused by regression toward the mean in studying the relation between change and initial value. J Clin Periodontol 14:34–37.[Medline]

Bryk AS, Raudenbush AW (1992). Hierarchical linear models: applications and data analysis methods. London: Sage, pp. 66-70.

Egelberg J (1989). The impact of regression towards the mean on probing changes in studies on the effect of periodontal therapy. J Clin Periodontol 16:120–123.[Medline]

Falk H, Laurell L, Ravald N, Teiwik A, Persson R (1997). Guided tissue regeneration therapy of 203 consecutively treated intrabony defects using a bioabsorbable matrix barrier. Clinical and radiographic findings. J Periodontol 68:571–581.[Medline]

Gill JS, Zezulka AV, Beevers DG, Davies P (1985). Relation between initial blood pressure and its fall with treatment. Lancet 1:567–569.[Medline]

Gilthorpe MS, Maddick IH, Petrie A (2000). Introduction to multilevel modelling in dental research. Community Dent Health 17:222–226.[Medline]

Gilthorpe MS, Griffiths GS, Maddick IH, Zamzuri AT (2001). An application of multilevel modelling to longitudinal periodontal research data. Community Dent Health 18:79–86.[Medline]

Granton JT, Walley KR, Phang PT, Russell JA, Lichtenstein S (1998). Assessment of three methods to reduce the influence of mathematical coupling on oxygen consumption and delivery relationships. Chest 113:1347–1355.[Abstract/Free Full Text]

Gunsolley JC, Yeung GM, Butler JH, Waldrop TC (2001). Is loss of attachment due to root planing and scaling in sites with minimal probing depths a statistical or real occurrence? J Periodontol 72:349–353.[Medline]

Halverson JD, Koehler RE (1981). Gastric bypass: analysis of weight loss and factors determining success. Surgery 90:446–455.[Medline]

Hayes RJ (1988). Methods for assessing whether change depends on initial value. Stat Med 7:915–927.[Medline]

Kirkwood BR (1992). Essentials of medical statistics. London: Blackwell.

Laurell L, Gottlow J, Zybutz M, Persson R (1998). Treatment of intrabony defects by different surgical procedures. A literature review. J Periodontol 69:303–313.[Medline]

Lowrie EG (1996). Thoughts about judging dialysis treatment: mathematics and measurements, mirrors in the mind. Semin Nephrol 16:242–262.[Medline]

Machtei EE, Dunford R, Norderyd OM, Zambon JJ, Genco RJ (1993). Guided tissue regeneration and anti-infective therapy in the treatment of Class II furcation defects. J Periodontol 64:968–973.[Medline]

Machtei EE, Cho MI, Dunford R, Norderyd J, Zambon JJ, Genco RJ (1994). Clinical, microbiological, and histological factors which influence the success of regenerative periodontal therapy. J Periodontol 65:154–161.[Medline]

Mombelli A, Lehmann B, Tonetti M, Lang NP (1997). Clinical response to local delivery of tetracycline in relation to overall and local periodontal conditions. J Clin Periodontol 24:470–477.[Medline]

Mora F, Etienne D, Ouhayoun JP (1996). Treatment of interproximal angular defects by guided tissue regeneration: 1 year follow-up. J Oral Rehabil 23:599–606.[Medline]

Moreno LF, Stratton HH, Newell JC, Feustel PJ (1986). Mathematical coupling of data: correction of a common error for linear calculations. J Appl Physiol 60:335–343.[Abstract/Free Full Text]

Newcomb GM, Nixon KC (1990). Microbiological assays as predictors of the response to periodontal therapy. Aust Dent J 35:261–265.[Medline]

Oldham PD (1962). A note on the analysis of repeated measurements of the same subjects. J Chronic Diseases 15:969–977.[Medline]

Pini Prato G, Clauser C, Cortellini P, Tinti C, Vincenzi C, Pagliaro U (1996). Guided tissue regeneration versus mucogingival surgery in the treatment of human buccal recessions. A 4-year follow-up study. J Periodontol 67:1216–1223.[Medline]

Rasbash J, Browne W, Cameron B, Charlton C (2000). MLwiN version 1.10.0006. London: Multilevel Models Project, Institute of Education.

Selvig KA, Kersten BG, Wikesjö UME (1993). Surgical treatment of intrabony periodontal defects using expanded polytetrafluoroethylene barrier membrane: influence of defect configuration on healing response. J Periodontol 64:730–733.[Medline]

Yu M, Burchell S, Takiguchi SA, McNamara JJ (1996). The relationship of oxygen consumption measured by indirect calorimetry to oxygen delivery in critically ill patients. J Trauma 41:41–48.[Medline]

Yudkin PL, Stratton HH (1996). How to deal with regression to the mean in intervention studies. Lancet 347:241–243.[Medline]




This article has been cited by other articles:


Home page
 Stat Methods Med ResHome page
A. Blance, Y.-K. Tu, and M. S Gilthorpe
A multilevel modelling solution to mathematical coupling
Statistical Methods in Medical Research, December 1, 2005; 14(6): 553 - 565.
[Abstract] [PDF]

Home page
 Am J EpidemiolHome page
Y.-K. Tu, R. West, G. T. H. Ellison, and M. S. Gilthorpe
Why Evidence for the Fetal Origins of Adult Disease Might Be a Statistical Artifact: The "Reversal Paradox" for the Relation between Birth Weight and Blood Pressure in Later Life
Am. J. Epidemiol., January 1, 2005; 161(1): 27 - 32.
[Abstract] [Full Text] [PDF]

Home page
 Int J EpidemiolHome page
M. S Gilthorpe and Y.-K. Tu
Mathematical coupling: a multilevel approach
Int. J. Epidemiol., December 1, 2004; 33(6): 1399 - 1400.
[Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Data Supplement
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (21)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Tu, Y.-K.
Right arrow Articles by Griffiths, G.S.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Tu, Y.-K.
Right arrow Articles by Griffiths, G.S.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
IADR Journals Advances in Dental Research ®
Journal of Dental Research ® Critical Reviews (1990-2004)