| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
RESEARCH REPORT |
1 Department of Pediatric Dentistry, School of Dentistry, CB 7450 Brauer Hall, and 2 Department of Health Policy and Administration, School of Public Health, University of North Carolina, Chapel Hill, NC 27599-7450, USA;
* corresponding author, jessica_lee{at}dentistry.unc.edu
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMATERIALS & METHODSRESULTSDISCUSSIONREFERENCES |
|---|
KEY WORDS: dental use • selection bias • endogeneity • WIC • non-randomization
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS & METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Issue of Endogeneity
To understand how the two-stage technique can address selection bias, let us consider the study by McClellan and colleagues (1994) on how catheterization for myocardial infarction affects mortality. Catheterization, a diagnostic procedure, is used only on patients healthy enough to be considered for subsequent surgery. Therefore, a simple single-stage regression will overestimate the effectiveness of catheterization on mortality, because patients who undergo catheterization are healthier than the average myocardial infarction patient. If this were a randomized controlled trial, the catheterization treatment would be determined randomly, but this is not a feasible option for this study. Although the decision to undergo catheterization is based partly on health status, it is also based on a seemingly random variable—the difference in distance from home to either (1) the nearest hospital that does catheterizations, or (2) the nearest hospital (that may or may not do catheterizations). These seemingly random variables can be used as instrumental variables that will use random variation in where people live, which is strongly related to their willingness to undergo catheterization, to control for selection bias. Using the two-stage technique, McClellan and colleagues (1994) were able to control for selection bias and reported the differences in effects of catheterization to be only –0.3%, according to the two-stage method. This is compared with –5.9% according to a conventional single-stage analytical method.
Many public health programs are designed to help increase access to health services. Because of practical and ethical concerns, evaluation studies of their effectiveness are usually unable to rely on randomization; instead, many rely on observational or quasi-experimental designs. The possibility therefore exists that any observed effects of the intervention on dental use are due to selection bias, and not to the program or intervention itself. Under these conditions, a standard, single-stage analysis of the effect of the program on use of services will overestimate the program’s positive effects, and a case of endogeneity may arise. To address this design problem, we implemented a system of simultaneous equations that explicitly model participation in The Supplemental Program for Women, Infants, and Children’s (WIC), and the error correlation structure, in the empirical analysis for this study. Using the Anderson and Aday (1974) conceptual framework (Fig.
), we hypothesized not only that the WIC has a direct effect on the use of dental services, but also that the use of dental services may have an effect on WIC participation; thus, a case selection bias arises. We further hypothesized that the two-stage method could be implemented to control for this selection bias.
|
This paper presents the two-stage technique of controlling for selection bias during examination of the role of child WIC participation in the probability of having a Medicaid-reimbursed dental visit. We focused on the methodological issues arising from this analytical approach. Substantive findings for the effect of WIC on dental services use have been reported elsewhere (Lee et al., 2004).
|
MATERIALS & METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS & METHODSRESULTSDISCUSSIONREFERENCES |
|---|
All children born in NC in 1992, and who were enrolled in the Medicaid program, were eligible for inclusion in the study. Children were excluded if they had more than one Medicaid ID in their records or if they had recorded periods of Medicaid enrollment indicated prior to the date of birth. A sample size of 49,795 was established. A Medicaid enrollment history was created for each child in which enrollment status was indicated for each month of life from birth to age five years (months 01–60). A dental visit was defined as having one or more dental claims filed through Medicaid.
Correcting for the Endogeneity of WIC— The Two-stage Method
The two-stage model provides consistent estimates as long as there are valid instrumental variables. In contrast, the single-stage logit usually provides the smallest mean square error but biased results in cases of a potentially endogenous variable, suggesting a trade-off between bias and mean square error (Rivers and Vuong, 1988; Blundell and Smith, 1989). Most dental studies use the single-equation method exclusively, leading to precise, but biased, estimates (Grembowski et al., 1985; Griffen et al., 2000; Kanellis et al., 2000). The two-stage method was applied to our estimating equations. Before estimating the second-stage model, we predicted the endogenous variable, which, in our example, was WIC participation as a function of control variables and the instrumental variables. All instrumental variables were excluded from the second-stage model. The residual from the first stage was then added as an explanatory variable to our second-stage model. Using the residual from our first-stage equation captured (equation 1.1) the unobservable non-random component and allowed us to control for selection bias.
First-stage OLS Regression (Table 2, column 1)
 | (1.1) |
|
Second-stage Logit Equation (Table 3, column 2)
 | (1.2) |
|
 |
 |
WIC Participation was measured as the number of months in the WIC. Dental Visit was measured as the child having a Medicaid-reimbursed dental visit, and Control Variables included Medicaid enrollment, maternal age, maternal education, race, and dentist-per-population ratio. Instrumental variables included the number of full-time WIC clinics, multiple sites, and hours open per month.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS & METHODSRESULTSDISCUSSIONREFERENCES |
|---|
).
|
). The first test applied was the test for endogeneity, to determine if participation in the WIC was endogenous in relation to the use of dental services. The error term from the first stage should not be a significant predictor in the second-stage model. The Hausman specification test, with the t statistic used on the coefficient of the predicted error, when included in the single equation, showed that the null hypothesis of ‘no endogeneity present’ was rejected (p < 0.01). These results indicated that WIC participation was endogenous (Table 3, column 2). Next, we examined the strength of the instrumental variables and for sufficient explanatory power in the first stage. The first-stage OLS regression (Table 2, column 1) indicated that the t test and F test for our proposed instrumental variables were significant (p < 0.05), and that we had strong instrumental variables, as long as they could be excluded from our second-stage dental use model. The two-stage method is preferred over a single-equation logit, if the R-squared in the first equation is greater than 0.3, or correlation between two error terms is large and the R-squared of the first equation is greater than 0.1, or if the percentage of explanatory variables in the first-stage regression that are instruments is greater than 0.25 (Bollen et al., 1995). Nearly one-third of the variables in our first equation were instruments, and the R-squared was 0.20. The next test examined whether the WIC instrumental variables belonged in the second-stage equation. The test for overidentifying restrictions failed to reject the null hypothesis that the overidentifying restrictions were valid at the 0.05 level, and indicated that the instrumental variables could be excluded from the main equation of WIC and dental use. The likelihood ratio test was constructed under the null that the overidentifying restrictions would be examined for validity. The results (Table 2, column 2) indicated that the instrumental variables could be excluded from the main equation. The collective results of our specification testing indicated that WIC participation was endogenous and that we had good instrumental variables. Accordingly, the first-stage residual values were added to our second-stage random-effects logit model for dental visits.
Effects of the WIC on Oral Health Services Use
The results from the two-stage child-level random-effects logit estimation models for WIC participation and dental services use indicated that WIC participation was significant at the 0.05 level and had a positive effect on the likelihood of a dental visit (Table 3, column 2). In a prediction model of a ‘base case’ child (white, maternal age of 21 yrs, maternal education of 11th grade, married, 6.8 dentists/10,000, and enrolled in Medicaid for 7 mos/yr), children who participated in the WIC for 12 mos had the predicted probability of 21% of having a dental visit in the two-stage model (Table 3, column 2). In the model that did not control for endogeneity, the single-stage model, prediction results indicated that children who participated in the WIC for 12 mos had a predicted probability of 33% (Table 3, column 1). Thus, the results from the single equation overestimated the effects of the WIC program on dental services use by 36%.
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS & METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Our study is the first to examine a public health program (WIC) and dental health services utilization using the two-stage statistical modeling approach. A strong criticism of previous WIC child health studies has been their inability to control for the potential selection bias of enrollment in the WIC program (Besharov and Germanis, 2001). We conducted extensive tests for these sources of bias in the relationship between WIC participation and the use of oral health services, and found that selection bias did exist. Random assignment of families to WIC participation would be a stronger design and would help overcome any selection bias. However, the implementation of this strategy in a community setting would be difficult, and such a design is not ethically defensible. Our study demonstrates the feasibility of using the two-stage analysis to control for selection bias when examining the effects of a public health program on use of dental services.
Our results should also be considered in light of two major limitations. First, this study used claims data and can capture only the dental visits that were reimbursed by the Medicaid program. It has been well-documented that Medicaid children have disproportionately more dental disease than other children, and that they also have the most unmet dental needs (Davidoff et al., 2000; Newacheck et al., 2000). Therefore, the likelihood of Medicaid children getting care outside the program is low. Second, we recognize that the two-stage procedure can be done only if good instrumental variables exist, and their availability may be limited for some studies.
|
ACKNOWLEDGMENTS |
|---|
Received April 14, 2004; Last revision May 9, 2005; Accepted June 22, 2005
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS & METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Besharov D, Germanis P (2001). Rethinking WIC: an evaluation of the Women, Infants and Children Program. Washington, DC: AEI Press.
Blundell R, Smith R (1989). Estimation in class of simultaneous equation limited dependent variable models. Rev Econom Stud 56(5):37–58.
Bollen KA, Guilkey DK, Mroz TA (1995). Binary outcomes and endogenous explanatory variables: tests and solutions with an application to the demand for contraceptive use in Tunisia. Demography 32(1):111–131.[ISI][Medline]
Buescher PA, Horton SJ, Devaney BL, Roholt SJ, Lenihan AJ, Whitmire JT, et al. (2003). Child participation in WIC: Medicaid costs and use of health care services. Am J Public Health 93:145–150.
Davidoff AJ, Garrett AB, Makuc DM, Schirmer M (2000). Children eligible for Medicaid but not enrolled: how great a policy concern. The Urban Institute. http://www.urban.org/AmyJDavidoff
Deeks JJ, Dinnes J, D’Amico R, Sowden AJ, Sakarovitch C, Song F, et al. (2003). Evaluating non-randomised intervention studies. Health Technol Assess 7(27):iii–x, 1–173.[Medline]
Edelstein BL, Manski RJ, Moeller JF (2000). Pediatric dental visits during 1996: an analysis of the federal Medical Expenditure Panel Survey. Pediatr Dent 22:17–20.[Medline]
Grembowski D, Conrad D, Milgrom P (1985). Utilization of dental services in the United States and an insured population. Am J Public Health 75:87–89.
Griffen SO, Gooch BF, Beltran E, Sutherland JN, Barsley R (2000). Dental services, costs, and factors associated with hospitalization for Medicaid-eligible children, Louisiana 1996–97. J Public Health Dent 60:21–27.[ISI][Medline]
Ioannidis JP, Haidich AB, Lau J (2001). Any casualties in the clash of randomised and observational evidence? BMJ 322:879–880.
Jones CM, Tinanoff N, Edelstein BL, Schneider DA, DeBerry-Sumner B, Kanda MB, et al. (2000). Creating partnerships for improving oral health of low-income children. J Public Health Dent 60:193–196.[ISI][Medline]
Kanellis MJ, Damiano PC, Momany ET (2000). Medicaid costs associated with the hospitalization of young children for restorative dental treatment under general anesthesia. J Public Health Dent 60:28–32.[ISI][Medline]
Lee JY, Rozier RG, Norton EC, Kotch JB, Vann WF Jr (2004). Effects of WIC participation on children’s use of oral health services. Am J Public Health 94:772–777.
Mayer ML, Stearns SC, Norton EC, Rozier RG (2000). The effects of Medicaid expansions and reimbursement increases on dentists’ participation. Inquiry 37:33–44.[ISI][Medline]
McClellan M, McNeil BJ, Newhouse JP (1994). Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. J Am Med Assoc 272:859–866.[Abstract]
Newacheck PW, Hughes DC, Hung YY, Wong S, Stoddard JJ (2000). The unmet health needs of America’s children. Pediatrics 105:989–997.
North Carolina Food and Nutrition Services (1999). The Supplemental Program for Women, Infants, and Children Training Manual. Raleigh: North Carolina Food and Nutrition Services.
Rivers D, Vuong Q (1988). Limited information estimators and exogeneity tests for simultaneous probit models. J Health Econ 39:347–366.
Rush D, Horvitz DG, Seaver WB, Leighton J, Sloan NL, Johnson SS, et al. (1988). The national WIC evaluation: evaluation of the Special Supplemental Food Program for Women, Infants, and Children. IV. Study methodology and sample characteristics in the longitudinal study of pregnant women, the study of children, and the food expenditures study. Am J Clin Nutr 48(2 Suppl):429–438.
Terza J (2005). Endogeneity in nonlinear parametric models: a guide for applied researchers in health economics. Charleston, SC: Center for Health & Economic Policy Studies, University of South Carolina.
US General Accounting Office (1992). Early intervention: Federal investments like WIC can produce savings. Document HRD 92-18. Washington, DC: US General Accounting Office.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| IADR Journals | Advances in Dental Research ® |
| Journal of Dental Research ® | Critical Reviews (1990-2004) |
