• About
    • Login
    View Item 
    •   Institutional Repository Home
    • Electronic Theses and Dissertations
    • Electronic Theses and Dissertations
    • View Item
    •   Institutional Repository Home
    • Electronic Theses and Dissertations
    • Electronic Theses and Dissertations
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Browse

    All of Institutional RepositoryCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

    My Account

    LoginRegister

    A comparative analysis of non-experimental methods for estimating the impact of magnet school enrollment on student achievement

    Stuit, David Alan
    : https://etd.library.vanderbilt.edu/etd-10052009-210839
    http://hdl.handle.net/1803/14275
    : 2009-10-15

    Abstract

    The objective of this dissertation is to understand the circumstances under which non-experimental methods yield unbiased estimates of the effect of magnet school attendance on student achievement. This dissertation has two main analyses. In the first analysis, non-experimental estimates (via multiple regression with observed covariates, analysis of covariance with student fixed effects, and propensity score matching) of the effect of attending one academically selective magnet school on 5th and 6th grade math and reading achievement are compared to experimental estimates found using lottery status as an instrumental variable (IV) for magnet school attendance. This analysis finds that multiple regression and propensity score matching yield estimates with sizeable positive bias that would likely lead a policymaker conclude the magnet school is more effective than it really is; in some cases, this bias represents over half a school year’s worth of learning. Student fixed-effects modeling performs the best of the non-experimental methods and in reading yields estimates of the magnet effect on 5th and 6th grade achievement that are not meaningfully different from the experimental IV estimates. The second analysis tests how well the experimental and non-experimental methods perform under various forms and rates of sample attrition. To investigate this issue I create a variety of samples with different forms and rates of artificial attrition among lottery winners, lottery losers, and non-participants and then run the experimental and non-experimental estimators on these simulated samples. The second analysis finds that the experimental IV estimates are less biased than the non-experimental estimates in almost all scenarios. The one exception is the student fixed-effects estimator, which performed as well or better than the experimental IV estimator as attrition rates exceeded 40%. Collectively, the findings raise caution against using multiple regression and propensity score matching to evaluate the causal impact of school choice programs, even under situations where attrition in the experimental sample is severe. Student fixed-effects modeling shows promise, particularly in reading, but only under high rates of sample attrition can one expect it to perform better than an analysis using randomly assigned comparison groups.
    Show full item record

    Files in this item

    Icon
    Name:
    Stuit_dissertation.pdf
    Size:
    696.9Kb
    Format:
    PDF
    View/Open

    This item appears in the following collection(s):

    • Electronic Theses and Dissertations

    Connect with Vanderbilt Libraries

    Your Vanderbilt

    • Alumni
    • Current Students
    • Faculty & Staff
    • International Students
    • Media
    • Parents & Family
    • Prospective Students
    • Researchers
    • Sports Fans
    • Visitors & Neighbors

    Support the Jean and Alexander Heard Libraries

    Support the Library...Give Now

    Gifts to the Libraries support the learning and research needs of the entire Vanderbilt community. Learn more about giving to the Libraries.

    Become a Friend of the Libraries

    Quick Links

    • Hours
    • About
    • Employment
    • Staff Directory
    • Accessibility Services
    • Contact
    • Vanderbilt Home
    • Privacy Policy