IS THERE A DIFFERENCE IN PERFORMANCE IN URBAN AND NON-URBAN CHARTER SCHOOLS?

Responsible researcher: Eduarda Miller de Figueiredo

Authors: Joshua D. Angrist, Parag A. Pathak and Christopher R. Walters

Original name: Explaining Charter School Effectiveness

Intervention Location: Massachusetts - United States

Sample Size: 23 schools

Sector: Education

Variable of Primary Interest: MCAS Test Score

Intervention Type: Charter Schools

Methodology: 2SLS and LATE

Summary

Charter schools are public schools that operate independently and have generated increasing interest in analyzing their performance and the heterogeneity of results. charter schools take their students well beyond the performance levels characteristic of the districts urban public schools, while charters leave their students' performance unchanged.

1. Policy Problem

Interest in charter has grown rapidly, with growing evidence suggesting that charter in urban settings have the potential to generate impressive achievement gains, especially for minority students living in high-poverty areas. The presence of these types of schools in Boston and Lynn increases student performance in mathematics and English language arts (ELA), in addition to school attendance. These positive effects have also been reported in Washington, Chicago, and New York (Hoxby and Rockoff, 2004; Hoxby, Murarka, and Kang, 2009; Dobbie and Fryer, 2011).

The current curiosity, however, is in charter schools in school districts outside the urban environment, as the results for this set of schools are more mixed (Hu, 2011). There was little evidence of performance gains in schools outside of high-poverty urban areas (Angrist, Pathak, and Walters, 2011).

The article analyzed here analyzed the heterogeneity of the treatment effect in charter in Massachusetts, through explanatory variables at the student and school level.

2. Implementation and Evaluation Context

elementary charter schools secondary charter In which urban schools are younger than non-urban schools. charter schools also have a longer day and school year than non-urban schools, where the extra time appears to go toward increasing math and literature instruction, and yet 19% of charter urban areas have classes on Saturdays.

charter schools charter schools spend less. Another detail is that the student-to-teacher ratio in charter is generally lower than the staff ratio in traditional public schools, and charter have the smallest class sizes.

When analyzing the sample, the authors noticed that urban students are more likely to be black or Hispanic and have much lower baseline test scores than other public school students. charter schools serve a higher proportion of students of color than urban public schools. charter school students in urban and non-urban areas have slightly higher baseline test scores than their traditional public school counterparts.

• Policy/Program Details

            A charter school is a public school that operates independently, having the freedom to design classrooms that meet the needs of its students. Operating under a contract with an “authorizer” (NGO, government agency, or university) who holds them accountable to high standards, is exempt from state or local regulations related to operation and management, but adheres to public school regulations by charging no tuition and nor being affiliated with a religious institution. The charter school commits to achieving specific educational objectives (National Charter School Resource Center, n.d.; National Alliance for Public Charter Schools, n.d.)[1].

4. Assessment Method

Lottery data were collected for the set of charter that have elementary and middle school grades, excluding closed schools, schools that opened after the 2009-2010 school year, and schools serving nontraditional populations. Thus, the resulting sample ended up with a total of 17 elementary schools and 6 high schools, with urban and non-urban schools.

The student data used for the research was captured through administrative records from the academic years 2001-2002 to 2010-2011. In which they have information on demographics, schools attended, Massachusetts Comprehensive Assessment System (MCAS) scores[2].

Through lottery-based identification that captures causal effects for candidates, using in the second-stage equation for lottery analysis the test score for student i in grade g in year t ( ) as the dependent variable. However, because ordinary least squares (OLS) estimates failed to capture the causal effects if the decision to apply for or attend a charter school is correlated with unmeasured ability, motivation, or family background, the authors used a binary variable that indicated the Lottery offering as an instrumental variable for time spent at charter school. Thus, estimating the results through a two-stage least squares (2SLS) model.

A two-part local average treatment effect (LATE) estimate was also performed, with the first capturing differences in potential outcomes in the treated state and the second capturing differences in potential outcomes in the untreated state. That is, the authors compared performance in each area using the non-chartered local average.

5. Main Results

Second-stage estimates show that one year of attendance at a lottery elementary charter and ELA scores by 0.08. The 2SLS estimates for high schools reveal larger causal effects, with score gains of 0.27 per year in math and 0.21 per year in ELA.

Furthermore, the results found masked considerable heterogeneity in relation to urban and non-urban status. Where estimates for urban high schools suggest that these schools generate gains of 0.32 in math and 0.15 in ELA, and for non-urban they are negative.

charter schools raise scores for most applicants, although not uniformly. In which girls made slightly greater gains in math, while boys made greater gains in ELA. Black and Hispanic students benefit considerably from urban charter attendance in high school, but math gains for whites are smaller. Attending urban high schools increases math and reading scores across all groups.

From LATE estimates, the ELA scores of non-urban elementary students in public schools exceed the performance level in the non-urban environment by 0.10 . The results for non-urban high school students show more positive selection than in elementary school. These results show an advantage for these schools, in that urban elementary schools take their students' scores from a typically low level to a level much closer to that performed by non-urban students.

charter schools take their students well beyond the performance levels characteristic of urban public school districts, while charters leave their students' performance unchanged or diminished from a higher starting point.

6. Public Policy Lessons

Massachusetts' urban charter schools generate impressive performance gains, while non-urban charter schools are largely ineffective and appear to reduce performance.

Reference

Angrist, Joshua D., Parag A. Pathak, and Christopher R. Walters. 2011. “Explaining Charter School Effectiveness.” National Bureau of Economic Research (NBER) Working Paper 17332.

[1] For more information, see: https://www.publiccharters.org/about-charter-schools/what-charter-school and https://charterschoolcenter.ed.gov/what-charter-school . Accessed on: 09 Jan. 2023.

[2] Massachusetts Comprehensive Assessment System (MCAS).