Urban Charter Schools Often Succeed. Suburban Ones Often Don’t.

22view-web-articleLargeThe Match Charter Public High School in Boston. Charter schools in Boston have produced big gains in test scores, research shows. KAYANA SZYMCZAK FOR THE NEW YORK TIMES

Charter schools are controversial. But are they good for education?

Rigorous research suggests that the answer is yes for an important, underserved group: low-income, nonwhite students in urban areas. These children tend to do better if enrolled in charter schools instead of traditional public schools.

There are exceptions, of course. We can’t predict with certainty that a particular child will do better in a specific charter or traditional public school. Similarly, no doctor can honestly promise a patient she will benefit from a treatment.

Social scientists, like medical researchers, can confirm only whether, on average, a given treatment is beneficial for a given population. Not all charter schools are outstanding: In the suburbs, for example, the evidence is that they do no better than traditional public schools. But they have been shown to improve the education of disadvantaged children at scale, in multiple cities, over many years.

Charter schools are publicly funded but not bound by many of the rules that constrain traditional public schools.

Charters, for example, can easily try new curriculums or teaching strategies, or choose to have a longer school day. They have more autonomy than traditional public schools in hiring and firing teachers, who have voted to form unions at only a handful of charters. Perhaps as a result, teachers’ unions have often opposed charter schools, saying they compete unfairly with traditional public schools and are not held to the same standards.

Measuring the effectiveness of any school is challenging. Parents choose their children’s schools, either by living in a certain school district or by applying to a private or charter school.

Some schools are filled with students — say, the children of highly motivated parents — who would perform well in almost any setting. This could mislead us into thinking these schools provide an exemplary education, when the truth is they attract strong students.

This is so-called selection bias, the greatest challenge in evaluating the effectiveness of schools. Stuyvesant High School in New York City, to which entry is granted through a competitive exam, is filled with smart students who might succeed anywhere. When those students do well, is it because of the school or the students or both? How about Harvard, or any other school?

In the case of charter schools, researchers have found an innovative way to overcome selection bias: analyzing the admission lotteries that charters are required to run when they have more applicants than seats.

Each lottery serves as a randomized trial, the gold standard of research methods. Random assignment lets us compare apples to apples: Lottery winners and losers are identical, on average, when they apply. Any differences that emerge after the lottery can safely be attributed to charter attendance.

One concern with this approach is that charters might push out difficult students after the lottery. News accounts indicate that some schools have engaged in this practice, including one school of the Success Academy, a high-performing charter school network in New York City.

That’s one reason the lottery studies don’t compare students who are and are not enrolled in charter schools, but instead compare students who win and lose the lotteries. If a student wins a lottery but declines to attend, or transfers out, her test scores are still assigned to the charter for the analysis. This means that the estimates are not biased by transfers after the lottery takes place.

Tracking thousands of students across hundreds of schools is made possible by student-level databases that states provide to approved researchers under stringent security agreements. Whether a child stays in a given school, transfers within a district or moves to a different district, these databases record her test scores, grade progression and, in some states, college attendance.

A consistent pattern has emerged from this research. In urban areas, where students are overwhelmingly low-achieving, poor and nonwhite, charter schools tend to do better than other public schools in improving student achievement. By contrast, outside of urban areas, where students tend to be white and middle class, charters do no better and sometimes do worse than public schools.

This pattern — positive results in low-income city neighborhoods, zero to negative results in relatively affluent suburbs — holds in lottery studies in Massachusetts as well in a national study of charter schools funded by the Education Department.

My own research, conducted with colleagues at the Massachusetts Institute of Technology and Harvard, shows that charter schools in Boston produced huge gains in test scores. A majority of students at Boston’s charters are African-American and poor. Their score gains are large enough to reduce the black-white score gap in Boston’s middle schools by two-thirds.

Boston’s charters also do a better job at preparing students for college. Charter students are twice as likely to take an Advanced Placement exam as similar students in Boston’s other public schools. Ten percent of charter students pass an A.P. calculus test, compared with just 1 percent of similar students in other public schools. This stronger preparation means that these charter students are far more likely than similar students in traditional public schools to attend a four-year college.
Perhaps only the best charters are popular, and that’s why the lottery studies produce such positive estimates. We can’t use the lottery approach to assess a school that does not have high demand for its seats.

In Boston, we used alternative statistical methods to examine the charters that are not oversubscribed. We found smaller but still positive results. A Stanford study examined student performance in 41 cities, and also concluded that their charters outperformed their traditional public schools. A caution: Without randomization, we can’t be as certain these nonlottery studies have eliminated selection bias.

Not all charters are successful, of course, but we should not expect them to be. Charters are a place for educators to try out new methods. Some of these experiments produce great results. Others don’t. It’s the job of government to distinguish between the successful schools and the failures, and to shut down the failures.

In Massachusetts, charter schools with a proven track record are allowed to add more grades or open more schools. The Match charter in Boston started as a high school but now also enrolls students from kindergarten through middle school. But in some states, including Ohio, oversight is weak, and poorly performing charter schools have been allowed to expand.

A charter gives schools the autonomy to innovate. But it also obliges state and local governments to hold schools accountable. If schools that don’t educate children well are allowed to stay open, it’s a failure of government, and not of the charter experiment itself.

Susan M. Dynarski is a professor of education, public policy and economics at the University of Michigan. Follow her on Twitter at @dynarski.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.