How to improve educational outcomes in developing countries?

Responsible researcher: Viviane Pires Ribeiro

Article title: IMPROVING EDUCATIONAL OUTCOMES IN DEVELOPING COUNTRIES: LESSONS FROM RIGOROUS EVALUATIONS

Article authors: Alejandro J. Ganimian and Richard J. Murnane

Location of intervention: 58 low- and middle-income countries

Sample size: 223 reviews

Sector: Education

Type of Intervention: Assessment of educational outcomes

Variable of main interest: Educational outcomes

Evaluation method: Others- Analysis of evaluations of educational interventions

Assessment Context

Developing countries have become increasingly interested in adopting educational policies that are supported by rigorous evidence of effectiveness. This interest in “evidence-based” policies has resulted in an increasing number of impact evaluations since the turn of the century. Thus, several researchers have attempted to summarize the evidence from these assessments.

Such analyzes aim to support the evidence for policymakers, who do not have the time (and sometimes specialized training) to sift through hundreds of academic articles. Evaluations seek to identify promising interventions as well as point to puzzles to be examined in future research. However, many of the analyzes reach seemingly conflicting conclusions about which educational interventions would improve educational outcomes. Disagreements can arise for several reasons (for example, some authors focus on studies in a specific geographic region, others on studies with cost-effectiveness data and others on randomized trials). However, even reviews with similar inclusion criteria consider different sets of studies. Furthermore, the different ways in which reviews categorize interventions result in some conflicting conclusions, even when considering the same studies.

Intervention Details

Murnane and Ganimian (2014) analyze and interpret evidence from 223 rigorous impact evaluations of educational initiatives conducted in 58 low- and middle-income countries. All recent studies at the time that reached seemingly conflicting conclusions about which interventions would improve educational outcomes were considered. To this end, the interventions were grouped based on their theory of action.

The authors reviewed evaluations of educational interventions in pre-primary, primary, and secondary schools that took place in low- and middle-income countries from 2000 to 2015. The studies included at least one of the following education-related outcomes: enrollment, attendance, repetition, dropout, retention, or measures of student performance or cognitive abilities.

Studies of educational interventions, such as offering computer-assisted learning or changing incentives for teachers, as well as studies of health interventions, such as offering medications and iron supplements, were included.

Methodology Details

To conduct the analysis, the authors included only studies that take advantage of exogenous variation in intervention receipt, either by randomly assigning individuals to a treatment or by exploiting quasi-random variation in treatment assignment from natural experiments. Included studies based on natural experiments use differences-in-differences, regression discontinuities, and/or instrumental variables to estimate the causal impacts of interventions. Studies that attempt to estimate counterfactual results were not considered, whether through the use of matching methods or fixed effects for individuals in pre- and post-comparison.

Results

Murnane and Ganimian (2014) report the effect of interventions on student participation, student performance, or teacher attendance for every 100 US dollars. Thus, the authors highlight four lessons from the studies they reviewed:

1. Reducing the cost of going to school and expanding educational options increases attendance and achievement, but does not consistently increase student achievement;

2. Providing information about school quality, developmentally appropriate parenting practices, and the economic returns to schooling affects parental actions and the performance of children and adolescents;

3. More or better resources improve student performance only if they result in changes in children's daily experiences at school;

4. Well-designed incentives increase teacher effort and the performance of very low-level students, but low-skilled teachers need specific guidance to achieve minimally acceptable levels of instruction.

Public Policy Lessons

The study carried out by Murnane and Ganimian (2014) yields some useful lessons for improving educational provision. In other words, increasing the quantity or quality of resources, at home or at school, had at best modest effects on student achievement. The same is true for initiatives that have increased the school load per day.

There is evidence that addressing student learning needs individually, helping teachers personalize instruction, providing additional assistance to students, or letting students learn at their own pace increases student achievement. Perhaps equally important, these strategies are often more effective in increasing the skills of low-performing students.

Increasing teacher and/or principal effort, involving parents and communities in school management, has not been effective in improving student outcomes — especially in environments where pre-existing capacity is low. Offering incentives to principals and/or teachers improved school performance. However, it has proven difficult to design incentive systems that do not produce dysfunctional responses. Hiring teachers on fixed-term, renewable contracts has increased student performance in the short term, but taking this policy to scale and maintaining it over time presents many challenges.

Professional development aimed at increasing teachers' knowledge of content and pedagogy did not increase student achievement. Pre-prepared teacher lessons have improved student performance in low teacher capacity environments.

The consequences of any school improvement strategy will likely depend on the circumstances of the specific environment. In environments where many children are out of school, it may make sense to focus on demand-side interventions. However, in countries where most children go to school and learning is low, improving instruction is crucial. Likewise, in environments where teacher capacity is very low, making their jobs easier—for example, by tracking students by ability or preparing lessons for teachers—can produce incremental improvements. In countries where teachers have moderate ability but low effort, aligning the incentives of students, parents, and teachers may make more sense, as long as there are mechanisms to discourage, identify, and correct dysfunctional responses.

Much of the information about interventions concerns short-term outcomes for students. Examining the long-term consequences of promising interventions in developing countries is an important goal for future research.

References

MURNANE, Richard J.; GANIMIAN, Alejandro. Improving educational outcomes in developing countries: Lessons from rigorous impact evaluations. NBER Working Paper, no. w20284, 2014.