HOW DOES THE AWARD FOR ACADEMIC PERFORMANCE AFFECT STUDENTS’ RESULTS IN PROFICIENCY EXAMS?

Responsible researcher: Bruno Benevit

Authors: Joshua Angrist and Victor Lavy

Intervention Location: Israel

Sample Size: 12,188 children

Sector: Education

Variable of Main Interest: High School Certification

Type of Intervention: Award for certification

Methodology: OLS and Logit

Summary

High school graduation exam certificates are usually associated with entry into higher education, productivity gains and income. Despite the potential economic value attached to these certificates, some societies experience large differences in completion rates resulting from regional and socioeconomic differences between individuals. To test how prizes for obtaining the Israeli Bagrut would affect students' chances of obtaining the certificate, the authors of this study conducted an experiment with randomly defined schools. Adopting the OLS and Logit methods, the results demonstrated that only girls from the included schools had an increase in their chances of certification, resulting in an increase in the probability of entering colleges. This result was partially explained by the increase in time dedicated to preparing for the exam. No significant effects were identified for boys.

1. Policy Problem

High school graduation certification is a formal prerequisite for admission to universities in Israel, called Bagrut. According to Angrist & Lavy (2009), certification holders have 25% higher salaries. However, there are large differences in Israeli society in Bagrut between regions and arising from students' socioeconomic background. Israeli educators have adopted several programs aimed at changing this scenario, but without apparent effect. Therefore, policymakers are challenged to identify which economic incentives are associated with this phenomenon.

2. Implementation and Evaluation Context

Obtaining the Bagrut consists of a series of national exams in basic and elective subjects that begin in the tenth grade and end in the twelfth grade. Bagrut certificates are usually obtained at the end of the final year (or later), the year in which most exams are taken. Students choose tests with varying proficiency levels. Each test awards one to five units of credit per subject, depending on the difficulty selected. To qualify for a Bagrut a minimum of 20 units of credits is required, although some study programs require more and/or a minimum credit distribution requirement. According to the Israeli Ministry of Education (2001), approximately 52% of all high school students received a certificate of enrollment in the 1999 and 2000 cohorts.

This article conducted a randomized experiment with Achievement Awards , a project that offered cash prizes to low-performing high school students in Israel who completed and performed well on the Bagrut . The need to enter the job market and the uncertainties related to the returns to education may explain this phenomenon. In this sense, immediate financial provision can be an effective tool to encourage the involvement of this group of students in Bagrut certification.

3. Policy/Program Details

The selection of the experiment took place in December 2000 with the selection of 40 non-professional secondary schools with the lowest Bagrut in 1999 in a national ranking. Additionally, only schools above the minimum rate of 3% were chosen, as schools with low rates would probably not benefit from the program. The total number of schools treated was determined by the program's budget constraint, about $650,000. The list of participating schools included 10 Arab religious schools, 10 Jewish religious schools and 20 Jewish secular schools. The allocation of treatment, the offering of prizes to the Bagrut , took place randomly. Only schools selected for treatment were informed about the program. Selection used a matching strategy that paired treatment and control schools based on lagged values ​​of the 1999 Bagrut

4. Method

The baseline data January 2001, and the main outcome for the cohort of schools that underwent the intervention were collected in June 2001. Additional surveys were carried out between 2001 and 2002 to collect data. data on the results of the examination. Bagrut results , school characteristics, basic socioeconomic information and students' lagged academic performance scores. The sample has 12,188 students distributed among the 39 schools that participated in the experiment, of which 6,267 are boys and 5,921 are girls, covering the period from 2000 to 2002.

The article presents several analyzes to investigate the effects of the experiment. Two econometric models were established. In the first model, the coefficients for the intervention of the school certification premium (treatment), school characteristics and social characteristics of the students' families were considered. The second model incorporated new variables from lagged measures of credit scores in pre-2001 exams. In the first analysis, the authors verified the marginal effects of the experiment using the OLS and Logit methods to estimate the two econometric models with and without peer effects. of schools. The second analysis applied the Logit method to identify the profile of students with more marginal chances of benefiting from the intervention among the group of schools that did not undergo intervention in the experiment for the year 2001.

In order to identify the profile of the most benefiting students, several other analyzes were carried out applying Logit models with different specifications, disregarding mothers' education or incorporating school fixed effects. Additionally, the effects were estimated for several subgroups taking into account the distributions of the lagged score or the probability of certification.

To understand which channels caused changes in the chance of certification, the study also analyzed the effect of the experiment on outcome variables relating to students' academic performance: credit attempts, exam performance and chances of satisfying Bagrut requirements . Finally, the authors verified whether the increase in the chance of certification was reflected in a greater probability of entering higher education.

• Main Results

Results across all analyzes demonstrated that offering awards to students in a low-performing school had no effect on boys' certification rates. However, the results for girls indicate an increase in the certification rate. This effect is due to the subgroup of students with more chances of certification, generating an increase of around 10% in models that allow the omission of fixed school effects. Analyzing data from the Ministry of Education's monitoring survey (2001) suggests that this effect can be explained by the greater dedication of time to studies by girls in preparatory courses for Bagrut .

Regarding the mechanisms that explain how the change in incentives modified the students' study process, the results show positive effects for girls. Increases were seen in the attempt to obtain 20 units of credits, in the chance of obtaining 18 and 20 units of credits, and in the distribution requirements in mathematics and writing tests. Again, no significant effects were identified for boys. The difference between the response to financial incentives for higher education may explain such behavior.

The observed increase in female student certification rates translated into increased enrollment in higher education. This effect occurred in the groups in the upper half of the lagged score distribution, with an increase of 12 to 13 percentage points in the fourth quartile and around 7 percentage points in the upper half.

6. Public Policy Lessons

In this article, the authors carried out a randomized experiment in schools to test the impact of granting prizes to all students in the treated schools who achieved the secondary school leaving certificate. The results showed a substantial increase in certification rates for girls, but no significant effect was observed for boys. The affected students were already part of the group with a high chance of certification, indicating that the experiment was not effective in increasing the chance of certification for students with low academic performance. The increase in girls' certification rates has resulted in an increase in the probability of entering college. Additionally, the increase in girls' certification rates was partly due to students in the targeted schools spending more time preparing for exams. No evidence of negative effects resulting from offering prizes to students was identified.

References

Angrist, J. and Lavy, V. (2009), “The Effects of High Stakes High School Achievement Awards: Evidence from a Randomized Trial”, American Economic Review , Vol. 99 No. 4, pp. 1384–1414.

Israel Ministry of Education. 2001. Statistics of the Matriculation Examination (Bagrut) Test Data, 2000. Jerusalem: Ministry of Education Chief Scientist's Office.