Ruhr Economic Papers
Ruhr Economic Papers #603
The Effect of Supplemental Instruction on Academic Performance – An Encouragement Design Experiment
by Alfredo Paloyo, Sally Rogan and Peter Siminski
RWI, 02/2016, 42 S./p., 8 Euro, ISBN 978-3-86788-700-7 DOI: http://dx.doi.org/10.4419/86788700download
While randomized controlled trials (RCTs) are the “gold standard” for impact evaluation, they face numerous practical barriers to implementation. In some circumstances, a randomized-encouragement design (RED) is a viable alternative, but applications are surprisingly rare. We discuss the strengths and challenges of RED and apply it to evaluate a mature Supplemental Instruction (SI) or PASS (Peer Assisted Study Session) program at an Australian university. A randomly selected subgroup of students from first-year courses (N = 6954) was offered large incentives (worth AUD 55,000) to attend PASS, which increased attendance by an estimated 0.47 hours each. This first-stage (inducement) effect did not vary with the size of the incentive and was larger (0.89) for students from disadvantaged backgrounds. Instrumental variable estimates suggest that one hour of PASS improved grades by 0.065 standard deviations, which is consistent with the non-experimental literature. However, this estimate is not statistically significant, reflecting limited statistical power. The estimated effect is largest for students in their first semester at university.
JEL-Classification: C93, I21, I23, I24
Keywords: Australia; randomized-encouragement design; student outcomes; peer-assisted study session; supplemental instruction; selection bias