Research

Publications

Rank Response Functions in an Online Learning Environment

December 2021 with Valentin Wagner and Isabell Zipperle
Economic Letters
We estimate rank response functions after receiving rank-order feedback in an online learning platform. We find that the shapes of the rank response functions depend on the outcome measure under consideration. For our effort measure, i.e., whether learners continue to interact with the platform, we can reject a linear rank response function and find significant evidence of a U-shaped relationship. For our performance measure, i.e., correctly solved exercises, we find no clear pattern overall but suggestive evidence for a linearly decreasing rank response function for individuals in the lower half of the ability distribution, i.e., the lower the rank the lower the performance.

Projects and Working Papers

A Field Experiment on Adaptive Learning Applying a Machine Learning Algorithm

in the field, first version in 2022 with Marius Köppel, Daniel Schunk, and Isabell Zipperle
It remains an open question how to adapt and individualize learning contents. To tackle this in a digital learning context, we developed an algorithm based on a convolutional neural network that assigns tasks to the learners. Our application is a large online learning platform in which we run a randomized controlled trial. Participants are randomized into three groups: two treatment groups – a group-based adaptive treatment group and an individualized adaptive treatment group – and one control group. We analyze the difference between the three groups with respect to effort learners provide within the platform, their performance within the platform, formative assessments within the platform and their final high-stake standardized exams (summative assessment).

Feedback in Homogeneous Ability Groups: A Field Experiment

June 2021
Relative performance feedback (RPF) often increases effort and performance on average. However, in the context of education, learners with low ability are observed to reduce effort and performance when RPF is provided. In a randomized field experiment with 7352 learners, we sort treated learners into anonymous homogeneous ability feedback groups. These learners receive RPF in groups of learners with the same ability level. Futher we observe a control group that does not receive feedback and a group that face RPF in anonymous heterogeneous ability feedback groups. We find that on average RPF increases learning effort by 0.11 standard deviations with no additional effect average effect of homogeneous ability groups. Further, we find that weak learners in homogeneous ability feedback groups learn insignificantly more and strong learners insignificantly less compared to learners in more heterogeneous groups. This is in line with a flatter skill formation curve or insignificantly less inequality due to feedback in homogeneous ability groups.

Shedding light on the effects of adaptive learning: A randomized field experiment in an online learning platform

Mai 2021 with Daniel Schunk
In the context of the digital transformation of teaching and learning, digitalization enables a high degree of adaptivity, thus potentially improving the effectiveness of education. However, conclusive evidence on the effect of adaptive digital teaching approaches on learning processes and learning success is still scarce. Here, we first develop an algorithm that operationalizes adaptive learning based on the difficulty levels of learning tasks. Then, we conduct a randomized controlled trial in the context of a large digital learning platform and compare a non-adaptive control group with two treatment groups that differ in their degree of adaptivity. We find that adaptivity significantly increased learning effort in both treatment groups but did not lead to better test scores in formative or summative assessments. We discuss reasons for these findings to shed light on the complex challenges associated with effectively operationalizing adaptive digital education approaches.

Investment into Identity in the Field – Nudging Refugees’ Integration Effort

January 2021 with Nora Grote and Mario Scharfbillig
Revise and Resubmit at the European Journal of Political Economics
Social identity greatly affects behavior. However, less is known about individual’s investment into identification, i.e. into belonging to a social group. We design a field experiment that allows us to make effort as an investment into a new group identity salient. The social identity in our treatment is refugee’s identification with the host society. We modified a mailing to 5600 refugees who use an online language-learning platform to learn the host countries’ language. These treatment emails make salient that improving the host country’s language ability increases the belonging to the host society. Our analysis reveals that the treatment has a significant positive effect on the effort exerted on the language-learning platform, leading to more completed exercises and more time spent learning the host country’s language. This suggests that refugees’ invest into being part of the host country’s society for its social identity component.

Conferences

  • Field Days: Experiments outside the laboratory, online, Dec 2020
    Presenting “Investment into Identity in the Field – Nudging Refugees’ Integration Effort”
  • Advances with Field Experiments, online, Oct 2020
    Presenting “Feedback in Homogeneous Ability Groups: A Field Experiment”
  • Field Days and INSEAD RCT days: Experiments outside the laboratory, INSEAD Europe Fontainebleau, Nov 2019
    Presenting a poster on “Understanding Adaptive Learning with a Field Experiment”
  • Advances with Field Experiments, Boston University, Oct 2018
    Presenting “Preference for Identification in the Field – Nudging Refugees’ Integration Effort”
  • Nordic Conference on Behavioral and Experimental Economics, USD Odense, Sep 2018
    Presenting “Preference for Identification in the Field – Nudging Refugees’ Integration Effort”
  • Current Trends in Public Sector Research, Masaryk University, Brno, Jan 2018
    Presenting “Preference for Identification in the Field – Nudging Refugees’ Integration Effort”
  • International Workshop Economics of Education and Self-Regulation, University of Mainz, Oct 2015
    Participant