Connections Matter: How Interactive Peers Affect Students in Online College Courses
GAICIL Insights
The chat log of an online course is a goldmine that most universities never open.1 Connections Matter: How Interactive Peers Affect Students in Online College Courses, by Eric Bettinger, Jing Liu and Susanna Loeb, opens it. Peers shape productivity in workplaces, classrooms and teams, and the authors measure peer effects using large-scale language data from online college courses. Students are quasi-randomly assigned to peers, overcoming selection biases from endogenous peer grouping, and rich interaction data help mitigate reflection bias.
The findings reveal who talks and who benefits. Females and older students are more likely to engage in student interactions. Students are more likely to interact with peers of the same gender and from roughly the same geographic region. For students relatively less likely to engage in online discussion, exposure to more interactive peers increases their probability of passing the course, improves their grades and raises their likelihood of enrolling the following term. The quiet student gains most from a lively room.
The method is as significant as the result. Because student behavior is observable online, researchers can examine interactions directly and assess how peer behaviors, beyond background characteristics, affect learning and attainment. The authors go beyond counting posts to analyzing the content of student language, using natural language processing to create measures of interaction, an approach that could easily scale. It is the first study the authors know of to examine large-scale language data from online courses to assess peer effects within a rigorous causal framework.
That is a preview of artificial intelligence as a research instrument. Natural language processing turns the text students produce into measurable data about engagement and learning. Modern language models make such analysis far more powerful and accessible. Educators can study classroom processes at a scale and depth once impossible, identifying the interactions that support learning. The discussion board becomes a microscope.
The practical uses are immediate. If interactive peers help less-engaged students pass and persist, course designers can structure groups to expose such students to engaged peers. AI could measure interaction in real time and suggest groupings or prompts that foster productive discussion. The tendency to interact with similar peers suggests deliberate design may be needed to broaden connections across groups. Left alone, students sort; a nudge can mix them.
AI might play a more active role still. Conversational agents could stimulate discussion, respond to posts and model the kind of interaction that benefits less-engaged students. Whether an artificial interlocutor delivers the same benefits as a human peer is open, and the causal framework demonstrated here offers a method for answering it. A bot that posts more is not automatically a peer that helps.
Language data bring privacy and consent questions. Analyzing what students write yields valuable insights and requires safeguards. Institutions adopting such techniques should be transparent with students and protect their data. Clear rules on how text data are collected, stored and used make it easier to adopt these methods with confidence and sustain trust while education benefits from AI's analytic power.
For policy, online education should be designed around interaction. The benefits of online courses depend partly on the connections students form. Investments in course design, discussion structures and analytic tools that foster engagement can improve outcomes, especially for students at risk of disengaging. AI-based tools that monitor and support interaction could become a valuable part of that design.
Distilled to one claim: learning is social, even online. Peers who engage help others succeed, and the effects can be measured using the text students generate. AI has made such measurement scalable, opening new possibilities for understanding and improving education. This study combines natural language processing with rigorous causal inference to reveal how connections matter for student success.
A hard question for online program directors: the most valuable seat assignment in a virtual course may be placing a disengaged student next to a talkative one. That costs almost nothing, and the evidence suggests it pays off in passing grades and re-enrollment.
Introduction
Peers affect individual's productivity in the workforce, in education, and in other team-based tasks. Using large-scale language data from an online college course, we measure the impacts of peer interactions on student learning outcomes and persistence. In our setting, students are quasi-randomly assigned to peers, and as such, we are able to overcome selection biases stemming from endogenous peer grouping. We also mitigate reflection bias by utilizing rich student interaction data. We find that females and older students are more likely to engage in student interactions. Students are also more likely to interact with peers of the same gender and with peers from roughly the same geographic region. For students who are relatively less likely to be engaged in online discussion, exposure to more interactive peers increases their probabilities of passing the course, improves their grade in the course, and increases their likelihood of enrolling in the following academic term. This study demonstrates how the use of large-scale, text-based data can provide insights into students’ learning processes.
Takeaway
The rapid increase in online courses and the accompanying data creates opportunities to investigate peer effects from new perspectives. Because it is possible to observe student behaviors online, researchers can examine student interactions directly and assess how peer behaviors, not just peer background characteristics, affect student learning and educational attainment. Beyond measuring student actions – such as posting length and frequency – we look into the content of student language, capturing an element of student interaction. Using data from all online student postings, we use natural language processing techniques to create measures of student interaction, an approach that could be used easily on an even larger scale.1
We find that females and older students are more likely to engage in student interactions. In addition, students do not choose which peers to interact with randomly. Students are more likely to interact with peers who share the same gender and residential areas. They also are likely to exhibit other forms of homophily, but the student characteristic measures in this study are limited. In contrast, not all patterns reflect homophily; for example, younger students tend to nominate older peers. Given that students have varying engagement levels in student interactions, we examine the heterogeneous effects of peers’ interpersonal interaction on student learning outcomes. We find that for students who are relatively less likely to be engaged in online discussion, exposure to more active peers increases their probabilities of passing the course, earned grade and course points. These effects are not large, but they are meaningful. With engaging peers, students with a 50 percent chance of being referred to by peers perform 13 percent of a standard deviation higher in grades and increase their probability of enrollment in the following term by more than 6 percentage points. This paper provides evidence that peers can affect students by engaging them more in class discussions and this engagement can benefit their course performance. This result has direct implications for policies and practices that aim to improve student learning in online settings.
An increasing research literature indicates that students on the margin of taking online courses instead of face-to-face courses perform substantially worse in the online setting (Bettinger et al., 2015; Hart & Friedmann, 2014).2 A possible explanation for these negative effects, especially among lower performing students, is that the online setting does not engage them as well. Peer (and professor) actions that engage students hold promise for improving these students’ educational outcomes, especially in these online settings. While these settings may, on average, be less effective for students today, with improvements in engagement practices they may be able to equal in-person in engagement while maintaining the benefits of online courses, such as reduced travel time and flexible scheduling. As college online courses grow at an accelerating speed, policymakers and education administrators are searching to find effective policy instruments to enhance student experiences, and improve their learning outcomes. Online courses provide richer data because interactions are captured and can be used for analysis. They also are easier to manipulate than in-person courses, and, thus, can allow for research to assess the effectiveness of different approaches. Some of these approaches are likely to be specific to online settings, but others likely are generalizable to teaching and learning in higher education more generally. This paper finds positive causal effects of peers’ interpersonal interaction on less engaged students; and, thus, both provides an indication of the barrier that lack of engagement presents in online learning and some of the first evidence on productive interventions to engage students online.
As one of the first studies in this area, our measure of peer interaction has clear short-comings, and it is possible that we understate the power of peer interaction due to the limitation of our measure. Future research should build on the current work to develop constructs that can better capture the richness of peer interactions.
Notes
- Eric Bettinger, Jing Liu & Susanna Loeb, Connections Matter: How Interactive Peers Affect Students in Online College Courses, 35 J. Pol'y Analysis & Mgmt. 932 (2016), https://doi.org/10.1002/pam.21932. Takeaway source: DISCUSSION AND CONCLUSION, reproduced from a free author or repository copy (working-paper version). ↩
- Hart, Cassandra, Elizabeth Friedmann, and M. H. (2014). Online Course-Taking and Student Outcomes in California Community Colleges. In Association for Public Policy Analysis & Management. Albequerque, NM.. ↩
Cite as: Eric Bettinger, Jing Liu & Susanna Loeb, Connections Matter: How Interactive Peers Affect Students in Online College Courses, 35 J. Pol'y Analysis & Mgmt. 932 (2016), https://doi.org/10.1002/pam.21932
Header image: BM Reading Room, November 2024.jpg by 14GTR, CC0, via Wikimedia Commons.
