Twitter vs. Student Engagement and Grades

In a recent article in the Journal of Computer Assisted Learning, Junco, Heiberger and Loken1 examine the value of Twitter for improving student engagement and grades.  Junco et al. concluded that student engagement and semester GPAs were both improved through the addition of Twitter to a course.

The study design was somewhat peculiar.  Seven sections of a one-credit first-year pre-health seminar course formed the setting for the study.  Four of these sections were randomly assigned as the experimental group (74 students) and three sections were assigned to the control group (58 students).  Students in the experimental group used Twitter as part of their class, while the control group did not.  All sections used Ning instead of a standard LMS like Blackboard/WebCT/Moodle.  As students were not randomly assigned to conditions, the present study is a quasi-experiment.

Student composition is an important issue to explore here, and the specifics of this sample may have contributed to Junco et al.’s results.  98% of the sample was between 18 and 19 years old, 60% were female, 91% were white, and 72% had at least one parent with a college degree.  This creates a picture of a very specific kind of student – and perhaps a type that would be more receptive to a Twitter-enhanced course.

So how was Twitter actually used?  In the second week of the course, students received an hour-long training session on how to use Twitter – how they filled an hour, I have no idea, although students were required to send an introductory tweet during this session.  Q&A was then held over the next few weeks during class.  Twitter was then used for at least 11 purposes:

• Continuing class discussions online
• Providing a low-stress way to ask questions
• Discussing a required book
• Providing course-related reminders
• Providing campus-related reminders
• Providing information about campus events and resources (e.g. information on the campus tutoring center)
• Helping students connect in the context of the course
• Organizing service learning
• Organizing study groups
• Posting optional assignments (which required Twitter)
• Posting required assignments (which also required Twitter)

Here, we encounter a common problem with technology studies: there is no way to disentangle the effect of 1) providing additional learning opportunities in general and 2) using the technology.  These students got extra assignments, extra attention, and so on.  How do we know it was Twitter and not the additional opportunities provided that produced any observed results?

The results themselves were small but statistically significant.  The researchers used a mixed effects ANOVA with sections nested within condition to try to account for course section being confounded with treatment condition (remember, this is a quasi-experiment).  They found statistically significantly higher difference scores for the experimental group than the control group, indicating that engagement increased to a greater extent for the Twitter users than the non-Twitter users.

Differences in “semester GPA” were also found.  This may refer to overall semester GPA or just semester GPA for the course in question – this is not made clear.  But again, differences were found – 2.28 vs. 2.79.  The researchers also examined this with high school GPA as the dependent variable to provide some evidence against bias introduced by quasi-experimentation.  No statistically significant difference was found, somewhat supporting this assertion.

There is little information given on how the instructors of these courses accounted for experimenter bias, and the researchers admit this makes interpretation tricky.  The use of Twitter forced the instructors themselves to engage to a much greater degree than they did for the control group, which likely affected student perception, participation, and results.  To what extent Twitter helps students more than it motivates instructors engage with students is left to future research.

1. Junco, R., Heiberger, G., & Loken, E. (2011). The effect of Twitter on college student engagement and grades Journal of Computer Assisted Learning, 27 (2), 119-132 DOI: 10.1111/j.1365-2729.2010.00387.x [↩]