Skip to content

Examining Evidence for Leaderboards and Learning

2015 February 25
by Richard N. Landers

ResearchBlogging.orgAs I described in my last post, gamification is often misused and abused, applied in ways and in situations where it is unlikely to do much good. When we deploy new learning technologies, the ultimate goal of that change should always be clear, first and foremost. So how do you actually go about setting that sort of goal? In an upcoming issue of Simulation & Gaming, Landers and Landers[1] experimentally evaluate the Theory of Gamified Learning to explore not only how leaderboards can be used to improve learning but also to demonstrate the decision-making process that you should engage in before trying them out. This is important, because leaderboards are one of the more contested tools in the gamification toolkit.

On one hand, they are one of the oldest and still most common forms of gamification across contexts, including the learning context. If you’re over about 25, you probably remember a high school teacher or college professor posting grades outside the classroom with your name and your grade. That’s a leaderboard. We’ve been using them to motivate students for a very long time, although the way we use them has been changing recently.

On the other hand, leaderboards are often considered by gamification practitioners with the term PBL: points, badges, and leaderboards. PBL is often used disparagingly to refer to the most overused of all tools in the gamification toolkit. They are overused because they are the easiest to implement; PBL can be applied just about any situation, even if there’s not really a good reason to do so.

In their paper, the researchers followed the story of a course with a problem that gamification was used to solve. In this course, a semester-long project was assigned on a wiki. The goal of this project was for students to conduct independent research on an assigned topic and write a wiki article about their topic. The hope of the instructor for this project was that students would engage with the wiki project throughout the semester, learning along the way. Unfortunately, as any course instructor knows, students tend to procrastinate. In looking at usage data from past semester, the instructor realized that most of the class would only work on the project the week before it was due.

Given this, the instructor turned to the theory of gamified learning to identify what type of gamification would be best to increase the amount of time students spent working on their project. Leaderboards were chosen because they were persistent throughout the entire semester and could be used to provide very clear, well-defined goals related to the amount of time spent working on the project.

If you’ve taken a look at theory of gamified learning, you’ll recognize this as a gamification effort that can increase a learner behavior or change a learner attitude that we already know is important to learning. We already know that spending more time on an assignment is good for learning! That means the next step is to find a gamification technique that will increase that amount of time.

To test if this actually happened, the researchers randomly split the course in half and assigned the halves to either a wiki gamified with a leaderboard or a wiki without a leaderboard.  This experimental design was important in order to conclude that the leaderboard actually caused changes in learning.  The results of this study revealed that this approach worked just as expected. In statistical terms, the amount of time spent on the project mediated the relationship between gamification and project scores.  On average, students experiencing leaderboards made 29.61 more edits to their project than those without leaderboards.

In actually implementing the leaderboard, two sets of decisions were key.  First, all entries on the leaderboard were specifically targeted at increasing the amount of time spent on the leaderboard, which was the focal behavior chosen before the project began.  For example, several leaderboard items tracked who had edited their entry the most times as of multiple time points in the course – after one month, after two months, and by the end of the course.

Second, the leaderboard was optional.  Key to all gamification efforts is that participants must feel that they have a choice to participate.  Just like organizational citizenship behaviors, once you require participation in a game, it’s no longer a game. The same applies to gamification. None of the tasks on the leaderboard were explicitly required to earn a high grade; instead, they simply encouraged students to focus their attention and return frequently to their project, as shown below.

It’s important to be very clear that this project does not demonstrate that leaderboards always benefit learning. Instead, leaderboards are just one example of the many tools available to gamification designers that can be used depending upon the specific goal of gamification. This time it was leaderboards; next time, it may be action language or game fiction. The key takeaway is that you need to decide exactly what you’re trying to change before you dive into choosing your game elements, and then choose an element to meet your instructional goals.

Leaderboard from Landers & Landers (2015)

Leaderboard from Landers & Landers (2015)

  1. Landers, R., & Landers, A. (2015). An Empirical Test of the Theory of Gamified Learning: The Effect of Leaderboards on Time-on-Task and Academic Performance Simulation & Gaming DOI: 10.1177/1046878114563662 []

How to Gamify Your Teaching: The Processes of Gamification

2015 January 28
by Richard N. Landers

ResearchBlogging.orgWhen any new technology is introduced purported to “revolutionize teaching,” people tend to get skeptical. Teaching has been the target of revolution many times, yet the best teaching now tends to resemble the best teaching millennia ago, at least at its core. Gamification is one of the more recent approaches, right there alongside MOOCs, tablets, and “modular content.” Gamification’s sudden popularity a few years ago led to everyone to gamify everything they could get their hands on, and it didn’t generally work that well.

Such gamification failures are most directly attributable to this sort of revolution-thinking. A course designer has a technology in hand. The course designer has heard that the technology is GREAT. But the course designer doesn’t actually have a clear reason to adopt it. So the course designer just arbitrarily slaps it on all sorts of situations that don’t make even a little bit of sense, observe that gamification doesn’t accomplish what it was never intended to accomplish, and declare it a failure.

That’s an awful shame to me, because gamification has a great deal of potential not necessarily to revolutionize teaching, but to improve it. To help people realize their educational goals a little less painfully, and perhaps with a little more fun.

In an upcoming issue of Simulation & Gaming, Landers[1] presents the Theory of Gamified Learning, which provides a framework by which to deploy gamification successfully in instruction. There are two major ways to do it.

First, gamification efforts can increase a learner behavior or change a learner attitude that we already know is important to learning. For example, we already know that students who take a moment to pause and think about their learning while studying (called meta-cognition) tend to learn more. So a gamification effort targeted at increasing meta-cognition is likely to improve learning. It’s not necessarily going to be transformative or revolutionary or powerful or whatever other gag-inducing word you want to come up with.  Instead, it’s quite simple: when we use game elements to help people do what they know they should do anyway, everyone wins.

A great example of this sort of process are gamified fitness apps. Everyone knows they should exercise. We’re bombarded with messages about obesity epidemics and shorter lifespans and all sorts of horrible things. We know this. It’s quite clear at this point – exercise is good for you, vital even. Yet despite that clarity of purpose, it can be quite the fight uphill to drag oneself to the gym. Fitness apps make that a little less of a drag by promising a quick and easy reward/recognition once the workout is complete.  It’s not compelling you to act differently.  It’s not forcing you to play. It just encourages you to do something that you know you should be doing anyway.

Effective gamification in education that takes this first approach follows the same idea. Effective gamification in education encourages learners to do things they know they should be doing anyway, or perhaps even a step further, it encourages learners to try things they might otherwise be too afraid or indifferent to try.

This is why one of the motivational principles of the Theory of Gamified Learning is that you shouldn’t force learners into participating. Gamification should always recognize and encourage behaviors that are helpful, but not critical. If an activity is critical to learning, it’s not a game. But if it’s something that would be good for learning, game away.

Second, gamification efforts can increase a learner behavior that makes existing instruction more effective. For example, imagine you’ve spent hours developing what is a fabulous set of review questions. You’ve made amazing connections that will make everything so clear! But your learners just aren’t into it. Engagement is low. So we need to increase engagement. This is the process by which review games work – it’s not that the review game itself teaches you anything. Instead, by presenting the review questions as a game, you encourage students to really think about the answers to the questions in a way that they would not otherwise have tried.

Either approach can be effective. But it must be targeted. You can’t simply throw gamification at learning and hope it to stick. One of these two processes must be targeted, which are summarized in the figure below.

Theory of Gamified Learning

The Theory of Gamified Learning, per Landers

So now that we have a process identified, how do we actually go about gamifying? Fortunately, prior research has already explored which aspects of serious games are typically manipulated in order to influence learning. If our goal is to extract some aspect of these games in order to change something about learners, this sounds like a great place to start. These aspects and examples of gamification appear below. Some of these are likely to work better than others, or in combination. But that is where the research is going next!

Game AttributeDefinitionExample of Gamification
Action LanguageThe method and interface by which communication occurs between a player and the game itselfTo participate in an online learning activity, students are now required to use game console controllers (e.g. a PlayStation controller).
AssessmentThe method by which accomplishment and game progress are trackedIn a learning activity, points are used to track the number of correct answers obtained by each learner as each learner completes the activity.
Conflict/ChallengeThe problems faced by players, including both the nature and difficulty of those problemsA small group discussion activity is augmented such that each small group competes for the “best” answer.
ControlThe degree to which players are able to alter the game, and the degree to which the game alters itself in responseA small group discussion activity is restructured such that each decision made by each small group influences the next topic that group will discuss.
EnvironmentThe representation of the physical surroundings of the playerA class meeting is moved from a physical classroom to a 3D virtual world.
Game FictionThe fictional game world and storyLectures, tests, and discussions are renamed adventures, monsters, and councils, respectively.
Human InteractionThe degree to which players interact with other players in both space and timeLearners participate in an online system which reports on their assignment progress to other students as they work.
ImmersionThe affective and perceptual experience of a gameWhen learning about oceanography, the walls of the classroom are replaced with monitors displaying real-time images captured from the sea floor.
Rules/GoalsClearly defined rules, goals, and information on progress towards those goals, provided to the player.When completing worksheet assignments on tablet computers, a progress bar is displayed to indicate how much of the assignment has been completed (but not necessarily the number of correct answers, which would fall under “Assessment”).
  1. Landers, R.N. (2015). Developing a Theory of Gamified Learning: Linking serious games and gamification of learning Simulation & Gaming DOI: 10.1177/1046878114563660 []

Psychological Theory and Gamification of Learning

2015 January 15
by Richard N. Landers

ResearchBlogging.orgGamification, the use of game elements in non-game contexts, is increasingly being implemented in both student and organizational learning initiatives. Many of these efforts are atheoretical, meaning that the teachers using them don’t necessarily have a well-grounded reason for gamifying. Instead, they often gamify with the intention of making learning more “fun.”

Unfortunately, 1) not all gamification is fun and 2) fun is sometimes counterproductive. It is common belief by educators that I’ve spoken with that fun and learning are opposed – as you increase the fun of a learning activity, its impact on learning goes down.

Personally, that doesn’t make a lot of sense to me. Fun and learning should go hand in hand. It seems much more likely that the ways learning activities are being made fun are the problem – not the fun itself.

So if an instructor does want to gamify learning, what theories might be used to guide it? How can gamification be used that won’t potentially harm learning, fun or not?

Recent research by Landers, Bauer, Callan and Armstrong[1] sheds some light on this question by exploring several psychological theories of learning in relation to gamification. Specifically, they identify categories of theories that speak to gamification.

  1. Theory of Gamified Learning.  First, and perhaps most relevant, is the theory of gamified learning[2]. This theory proposes that gamification can affect learning via one of two processes and is intended to guide decision-making when creating gamified activities. Critically to both, gamification should not be intended just to “get people to learn,” and gamification cannot replace high-quality instruction.  Instead, it should be targeted at learner behavior and attitudes.
    1. Gamification can target a behavior or attitude that we already know affects learning. For example, we already know that students who spend more time engaging in meta-cognition (thinking about how they learn) tend to have higher grades. Thus, gamification might be used to increase meta-cognition (e.g., a mobile app might be used to reward students who “check in” to studying).
    2. Gamification can target a behavior or attitude that makes existing instruction more effective. We might have a great lesson plan to teach oceanography, but students might be bored.  To increase their interest, we might bring in an interactive demonstration to illustrate key points.  In such cases, the demonstration doesn’t actually teach anything new – it is a type of gamification intended to increase student engagement.
  2. Conditioning.  Classical and operant conditioning are classic psychological theories of learning. Although they have been generally replaced and supplemented with many other theories over the years, they continue to explain a great deal of behavior, especially in children. The most successful type of psychological therapy, cognitive behavioral therapy, is even largely based upon operant conditioning principles (the “behavioral” part).  At its core, conditioning is quite simple – the goal of the instructor is to create a positive association with a beneficial activity (like studying).
  3. Expectancy Theories. Expectancy theory is used to describe human motivation.  It actually describes three separate motivational processes, listed below. Any particular learning activity can be described as the product of these three processes – if any of the three are low, there will be low motivation.
    1. Expectancy, which is how likely you believe your actions will lead to a consequence. For example, if I work hard, I’ll score high on the leaderboard.
    2. Instrumentality, which is how likely you believe that consequence will lead to a reward. For example, if I score high on the leaderboard, I’ll feel good about myself.
    3. Valence, which is the value you place on that reward. For example, feeling good about myself is very important.
      In combination, we might conclude that a leaderboard is likely to be unsuccessful 1) if students don’t think their effort will lead to a high position on the leaderboard, and 2) if students don’t think that a high position on the leaderboard will lead to anything they want.
  4. Goal-Setting Theory. Goal-setting theory is one of the most well-supported theories of motivation in psychology, and it applies well to learning.  People are motivated by SMART goals: specific, measurable, attainable, realistic, and time-bound.  If you incorporate SMART goals in gamification, you are much more likely to get students to do what you want them to do.
  5. Self-Determination Theory. One of the more recent theories of motivation is self-determination theory, which posits that all humans are motivated by a drive to self-determine – to identify a path for themselves forward through life. This is done by meeting three needs: feeling competent in the tasks you attempt, feeling that you have accomplished those tasks without the influence of others, and feeling that your life is connected to those around you.  SDT also defines two types of motivation: intrinsic, which refers to the motivation needs met by satisfaction of those three needs, and extrinsic, which is motivation that is not self-determined.  Other-determined motivation is most often considered synonymous with incentives, like gold stars, grades, and recognition from others.  Recent work in SDT has revealed that intrinsic and extrinsic motivations work together – that intrinsic motivation occurs when tasks have been internalized, whereas extrinsic motivators are most useful to get people to try new tasks. For example, a person being forced to play piano by a parent might hate it at first but do it anyone to make her parents happy, but later, as she becomes highly competent and autonomous playing, that playing becomes enjoyable on its own. Gamification can be used the same way, to introduce a person to something they don’t have any experience with or know they are good at so that they develop intrinsic motivation later.

Overall, no single theory is going to explain gamification or be the magic bullet for successful gamification. But these theories provide a strong foundation on which to build such efforts.

  1. Landers, R.N., Bauer, K.N., Callan, R.C., & Armstrong, M.B. (2015). Psychological theory and the gamification of learning Gamification in Education and Business, 165-186 []
  2. Landers, R. N. (2015). Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45, 752-768. []