Label It a “Game” to Get Gamification Benefits

Gamification, which refers to the use of game elements in non-game contexts, is commonly used as a way to influence the motivation of people in a variety of contexts, including consumer behavior, employee behavior, and student behavior. Much prior research on gamification has been imprecise in which particular game elements are adopted; for example, a study might implement 3 or 4 game elements simultaneously and compare performance in a “gamified” group to a control group. This generally results in difficult-to-interpret and impractical results; how do you know what aspect of gamification actually changed behavior?  And beyond this challenge, the simple framing of an activity as a “game” can potentially alter behavior even further.  How can we disentangle the effects of game elements from the effect of game framing?

In an upcoming article in Games and Culture, Lieberoth1 sought to explore the effect of such framing empirically, which he called shallow gamification.  In this study, all participants participated in a 30-minute task where they discussed a variety of questions related to solving a “business problem” in groups of 6 – in this case, understanding why student satisfaction statistics were low in the researcher’s department’s recent student evaluation surveys.  Deception was involved.  After the task, participants were told that the experiment was over but that they needed to stick around for an additional 20 minutes to wait for another group to finish, during which time they were able to continue engaging in the task.  Within this procedure, they were randomly assigned to one of three conditions:

• In the control condition, 23 participants just did the task as described.
• In the shallow gamification condition, 22 participants completed the control task but were additionally provided a game board, but no additional game mechanics were used – players simply progressed along the board as they provided answers.
• In the deep gamification condition, 25 participants completed the control task but were additionally provided a game board, and some game mechanics were introduced; specifically, the rating of their discussion was used to determine how many spaces they progressed.

I don’t personally agree with this characterization; I would say that the shallow gamification condition simply incorporates fewer game elements than the deep gamification condition.  I would argue that because the shallow/deep dichotomy is quite artificial – just how much gamification is needed to move from one to the other?

Regardless, did more gamification produce better results than less?  Unfortunately, presumably due to the small sample size, the researchers did not use a hierarchical analytic approach despite the grouping.  This is problematic because there may have been an effect of the composition of gameplay groups – and if you count the number of groups, there were only 3 to 4 per condition, which is a tiny sample.  Instead, the researcher ignored group membership and instead focused upon individual-level effects.  That may or may not have mattered; there’s no way to know given what was reported.

At the individual level though, some interesting results appeared.  Specifically:

• Gamification conditions were rated as more interesting than the control conditions, although the gamification conditions were not significantly different from each other; this may be a power issue, however, given the small sample size (there was about a .2 SD difference between these conditions, with higher scores for the “deep” conditions).  There were no differences on any other dimensions of intrinsic motivation.
• The control condition addressed more items in the task than either of the gamification conditions.
• There were no other differences in behavior between conditions, i.e. time spent on task.

The researcher concluded from this that “framing has a significant effect on enjoyment.”

My major concern with this conclusion is that framing is not really what changed here.  To make conclusions about framing, I would have rather the researchers only changed one thing about the study: did they call the activity a game?  Instead, they presented a game board, which is itself a type of gamification.  The major difference between gamification conditions to me is not that one is shallow and one is deep, but instead that one is gamified with one element (the board) and the other with two (the board and a set of rules).  Would we have seen the same effect with simple framing, i.e., “You’re about to play a game.”?  Would we have seen the same effect with a different gamification element?  What if, for example, only rules had been used, and players had been asked to record their scores on paper?  There is no way to know from this study alone.

Regardless, this study provides a compelling example that relatively simple changes inspired by gaming – which I would argue is the heart of “gamification” – can produce measurable effects on motivation.  Interestingly, the number of discussion items addressed decreased as a result of such gamification.  The researcher suggested that this was because the game framing reduced the feeling of this being a “serious” task.  As the researcher put it:

I surmise that that [sic] adding a playful frame to the task actually took away some of the grit and output orientation of more goal-oriented work.

If this type of gamification reduces performance in some contexts, this is certainly an important starting point for future research. But I am hesitant to attribute this to “shallowness” or “depth” alone.

1. Lieberoth, A. (2014). Shallow gamification: Testing psychological effects of framing an activity as a game Games and Culture DOI: 10.1177/1555412014559978 [↩]