Setting the Difficulty of Serious Training Games

A recent article in GamePro discusses the continuing disconnect between gamers and game studios: gamers say they want more challenging games, game studios produce challenging games, and then those games often don’t perform as well on the market as they should.  The idea is summed up here:

The problem is, the vast majority of gamers don’t really behave the way they say they do. How do we know this? Because an increasing number of games incorporate telemetry systems that track our every action. They measure the time we play, they watch where we get stuck, and they broadcast our behavior back to the people that make the games so they can tune the experience accordingly.

Every studio I’ve spoken to that does this, to a fault, says that many of the games they’ve released are far too big and far too hard for most players’ behavior. As a general rule, less than five percent of a game’s audience plays a title through to completion. I’ve had several studios tell me that their general observation is that “more than 90 percent” of a games audience will play it for “just four or five hours.”

This provides an important insight for the training difficulty regarding difficulty level.  When designing training games, they can’t be too hard, or you risk the player becoming frustrated and quitting.

Or perhaps even worse, because trainees are typically a captive audience, instead of quitting they will simply disengage.  Your trainees will only go through the motions only because they are required to, and a result, will not internalize as much of the training material as those that are engaged.  This is especially troublesome because the utility/ROI of your already pricey training system will drop.  As one of the most expensive varieties of training system, you need the highest returns you can get.

How to appropriately design a training game?  Extensive pilot testing.  Take 10 to 20 trainees, put them through the training game, survey their frustration with the game, and also plot a histogram of their performance levels.  Your target for the frustration distribution is a moderate skew towards the low end, and your target for the performance distribution is a normal curve.

Why do we want moderate skew to frustration?  If it’s normally distributed, that means roughly half of your trainees are frustrated with your training program, and that’s not what you want.  If it’s heavily skewed, that means your performance distribution is likely skewed too; it’s too easy, and no one is being challenged.  The key, like in the retail games industry, is finding the sweet spot.