Cheating Base Rates
One of the most difficult aspects of studying cheating on selection/promotion/training tests and otherwise dishonest behavior in either work or school is figuring out what percentage of employees/students actually engage in such behavior.
Statistical power is often low in research on organizations because sample sizes are restricted to whatever number of employees the researcher has access to. Now imagine if the only a small percentage of those already small samples were the people you were really interested in. Would it be worth it to conduct research on cheating with a sample of 400? How many people engaging in cheating activities would you really get? This is where base rates come in.
In a recent article from Network World, Professor Ed Lazowska, the Computer Science and Engineering department chair at the University of Washington says that roughly 1% – 2% of submitted computer science assignments involve cheating. Computer science is a particularly interesting case because of the nature of the training material; computer science students submit programs and algorithms via computer that can be easily examined for cheating through automated methods. This makes the tracking of cheating, even with such a low base rate, much easier – in this case, by automatically examining the assignment of roughly 2750 students per year.
The 1% – 2% figure is portrayed in the article as a high estimate, but is actually lower than the estimate that two colleagues and I found just a few years ago (Retesting after initial failure, coaching rumors, and warnings against faking in online personality measures for selection). There, we found evidence that roughly 3% of people completing online personality tests for selection purposes engaged in a particular purposeful attempt to increase their scores (and thus their probability of getting hired). But it’s impossible at this point to determine why those two estimates are different, to what extent these base rates change by environment, how test-taker motivation applies, and so on.
So how many cases do we need to investigate what psychological attributes might be relevant to cheating? Well, the depressing thing is that this research gives us a fairly small range of low base rates for cheating behavior. If you want 100 cheaters to study, it looks like you need an overall sample of between 3000 and 10000. Good luck!
|Previous Post:||SIOP 2010: End of Day 3|
|Next Post:||Your Professor Serves at Applebee’s (Or: Why Profs Hate Texting In Class)|