All Current Evidence for Second Life in Business and Education

I recently got into a discussion with Flemming Nielson on Anja Patulski Nielson’s Weblog about what evidence is available to support the use of virtual worlds for training and education.  Flemming argued that a number of high-profile case studies are available (especially from IBM); I argued that there is little scientific support.  But I soon realized that all I knew about was the lack of scientific support in my home research area, industrial/organizational psychology, the application of psychology to business principles specifically.  What about the wider psychological literature, especially relating to education?  What evidence is there that virtual worlds produce learning outcomes as strong as traditional classrooms and instructional methods?

A simple search for [“Second Life” or “virtual world”] produced the following psychological studies relevant to my question.  Criteria for inclusion in this list:

1. Considers adult learning in some regard (college courses, training and development, includes lab studies)
2. Collects empirical data (research volunteers, trainees, or students)
3. Is in English  (I have to be able to read it)
4. Can be retrieved easily (2-3 articles that might have otherwise fit in this list would have taken several days of inter-library loan to retrieve)
5. Is indexed by APA’s PsycINFO
6. Has been peer-reviewed
7. Discusses true 3D virtual worlds (not just massively multiplayer games or multi-user learning systems).  It should be noted that there is some confusion in the scholarly literature as to what exactly a virtual world is.  Virtual worlds differ primarily from massively multiplayer games in that there are no set objectives in a virtual world – Second Life qualifies, World of Warcraft does not.
8. Some effort must have been made to prove virtual worlds as a reasonable alternative or complement to traditional education/training practices.  I would have included “unreasonable” as well, except that no study study set out with this objective.

My initial search turned up a couple hundred, but few of those met these qualifications.  Every study I could locate in psychology within those guidelines is in the following list:

• De Lucia, Francese, Passero & Tortora (2009)1 examined the reactions of 26 students participating in 3 virtual SL lectures.  Unfortunately, evaluation data is never recorded, so we don’t really know how Second Life compared to traditional instruction.  They did find that participants had some degree of a “sense of presence,” but that sense is not connected to any real outcomes (like learning).
• Jarmon, Traphagan, Mayrath & Trivedi (2009)2 conducted a qualitative study that I think is typical of this literature; a virtual world was used as part of an assignment for 5 graduate students to examine communications, and all sorts of stories and personal explanations were collected.  It’s fantastic that SL worked well in this context; the question that this study cannot answer is “will it work in other contexts?”
• Jamaludin, Chee and Ho (2009)3 randomly assigned to groups which role-played in the virtual world.  While I agree this is certainly a good educational use for Second Life that takes advantage of the technology, no effort is made to compare it to traditional in-person role-playing.  Without it, there’s no way know if SL is a reasonable substitute for traditional instruction.
• Lester and King (2009)4 is one of the better designed studies I saw.  An online course held in Second Life was compared with a traditional course in terms of both student reactions and grades.  Scores were similar across all grades, which is promising, although standard deviations (and standardized effect sizes) were not reported, so it is impossible to say just how similar they really were.
• Good, Howland and Thackray (2008)5 is another qualitative study examining the use of SL in a classroom setting.  No comparison group, no statistics.  Just a story, albeit an interesting one.
• Edirisingha, Nie, Pluciennik & Young (2009)6 describe yet another qualitative study.  This one is not done through interviews, however, but through the analysis of chat logs.  Again, interesting, but doesn’t help address my core question.
• Herold (2009)7 is yet another qualitative study of SL, this time for a Media Studies course in Hong Kong.
• McVey (2008)8 conduct a qualitative study of 10 students, and then generalizes that to several “lessons” of education regarding Second Life.
• Livingstone, Kemp & Edgar (2008)9 takes a similar qualitative approach.
• Dickey (2005)10 is the oldest reference I could find, with a discussion of two case studies of classes using Active Worlds as their primary course delivery mode.  Again, interesting, but limited in generalizability.

The especially depressing thing about these studies is that most decry poor study design and the overabundance of theoretical work in the area, saying data is the only path to understanding the best way to use virtual worlds.  So where is that data?  On the bright side, a number of unpublished dissertations appeared in my list as well, which is promising – younger researchers have often had stats more recently.  All that we can do is hope.  Frankly, If anyone reading this has a virtual world they’d like to “prove” is effective, just let me know, and my team will freely help you evaluate it (it really is that easy!).

Several bits of this evidence come from the same place – a special issue of ALT-J, Research in Learning Technology, volume 16, number 3.  If you want further reading, that would be a good place to start.

1. Delucia, A., Francese, R., Passero, I., & Tortora, G. (2009). Development and evaluation of a virtual campus on Second Life: The case of SecondDMI Computers & Education, 52 (1), 220-233 DOI: 10.1016/j.compedu.2008.08.001 [↩]
2. Jarmon, L., Traphagan, T., Mayrath, M., & Trivedi, A. (2009). Virtual world teaching, experiential learning, and assessment: An interdisciplinary communication course in Second Life Computers & Education, 53 (1), 169-182 DOI: 10.1016/j.compedu.2009.01.010 [↩]
3. Jamaludin, A., Chee, Y., & Ho, C. (2009). Fostering argumentative knowledge construction through enactive role play in Second Life Computers & Education, 53 (2), 317-329 DOI: 10.1016/j.compedu.2009.02.009 [↩]
4. Lester, P.M. & King, C.M. (2009). Analog vs. Digital Instruction and Learning: Teaching Within First and Second Life Environments Journal of Computer-Mediated Communication, 14 (3) : 10.1111/j.1083-6101.2009.01449.x [↩]
5. Good, J., Howland, K., & Thackray, L. (2008). Problem-based learning spanning real and virtual words: a case study in Second Life ALT-J, 16 (3), 163-172 DOI: 10.1080/09687760802526681 [↩]
6. Edirisingha, P., Nie, M., Pluciennik, M., & Young, R. (2009). Socialisation for learning at a distance in a 3-D multi-user virtual environment British Journal of Educational Technology, 40 (3), 458-479 DOI: 10.1111/j.1467-8535.2009.00962.x [↩]
7. Herold, D. (2010). Mediating Media Studies – Stimulating critical awareness in a virtual environment Computers & Education, 54 (3), 791-798 DOI: 10.1016/j.compedu.2009.10.019 [↩]
8. McVey, M. (2008). Observations of expert communicators in immersive virtual worlds: implications for synchronous discussion ALT-J, 16 (3), 173-180 DOI: 10.1080/09687760802526673 [↩]
9. Livingstone, D., Kemp, J., & Edgar, E. (2008). From Multi-User Virtual Environment to 3D Virtual Learning Environment ALT-J, 16 (3), 139-150 DOI: 10.1080/09687760802526707 [↩]
10. Dickey, M. (2005). Three-dimensional virtual worlds and distance learning: two case studies of Active Worlds as a medium for distance education British Journal of Educational Technology, 36 (3), 439-451 DOI: 10.1111/j.1467-8535.2005.00477.x [↩]