Thoughts of a Neo-Academic

Several pieces have been floating around in the last week or so on virtual crime.  GamePolitics discussed a paper by the Australian government’s Institute of Criminology discussing the nature of criminal activity in virtual worlds.  It centers around a basic question: is interpersonal crime committed in a virtual space really crime if there is no real-world corollary?

The question of whether real-world notions of interpersonal harm apply to virtual assault or sexual assault is unresolved. This complicates the question of regulation within virtual worlds.

There is some precedent for virtual crime that also occurs in the real world to be considered criminal activity.  For example, if you were to illegally extort another person out of Linden Dollars in Second Life, you could be prosecuted for that crime in the real world because Linden Dollars have a real-world monetary equivalent (currently, roughly L$260 = US$1).

But criminal activity that occurs only in a virtual space is a gray area. Consider the cases described by Inside Higher Ed of “avatar rape.”  I put that in quotes because there is some disagreement on what specific term should be used.  You see, it’s possible for one avatar in Second Life to sexually assault another avatar.  Because consent is required to participate realistically in any activity in Second Life, the mechanics of this activity are kind of odd:

• Example 1: An avatar might equip a male genitalia add-on and then walk into another person.  There is no forceful penetration, and in many cases, the victim avatar doesn’t necessarily experience anything physical – it is more of a symbolic attack.  It can be remedied by either walking away or teleporting elsewhere.
• Example 2: In Second Life, there are objects called “pose balls.”  When an avatar right-clicks and selects “Sit” on a pose ball, it loads an animation for that avatar.  Sex in Second Life occurs when two avatars are sitting on two pose balls, the animations of which are synced together to simulate some sort of sexual activity.  One way that virtual rape can thus occur is by tricking another (usually new) user into sitting on a pose ball that engages them in an unexpected sexual activity.  It’s easy to stop such an activity – simply click “Stand Up.”

The commenters provide some interesting perspectives on this piece:

Calling this rape is sensationalistic, yellow journalism. Exposure to unwanted sexual imagery (which is the closest you might call this) is sexual harassment, at worst. This is certainly a non-trivial issue, but to call it rape is irresponsible. In doing so, you are implying that actual rape victims have similar feelings and reactions to Second Life “rape” victims, and that is frankly insulting. Psychologically, I expect this sort of behavior’s effect is closer to flashing/exhibitionism.

Regardless of the specifics, it is clear that Second Life is to some degree an untamed frontier, where the specific rules of law and order in relation to “the real world” are not clear.  Is it ethical to expose students or employees to such an environment?

In my research laboratory, where we are currently testing Second Life-type training interventions, we never allow research participants onto the open Second Life grid.  It’s too dangerous for two reasons.  First, the Second Life environment is difficult to control unless you have a private island – any random jackass can come by an interrupt your experiment, although this is uncommon.  Second, we would be exposing participants to everything in Second Life, and I don’t believe it’s necessarily appropriate/ethical for researchers to make that decision for them. Or at the least, it’s not a risk that I want to take yet.

But the classroom is a different environment.  To some degree, isn’t it the responsibility of instructors to expose students to new ideas and ways of thinking?

The training room is a different story, which I think is the impetus for Linden Lab’s behind-the-firewall product, which allows companies to run their own Second Life grid inaccessible to the Internet at large.  Exposing your employees to material not controlled by your company is seem by many companies simply as an unnecessary risk.  The only problem is that this product appears to be ridiculously expensive.

There’s been a lot of interest in one of the cases I mentioned last week in which it was discovered that school officials were spying on kids and their families in their own homes by surreptitiously activating webcams on school-provided laptops.

There’s a lot of conflicting information on what exactly happened and what will happen regarding this case, so I set out to collect all of it into a single story.

• First of all, the case came to light through the filing of a class-action lawsuit, which you can find in this PDF.  The basic charge is straightforward: the school was spying on children without their consent.  It came to light when a student was accused of dealing drugs based on images from the webcam feed, but it turns out that he was just eating Mike & Ikes.
• A response to parents from school district sheds a little more light on the specifics of the situation.  The remote-webcam-enabling feature of the laptops was implemented as a security measure – if the laptop was reported stolen and was then used, the webcam would assist in recovering it.  This is a not-so-uncommon feature of laptop security programs, and from other sources, I’ve deduced that the specific program in question is called LANrev.
• The school has now been barred from speaking with students or parents about the case under court order, which may explain the sudden lack of information coming out of the school district.  You might think this a recommendation from the school system’s own lawyers, but it’s actually from the judge – any formal communication must be approved by the prosecution’s lawyers before it can be sent out officially.  They are also forbidden from changing anything about any of the laptops currently with the software under similar guidelines.
• The school’s records indicate that webcams have been activated only 42 times in roughly the past year, and only in incidents where it was believed that the laptops were stolen.
• In addition to the civil matter, the FBI has begun an investigation to determine if any violations of federal laws regarding wiretapping and computer-intrusion have taken place.
• Witold Walczak, legal director of the PA ACLU commented, “This is an age where kids explore their sexuality, so there’s a lot of that going on in the room.”  Now we can add child pornography to the list of problems associated with this case.  Consider the teen that leaves their laptop open on a desk while they are in their bedroom.
• Spying on kids through webcams is apparently not all that uncommon.  Scan ahead to 4:37 of this video from PBS to find a school administrator proudly explaining how he watches the desktops and webcams on his students in order to keep them on task when they’re working.  Through webcam snooping, they discovered, for example, that many students use their webcam as a mirror – they will turn the webcam on, style their hair/makeup/etc, and then turn it off.  In one example, the administrator surprise-instant-messages a student that he’s been spying on to tell her to get back to work.  What’s especially troubling about this video is the total lack of regard for the privacy of the kids – this is treated as an innovative approach to policing child behavior.

Here is the main problem: The school claims they only activate webcams when the laptops are stolen, but the central kid’s laptop was never reported stolen.  Thus, either some violation of the school’s own monitoring policies took place, or such policies were never really in place to begin with.  Additionally, the school has not come out with a flat-out denial of wrongdoing, only saying that the program had a purpose and was not abused, while simultaneously shutting that program down.

My suspicion is that while official school policy was against spying, the temptation for school officials wanting to sniff out illicit activity was simply too great.

Thanks to Jeremy Anglim’s Blog, I have discovered a section of the UK’s The Guardian that I did not know existed:  DataBlog.

It’s practically the scientist’s news wet dream.  DataBlog collects raw data, analyzes it, and gives summary information and an interpretation.  Already, it is miles ahead of most news outlets, many of which simply collect an interview or two and spin a story for there.

But DataBlog takes it a step further – they provide a copy of the raw data tables that they used to create their story.  Amazing, right?  As a reader, you can then take it upon yourself to reanalyze the data and come to your own conclusions, whatever the outcome might be.  You could contradict them or support them; both are encouraged.  If you think they faked their data, it’s right there for you to verify.  It reminds me of the editorial standards in Science.

If there’s a picture of more responsible journalism, I’ve never seen it.  Now the only question that remains – where’s the American equivalent?

Three privacy and online rights pieces have come up in the last couple of weeks, and I thought I’d share them all at once.

1. Wired has an interesting piece on how mobile phones are wreaking havoc with the justice systems.  You see, the U.S. justice system is built around the idea that a jury is impartial until they have heard evidence that has been vetted for presentation in court.  So when Juror #5 decides to log into some forums to check what the random jackasses on the Internet have to say about the case he’s sitting on, there’s a little problem.
2. Imagine if you were sent a captured picture or video of your child engaging in illicit activity by your school district.  Imagine then that the picture was shot in your own home.  How might that have happened, you ask?  Well, it might happen if your school district issued your child a laptop and then used the webcam on the laptop to spy on you and your family.  Surprise, surprise – there’s a class-action lawsuit.
3. Because of a series of conflicting court rulings, it is unclear if speaking ill of others in online social networks is protected speech.  This is vitally important question in many domains.  Can a child rant against her teacher in MySpace?  Can an employee complain that his boss is an a-hole on Facebook?  Or more accurately, can these things be done without fear of repercussion?  The disturbing answer so far?  We don’t know.  But so far, the courts are leaning towards defining it as free speech.

As new as the Internet is, it will be a while for case law and the Supreme Court to react to all of the new legal implications of that fact.  Can you legally use social networks for hiring?  Can you record your employee’s online shenanigans and use them in promotion, raise and firing decisions later?  Can you offer employment training only online knowing that there are differences in Internet access across protected classes?  Keep an eye here for the latest developments.

Now, only two days after I recommend holding off on adopting new technologies until a need is demonstrated, I give you a warning that might seem contradictory: don’t fear new technology.

A recent article in Slate discusses historical fear of new technology.  Consider this quote:

A respected Swiss scientist, Conrad Gessner, might have been the first to raise the alarm about the effects of information overload. In a landmark book, he described how the modern world overwhelmed people with data and that this overabundance was both “confusing and harmful” to the mind. The media now echo his concerns with reports on the unprecedented risks of living in an “always on” digital environment. It’s worth noting that Gessner, for his part, never once used e-mail and was completely ignorant about computers. That’s not because he was a technophobe but because he died in 1565.

New technology itself should not be feared; only the implications of implementing that technology blindly.  There’s a big difference between 1) clinging onto your training program because it worked ten years ago and 2) choosing to stay with your training program because you conducted a needs analysis, identified no technology that would do the job any better, and decide not to change anything because it’s the right decision for your business.

So the message is really the same as last time: don’t jump in blindly, but also don’t fear the unknown.  The trick is to stay familiar with new technologies such that you are aware of what they offer in case you need them while still maintaining a level head when making implementation decisions.

There’s unfortunately no magic bullet to any organizational problem, and the sooner you realize that, the better off your business will be.

A recent article in THE Journal revealed the $300 million plan to install 3D projectors in Colorado’s Boulder Valley School District.  Why?  Because it’s the future.

While preparedness is a noble goal, this is not a good idea.  There’s no evidence that 3D projectors give students any positive learning outcomes or really even offer any benefit whatsoever to schoolchildren.  Having said that, modern 3D projection systems are an incredible technology.  But that’s all they are.

It brings to mind a tendency of businesses to adopt new technologies without 1) identifying a purpose for such a change and 2) gathering evidence that such a change would actually bring about the outcomes desired.

Let’s use web-based training systems as an example.  Online training was adopted originally because it was new and exciting – just like the 3D projectors above.  But there was no particularly compelling reason to do so.  At that time, web-based instruction was more expensive to implement than traditional instruction because of the learning curve, and additionally, access was much less common than it is now, limiting its reach.  What was the point?  There really wasn’t one.

Fast forward to today – web-based training is still being implemented, and if anything, it’s as effective as traditional instruction.  High quality instructional design still takes the front seat in terms of the magnitude of the effect.  A poorly designed web-based instructional program will most likely encourage less student learning than a well-designed lecture.

But how many job listings do you see for “corporate trainer” without keywords “web,” “Internet,” or “online?”  Virtually none.

Let this 3D projector fiasco be a reminder to us all – hype about “the future” is no reason to invest your limited funds into any organizational program.  Develop a clear picture of what your organization needs, and only then find a technology that meets your needs.  To do so in reverse is just plain silly.

If you need further evidence of the enthusiasm and engagement that Second Life can enable, just observe this labor-of-love: an appropriately-timed Valentine’s music video captured in Second Life, singing a love song between two auditors.

The video was a group effort from various accounting bloggers and the staff of the MACPA.  A great amount of time was obvious spent – production values (in Second Life terms) are pretty high!  It is full of accounting in-jokes, so don’t be surprised if a few go over your head, but it is definitely worth checking out.

Scientific publishing culture is such that typically only studies with well-thought out theoretical backing and empirical evidence supporting that theory tends to get published.  The reason is not necessarily that studies without empirical support are worthless – rather, the problem is that there is limited journal space available.  Studies that find what they set out to find are seen as more valuable than those that don’t.

The problem is that all these unpublished studies, despite providing potentially useful information, are just that – unpublished, and difficult to find.  Imagine this situation:

1. Researcher A uses Technique to address the Problem
2. Technique fails to address the Problem
3. Researcher A tries to publish results, but they are negative, and cannot do so
4. Researcher B uses Technique to address the Problem

And the cycle continues.  In the social sciences (like psychology), things aren’t quite so clear cut, since we more typically are trying to explain more subtle processes.  But the problem is the same – unpublished studies are useful, but difficult to locate.

This led to me creating filedrawer,org, an online unpublished papers database in psychology.  It’s fairly empty, but it fills a need.  It essentially allows researchers to make their unpublished papers available to a wider audience.  There is no peer review, because it is just an online database.  It’s called “filedrawer” as a tribute to the so-called file drawer effect: the problem that unpublished (although still valid) studies usually bias meta-analytic effect estimates away from zero.

In the computational and life sciences, a new journal designed for such data has been created: the Journal of Serendipitous and Unexpected Results.  JSUR aims to put such studies through peer review, and if their methods are sound, publish them.  In Biology, the Journal of Negative Results in Biomedicine has been around since around 2002, and the Journal of Negative Results has been publishing such data in ecology and evolutionary biology since 2004.

Where are the peer-reviewed journals in the social sciences?  Well, as far as I can tell, they don’t exist.  I’m not really sure why.  I guess filedrawer.org will have to do for now…

I recently got into a discussion with Flemming Nielson on Anja Patulski Nielson’s Weblog about what evidence is available to support the use of virtual worlds for training and education.  Flemming argued that a number of high-profile case studies are available (especially from IBM); I argued that there is little scientific support.  But I soon realized that all I knew about was the lack of scientific support in my home research area, industrial/organizational psychology, the application of psychology to business principles specifically.  What about the wider psychological literature, especially relating to education?  What evidence is there that virtual worlds produce learning outcomes as strong as traditional classrooms and instructional methods?

A simple search for ["Second Life" or "virtual world"] produced the following psychological studies relevant to my question.  Criteria for inclusion in this list:

1. Considers adult learning in some regard (college courses, training and development, includes lab studies)
2. Collects empirical data (research volunteers, trainees, or students)
3. Is in English  (I have to be able to read it)
4. Can be retrieved easily (2-3 articles that might have otherwise fit in this list would have taken several days of inter-library loan to retrieve)
5. Is indexed by APA’s PsycINFO
6. Has been peer-reviewed
7. Discusses true 3D virtual worlds (not just massively multiplayer games or multi-user learning systems).  It should be noted that there is some confusion in the scholarly literature as to what exactly a virtual world is.  Virtual worlds differ primarily from massively multiplayer games in that there are no set objectives in a virtual world – Second Life qualifies, World of Warcraft does not.
8. Some effort must have been made to prove virtual worlds as a reasonable alternative or complement to traditional education/training practices.  I would have included “unreasonable” as well, except that no study study set out with this objective.

My initial search turned up a couple hundred, but few of those met these qualifications.  Every study I could locate in psychology within those guidelines is in the following list:

• De Lucia, Francese, Passero & Tortora (2009)^[1] examined the reactions of 26 students participating in 3 virtual SL lectures.  Unfortunately, evaluation data is never recorded, so we don’t really know how Second Life compared to traditional instruction.  They did find that participants had some degree of a “sense of presence,” but that sense is not connected to any real outcomes (like learning).
• Jarmon, Traphagan, Mayrath & Trivedi (2009)^[2] conducted a qualitative study that I think is typical of this literature; a virtual world was used as part of an assignment for 5 graduate students to examine communications, and all sorts of stories and personal explanations were collected.  It’s fantastic that SL worked well in this context; the question that this study cannot answer is “will it work in other contexts?”
• Jamaludin, Chee and Ho (2009)^[3] randomly assigned to groups which role-played in the virtual world.  While I agree this is certainly a good educational use for Second Life that takes advantage of the technology, no effort is made to compare it to traditional in-person role-playing.  Without it, there’s no way know if SL is a reasonable substitute for traditional instruction.
• Lester and King (2009)^[4] is one of the better designed studies I saw.  An online course held in Second Life was compared with a traditional course in terms of both student reactions and grades.  Scores were similar across all grades, which is promising, although standard deviations (and standardized effect sizes) were not reported, so it is impossible to say just how similar they really were.
• Good, Howland and Thackray (2008)^[5] is another qualitative study examining the use of SL in a classroom setting.  No comparison group, no statistics.  Just a story, albeit an interesting one.
• Edirisingha, Nie, Pluciennik & Young (2009)^[6] describe yet another qualitative study.  This one is not done through interviews, however, but through the analysis of chat logs.  Again, interesting, but doesn’t help address my core question.
• Herold (2009)^[7] is yet another qualitative study of SL, this time for a Media Studies course in Hong Kong.
• McVey (2008)^[8] conduct a qualitative study of 10 students, and then generalizes that to several “lessons” of education regarding Second Life.
• Livingstone, Kemp & Edgar (2008)^[9] takes a similar qualitative approach.
• Dickey (2005)^[10] is the oldest reference I could find, with a discussion of two case studies of classes using Active Worlds as their primary course delivery mode.  Again, interesting, but limited in generalizability.

The especially depressing thing about these studies is that most decry poor study design and the overabundance of theoretical work in the area, saying data is the only path to understanding the best way to use virtual worlds.  So where is that data?  On the bright side, a number of unpublished dissertations appeared in my list as well, which is promising – younger researchers have often had stats more recently.  All that we can do is hope.  Frankly, If anyone reading this has a virtual world they’d like to “prove” is effective, just let me know, and my team will freely help you evaluate it (it really is that easy!).

Several bits of this evidence come from the same place – a special issue of ALT-J, Research in Learning Technology, volume 16, number 3.  If you want further reading, that would be a good place to start.

1. Delucia, A., Francese, R., Passero, I., & Tortora, G. (2009). Development and evaluation of a virtual campus on Second Life: The case of SecondDMI Computers & Education, 52 (1), 220-233 DOI: 10.1016/j.compedu.2008.08.001 [↩]
2. Jarmon, L., Traphagan, T., Mayrath, M., & Trivedi, A. (2009). Virtual world teaching, experiential learning, and assessment: An interdisciplinary communication course in Second Life Computers & Education, 53 (1), 169-182 DOI: 10.1016/j.compedu.2009.01.010 [↩]
3. Jamaludin, A., Chee, Y., & Ho, C. (2009). Fostering argumentative knowledge construction through enactive role play in Second Life Computers & Education, 53 (2), 317-329 DOI: 10.1016/j.compedu.2009.02.009 [↩]
4. Lester, P.M. & King, C.M. (2009). Analog vs. Digital Instruction and Learning: Teaching Within First and Second Life Environments Journal of Computer-Mediated Communication, 14 (3) : 10.1111/j.1083-6101.2009.01449.x [↩]
5. Good, J., Howland, K., & Thackray, L. (2008). Problem-based learning spanning real and virtual words: a case study in Second Life ALT-J, 16 (3), 163-172 DOI: 10.1080/09687760802526681 [↩]
6. Edirisingha, P., Nie, M., Pluciennik, M., & Young, R. (2009). Socialisation for learning at a distance in a 3-D multi-user virtual environment British Journal of Educational Technology, 40 (3), 458-479 DOI: 10.1111/j.1467-8535.2009.00962.x [↩]
7. Herold, D. (2010). Mediating Media Studies – Stimulating critical awareness in a virtual environment Computers & Education, 54 (3), 791-798 DOI: 10.1016/j.compedu.2009.10.019 [↩]
8. McVey, M. (2008). Observations of expert communicators in immersive virtual worlds: implications for synchronous discussion ALT-J, 16 (3), 173-180 DOI: 10.1080/09687760802526673 [↩]
9. Livingstone, D., Kemp, J., & Edgar, E. (2008). From Multi-User Virtual Environment to 3D Virtual Learning Environment ALT-J, 16 (3), 139-150 DOI: 10.1080/09687760802526707 [↩]
10. Dickey, M. (2005). Three-dimensional virtual worlds and distance learning: two case studies of Active Worlds as a medium for distance education British Journal of Educational Technology, 36 (3), 439-451 DOI: 10.1111/j.1467-8535.2005.00477.x [↩]

Consider the case of Denver Broncos wide receiver Brandon Stokley who recently demonstrated what is perhaps the best evidence of transfer of training from a video game to the workplace that I have ever seen.  In this case, the workplace is the football field, but the principle remains the same.

According to a recent article in Wired, in the first game of the 2009 season, Stokley performed a surprising game-winning play – a touchdown that started 87 yards down the field.  But those sorts of miracles happen in a lot of games – what doesn’t usually happen is Stokley’s move right before the end zone.  Rather than running straight in for the touchdown, he cuts 90 degrees and runs parallel to the end zone before cutting in for the goal.

In football played on pro fields, that’s a little odd.  Normally, if you’re about to score a goal, you simply run right into the end zone: the goal-scoring area.  Instead, Stokley replicated something he’d done countless times before, but not on the field – in a video game called Madden NFL.

In training terms, this is quite compelling.  Madden NFL is essentially a football simulator.  It replicates the field, player actions, coach strategies and so on down to the finest details.  In playing Madden, Stokley was testing theories about what kinds of plays would work and seeing them in action, all without having to leave his couch.  And the result?  According to the Wired article:

When I caught up with Stokley by telephone a few weeks later, I asked him point-blank: “Is that something out of a videogame?” “It definitely is,” Stokley said. “I think everybody who’s played those games has done that” — run around the field for a while at the end of the game to shave a few precious seconds off the clock. Stokley said he had performed that maneuver in a videogame “probably hundreds of times” before doing it in a real NFL game. “I don’t know if subconsciously it made me do it or not,” he said.

He tested and perfected a technique in a video game and then applied it on the field.  What better evidence of transfer of training from video games can there be?