A report has come out about various medical schools running in Second Life, where students can participate in a wide variety of simulations of real medical scenarios. Imperial College London, San Jose State University, and the University of Auckland are mentioned specifically. Now, it may seem like the fidelity of such simulations couldn’t be very high, but consider this snippet about the Nursing Education Simulation (NESIM):
For sims like these, students wear a heads-up display, similar to those used by pilots, which shows data like the patient’s blood pressure, heart rhythm, and medical history. Then they click on objects such as a medication dispenser or the controls of an IV pump. When an object is clicked on, it triggers another heads-up display, allowing students to select, for instance, a certain medication, the dosage, and how to administer it (by mouth, injection, etc.). Importantly, the patient avatar will react realistically—if the student gives him too much nitroglycerine, for example, the avatar’s blood pressure will sink and he’ll go into shock.
Only two things seem different to me: 1) consulting actual machine output has been replaced by referencing the HUD and 2) instead of interacting with a person next to you, interaction takes place virtually.
I don’t think #1 would really inhibit transfer (the application of principles learned in training to the “real world”), so the concern is around #2. But medicine has been using virtual interaction with patients for a long time – after all, you can’t force a person to experience a postpartum-hemorrage (the subject of the above quote) so that students can experience in a controlled environment. So instead, dummies and CD-ROMs have been the norm in medical education for quite some time, and perhaps Second Life isn’t all that different.
One of the biggest advantages to Second Life is cost. Dartmouth Hitchcock Medical Center, for example, has a state-of-the-art training facility for medical education. And although its fidelity is probably quite high, so was its pricetag – $2.5 million. A Second Life simulation by comparison is pennies, as a decent programmer can often be hired to build a simulation for $1000 or less, not to mention the saving on the cost of running the physical facility.
Of course, having said all of this, research is still lacking in the effectiveness of virtual world training. A few scattered studies have tackled it, but nothing too substantial just yet. I suppose that just means my lab will be busy in the fall!
Hot on the heels of Montana wanting to know what’s in your Facebook profile is Best Buy requiring 250 twitter followers to be for a marketing position.
Now, I understand that a position titled “senior social media manager” is likely to involve heading out to the interwebs to check on the social scene, but this requirement’s a bit specific, isn’t it? And where are the other social technologies? Twitter isn’t the only game out there, and it will likely fade into the mists in three or four years like all trendy things eventually do.
So then what happened? I think this is a textbook example of a company creating hiring requirements with only a vague idea of what they want from the person they hire. I imagine something like the following occurred:
- Executive 1 says, “A lot of people are talking about social networking. Let’s get a social networking person.”
- Executive 2 says, “I’ve been reading about Twitter. Let’s make sure they know how to use Twitter.”
- HR says, “OK.”
Now, I’m not saying that requiring familiarity with Twitter is necessarily a bad thing. It is clearly grabbing a great deal of attention right now. But what is the underlying skill that these executives are really interested in? Social media fluency perhaps?
The best way to have gone about this would be the following:
- Conduct a job analysis if this position already exists to determine what KSAOs (knowledge, skills, abilities, and other characteristics) are actually relevant to the job. If it doesn’t already exist, consult those that want to hire this person to find out what tasks they might be involved in.
- After you determine the KSAOs relevant to the job, consult SMEs (subject matter experts) to determine how you might test for these KSAOs. Presumably, if you’re hiring a senior manager, you already have some other managers – these are your experts. Are there standard tests for some of these skills? Could your experts make one?
- Use the test as part of your selection process.
This is likely to produce a much better result than simply hiring people with an arbitrary number of followers on an arbitrarily chosen website. Imagine the person that you hire on Twitter experience alone – how can you know that this Twitter expert will be good at anything else?
Unfortunately, the position has already been filled, so there’s no way to see what other requirements Best Buy had. I just hope their selection process here involved a little more than “check his tweets.”
There are many factors to consider in the traditional graduate school application; standardized test scores, recommendation letters, personal statements, college credits, and research experience are just some of the considerations to weigh when picking who will be the next great Ph.D. Among psychology programs, ETS‘s Graduate Record Examination (GRE) is the most popular standardized test by which to judge the promise of candidates. It is currently split into three components: verbal, quantitative, and analytical writing.
The GRE predicts graduate student success. In a meta-analysis of up to 1231 (k) studies covering 45,617 (N) students, Kuncel and Hezlett (2007) report a positive correlation (ρ = .41) between 1st-year graduate student GPA and total GRE scores. A variety of other predictors relevant to graduate school selection are reported:
- Total GRE vs. 1st Year GPA: ρ = .41, k = 1231, N = 45617
- Total GRE vs. Overall GPA: ρ = .37, k = 103, N = 14291
- Total GRE vs. Qualifying Exam Scores: ρ = .40, k = 11, N = 1196
- Total GRE vs. Degree Completion: ρ = .22, k = 32, N = 6304
- Total GRE vs. Research Productivity: ρ =.11, k = 18, N = 3328
- Total GRE vs. Citations to Publications: ρ = .23, k = 12, N = 2306
- Total GRE vs. Faculty Ratings of Students: ρ = .50, k = 34, N = 4939
Now, most of these shouldn’t be too surprising. The GRE measures an amalgamation of many constructs, prominently including general mental ability (g), which we know to be correlated with a wide variety of life outcomes (lest the controversy surrounding this construct swallow me whole, I shall simply point you here without discussing it more than that). If the GRE partially measures g, and GPA and exam scores are driven largely by g, then it is perfectly logical that these correlations would be sizable and consistant.
But unfortunately, like virtually all psychological constructs, such scores do not predict perfectly. The largest of these relationships is the GRE in predicting 1st-year graduate school GPA, and this value is only .41 – only 16.8% of the variance in GPA can be explained by the GRE alone. And although that’s a substantial chunk successfully predicted, it leaves 83.2% of the variance unaccounted for. Thus, while we could select on GRE scores alone, we would only have a single piece of the puzzle as to what makes a successful graduate student. And depending on your metric, that number might be as high as 98.8% (for research productivity).
In an effort to measure what psychologists traditionally call “noncognitive” factors, most instutitions try to tap that 83.2 – 98.8% by including the supplemental application information listed above: recommendation letters, personal statements, and so on. Each document is a window into the applicant: lazy or committed, neurotic or stable, friendly or hostile. Not to be outdone, and in the name of standardization, ETS is adding a personality measure to its battery of available tests for graduate school admissions, called the Personal Potential Index.
Of course, since it is so new, we know very little about this personality test or its true value, despite assurances from ETS. With apparently years of development behind it, it is only 24 questions long, and includes questions on whether an applicant “produces novel ideas,” “meets deadlines,” “works well under stress” and “is worthy of trust from others.” At the very least, we’ll have plenty of new fodder for faking research.
Whatever the value of this new personality test, the fact remains that poorly selecting students into graduate school is expensive for everyone and must be addressed somehow. According to the Council of Graduate Students, graduation rates for doctoral students hover around only 50-60%. Every graduate student that doesn’t finish is a waste of their own time and money, their advisor’s time and probably money, and the university resources that could have gone to support another student that will finish. That’s a lot of waste from poor selection procedures that couldn’t weed out who didn’t belong in graduate school in the first place.