SIOP 2014: Reflections on SIOP Honolulu

SIOP 2014 Coverage: Schedule Planning | Live-Blog | Reflections

Normally, SIOP is held during the final weeks of April. It’s probably the worst conceivable timing for the academic community – immediately before and sometimes during finals of the Spring semester. This year, SIOP was unusually scheduled due to its unusual location: Honolulu, Hawaii. With the wise assumption that many SIOP attendees would dual-purpose their trip to Honolulu into a vacation, the conference was pushed forward to mid-May (and post-semester). The daily conference schedule was also pushed forward, starting at 7AM and ending around 3PM, in order to allow people to enjoy their afternoons in Hawaii. That resulted in a very different SIOP feel than usual. Not really better or worse – just different.

My personal conference schedule was packed full, with 8 separate events (9 total presentations) spread over all three days of the conference. That didn’t leave a whole lot of time to attend other people’s sessions, to listen and reflect, which is really the best part of SIOP for me. My overall goals in attending SIOP are to be inspired in my own research, to incorporate new ideas into my thinking, and to remind myself of the excitement of research and why I took a research career in the first place. Even with my schedule, I think these goals were met.

In terms of specific content, I also use SIOP as a barometer of the tech sophistication of the practitioner HR consultant crowd. I used to think the Academy of Management conference would be the best place for this, since they do study management after all, but after attending AOM for a few years, I realized that the Academy crowd is not generally very concerned with people are actually doing in real organizations. So now I stick with SIOP to learn what I need to know.

In general, things are looking good for technology in IO. At a minimum, people are paying attention and heeding the calls I’ve been making for years – namely that technology is becoming an overwhelming force of marketing in the HR consulting community, with both good and bad implications, and we need to pay attention.

On the good side, technology enables us to do a lot we couldn’t do before. One of the most fascinating examples I saw was a manufacturing simulation in which consultants could simulate an entire car-building process, recording every mouse click, every physical movement, every eye glance of the operators for the purpose of either developmental or administrative assessment. For softer skills, assessment is improving but isn’t yet great. Leadership assessment, for example, usually takes place online and is video-based, in the tradition of situational judgment tests (SJTs). That’s more convenient than an in-person role-play, but we can do better. The line between SJTs and assessment centers is blurring, and I think this will ultimately force purveyors of both to really think about whether they are really unique concepts and what unique value each brings, if any.

On the bad side, technology is edging out the more important part of the equation – the underlying psychological mechanisms that enable these technologies to work in the first place. In many cases, assessment technology is more about technology than assessment; learning technology is more about technology than learning. That is not the way things should be, and the IO crowd is starting to take notice. We are the ones to make that stand, and we need more people fighting the good fight!

A great example of this is the recent highly visible gamification effort by Knack called Wasabi Waiter. It is essentially a casual web browser game (reminds me of classic casual game Diner Dash), but it purportedly can be used to select high performers on “social intelligence” among other constructs. The media has gone crazy over this thing. Yet there is no published psychometric evidence or even a white paper describing the construct validity of whatever it is that this game is supposed to assess. Without that, we should not trust this game – it may be all flash and no substance – yet the press is impressed with flashy, and that’s what people remember. IO should bring the science!

Another assessment gamification effort called Insanely Driven provides another case study that I learned about at SIOP. This game supposedly assesses candidates on… something. I honestly don’t know. And I’m not convinced the creators do either! It’s an entertaining experience, and personality profiles are spit out at the end, depending upon your performance, but is it valid? Again – nothing published, nothing shared, so who knows?

In any case, next year’s SIOP is not anywhere quite so exciting (Philadelphia!), so we’ll see then if the tech enthusiasm gets even stronger!