One of the most significant challenges currently facing IO psychology researchers is ensuring the relevance of our research to real-world HR and OD practices. Unfortunately, over the last few decades, the IO research in our top journals has become increasingly specialized and decreasingly applicable to solving real world problems.
There are many potential causes of this problem. I personally blame the siren song of “methodological rigor.” By demanding only the absolute most rigorous tests of theories for publication in top-tier journals, authors are actively discouraged from investigating new, relevant organizational phenomena because the risk that such research will be unpublishable in those journals is extremely high. When rejection rates are around 95%, as they are currently for Journal of Applied Psychology, editors and reviewers use absolutely any weakness to justify rejection. As any IO psychologist can tell you, if methodological weakness is correlated even weakly with practical relevance, fewer relevant papers will be published.
More troubling, such practices also encourage researchers to engage in a wide variety of questionable research practices to get past such barriers. When a paper could be rejected due to unsupported hypotheses, researchers may be inclined to modify or eliminate hypotheses or analyses based upon the data (i.e., p-hacking). When a paper could be rejected due to lack of complex theory, researchers may be included to overdevelop theory to the point that it provides little real-world value. When only new theory is published, the very foundation of science, replications, become actively discouraged. When faced with developing a new theory or developing a new theory and empirically testing it, developing an untested theory becomes increasingly attractive – you can’t have flaws with your method, results, and their interpretation when you don’t have any data. These are all significant problems for IO psychology, and most start and end with our publication process.
Fortunately, there are a few IO psychology journals making an active effort to fix this problem, in a variety of ways. The key for us is to prioritize these journals as both authors and consumers of research. If you’ll only publish in journals that actively work to repair the problems in our field, eventually those journals and their practices will hold the most influence over our literature. What follows is a brief and partial list of three, based upon the journals I’m regularly personally exposed to, so if you have more that qualify, let me know in the comments.
1. Journal of Business and Psychology
Editor: Steven G. Rogelburg
For me, JBP is the most prominent do-gooder in the area of IO publishing. As a few examples of the innovative initiatives pursued by JBP:
- These nuggets from their guide for authors are revealing: “We very rarely publish uninvited conceptual or theoretical pieces unless highly impactful and ground-breaking” and “The Journal of Business and Psychology is…dedicated to bridging the science/practice divide…striving to create interdisciplinary connections.” This ain’t no Academy of Management Review.
- Each year, JBP publishes a special feature edition, and some planned editions are directly targeted at improving the research-practice gap. As described on their website, this planned edition is:
a “State of the Practice” edition. This edition would have about 12 pieces (around 3000 words each), typically written by well-known scientist-practitioners. Each peer-reviewed piece would discuss best practices in a particular practice area that are extremely relevant in today’s business world.
If you’re at all familiar with scholarly publishing, you’ll realize just how unusual such an effort would be.
- JBP did something completely unfamiliar to most IO psychologists when introducing its hybrid registered reports initiative. When taking this approach, researchers submit only the introduction and method for initial review. If the reviewers and editor approve the manuscript in this form, it becomes untouchable, like a dissertation proposal. If not one hypothesis is supported but the analyses are sound in the full version of the manuscript, that is still a publishable paper.
2. Personnel Assessment and Decisions
Editor: Scott Highhouse
PAD is a new journal, the official journal of the International Personnel Assessment Council (IPAC). In addition to being open-access and therefore accessible to practitioners who often don’t have the journal subscriptions that universities do, PAD has a submission model unlike anything else in our field:
PAD is a unique short reports journal in industrial-organizational psychology. Its aim is to publish concise reports of empirical studies that provide meaningful contributions to our understanding of staffing organizations and assessing and developing its members. PAD strives to publish innovative, cutting-edge, and impactful research. It is geared toward a speedy review and publication process to allow innovative research to quickly become part of the applied and scientific discourse. Articles cannot exceed 4,000 words (excluding references), and may present new theory, new data, new methods, or any combination of these.
Short, impactful, practice, and cutting-edge. Not words commonly associated with scholarly publishing!
3. Industrial/Organizational Psychology: Perspectives on Science and Practice
Editor: John Scott
IO Perspectives is the original “let’s try something different” journal in IO Psychology. Its model is also quite unique:
Industrial and Organizational Psychology focuses on interactive exchanges on topics of importance to science and practice in our field. The journal features focal articles which present new ideas or different takes on existing ideas that stimulate conversation on an important issue for the field (or potentially a pair of papers taking opposite sides in a debate).
One of the side effects of the mainstream scholarly publishing model is that it’s much easier and thus more common to publish papers making small and incremental contributions. But sometimes what science needs is an about-face. IO Perspectives provides the opportunity for scholars to present controversial or tentative viewpoints, inviting other scholars to submit commentaries refuting or supporting that paper. Perhaps no opinions will change as a paper, or perhaps it will trigger a revolution. There’s only one way to find out.
Importantly, the current editor is a highly experienced practitioner. In that position, he can easily direct the journal toward those topics of most interest to both the academic and practitioner communities. And because access to IO Perspectives is an included benefit for membership in SIOP, IOs have no excuse not to read it!
The only way that new journals or journals with new initiatives like these succeed is if people like you read, cite, and submit to them. These are the journals to watch, and let’s hope more follow soon in their footsteps. As long as we hold in high esteem those journals that encourage a wider science-practice gap, that is exactly the sort of IO psychology research that will be published. Time to get to work!
As a companion to my grad school series, I will be hosting a week-long AMA in the IO Psychology subreddit. Although it’s an AMA, I suspect that most of my time will be spent responding to questions about getting into graduate school in I/O psychology! My post about choosing between the Master’s and Ph.D. already has 400+ comments, so I guess people like my advice! This will be a relaxed sort of AMA, taking place over a week, from Wednesday, Oct 21, 2015 around 6PM through about the same time on Tuesday, Oct 27.
Here is the text of the announcement on Reddit:
We are proud to announce our next AMA will be Richard Landers starting on Wednesday October 21st at around 6 pm. He will be keeping the AMA thread going through Tuesday October 27th so there will be a lot of time for people to ask their questions. Feel free to tell your friends to bring their questions too! With the extended timeline it is reasonable to expect responses back will be at least a bit delayed from our more compact AMAs, so do note that.
Below is a Bio and AMA information from Richard:
Richard Landers is a very recently tenured Associate Professor of Industrial/Organizational Psychology at Old Dominion University. After graduating from the University of Minnesota in 2009, he began his research program on improving the use of Internet technologies in talent management, especially the measurement of knowledge, skills and abilities, the selection of employees using innovative technologies, and learning conducted via the Internet. Recent topics have included game-based learning, game-based assessment, gamification, unproctored Internet-based testing, mobile devices including smartphones and tablets, immersive 3D virtual environments and virtual reality, and social media and online communities. His research and writing has been featured in Forbes, Business Insider, Science News Daily, Popular Science, Maclean’s, and Chronicle of Higher Education, among others. He currently serves as Associate Editor of the International Journal of Gaming and Computer-Mediated Simulations, on the editorial board of Technology, Knowledge and Learning, and is part of the steering committee for the Coalition for Technology in Behavioral Science. He was Old Dominion University’s 2014 and 2015 nominee for the State Council for Higher Education in Virginia Rising Star Outstanding Faculty Award. He is also author of a statistics textbook, A Step-by-Step Introduction to Statistics for Business (SAGE), and editor of the upcoming Social Media in Employee Selection (Springer) alongside sub-reddit moderator iopsychology.
If you’ve been looking into graduate school in I/O Psychology, you’ve probably seen Richard’s blog, which is the most popular resource on the Internet for getting into I/O grad school: http://iopsychology.info
On that website, he’s compiled a lot of advice on the most effective strategies for getting into both Master’s and Ph.D. programs, including a compilation of available PhD program rankings and more general guidance. The most popular page on the site (http://neoacademic.com/2011/06/14/grad-school-should-i-get-a-ph-d-or-masters-in-io-psychology/) currently has 431 comments, mostly questions from grad school hopefuls, about half of which are replies from Richard.
In this AMA, Richard will be primarily giving honest advice and answering questions about getting into graduate school in I/O from the perspective of an I/O faculty member selecting students into a fairly competitive I/O PhD program, but it’s an AMA, so any questions not covered by an NDA are fair game!
A few days ago, I was looking at numbers from the National Center for Education Statistics in order to get a better sense of the gender bias in some STEM fields. Specifically, as part of a grant application, I wanted to know how much greater the rate of degrees awarded to males was than to females in a few scientific fields relevant to my project. The basic logic is that women make up 50% of the pool of degree candidates, so they should be around 50% of the number of degrees awarded. This sort of comparison is often used as a piece of the justification for allocating resources to increase the representation of women in STEM. As I expected, quite a few STEM fields have a significant gender imbalance; for example, only 14% of Master’s degrees in nuclear engineering go to women.
Out of curiosity, I decided to check the stats for Psychology and found something surprising: women are overrepresented, sometimes dramatically, in Master’s and Ph.D. degrees awarded. Far more women are pursuing graduate education in psychology than men, across domains. This certainly reflects my undergraduate classrooms, which are often quite lopsided, but I did not expect it at the graduate level.
I’m honestly not sure if this is a problem or not. We have evidence that women are systematically discouraged from pursuing most STEM careers in ways that are often quite explicit. Men do not generally face these sorts of problems. Although societally-prescribed social roles in part shape career paths for both men and women, women face unique problems not faced by men, and there are active efforts to address these problems.
Recognition of this gender imbalance in psychology, although not in the direction we usually hear about, is not new. Apparently the gender balance flipped around 1986 and became more extreme over the years. But we don’t seem to be doing anything about it. If the degree of imbalance seen here was in the opposite direction, in a natural science, it would be a problem to be solved. Are the same forces at play here? Is psychology seen as stereotypically feminine, discouraging men to pursue it, in the same way that stereotypically masculine fields discourage women? (Although assuredly, men probably do not face the same sort of outright harassment.)
I’ve created a couple of charts below to illustrate the extent of the imbalance in both Master’s and Ph.D. degrees awarded. My own area, I/O Psychology, is one of the more gender-balanced, with 36% male at the Master’s level (versus 50%, p < .001) and 42% male at the doctoral level (versus 50%, p = .056). In developmental psychology, around 1 in 10 graduate degrees awarded are to males. Should we be concerned?