30,859 Pieces of Hell

The goal of my dissertation is to identify underlying differences between traditional and web-based training programs.  To do this, I went back through hundreds of published and unpublished articles, dissertations, and assorted scholarly sources that have attempted to examine such differences.  By aggregating results across all of these studies (using a statistical technique called meta-analysis), I am better able to capture the true differences between the two modalities.  Without such aggregation, statistical forces are at work that make the interpretation of a single study in this domain a little difficult to interpret.

The problem is that in order to examine anything across studies, you have to read and record information from those studies.  That’s not so bad if you’re only looking at a handful of studies or are looking at a handful of variables of interest.  It’s more of a problem if you are looking at, for example, 132 studies with up to 250 pieces of information per sample, and each study may contain multiple samples.  That’s tens of thousands of pieces of information that have to be recorded by hand, not including the articles that you think have data you can use but ultimately do not.  This process is called “coding.”  Coding is, generally speaking, not something that can be automated – expert human judgment is needed to quantify the words in the article .

So as you might have guessed, I coded 250 variables across 132 studies for my dissertation, which produced 30,859 pieces of data.  All of which I coded by hand.  It was painful.  And took forever.  But now it’s done!  You can see a zoomed-out shot of about 40% of my Excel spreadsheet above.  Yes, I had to enter every character that you can (barely) see there.

Of course, my wife has to do a 20% reliability recode (to ensure that the values that I’m finding for each study aren’t entirely arbitrary), which means she gets to code the same 250 variables, but only across 26 studies.  It still takes forever.

On the bright side, while I wait for her to finish, I get to put together my posters for the Society for Industrial and Organizational Psychology (SIOP) conference in New Orleans.  To save about $300, we are driving to Madison, WI on Tuesday night to catch a flight early Wednesday morning, which is where I will be for the rest of the week.  Only a week after the conference, I hope to have my first draft of my completed dissertation to Paul, hopefully so that I can defend in the first few weeks of May.

It’s going to be a busy couple of weeks…