GRE: The Personality Test

There are many factors to consider in the traditional graduate school application; standardized test scores, recommendation letters, personal statements, college credits, and research experience are just some of the considerations to weigh when picking who will be the next great Ph.D.  Among psychology programs, ETS‘s Graduate Record Examination (GRE) is the most popular standardized test by which to judge the promise of candidates.  It is currently split into three components: verbal, quantitative, and analytical writing.

The GRE predicts graduate student success.  In a meta-analysis of up to 1231 (k) studies covering 45,617 (N) students, Kuncel and Hezlett (2007) report a positive correlation (ρ = .41) between 1st-year graduate student GPA and total GRE scores.  A variety of other predictors relevant to graduate school selection are reported:

1. Total GRE vs. 1st Year GPA: ρ = .41, k = 1231, N = 45617
2. Total GRE vs. Overall GPA: ρ = .37, k = 103, N = 14291
3. Total GRE vs. Qualifying Exam Scores: ρ = .40, k = 11, N = 1196
4. Total GRE vs. Degree Completion: ρ = .22, k = 32, N = 6304
5. Total GRE vs. Research Productivity: ρ =.11, k = 18, N = 3328
6. Total GRE vs. Citations to Publications: ρ = .23, k = 12, N = 2306
7. Total GRE vs. Faculty Ratings of Students: ρ = .50, k = 34, N = 4939

Now, most of these shouldn’t be too surprising.  The GRE measures an amalgamation of many constructs, prominently including general mental ability (g), which we know to be correlated with a wide variety of life outcomes (lest the controversy surrounding this construct swallow me whole, I shall simply point you here without discussing it more than that).  If the GRE partially measures g, and GPA and exam scores are driven largely by g, then it is perfectly logical that these correlations would be sizable and consistant.

But unfortunately, like virtually all psychological constructs, such scores do not predict perfectly.  The largest of these relationships is the GRE in predicting 1st-year graduate school GPA, and this value is only .41 – only 16.8% of the variance in GPA can be explained by the GRE alone.  And although that’s a substantial chunk successfully predicted, it leaves 83.2% of the variance unaccounted for.  Thus, while we could select on GRE scores alone, we would only have a single piece of the puzzle as to what makes a successful graduate student.  And depending on your metric, that number might be as high as 98.8% (for research productivity).

In an effort to measure what psychologists traditionally call “noncognitive” factors, most instutitions try to tap that 83.2 – 98.8% by including the supplemental application information listed above: recommendation letters, personal statements, and so on.  Each document is a window into the applicant: lazy or committed, neurotic or stable, friendly or hostile.  Not to be outdone, and in the name of standardization, ETS is adding a personality measure to its battery of available tests for graduate school admissions, called the Personal Potential Index.

Of course, since it is so new, we know very little about this personality test or its true value, despite assurances from ETS.  With apparently years of development behind it, it is only 24 questions long, and includes questions on whether an applicant “produces novel ideas,” “meets deadlines,” “works well under stress” and “is worthy of trust from others.”  At the very least, we’ll have plenty of new fodder for faking research.

Whatever the value of this new personality test, the fact remains that poorly selecting students into graduate school is expensive for everyone and must be addressed somehow.  According to the Council of Graduate Students, graduation rates for doctoral students hover around only 50-60%.  Every graduate student that doesn’t finish is a waste of their own time and money, their advisor’s time and probably money, and the university resources that could have gone to support another student that will finish.  That’s a lot of waste from poor selection procedures that couldn’t weed out who didn’t belong in graduate school in the first place.