GRE Scores Are Not Good Predictors of Success in STEM Doctoral Programs

1) META-ANALYSES:

In a meta-analysis of 1753 independent studies conducted over 50 years, Kuncel et al. [11] found mostly weak or negative correlations between the scores on the general GRE and indices of success in graduate school.

For Life Science disciplines (top graph), GRE scores and graduate GPA showed the highest correlation but GREs explained less than 8% in the variability in that measure.   For mathematics and physical sciences disciplines, faculty ratings of students were the most highly correlated with the GRE Quantitative score and that score explained only 11% of the variability.   

The data show correlations between GRE scores and success in graduate school, but correlations are generally weak or negative.   The data are presented below showing the percentage of the entire variability (100%) of measures of success explained by GRE scores.  Particularly concerning is the lack of the ability of this high stakes examination to predict degree completion.  A more recent meta-analysis of similar data [12] showed somewhat higher correlations between GRE scores and faculty ratings, citation counts, GGPA and first-year GPA, but only 6% of the variability in degree completion is related to GRE scores. 

2) MULTI-INSTITUTIONAL ANALYSIS:

A) Burton and Wang [5] conducted a study that included four institutions and 21 departments and investigated the correlation of graduate school outcomes with GRE scores and undergraduate grade point average. They also found no correlation between GRE scores and attaining candidacy or PhD degree (graphical depiction of their data is shown). They examined other outcomes, but in those analyses, they combined data from master’s and doctoral students.

B) A collaborative study between Campbell-Kibler Associates (Patricia Campbell) and the Northeast Alliance for Graduate Education and the Professoriate (Sandra Petersen, Director) studied the relationship between GRE scores and PhD completion in science technology, engineering and mathematics (STEM) fields. Data were gathered from nearly 1955 doctoral students in four state flagship institutions using a standardized template. We divided cohorts at each institution into quartiles and combined data across institutions.

In preliminary analyses, we found that women had lower GRE Quantitative scores than men, but graduation rates did not differ overall. In addition, women who left without a degree had similar GRE Verbal and Quantitative GRE scores. In contrast, men in the lowest quartile for GRE Quantitative, and to a lesser extent Verbal scores, had significantly higher completion rates than those in the two highest quartiles (see figure).

3) BIOMEDICAL PhD PROGRAMS

A study of 52 graduate students in the Tetrad biomedical sciences program at the University of California at San Francisco divided students into highest- and lowest-ranked based on faculty assessments. They found that GRE scores (Verbal, Quantitative or Analytical) did not differ significantly between the groups of highest- and lowest-ranked students. They also found that neither GPA nor the undergraduate institution (top 10 life sciences vs other universities) predicted ranking. The number of years of previous research experience and the subject test GPA did have some predictive value (15 points different between highest- and lowest-ranked students).

Pacheco and colleagues [13] studied various strategies for identifying students who would persist in the Biomedical Science PhD program at the Ponce Health Science University (Ponce, Puerto Rico) and earn PhD degrees. They found no relationship between persistence/completion and either undergraduate GPA (overall or science) or GRE scores. A composite score including the undergraduate GPA (UGPA), GRE score, and research experience, post-baccalaureate courses in medical sciences, presentations and publications had better predictive value. Importantly, the GRE did not increase the predictive power of the composite score.

Hall, Connell and Cook [14] examined the ability of various components of currently used admission procedures to predict productivity (number of first-author papers) of 280 doctoral students in the Biological and Biomedical Science Program at the University of North Carolina at Chapel Hill. They found that neither undergraduate GPA nor GRE (Verbal, Quantitative and Analytical Writing) scores differed significantly among students with varying publication numbers. They also found that there were no differences in amount of previous research experience between their most and least productive students. In contrast, the assessment of applicants by those who wrote letters of recommendation did have predictive value. As a side note, this study also found that in their cohort, women and minorities had lower GRE scores than males or Asian and White students.

Moneta-Koehler et al. [9] studied the importance of the GRE (General Test) in predicting a range of outcomes for 683 doctoral students in the Vanderbilt Interdisciplinary Graduate Program, an umbrella program for a number of biomedical disciplines. They found no significant correlation between GRE scores and graduation, passing the qualifying exam, time to defense, presentation count, first author publication count or securing independent grants and fellowships. There was a weak correlation between GRE Quantitative or Verbal scores and first semester grades and between the GRE Verbal score and graduate GPA. In addition, higher GRE Verbal scores predicted somewhat better faculty evaluations. These authors conclude that “there is little objective information in the application to reliably identify future outstanding performers in research.

Works Cited

  1. Ph.D. Completion Project. 2017; Available from: www.phdcompletion.org/quantitative/book1_quant.asp.
  2. National Science Foundation: Science & Engineering Doctorates; Table 18 Doctorate Recipients from U.S. Universities: 2016. 2018; Available from: https://www.nsf.gov/statistics/2018/nsf18304/data.cfm.
  3. Posselt, J.R., Inside Graduate Admissions: Merit, Diversity, and Faculty Gatekeeping. 2016, Cambridge, MA: Harvard University Press. 250.
  4. Sternberg, R.J. and W.M. Williams, Does the graduate record examination predict meaningful success in the graduate training of psychologists? American Psychologist, 1997. 52(6): p. 630-641.
  5. Burton, N.W. and M. Wang, Predicting long-term success in graduate school: A collaboratie validity study. 2005: GRE Board Report. p. 61.
  6. Walpole, M.B., et al., Selecting successful graduate students: In-depth interviews with GRE users., in GRE Board Research Rep. . 2002, ETS: Princeton, NJ.
  7. GRE General Test Score Information by Ethnicity/Racial Groups, 2009-2010. . Educational Testing Service, 2012.
  8. Miller, C. and K. Stassun, A test that fails. Nature, 2014. 510(12 June ): p. 303-304.
  9. Moneta-Koehler, L., et al., The Limitations of the GRE in Predicting Success in Biomedical Graduate School. PLOS ONE, 2017. 12(1): p. e0166742.
  10. Sedlacek, W.E., Alternative admissions and scholarship selection measures in higher education. (Assessment in Action). Measurement and Evaluation in Counseling and Development, 2003. 35(4).
  11. Kuncel, N.R., S.A. Hezlett, and D.S. Ones, A comprehensive meta-analysis of the predictive validity of the Graduate Record Examinations: Implications for graduate student selection and performance. Psychological Bulletin, 2001. 127(1): p. 162-181.
  12. Kuncel, N.R. and S.A. Hezlett, Standardized tests predict graduate students' success. Science, 2007. 315: p. 1080-1081.
  13. Pacheco, W.I., et al., Beyond the GRE: Using a composite score to predict the success of Puerto Rican Students in a biomedical PhD program. CBE--Life Sciences Education, 2015. 14(Summer 2015): p. 1-7.
  14. Hall, J.D., A.B. O'Connell, and J.G. Cook, Predictors of Student Productivity in Biomedical Graduate School Applications. PLoS One, 2017. 12(1): p. e0169121.