AER-equivalent score, by percentile of graduates, after six years
“IF THE objective of graduate training in top-ranked [economics] departments is to produce successful research economists, then these graduate programmes are largely failing.” That’s the startling message from a recent paper published in the Journal of Economic Perspectives.
How did the authors of this paper reach such a pessimistic conclusion? They look at a 14,300 people who received an economics PhD from 154 American and Canadian institutions. They then find a massive database of academic papers published over a two-decade period.
From that, they are able to tell how many papers each PhD graduate has produced in the six years after leaving graduate school. (Six years, by the way, is about the average time it takes for a newly-minted PhD to get tenure).
Of course, quantity is not the only measure of success. One great paper is worth more than three bad ones. So the authors create an index that adjusts the number of publications by the quality of the journal it appears in.
The authors end up with what they call the “American Economic Review-equivalent”. To get published in the AER is a dream for any economist and so other journals are indexed in relation to it. An article in the Journal of Political Economy, for instance, is worth 0.67 papers in AER. A paper in Economic Theory is worth a quarter.
Some of the results are not terribly surprising. Graduates from the big-hitting universities can be extremely productive. The graduate in the 99th percentile from Harvard or MIT—that is, right at the very top of the graduating class—produces over 4 AER-equivalent papers over six years.
But the vast majority of PhD students, even at top universities, produce nowhere near that much (see chart). The number of AER-equivalent papers of the median PhD student, six years after graduation, is below 0.2 for all universities. Yes, all—even Harvard, MIT and Chicago.
The 50th percentile at almost all universities has a score of 0.1. That’s equivalent to publishing one paper in a second-tier field journal over six years.
What are the implications of these results? Even if you have been accepted into a top economics department, there is no guarantee that you will be a successful researcher. In fact top researchers come from a range of institutions, not just the best ones.
The researcher in the 99th percentile of the typical “non-top-30” institution—that is, the 124 other universities in the authors’ sample—is better than her equivalent from a range of big-hitting institutions like Penn State and the University of Texas at Austin.
The paper probably says something about how economics PhD programmes are taught. Professors may give a disproportionate amount of time to the students that they think are most naturally gifted, while leaving the majority behind. As a result that lucky student is much more likely to have a successful publication record.
Now: the crucial question is whether economics PhD students want to be successful researchers. The authors see this as self-evident:
Our experience suggests that most students, especially at the better programs, enter graduate school planning to seek academic jobs, or at any rate, jobs that require research.
I’m not so sure: many econ PhDs that I know have no intention of becoming an academic but instead want to work for government or an NGO. And lots of people working in the upper echelons of business and government may produce research, but may not publish it in a peer-reviewed journal.
Take economists at the IMF, for example, who produce working papers that may never become “proper” academic articles. The same goes for government employees who produce policy analysis.
For the vast majority of economics PhDs there is little point in being more productive. As we have shown before, there are far more PhDs produced each year than there are job openings.
America produced more than 100,000 doctoral degrees between 2005 and 2009; in the same period there were just 16,000 new professorships. What’s the point in killing yourself to be a productive researcher when finding an academic job is so hard?
Published in: The Economist, London | November 2014