Research impact

I recently learned of the death of two people who, in very different ways, played important roles in my early academic career. In reflecting on their passing, I have gained clarity over my frustrations with contemporary research. I shall explain.

Walter E. Grinder was a key part of the organisational and intellectual rehabilitation of classical liberalism that occurred in the 1970s. He was a true ideational entrepreneur and although he made significant scholarly contributions of his own, his biggest impact came from linking people to ideas and to each other.

Walter E. Grinder (1938-2022)

For example, the Mercatus Center at George Mason University is the definitive model of a public policy arm of a reputable academic establishment. It emerged out of the Centre for Study of Market Processes, which originated at Rutgers University and Walter was there for that. When the Institute for Humane Studies moved to George Mason in 1985, Walter (as vice-President) was the impetus. Walter had studied under Hans Sennholz which gave him direct lineage to my favourite economist of all time  (Ludwig von Mises) and he even attended Mises’s seminar at NYU. This gave him access to an intellectual tradition that was severely waning, and he helped nurture and steer the careers of people like Tyler Cowen, Peter Boettke and Larry White. As Alberto Mingardi has said, “Never underestimate the role of the person who puts a good book in your hands”.

As a graduate student, I was well aware of the key personnel behind the second “Austrian revival” and was proud to join Walter’s infamous email list in 2006. I also knew that he was based in Mountain View so when I visited San Jose as a Fulbright Scholar in 2011 I reached out to him. We went for coffee and had a long and wonderful conversation. I knew that he had health problems, and did not expect to be able to meet him again. I thanked him for the role he had played in creating the intellectual and professional landscape that I had entered. It was a meaningful experience, and on behalf of my generation of economists, I am glad I was able to say that in person.

For more about Walter see this collection of resources, published by Cato on the occasion of his 75th birthday.

It was with much greater shock that I also recently heard that Jeffrey Friedman had passed away (see this obituary by Ilya Somin). Jeff had a thing: the epistemological foundations of political economy. He thought about it more than anyone else, and founded and steered a wonderful journal, Critical Review, with that as its core focus. In 2008 I wrote to Jeff because I’d heard that he was publishing a special issue about Bryan Caplan’s ‘The Myth of the Rational Voter‘, and I had a paper outlining a subjectivist critique. I got an enthusiastic and incredible detailed and helpful reply which prompted a correspondence that led to him soliciting my work, offering his guidance, and ultimately becoming a co-author. I believe that relationship was common, and Jeff’s infectious obsession has simultaneously frustrated and prompted many early stage careers! Our article was eventually published in 2011, called, ”Search” vs “browse”: A theory of error grounded in radical (not rational) ignorance” and the parts of my original article that he didn’t like became  “A subjectivist’s solution to the limits of public choice” (published in 2014 by the Review of Austrian Economics).

Jeffrey Friedman (1959-2022)

Each of these two men revealed an appealing vision of academic life, and could easily be treated as role models. A modern Walter would manage an email list (or substack) alerting select people to new works and commentary – a mentor and connector. Someone who also contributes to funding decisions and arranges conferences and professional workshops. And Jeff’s example of a scholar who creates and nurtures a journal that carries an editorial focus in every issue, with spin off books and a distinct and unique voice, remains underappreciated and much needed. Yet both of these models would constitute career suicide for young academics, and be indulgent and in tension with the professional incentives or demands for mid career ones.

~

Over recent years I’ve made a deliberate decision to clean up my research “pipeline” and dispense with long standing research topics in order to create scope to assess my direction and sharpen my focus. And I’ve been reflecting on what our objectives, as academic researchers, should be. To me, there are four areas where we can establish our impact but each of these areas imply separate audiences, and therefore have different objectives.

  1. The first and most obvious dimension is academic – and should of course be the foundation. To some extent other dimensions flow from this (hence the different border colour) but I don’t want to propagate the myth that research impact must escape the ivory tower in one leap. In this dimension, your audience is other academics and the objective is to make meaningful contributions to science. Elite academics will have heavily cited books and articles, well placed PhD students who continue that intellectual work, and recognition from their peers in terms of awards and prestigious appointments. Other academics will publish, but struggle to justify whether that, in itself, has any wider impact on society.
  2. We can also have policy impact – and see how our research improves the functioning of society. In this case, the audience is policymakers and the objective is to influence policy.
  3. Many of us seek public impact – and reaching the hearts and minds of “the man on the street” can be a noble aim. Here, the audience is the general public and the objective is to become a well known public intellectual.
  4. Practical research can also have a corporate impact – particularly those of us who work in a business school might want to see our research improve the performance of organisations. The audience is executives and actual managers and the objective is to inform management practice.

An important factor in all this is legitimacy. Academics face competition from an array of different professions within each area. For academic status we compete with journalists and commentators. Our academic books are increasingly aimed at explaining research findings for a non-academic audience, and a lot of published articles are merely under researched think pieces. (Notice the similarity, in content and style, between two recent books on informational authoritarianism. One by Gideon Rachman, an FT journalist. And the other by Sergei Guriev and Daniel Treisman, tenured and elite academics.)

For policy status we compete with commentators, think tankers and politicians (who, like journalists, tend to be young, hungry, and in plentiful supply).

For public status we compete with commentators and public intellectuals.

And for corporate status we compete with executives and business gurus.

It is hard for academics to make a mark in these competitive arenas, especially when scholarly integrity can weigh us down and it is merely our academic title that delivers status. It’s no surprise that the industry is full of charlatans of all stripes – people who are too lazy and ill disciplined to be journalists (!), too idiosyncratic and out of touch with public opinion to be politicians, and without the management skills to do well in business.  It really is remarkable how academics get encouraged to enter each of these territories, as if there is merit in doing so.

To maintain legitimacy, however, academics must have some scholarly achievements to fall back on. I’ve written elsewhere about whether media engagements should have a basis in scholarly expertise, but the bottom line is that without peer-reviewed publications we are merely hacks (bad journalists), charlatans (anonymous pundits), or grifters (failed business people).

At various points in my career, I’ve been all four. On each dimension I’ve felt some success, and glimpsed what it might take to achieve genuine impact.

  1. Academic impact – I’ve had over 40 peer-reviewed journal publications including two from the FT 50 list. And yet those two publications were two of the worst academic articles I’ve written, and reflected an opportunistic and somewhat degrading effort to reach that list rather than achieve any scholarly accomplishment. My article with Jeff Friedman was one of my best, most cited, and appeared in a journal on the Social Science Citation Index, yet it has zero weight on internal metrics of research performance (because the journal doesn’t contribute to standard business school rankings lists). I’ve been discouraged from submitting to field journals even though they tend to be more widely read, and contribute more to my professional reputation, than higher listed ones. As a graduate student you crave that first journal publication and each hit is something to celebrate. I’m proud that I’ve had at least one publication in every year since I obtained my PhD. And yet I don’t feel much professional advancement as a consequence. This may be a reflection of research strategy, and my preference for single authored papers on a wide range of topics. But to some extent I feel that I’ve played the “publish or perish game”, done ok, but think “and what about it?” I don’t have PhD students and therefore don’t feel I’m in a position to establish whether I have the capabilities to achieve an elite research career or not. But publish for the sake of publishing? Sacrificing your life to become an unpaid content creator for Elsevier? Really?
  2. Policy impact – I’ve had policy proposals that have received widespread media coverage (including front pages of broadsheet newspapers and even page 3 of The Sun) and have made legitimate contributions to actual policy reforms (such as the rate of corporation tax). And yet the UK policy environment is cut throat and fast moving. I knew that my voice was more powerful due to my academic credentials than my scholarly expertise, and that many examples of media coverage compromised my integrity. I don’t have the stamina or inclination to become a go to figure for broadcast media, and am saddened by the fact that on the occasion where we had a PM that was keen to promote the very ideas I’m passionate about the whole thing was an epic clusterfuck. Ok, we can meme the PM to ban certain dog breeds. But should we?
  3. Public impact – I’ve had a weekly column in a large circulation newspaper that ran for over a year and have written a practical and accessible guide to economics. I ran a social media workshop for faculty to engage in public conversations and we all boosted our Klout score. I have more LinkedIn followers than many full time LinkedIn influencers. And yet I have seen no real tangible consequence to that. I don’t make much money from book sales and speaking engagements dried up some time ago. I find social media fairly boring and am back to reading The Economist, like a disaffected undergraduate.
  4. Corporate impact – I was one of the first and only academics to be invited to an internal training program for a company that utilises concepts and frameworks that are at the heart of my academic interests. And yet I’ve not been able to leverage that either within a standard business school curriculum or by providing in house training. I have a good understanding of what topics are in demand and routinely deliver training programmes for executives. But I have no market power (more pay requires more hours on the ground) and not a single former student has seen any benefit in my involvement in their company decision-making. Should I degrade myself by contributing to my own “company”, and giving students the perceived legitimacy of an instructor that runs a barely operational side hustle? I don’t think so.

For purely academic impact, we can look at several key areas: research output, research coverage, and leadership and collegiality. As the table below shows, we can consider various success levels. As we move through our careers we should progress towards the right on many of these dimensions. This is, I think, standard, familiar, and achievable. But it does not constitute the type of wider impact many of us attempt to talk about.

~

So what to do?

If you ask a decent academic what motivates your research they’ll say ideas. But no-one will pay you to do that. Impact matters, and each dimension has a metric. Ultimately, you have to chase one of the following:

  • Citations – academic prowess.
  • Influence – demonstrable impact on policy reform.
  • Clicks – social media metrics.
  • Consulting gigs – being hired.

So pick your master.