The academic backlog

Photo from http://www.pa-legion.com

Here’s an interesting question to ask any scientist: If you were to receive no more research funding, and just focus on writing up the data you have, how long would it take? The answer tends to go up with seniority, but a typical answer is 3 to 5 years.

I don’t have any hard data on this – just my own experience and that of colleagues – and I suspect it varies from discipline to discipline. But my impression is that people generally agree that the academic backlog is a real phenomenon, but they disagree on whether it matters.

One view is that completed but unpublished research is not important, because there’s a kind of “survival of the fittest” of results. You focus on the most interesting and novel findings, and forget about the rest. It’s true that we’ve all done failed studies with inconclusive results, and it would be foolish trying to turn such sow’s ears into silk purses.  But I suspect there’s a large swathe of research that doesn’t fall into that category, but still never gets written up.  Is that right, given the time and money that have been expended in gathering data? Indeed, in clinical fields, it’s not only researchers putting effort into the research – there are also human participants who typically volunteer for studies on the assumption that the research will be published.

I’m not talking about research that fails to get published because it’s rejected by journal editors, but rather about studies that don’t get to the point of being written up for publication. Interest in this topic has been stimulated by Ben Goldacre’s book Bad Pharma, which has highlighted the numerous clinical trials that go unreported – often because they have negative results. In that case the concern is that findings are suppressed because they conflict with the financial interests of those doing the research, and the Alltrials campaign is doing a sterling job to tackle that issue. But beyond the field of clinical trials, there’s still a backlog, even for those of us working in areas where financial interests are not an issue.

It’s worth pausing to consider why this is so. I think it’s all to do with the incentive structure of academia. If you want to make your way in the scientific world, there are two important things you have to do: get grant funding and publish papers. This creates an optimisation problem, because both of these activities take time, and time is in short supply for the average academic.  It’s impossible to say how long it takes to write a paper, because it will depend on the complexity of the data, and will vary from one subject area to the next, but it’s not something that should be rushed. A good scientist checks everything thoroughly, thinks hard about alternative interpretations of results, and relates findings to the existing research literature. But if you take too much time, you’re at risk of being seen as unproductive, especially if you aren’t bringing in grant income. So you have to apply for grants, and having done so, you have then to do the research that you said you’d do. You may also be under pressure to apply for grants to keep your research group going, or to fund your own salary.

When I started in research, a junior person would be happy to have one grant, but that was before the REF. Nowadays  heads of department  will encourage their staff to apply for numerous grants, and it’s commonplace for senior investigators have several active grants, with estimates of around 1-2 hours per week spent on each one. Of course, time isn’t neatly divided up, and it’s more likely that the investigator will get the project up and running and then delegate it to junior staff, then putting in additional hours at the end of the project when it’s time to analyse and write up the data. The bulk of the day-to-day work will be done by postdocs or graduate students, and it can be a good training opportunity for them. All the same, it’s often the case that the amount of time specified by senior investigators is absurdly unrealistic. Yet this approach is encouraged: I doubt anyone ever questions a senior investigator’s time commitment when evaluating a grant, few funding bodies check whether you’ve done what you said you’d do, and even if they do, I’ve never heard of a funder demanding that a previous project be written up before they’ll consider a new funding application.

I don’t think the research community is particularly happy about this: many people have a sense of guilt at the backlog, but they feel they have no option. So the current system creates stress as well as inefficiency and waste. I’m not sure what the solution is, but I think this is something that research funders should start thinking about. We need to change the incentives to allow people time to think. I don’t believe anyone goes into science because they want to become rich and famous: we go into it because we are excited by ideas and want to discover new things. But just as bankers seem to get into a spiral of greed whereby they want higher and higher bonuses, it’s easy to get swept up in the need to prove yourself by getting more and more grants, and to lose sight of the whole purpose of the exercise – which should be to do good, thoughtful science.  We won’t get the right people staying in the field if we value people solely in terms of research income, rather than in terms of whether they use that income efficiently and effectively.