One of the UK Research Councils, the Engineering and Physical Sciences Research Council (EPSRC) has put new measures in place to attempt to reduce the flood of applications. The other research councils are discussing how to do this, and the ESRC is to be congratulated for taking soundings from the academic community about possible ways forward.
Their document, however, creates an odd mental state in academic readers. We’ve all been told for years that getting grant funding is a Good Thing. Individuals who can bring in the funds will be rewarded with promotion, tenure, and glittering prizes. Universities pride themselves on their grant income, which is major factor in higher education rankings. Now, however, we are told that applying for grants is a Bad Thing, which the academic community needs to work with ESRC to cut back, with statements such as:
• (ESRC has) an ambitious target of halving the number of applications submitted through its standard grants scheme by 2014
• greater self regulation has the potential to significantly reduce the volume of applications submitted by institutions
• (self-regulation will) probably not go far enough achieve the 50% reduction that the Council is seeking to deliver
The five options outlined by ESRC (copied verbatim below) are:
• Researcher Sanctions: This involves limiting the number of proposals from individual researchers who consistently fail to submit applications that reach an agreed quality threshold;
• Institutional Sanctions: This involves introducing sanctions for HEIs whose applications fail to meet a certain success rate and/or quality threshold;
• Institutional Quotas for ‘managed mode’ schemes. This involves the introduction of institutional quotas for certain schemes (e.g. early career researcher schemes, Large Grants/Centres, Professorial Fellowships).;
• Institutional quotas for all schemes: This involves responsive as well as managed mode schemes;
• Charging for applications. Levying an agreed fee for institutions submitting applications, with the option that this levy is redeemable if the application is successful.
The pros and cons of each of these is discussed, and readers are asked to comment.
My response to ESRC is that they are looking for solutions in the wrong place. To fix the problem, they need to change the basic structure of university funding so that institutions and individuals are no longer assessed on amount of research funding, but rather on an output/input function, i.e. how much bang do you get for your buck.
I have argued in a previous blog that it makes no sense to reward people simply for securing large amounts of funding. Currently, a person who secures a £500K grant which leads to two publications in lower-ranking journals will be given more credit than one who generates five high-ranking publications over the same period with a £50K grant. Clearly, some sorts of research are much more expensive than others; the problem is that the current system discourages people from undertaking inexpensive research. In my own field of psychology, there are cases of people who have published an impressive body of work based largely on student projects: they do not, however, get much appreciation from their institutions. Meanwhile, it has become almost mandatory for psychology grant applications to include an expensive brain scanning component, even if this adds nothing to the scientific value of the research. The introduction of Full Economic Costing (FEC) has added to the problem, by introducing incentives for researchers to add collaborators to their proposals, as this will bump up the cost of the grant. In short, the combination of the RAE and FEC does the opposite of encouraging cost-effectiveness in grant applications - it makes people focus solely on cost, the higher the better.
The same incentives have not only encouraged an anti-thrift mentality in higher education institutions, they have also changed expectations about the numbers of grants that academics should hold. Doing research takes time, and applicants are typically asked to quantify this in grant applications in terms of hours per week spent on the research. As far as I know, nobody ever adds up the estimated hours of research time for a person holding grants by different bodies. I suspect that there are cases where, if one were to total up estimated time across all a researcher's grants, it would exceed the number of hours in a week. This is particularly true for grant-holders in lectureships, who presumably are expected to spend some time on teaching activities. The system as it stands will encourage an academic to apply for 5 grants, spending 1 hr per week on each of them, than to apply for a single grant, on which they propose to spend 5 hr per week. Yet I’d bet that the quality of research would be better in the latter case, because high quality research takes time and thought. Over-commitment is encouraged by the current system, yet causes stress and waste. Many research-active academics are overwhelmed by backlogs of research data, because they feel compelled to submit more grant applications rather than writing up what they have done.
I have three suggested solutions to the current crisis. The first involves such a radical change to funding structures of Universities that it is unlikely to be implemented. The other two are both feasible:
1. For Government: Ditch the current methods for allocating funding to Universities so that cost-effectiveness and thrifty use of research funds are rewarded, rather than punished.
2. For research councils: When evaluating research quality, be more focused on track record of outputs relative to income, so that funding is steered toward those who have demonstrated good cost-effectiveness. Given differences in costs and time-scale of outputs across disciplines, a generic metric would be unworkable, but just by asking grants panels to look at this would be a step forward. Obviously this would not apply to new investigators.
3. For Universities: Scrutinise all the research grants held by individuals to ensure that amount of time specified for research activities is realistic, taking into account other job demands. A person would be debarred from putting in a grant proposal if they were over a limit in terms of hr per week already allocated to research.
If all three could be implemented, we could achieve a situation where the pressure on research councils is relieved, academics would be able to do good research with less stress on continually applying for funds, and research quality would be enhanced.
P.S. 11th January 2011
My congratulating ESRC for undertaking consultation with academics may have been premature. It seems very few academics knew about this, at least in my discipline of psychology. One who made enquiries at his institution discovered that the consultation document had gone to members of the Association of Research Managers and Administrators (whose website is the only place with a link to the document). In Oxford University, the recipient of the document circulated it to academics for their views, but I suspect in many places this did not happen. So, if rumour is to be believed, those of us who actually write the grants don't get asked about a major change in funding policy - just those who administer grant applications. Also noteworthy that the document was released on 20th December, just before Universities close for a long break.
As I've said before/elsewhere (I recall you commented on the version that ran on the Times Science Online site), I think measures to reduce the number of applications for response-mode funding need to be accompanied by increase "baseline" funding (via Universities direct to academics). This would allow people to do some research WITHOUT having to get external funding.
ReplyDeleteWithout that, the only likely outcome of the sort of measures the Research Councils are putting in place, certainly in the biosciences, will be the Russell Group looking at making a lot of people with no, or thin / occasional, grant funding redundant.
Of course, these are likely to be precisely exactly the people who do "the most with the least", funding-wise.
Too many applications is certainly a problem. I think your proposals are reasonable. But it seems clear that major research universities derive a significant portion of their operating budget from indirects off of grants. The indirects level could be lowered (and probably should be), but then the universities have to find another source of funding to replace that lost funding. Is that going to be in the form of tuition increases? I'm asking as a serious question: what can universities do to diversify their income stream?
ReplyDeleteTrying to upload a comment without success.
ReplyDeleteI would propose an even more radical solution, in an extension of Austin’s comments, which would be to completely abolish the Research Councils and disburse the RC budget directly to the Universities. Universities would then be responsible for allocating the funds to their own researchers. The advantages of such a system would be far-reaching. No grant writing, no review panels, and a huge reduction in administrators - both at the RCs, and in Universities.
ReplyDeleteMuch state funding of research in German Universities works in this way, where formulaic methods have been devised to allocate funds internally. In my particular area of cell biology I have been competing for years with a friend and collaborator in a German university, always aware that he has funding for one to two post-docs, a PhD student, a technician and a secretary, for which he never has to write a grant.
To disadvantage UK scientists even further, the current system of grant writing and peer-review means that our best ideas end up being perused by EXPERTS IN OUR OWN FIELDS, the very people we would least like to know what we’re up to. Not so my friend in Germany, who can safely keep everything under wraps until publication.
When one considers that a proportion of RC-funded work leads to new intellectual property and/or findings with commercial potential, it is perverse that we operate a mechanism for funding research that is so anti-competitive for UK-PLC.
@stephenemoss on Twitter
ReplyDeleteI would propose an even more radical solution, which would be to completely abolish the Research Councils and disburse the RC budget directly to the Universities. Universities would then be responsible for allocating the funds to their own researchers. The advantages of such a system would be far-reaching. No grant writing, no review panels, and a huge reduction in administrators - both at the RCs and in Universities.
Much state funding of research in German Universities works in this way, where formulaic methods have been devised to allocate funds internally. In my particular area of cell biology I have been competing for years with a friend and collaborator in a German University, always aware that he has funding for two post-docs, a PhD student, a technician and secretary, for which he never has to write a grant.
To disadvantage UK scientists even further, the current system of grant writing and peer review means that our best ideas end up being perused by EXPERTS IN OUR OWN FIELDS, the very people we would least like to know what we're up to. Not so my friend in Germany, who can safely keep everything under wraps until publication.
When one considers that a proportion of RC-funded work leads to new intellectual property and/or findings with commercial potential, it is perverse that we operate a mechanism for funding research that is so anti-competitive for UK-PLC.
great post and I agree about papers per £. Are the ESRC inviting comments openly? Can you post a link to the policy document or tell us how to make our opinions known to them?ds
ReplyDeleteThis is the most sensible comment I've read for years. At the moment, response mode grant funding for basic research has all but vanished in some areas. At a recent MRC Neuroscience board, only 7% of applications were funded. If you have to write 14 grants to get one, you might as well pack up and go home.
ReplyDeleteThe appalling thing about this situation is that it is largely self-inflicted. The total amount of money has not (yet) decreased substantially, but the cost of each grant has increased considerably for the reasons you have listed. In addittion to them, research councils increasingly earmark large amounts for projects that seem to some committee to be worthy, but may well not actually achievable at the moment.
We need some sort of concerted effort to point out that smallish response mode grants give the best value for money. it would be a good start if evidence could be found about the bangs per buck given by grants of different sizes.
Excellent, thought-provoking post, as usual. One point: do we know whether the huge recent increase in number of applications is evenly distributed across the quality spectrum? Or might it be that there is a similar number of excellent applications as before, but a large increase in those that would never have been competitive? I suspect that a lot of the people you describe, who previously did quite nicely publishing inexpensive student project type research, are now being forced by their department heads into writing grants. With low motivation and little practice or help in writing their applications, the likelihood of success is low.
ReplyDeleteYour suggestions for tackling the problem are very good, but my fear is that they may move us even closer towards a situation in which the researcher is assessed rather than the research. There already seems to be pressure for funding to be channelled towards a limited number of high-profile 'stars', which cannot be good for innovation in the long run. I appreciate what you say about these suggestions not applying to new investigators, but it seems to me that there are dangers if funding decisions are based even more than they are already on factors like track record, rather than excellence of the proposed research.
@Coronal
ReplyDeleteMost of the Russell Gp Univs have long since stopped allocating departmental PhD studentships to people who don't have external grants. I don't think there are many labs left in the RG Univs that manage on the occasional PhD student - those guys went out of business years ago. And in my bit of the biol sciences I have never seen a lab in the last decade that managed to produce papers using just the work of undergraduate final yr project students.
In my institution the people scraping by with one PhD student, perhaps an MSc student for 6 mths, and the very occasional response mode grant - typically obtained after writing 10+ applications - are folk that publish in high class specialist journals like J Biol Chem, J Physiol, J Neurosci and so on. As an example of "doing more with less" it it unmatched. But the pressure on these people, who are usually high-class scientists in their late 30s recruited off 5-8 years postdoc experience, often outside the UK, is truly terrible. And now it is to be "one grant in fifteen", as David Colquhoun says. Basically, it is untenable.
I know this is sure to be seen as an overly simplistic analysis, but . . . economically, isn't there an advantage to having considerably more research proposals than grants available? Doesn't that situation tend to ensure that the quality of research will be higher than if the reverse were true?
ReplyDeleteJust asking . . . .
Morgan
@stephenemoss on Twitter
ReplyDeleteMorgan - having more proposals than grants doesn't make economic sense, for the following reasons. An unsuccessful grant is the result of an academic somewhere, having spent perhaps 4-6 weeks of their time, to which we can add on 3-5 reviewers each spending a few days of their time, plus the panel members who have to read it, and the administrators who have to do the costings, signatures that must be obtained, approvals etc.
Each unsuccessful grant is thus the culmination of an expensive, time-consuming, but ultimately futile exercise. I had a good year last year (4 out of 4), but in the two preceding years I had some 10 failed grant applications. Would the tax payer, who pays the bulk of an academic's salary, be satisfied to know that so much time and effort is expended on generating nothing.
Hence my earlier post - abolish the Research Councils, if you want to see our competitiveness and productivity rise.
@Morgan: What Steve Moss said. I will quote a bit from my article on funding (linked earlier in the comments thread)
ReplyDelete"...consider the current hidden cost of the response-mode [grant] system with its one-in-five to one-in-ten success rates*. Writing grants takes time. A lot of time. And time means salary costs. A recent study by two Canadian academics concludes that the cost of the grant system, and the time spent penning fruitless applications, exceeds the cost of simply giving every qualifying University scientist in Canada £20,000."
* Note that this number is out of date. Nowadays
in the UK it would be 1 in 10 to 1 in 15, with the costs rising in parallel with the decline in funding rate
The draconian measures introduced by the EPSRC had some knock-on effects. For instance, the BBSRC has seen a substantial fall in number of applications across the board,although they have not introduced any formal measures (at least so far).
ReplyDeleteSecondly, in the RAE Physics panel there was a suspicion that some departments were deliberately concentrating their recent hiring on astronomy (broadly interpreted) where there needed to be no local capital investment because the equipment is mainly large central facilities eg telescopes. This we termed 'suitcase science' if I recall correctly. It is another version of distortion that tight funding produces (I might be provocative and ask how many astronomers we need?), although with the meltdown that hit STFC it may have backfired quite badly.
Finally, we cannot ask university's to police fEC costs because it is in their interests to ask for as much as possible. Grant giving panels see PI's asking for inflated sums and the justification too often is 'my university required it'. Wakeham is going to require fEC costs to be cut by 5% on average, and there will thus be a new tension generated. The trouble is different research does indeed cost very different sums and you end up comparing apples and pears with little hard evidence to go on in a grant submission.
Thanks for all the comments.
ReplyDeleteAnonymous: I have now added a link to a page where you can download the consultation document. But I have to say it is all rather strange. Usually, I’d expect a document like this to be prominently available on the ESRC home page, but I had to do some inventive Googling to find this link.
Austin: not sure I agree with you about this. I think research funding should be directed at specific projects that individuals can make a case for. Otherwise, it’s easy to meander along in an unfocussed way. But I think it may be different for wet-lab science, where you need more long-term technical support
GamesWithWords: the fact that universities get funding relative to grant income is exactly my point. My suggestion is that we give them the same amount of funding, but that the formula that dictates this should be based on the research output/grant income ratio, rather than income alone. People aren’t going to stop putting in large numbers of grants if their universities are driving them to do so, and the universities won’t stop driving them to do so unless the criteria for university income change. My suggestion would lead to them appreciating and encouraging people who can do more research inexpensively – and may also encourage them to adopt the strategy that Austin proposes.
Steve: I think, again, that your comment is not so relevant for the social sciences supported by ESRC. I have never worried about having an expert in my field evaluate my proposals – my studies take months, often years, to do and it would be very difficult for someone to steal a march on me after reading the proposal. I guess there just aren’t the commercial spin-offs anyhow.
And I actually *like* the research councils. I feel they do a good job, given the resources they have. In contrast to funding agencies in other countries, they are highly efficient, there is no corruption (at least I’ve never seen any signs of it), and they do come up with creative approaches to funding – especially ESRC, who’ve had some really imaginative schemes, like the 1 year postdoctoral fellowships (just axed, alas).
I think that if you can get over the appalling bureaucratese in the their document, it is an honest attempt to explain what the problem is and suggest solutions: problem is they are all intensely unpalatable.
David: thanks for the supportive comments!
Coronal: I don’t think there are data on quality of projects, but my guess would be that it would be declining, because people are being pressured to write grant proposals by their institutions. I think there should be only one reason for writing a grant proposal, and that’s because you have a good idea and need funding to test it.
Morgan: I think you are right in principal, in that we need a kind of ‘survival of the fittest’ to ensure quality, but there’s a satuation point at which it just gets silly. As Steve points out, the costs for the people putting in proposals are very high. Also I’ve sat on grants boards where you end up feeling you might as well toss a dice to decide who gets funded, because there’s lots of excellent stuff on the table, but not enough money to go around.
Finally, I’ve been pondering further on the issue of how much time people specify for working on a grant. I’ve seen cvs of people who have 6 grants, but their time specification for any one of them is no more than 3 hr per week, and sometimes as low as 1 hr per week. If you are a collaborator that might be reasonable, but I reckon any grant needs at least one person putting in a *minimum* of 8 hr per week for it to be likely to succeed in terms of being of high quality and well written up. So maybe we should set a minimum time commitment for a PI on any one grant. You can delegate a certain amount to junior staff, but at the end of the day, the person awarded the grant should be putting their brains to work on the project. But maybe it is different in other disciplines?
Interesting ideas, but I'm not entirely sure about the research output/income ratio idea. How do you define research output? Number of publications (including retractions?), citations, impact factor. Are any of these measures satisfactory indicators of scientific quality or merit? Is the £500k grant that produces 1 paper necessarily less impressive in its scientific contribution that the £50l grant that produces 5 papers. Surely that depends on the longer-term scientific merit/value of the one paper - and how would you judge that over the short-term?
ReplyDeleteI did post another comment, but it seems to have been spam filtered. I think the most notable thing in it was a link to the Canadian study which estimated that, for the cost of the response-mode grant assessment system in Canada (i.e. just the cost of people writing and assessing all the proposals, plus the administration involved) you could have given every qualifying University academic in Canada about £ 20,000 a year.
ReplyDeleteThe situation in Germany is not so rosy as Stephen describes. Yes, there are Postdoc and PhD positions that come with professorships, but there is next to no intermediate level of lecturers and readers. Departments tend to be a set of narrow columns headed by professors working with their junior support staff, high turn-around included.
ReplyDeleteAnd every time a professor leaves, the support positions are in danger of being cut ...
OK, rant over.
I completely support David's suggestion of more smaller grants that give additional bang for the buck. These are ideal for nurturing young PIs who can build a reputation for high return on investment.
Finally, Dorothy said: "So maybe we should set a minimum time commitment for a PI on any one grant. You can delegate a certain amount to junior staff, but at the end of the day, the person awarded the grant should be putting their brains to work on the project."
Requiring the PI to put down a commitment of eight hours a week
is not necessary as long as the researcher co-investigators are sufficiently experienced. In a department with a significant number of senior researchers / career researchers, these people are often the ones who write the grants, do the work, manage the funds, etc., with the PI mostly acting as a figurehead, mentor, sounding board, and guide. Such a structure enables departments to build large research groups without overloading their permanent academic staff.
Thank you for the excellent post on an important topic. As other people have said smaller grants are good value for money and it is a pity that the RC are moving toward a system with fewer larger awards.
ReplyDeleteWith respect to the issue of relating income to output the natural place to do that would be the RAE/REF. Whatever criterion they will use to access the publications that score should be divided by the amount of funding. The objection may be that it would put people off getting money but that is unlikely to be the case given other pressures (eg from institutions) to bring in funding. It would act as a balancing factor.
This was a very sensible post. It seems to me that the simplest thing that could be done to relieve pressure would be to remove grant income as an independent contributor to the REF. The REF already assesses quality of publications. If you get a grant, and you use it properly, it should contribute to your quality publications. Including grant income again means that it is effectively being counted twice, with no real assessment of value for money.
ReplyDeleteVery thoughtful post, it's refreshing to hear these kinds of sensible discussions going on across all disciplines. Just a quick comment on Dorothy's idea that a PI should have a minimum time commitment on a grant - this happens, at least informally, in the environmental sciences, where I've seen negative comments from reviewers when the PI has committed insufficient time (usually 15-20% is considered the minimum). Given this, adding up people's committed time and preventing them from applying for more funding once a threshold is reached seems a sensible solution (and sufficiently obvious for me to have come up with it independently - honestly! - in my latest Mola mola blog over on Nature Network)
ReplyDeleteI do like the principle, especially the idea that small but effective grants should be encouraged. My concern is the practice. In a very sensible triad of recommendations, the problem is surely captured by:
ReplyDelete"Given differences in costs and time-scale of outputs across disciplines, a generic metric would be unworkable, but just by asking grants panels to look at this would be a step forward."
But how? We all know that different research will have different costs and timings, so how do you go about leveling out the playing field in order to make your cost-effectiveness assessment fair? Or do you necessarily end up discriminating against expensive research (my research, I confess, is at the very pricey end...)? What do you use to discriminate between an expensive grant that is "efficient" but does require expensive "things" and a grant overblown in the way you describe? I'm sure this can be done in extreme cases, but in the majority it is surely much less clear.
Further, some expensive (and therefore in the terms here, "non cost-effective") research is nevertheless worth doing.
So I really do think these are sensible proposals in theory, but without some outline of how you do it, I wonder if what you'd end up doing is just shutting down expensive science. Which of course is one way to go about things, but I'm not sure it's what you're driving at.
Looks like the ESRC has decided on its initial approach to demand management, which revolves around self-regulation. What chances this will be effective given the institutional (and personal, in some cases) pressures operating the other way?
ReplyDeletehttp://www.esrc.ac.uk/news-and-events/news/initial-programme-of-measures.aspx
We are offering website service & design and application for Android & IOS......
ReplyDeletewp plugin
IVR Application can be utilized for outbound calls, as IVR System are more wise than numerous prescient dialer frameworks, and can utilize call progress recognition to perceive diverse line conditions.
ReplyDelete