Ten things than can sink a grant proposal: Advice for a young psychologist

 © cartoonstock.com

So you’ve slaved away for weeks giving up any semblance of social or family life in order to put your best ideas on paper. The grant proposal disappears into the void for months during which your mental state oscillates between optimistic fantasies of the scientific glory that will result when your research is funded, and despair and anxiety at the prospect of rejection. And then it comes: the email of doom: “We regret that your application was not successful.” Sometimes just a bald statement, and sometimes embellished with reviewer comments and ratings that induce either rage or depression, depending on your personality type.

There are three things worth noting at this point. First, rejection is the norm: success rates vary depending on the funding scheme, but it’s common to see funding rates around 20% or less. Second, resilience in the face of rejection is a hallmark of the successful scientist, at least as important as intelligence and motivation. Third, there is a huge amount of luck in the grants process: just as with the journal peer review process, reviewers and grant panel members frequently have disparate opinions, and rejection does not mean the work is no good. However, although chance is a big factor, it's not the only thing.

This week I participated in a workshop on “How to get a grant” run by my colleague Masud Husain. We are both seasoned grant reviewers and have served on grants panels. Masud prepared some slides where he noted things that can lead to grant rejection, and I dug out an old powerpoint from a similar talk I’d given in 2005. There was remarkable convergence between the points that we highlighted, based on our experiences of seeing promising work rejected by grants panels. So it seemed worth sharing our insights with the wider world. These comments are tailored to postdocs in psychology/neuroscience in the UK, though some will have broader applicability.

1. Lack of clarity

The usual model for grant evaluation is that the proposal goes to referees with expertise in the area, and is then considered by a panel of people who cover the whole range of areas that is encompassed by the funding scheme. The panel will, of course, rely heavily on expert views, but your case can only be helped if the other panel members can understand what you want to do and why it is important. Even if they can't follow all the technical details, they should be able to follow the lay abstract and introduction.

It's crucial, therefore, that you give the draft proposal to someone who is not an expert in your research topic - preferably not a close friend, but someone more likely to be critical. Ask them to be brutally honest about the bits that they don't understand. When they give you this feedback, don't argue with them or attempt verbal explanations; just rewrite until they do understand it.

2.Badly written proposal

In an ideal world, funders should focus on the content of your proposal rather than the presentation, right? Things like spelling, formatting, and so on are trivial details that only inferior brains worry about, right?

Nope. Wrong on both counts. The people reading your grant are busy. They may have a stack of proposals to evaluate. I have, for instance, been involved in evaluating for a postdoctoral fellowship scheme where my task was to select the top five from a heap of forty odd proposals. The majority of proposals are very good, and so this is a task that is both difficult and important. You  can end up feeling like one of Pavlov's dogs forced to make ever-finer discriminations, and this can put you in a grumpy and unforgiving mood. You take a dim view of proposals where there are typos, spelling errors and missing references. I've seen grant proposals where the applicant failed to turn off 'track changes', or where 'insert reference here' is in the text. In this highly competitive context, there's a high chance that these will go on the 'reject' heap. Even if there are no errors in the text, a densely packed page of verbiage is harder for the reviewer to absorb than a well laid-out document with spacing and headings. You will usually feel that the word limit is too short, and it is tempting to pack in as many words as possible, but this is a mistake. Better ditch material than confront your reviewer with an intimidating wall of words. Judicious use of figures can make a huge difference to the readability of your text, and readability is key. I personally dislike it when numbers are used to indicate references, especially if the reference list then omits titles of referenced papers: people commonly do this to save space, but I like to be able to readily work out what references are being referred to.

Anyone can improve the presentation of a grant. Use of a spell-checker is obvious, but if possible, you should also look at examples of successful applications to see what works in terms of layout etc. You can also Google "good document layout" to find websites full of advice.

3. Boring or pointless proposal

This is a difficult one, because what one person finds riveting, another finds tedious. But if you find your proposal boring, then there's close to zero chance anyone will want to fund it. You should never submit a grant proposal unless you are genuinely excited by the work that you are proposing. You need to ask yourself "Is this what I most want to spend my life doing over the next 2-3 years?" If the answer is no, then rethink the proposal. If yes, then it's crucial to convey your enthusiasm.

4. Lack of hypotheses

This is a common reason for rejection of grant proposals. The phrase 'fishing expedition' is often used to dismiss research that involves looking at a large number of variables in an unfocussed way. As an aside, I remember an exasperated colleague saying that a fishing expedition was an entirely sensible approach if the aim was to catch fish! But funding bodies want to see clear, theoretically-driven predictions with an indication of how the research will test these. A hypothesis should have sufficient generality to be interesting, and usually will be tested by a variety of methods.  For instance, suppose I think that dyslexia may be caused by a particular kind of sensory deficit, and I plan to test children on a range of visual and auditory tasks. I could say that my hypothesis is that there will be differences between dyslexics and controls on the test battery, but this is too vague. It would be better to describe a particular hypothesis of, say visual deficit, and make predictions about the specific tasks that should show deficits. Better still one would set out a general hypothesis about links between the putative deficit and dyslexia, and specify a set of experiments that tested the predictions using a range of methods.

Also, ask yourself, is your hypothesis is falsifiable, and will it yield interesting findings even if it is rejected. If the answer is no, rethink.

5. Overambitious proposal

This is another common reason for rejection of proposals, particularly by junior applicants. In psychology, people commonly overestimate how many participants can be recruited (especially in clinical and longitudinal studies) and how much testing experimenters can do. Of course, you do sometimes see cases where the proposal does not contain enough. But that is much less common that the opposite.

If you are working with human participants, you need to demonstrate that you have thought about two things:
a) Participant recruitment

• Where will you recruit from?
• Have you liaised with referral sources?
• How many suitable people exist?
• What proportion will agree to take part?
• Overall, how many participants will you be able to include in a given period (e.g. 3 months/ 1 year)?
• Have you taken into account the time it will take to get ethics approval?
• Have you costed proposal to take into account reimbursements to partipants and travel?

b) Is your estimate of research personnel realistic?

• How long does it take to test one participant?
• Have you taken into account  the fact that researchers need to spend time on :
  - Scheduling appointments
  - Travelling
  - Scoring up/entering/analysing data
  - Doing other academic things (e.g. reading relevant literature, attending seminars)

If you are working with fancy equipment, then you need to consider things like whether you or your research staff will need training to use it, as well as availability.

For more on this, see my previous blogpost about an excellent article by Hodgson and Rollnick (1989): "More fun, less stress: How to survive in research", which details the mismatch between people's expectations of how long research takes and the reality.

6. Overoptimistic proposal

An overoptimistic proposal assumes that results will turn out in line with prediction and has no fall-back position if they don't. A proposal should tell us something useful even if the exciting predictions don't work out. You should avoid multi-stage experiments where the whole enterprise would be rendered worthless if the first experiment failed.

7. Proposal depends on untried or complex methods

You're unlikely to be funded if you propose a set of studies using a method in which you have limited experience, unless you can show that you have promising pilot data. If you do want to move in a new direction, try to link up with someone who has some expertise in it, and consider having them as a collaborator. Although funders don't want to take risk with applicants who have no experience in a new method, they do like proposals to include a training component, and for researchers to gain experience in different labs, even if just for a few months.

8. Overcosted (or undercosted) proposal

This one is easy: Ask for everything that you do need, but don't ask for things you don't need. This is not the time to smuggle in funding for that long-desired piece of equipment unless it is key to the proposal.

The committee will also be unimpressed if you ask for things the host institution should provide. But don't omit crucial equipment because of concerns about expense: just be realistic about what you need and explicitly justify everything.

9. Proposal is too risky

This is much harder one to call. Most funding bodies say they don’t want to fund predictable studies, but they are averse to research where there is high risk of nothing of interest emerging. A US study of NIH funding patterns came to the depressing conclusion that researchers who did high-impact but unconventional research often missed out on funding (Nicholson & Ioannidis, 2012). Funders often state that they like multidisciplinary research, but that runs the risk that, unless methodologically impeccable in all the areas that are covered, it will get turned down.

If you want to include a high-risk element to the proposal, take advice from a senior person whose views you trust - their reaction should give you an indication of whether to go ahead, and if so which aspects will need most justification. And if you want to include a component from a field you are not an expert in, it is vital to take advice from someone senior who does know that area.

It is usually sensible to be up-front about the risky element, and to explain why the risk is worth taking. If you are planning a high-risk project, always have a safety net - i.e. include some more conventional studies in the proposal to ensure that the whole project won't be sunk if the risky bit doesn't pan out.

10. Statistics underspecified or flawed

You need to describe the statistical analysis that you plan, even if it seems obvious to you - if only to demonstrate to the panel that you know what you are doing and have the competence to do it. If you are planning to use complex statistics, get advice from a statistician, and make it clear in the proposal that you have done so. If you don't have adequate statistical skill, consider having a statistician as consultant or collaborator on the grant. And do not neglect power analysis: underpowered studies are a common reason for grants to be rejected in biomedical areas.

Most grants panels are multidisciplinary, and there can be huge cultural differences in statistical practices between disciplines. I've seen cases where a geneticist has criticised a psychology project for lack of statistical power (something geneticists are very hot on), or where a medic criticises an experimental intervention study for not using a randomised controlled design. Don't just propose the analysis that you usually do: find out what is best practice to ensure you won't be shot down for a non-optimal research design or analytic approach.

***********************

Finally, remember that the proposed research is one of three elements that will be assessed: the others are the candidate and the institution. There's no point in applying for a postdoctoral fellowship if you have a weak CV: you do need to have publications, preferably first-authored papers. There's a widespread view that you don't stand a chance of funding unless you have papers in high impact journals, but that's not necessarily true, especially in psychology. I'm more impressed by one or two solid first-authored papers than by a long string of publications where you are just one author among many, and (in line with Wellcome Trust policy) I don't give a hoot about journal impact factors. Most funding agencies will give you a steer on whether your CV is competitive if you ask for advice on this.

As far as the institution goes, it helps to come from a top research institution, but the key thing is to have strong institutional support, with access to the resources you need and to supportive colleagues. You will need a cover letter from your institution, and the person writing it should convey enthusiasm for your proposal and be explicit in making a commitment to providing space and other resources.

Good luck!

Reference
Nicholson JM, & Ioannidis JP (2012). Research grants: Conform and be funded. Nature, 492 (7427), 34-6 PMID: 23222591