How to survive in psychological research

A Handbook of Skills and Methods in Behavioural Research is not the place you’d expect to find something to make you smile, but many years ago one of my graduate students pointed me to a wickedly funny piece by Ray Hodgson and Stephen Rollnick. Since then I’ve found myself loaning an increasingly dog-eared photocopy of the article to new generations of students and postdocs. Sadly, the article is not available electronically, though copies of the book can be tracked down. So here’s a summary of Hodgson and Rollnick’s laws, all of which are as pertinent to the older, seasoned researcher as to the intended readership of the ‘young, lively, questioning researcher who has great expectations but a lack of practical experience’:

Law #1. Getting started will take at least as long as the data collection

The barriers are various: perhaps the most salient for the newcomer is dithering induced by fear of commiting to a non-optimal design. Another barrier is having too many people involved; this just multiplies the dithering, as each person tries to include additional measures or graft on subsidiary projects. It’s vital to have someone who will take control for decision-making - a point emphasised in my previous post on the NationalChildren’s Study.

Hodgson and Rollnick also mention the need to get ethics approval, another topic that has featured on my blog. It’s got a lot worse in the years since they wrote their article: there’s even a kind of ‘meta-research’ in which the goal is to quantify the baleful influence of ethical scrutiny on research efficiency (e.g. Elwyn et al, 2005).

Law #2. The number of available subjects will be one-tenth of your first estimate

Note to young readers: ‘subjects’ are what we used to call ‘participants’ until someone decided that the term implied an unheathily controlling attitude to those taking part in experiment.

Quite simply, "as soon as somebody starts to research a particular condition, people with that condition leave the district". It’s totally true and totally mysterious.

Law #3. Completion of a research project will take twice as long as your last estimate and three times as long as your first estimate

This may be moderated by whether you are a pessimist or optimist, but no true pessimist would ever embark on a research project.

Law #4. A research project will change twice in the middle

Hodgson and Rollnick cite their experience with a one-year project to test the effectiveness of a Drinkwatchers program for problem drinking. “All we needed were thirty subjects from amongst the estimated 10,000 problem drinkers in South Glamorgan. One hundred and sixty problem drinkers answered the advertisement…Of these only eight volunteered to join a Drinkwatchers group, three turned up to the first meeting and one of these came to the second.” Since the study had been funded, the researchers decided the best they could do was an alternative study to discover what kinds of help problem drinkers really want. Needless to say, these days, such a change of plan would necessitate fresh ethics approval which would consume all the remaining time on the grant.

Law #5. The help provided by other people has a half-life of two weeks

Yes, yes, yes. Never do a study that depends on the kindness of strangers.

Law #6. The tedium of research is directly proportional to its objectivity

You really do need to know this when you start out in research. If you detest mundane, repetitive activities, try another career.

Law #7. The effort of writing up is an exponential function of the time since the data were collected

If the person who collected the data has left by the time you come to write it up, then it can be hard to remember exactly what was done, so you’d better be sure to have had a real obsessive in charge, who will document thoroughly every step of the research collection and data coding. Hodgson and Rollnick reckon that data that sit in a filing cabinet for 4 years will never escape.

On a more serious note, failure to get stuff written up is incredibly wasteful, especially if the funding for the study came from public funds. Sometimes the failure just comes from writer’s block, and sometimes because the researcher discovers a flaw that makes the study unpublishable. More commonly, though, the failure to write up is because the results are deemed uninteresting. This has the unfortunate effect of distorting the research literature, as null results are left in the file drawer. I'd like to see journal editors adopting a policy of determining ‘publishability’ of a paper on the basis of Introduction and Methods alone: if an interesting problem has been identified and the study is well-designed and adequately powered to answer it, then it should be published, regardless of the results.

Yet another reason for failure to publish is researchers who bite off more than they can chew. As I’ve suggested in a previous post, we need to move away from a system whereby the rewards for researchers are proportional to the amount of grant income they receive, to one that rewards thrift. And if research funders find themselves overwhelmed with far more proposals than they can fund, they should consider vetoing those who already have substantial funding, even if they are ace researchers. There is a limit to how much research someone can do and do well.

Law #8. Evidence is never enough

So you are lucky enough to get an interesting result, and are confident that this will change the field and make your reputation. And what happens? Nobody takes any notice. Hodgson and Rollnick note that research that conflicts with the prevailing view is likely to be ignored, but that’s not the only problem. You do also have to sell your science. But that does not need to mean cutting corners or distorting findings. But learn to write accessibly, get out there and give talks, start a blog (!) and, most important of all, focus on problems that are important.

Get hold of the original Hodgson and Rollnick chapter if you want positive tips on how to be a successful researcher. And for further advice, it’s hard to better Peter Medawar’s 1979 book Advice to a Young Scientist.

References
Elwyn, G. (2005). Ethics and research governance in a multicentre study: add 150 days to your study protocol BMJ, 330 (7495), 847-847 DOI: 10.1136/bmj.330.7495.847
Hodgson, R., & Rollnick, S. (1989). More fun, less stress: How to survive in research. In G. Parry & F.-N. Watts (Eds.), Behavioural and mental health research: A handbook of skills and methods (pp. 3-13). Hove, England: Lawrence Erlbaum Associates.