Research funders need to embrace slow science

Uta Frith courted controversy earlier this year when she published an opinion piece in which she advocated for Slow Science, including the radical suggestion that researchers should be limited in the number of papers they publish each year. This idea has been mooted before, but has never taken root: the famous Chaos in the Brickyard paper by Bernard Forscher dates back to 1963, and David Colquhoun has suggested restricting the number of publications by scientists as a solution more than once on his blog (here and here).

Over the past couple of weeks I've been thinking further about this, because I've been doing some bibliometric searches. This was in part prompted by the need to correct and clarify an analysis I had written up in 2010, about the amount of research on different neurodevelopmental disorders. I had previously noted the remarkable amount of research on autism and ADHD compared to other behaviourally-defined conditions. A check of recent databases shows no slowing in the rate of research. A search for publications with autism or autistic in the title yielded 2251 papers published in 2010; in 2019, this has risen to 6562. We can roughly halve this number if we restrict attention to the Web of Science Core database and search only for articles (not reviews, editorials, conference proceedings etc). This gives 1135 articles published in 2010 and 3075 published in 2019. That's around 8 papers every day for 2019.

We're spending huge amounts to generate these outputs. I looked at NIH Reporter, which provides a simple interface where you can enter search terms to identify grants funded by the National Institutes of Health. For the fiscal year 2018-2019 there were 1709 projects with the keyword 'autism or autistic', with a total spend of $890 million. And of course, NIH is not the only source of research funding.

Within the field of developmental neuropsychology, autism is the most extreme example of research expansion, but if we look at adult disorders, this level of research activity is by no means unique. My searches found that this year there were 6 papers published every day on schizophrenia, 15 per day on depression, and 11 per day on Alzheimer's disease.

These are serious and common conditions and it is right that we fund research into them – if we could improve understanding and reduce their negative impacts, it would make a dramatic difference to many lives. The problem is information overload. Nobody, however nerdy and committed, could possibly keep abreast of the literature. And we're not just getting more information, the information is also more complex. I reckon I could probably understand the majority of papers on autism that were published when I started out in research years ago. That proportion has gone down and down with time, as methods get ever more complex. So we're spending increasing amounts of money to produce more and more research that is less and less comprehensible. Something has to give, and I like the proposal that we should all slow down.

But is it possible? If you want to get your hands on research funding, you need to demonstrate that you're likely to make good use of it. Publication track record provides objective evidence that you can do credible research, so researchers are focused on publishing papers. And they typically have a short time-frame in which to demonstrate productivity.

A key message from Uta's piece is that we need to stop confusing quantity with quality. When this topic has been discussed on social media, I've noted that many ECRs take the view that when you come to apply for grants or jobs, a large number of publications is seen as a good thing, and therefore Slow Science would damage the prospects of ECRs. That is not my experience. It's possible that there are differences in practice between different countries and subject areas, but in the UK the emphasis is much more on quality than quantity of publications, so a strategy of focusing on quality rather than quantity would be advantageous. Indeed, most of our major funders use proposal forms that ask applicants to list their N top publications, rather than a complete CV. This will disadvantage anyone who has sliced their oeuvre into lots of little papers, rather than writing a few substantial pieces. Similarly, in the UK Research Excellence Framework, researchers from an institution are required to submit their outputs, but there is a limited number that can be submitted – a restriction that was introduced many years ago to incentivise a focus on quality rather than quantity.

The same people who are outraged at reducing the number of publications often rail against the stress of working in the current system – and rightly so. After all, at some point in the research cycle, at least one person has to devote serious thought to the design, analysis and write-up. Each of these stages inevitably takes far longer than we anticipate – and then there is time needed to respond to reviewers. The quality and impact of research can be enhanced by pre-registration and making scripts and data open, but extra time needs to be budgeted for this. Indeed, lack of time is a common reason cited for not doing open science. Researchers who feel that to succeed they have to write numerous papers every year are bound to cut corners, and then burn out from stress. It makes far more sense to work slowly and carefully to produce a realistic number of strong papers that have been carefully planned, implemented and written up.

It's clear that science and scientists would benefit if we take things more slowly, but the major barrier is a lack of researcher confidence. Those who allocate funds to research have a vested interest in ensuring they get the best return from their investment – not a tsunami of papers that overwhelm us, but a smaller number of reports of high-quality, replicable and credible findings. If things are to change we need funders to do more to communicate to researchers that they will be evaluated on quality rather than quantity of outputs.

More haste less speed in calls for grant proposals

Helpful advice from the World Bank

This blogpost was prompted by a funding call announced this week by the Economic and Social Research Council (ESRC)  , which included the following key dates:

• Opening date for proposals – 6 August 2018 
• Closing date for proposals – 18 September 2018 
• PI response invited – 23 October 2018 
• PI response due – 29 October 2018 
• Panel – 3 December 2018 
• Grants start – 14 February 2019 

As pointed out by Adam Golberg (@cash4questions), Research Development Manager at Nottingham University, on Twitter, this is very short notice to prepare an application for substantial funding:
I make this about 30 working days notice. For a call issued in August. For projects of 36 months, up to £900k - substantial, for social sciences. With only one bid allowed to be led from each institution, so likely requiring an internal sift. 

I thought it worth raising this with ESRC, and they replied promptly, saying:
To access funds for this call we’ve had to adhere to a very tight spending timeframe. We’ve had to balance the call opening time with a robust peer review process and a Feb 2019 project start. We know this is a challenge, but it was a now or never funding opportunity for us.
 
They suggested I email them for more information, and I’ve done that, so will update this post if I hear more. I’m particularly curious about what is the reason for the tight spending timeframe and the inflexible February 2019 start.

This exchange led to discussion on Twitter which I have gathered together here.

It’s clear that from the responses that this kind of time-frame is not unusual, and I have been sent some other examples. For instance this ESRC Leadership Fellowship (£100,000 for 12 months) had a call for proposals issued on 16th November 2017, with a deadline for submissions of 3 January. When you factor in that most universities shut down from late December until early January, and so this would need to be with administrators before the Christmas break, this gives applicants around 30 days to construct a competitive proposal. But it’s not only ESRC that does this, and I am less interested in pointing the finger at a particular funder – who may well be working under pressures outside their control - than just raising the issue of why this needs a rethink. I see five problems with these short lead times:

1. Poorer quality of proposals 
The most obvious problem is that a hastily written proposal is likely to be weaker than one that is given more detailed consideration. The only good thing you might say about the time pressure is that it is likely to reduce the number of proposals, which reduces the load on the funder’s administration. It’s not clear, however, whether this is an intended consequence.

2. Stress on academic staff 
There is ample evidence that academic staff in the UK have high stress levels, often linked to a sense of increasing demands and high workload. A good academic shows high attention to detail and is at pains to get things right: research is not something that can be done well under tight time pressure. So holding up the offer of a large grant with only a short time period to prepare a proposal is bound to increase stress: do you drop everything else to focus on grant-writing, or pass by the opportunity to enter the competition?

Where the interval between the funding call and the deadline occurs over a holiday period, some might find this beneficial, as other demands such as teaching are lower. But many people plan to take a vacation, and should be able to have a complete escape from work for at least a week or two. Others will have scheduled the time for preparing lectures, doing research, or writing papers. Having to defer those activities in order to meet a tight deadline just induces more sense of overload and guilt at having a growing backlog of work.

3. Equity issues 
These points about vacations are particularly pertinent for those with children at home during the holidays, as pointed out in a series of tweets by Melissa Terras, Professor of Digital Cultural Heritage at Edinburgh University, who said:
I complained once to the AHRC about a call announced in November with a closing date of early January - giving people the chance to work over the Xmas shutdown on it. I wasn't applying to the call myself, but pointed out that it meant people with - say - school age kids - wouldn't have a "clear" Xmas shutdown to work on it, so it was prejudice against that cohort. They listened, apologised, and extended the deadline for a month, which I was thankful for. But we shouldn't have to explain this to them. Have RCUK done their implicit bias training?

4. Stress on administrative staff 
One person who contacted me via email pointed out that many funders, including ESRC, ask institutions to filter out uncompetitive proposals through internal review. That could mean senior research administrators organising exploratory workshops, soliciting input from potential PIs, having people present their ideas, and considering collaborations with other institutions. None of that will be possible in a 30-day time frame. And for the administrators who do the routine work of checking grants for accuracy of funding bids and compliance with university and funder requirements, I suspect it’s not unusual to be dealing with a stressed researcher who expects them to do all of this with rapid turnaround, but where the funding scheme virtually guarantees everything is done in a rush, this just gets worse.

5. Perception of unfairness 
Adding in to this toxic mix, we have the possibility of diminished trust in the funding process. My own interest in this issues stems from a time a few years ago when there was a funding call for a rather specific project in my area. The call came just before Christmas, with a deadline in mid January. I had a postdoc who was interested in applying, but after discussing it, we decided not to put in a bid. Part of the reason was that we had both planned a bit of time off over Christmas, but in addition I was suspicious about the combination of short time-scale and specific topic. This made me wonder whether a decision had already been made about who to award the funds to, and the exercise was just to fulfil requirements and give an illusion of fairness and transparency.

Responses on Twitter again indicate that others have had similar concerns. For instance, Jon May, Professor in Psychology at the University of Plymouth, wrote:
I suspect these short deadline calls follow ‘sandboxes’ where a favoured person has invited their (i.e his) friends to pitch ideas for the call. Favoured person cannot bid but friends can and have written the call.
 
And an anonymous correspondent on email noted:
I think unfairness (or the perception of unfairness) is really dangerous – a lot of people I talk to either suspect a stitch-up in terms of who gets the money, or an uneven playing field in terms of who knew this was coming.

So what’s the solution? One option would be to insist that, at least for those dispensing public money, there should be a minimum time between a call for proposals and the submission date: about 3 months would seem reasonable to me.

Comments will be open on this post for a limited time (2 months, since we are in holiday season!) so please add your thoughts.

P.S. Just as I was about to upload this blogpost, I was alerted on Twitter to this call from the World Bank, which is a beautiful illustration of point 5 - if you weren't already well aware this was coming, there would be no hope of applying. Apparently, this is not a 'grant' but a 'contract', but the same problems noted above would apply. The website is dated 2nd August, the closing date is 15th August. There is reference to a webinar for applicants dated 9th July, so presumably some information has been previously circulated, but still with a remarkably short time lag, given that there need to be at least two collaborating institutions (including middle- and low-income countries)

, with letters of support from all collaborators and all end users. Oh, and you are advised ‘Please do not wait until the last minute to submit your proposal’.


Update: 17th August 2018
An ESRC spokesperson sent this reply to my query:

Thank you for getting in touch with us with your concerns about the short call opening time for the recently announced Management Practices and Employee Engagement call, and the fact that it has opened in August.

We welcome feedback from our community on the administration of funding programmes, and we will think carefully about how to respond to these concerns as we design and plan future programmes.

To provide some background to this call. It builds on an open-invite scoping workshop we held in February 2018, at which we sought input from the academic, policy and third-sector communities on the shape of a (then) potential research investment on management practices and employee engagement. We subsequently flagged the likelihood of a funding call around the topic area this summer, both at the scoping workshop itself, as well as in our ongoing engagements with the academic community.

We do our best to make sure that calls are open for as long as possible. We have to balance call opening times with a robust and appropriately timetabled peer review process, feasible project start dates, the right safeguards and compliances, and, in certain cases such as this one, a requirement to spend funds within the financial year. 

We take the concerns that you raise in your email and in your blog post of 11 August 2018 extremely seriously. The high standard of the UK's research is a result of the work of our academic community, and we are committed to delivering a system that respects and responds to their needs. As part of this, we are actively looking into ways to build in longer call lead times and/or pre-announcements of funding opportunities for potential future managed calls in this and other areas.

I would also like to stress that applicants can still submit proposals on the topic of management practices and employee engagement through our standard research grant process, which is open all year round. The peer review system and the Grant Assessment Panel does not take into account the fact that a managed call is open on a topic when awarding funding: decisions are taken based on the excellence of the proposal.

Update: 23rd August 2018
A spokesperson for the World Bank has written to note that the grant scheme alluded to in my postscript did in fact have a 2 month period between the call and submission date. I have apologised to them for suggesting it was shorter than this, and also apologise to readers for providing misleading information. The duration still seems short to me for a call of this nature, but my case is clearly not helped by providing wrong information, and I should have taken greater care to check details. Text of the response from the World Bank is below:

 

We noticed with some concern that in your Aug. 11 blog post, you had singled out a World Bank call for proposals as a “beautiful illustration” of a type of funding call that appears designed to favor an inside candidate. This characterization is entirely inaccurate and appears based on a misperception of the time lag between the announcement of the proposal and the deadline.

Your reference to the 2018 Call for Proposals for Collaborative Data Innovations for Sustainable Development by the World Bank and the Global Partnership for Sustainable Development Data as undermining faith in the funding process seems based on the mistaken assumption that the call was issued on or about August 2. It was not.

The call was announced June 19 on the websites of the World Bank and the GPSDD. This was two months before the closing date, a period we have deemed fair to applicants but also appropriate given our own time constraints. An online seminar was offered to assist prospective applicants, as you note, on July 9.

The seminar drew 127 attendees for whom we provided answers to 147 questions. We are still reviewing submissions for the most recent call for proposals for this project, but our call for the 2017 version elicited 228 proposals, of which 195 met criteria for external review.

As the response to the seminar and the record of submissions indicate, this funding call has been widely seen and provided numerous applicants the opportunity to respond.  To suggest that this has not been an open and fair process does not do it justice.

Here are the links with the announcement dates of June 19^th

World Bank Blog 

Official press release by the partnership on June 19^th

Tweet on June 19^th by DFID research which was retweeted by World Bank data.

Standing on the shoulders of giants, or slithering around on jellyfish: Why reviews need to be systematic

Yesterday I had the pleasure of hearing George Davey Smith (aka @mendel_random) talk. In the course of a wide-ranging lecture he recounted his experiences with conducting a systematic review. This caught my interest, as I’d recently considered the question of literature reviews when writing about fallibility in science. George’s talk confirmed my concerns that cherry-picking of evidence can be a massive problem for many fields of science.

Together with Mark Petticrew, George had reviewed the evidence on the impact of stress and social hierarchies on coronary artery disease in non-human primates. They found 14 studies on the topic, and revealed a striking mismatch between how the literature was cited and what it actually showed. Studies in this area are of interest to those attempting to explain the well-known socioeconomic gradient in health. It’s hard to unpack this in humans, because there are so many correlated characteristics that could potentially explain the association. The primate work has been cited to support psychosocial accounts of the link; i.e., the idea that socioeconomic influences on health operate primarily through psychological and social mechanisms. Demonstration of such an impact in primates is  particularly convincing, because stress and social status can be experimentally manipulated in a way that is not feasible in humans.

The conclusion from the review was stark: ‘Overall, non-human primate studies present only limited evidence for an association between social status and coronary artery disease. Despite this, there is selective citation of individual non-human primate studies in reviews and commentaries relating to human disease aetiology’(p. e27937).

The relatively bland account in the written paper belies the stress that George and his colleague went through in doing this work. Before I tried doing one myself, I thought that a systematic review was a fairly easy and humdrum exercise. It could be if the literature were not so unruly. In practice, however, you not only have to find and synthesise the relevant evidence, but also to read and re-read papers to work out what exactly was done. Often, it’s not just a case of computing an effect size: finding the numbers that match the reported result can be challenging. One paper in the review that was particularly highly-cited in the epidemiology literature turned out to have data that were problematic: the raw data shown in scattergraphs are hard to reconcile with the adjusted means reported in a summary (see Figure below). Correspondence sent to the author apparently did not achieve a reply, let alone an explanation.

Figure 2 from Shively and Thompson (1994) Arteriosclerosis and Thrombosis Vol 14, No 5. Yellow bar added to show mean plaque areas as reported in Figure 3 (adjusted for preexperimental thigh circumference and TPC-HDL cholesterol ratio)

Even if there were no concerns about the discrepant means, the small sample size and influential outliers in this study should temper any conclusions. But those using this evidence to draw conclusions about human health focused on the ‘five-fold increase’ in coronary disease in dominant animals who became subordinate.

So what impact has the systematic review achieved? Well, the first point to note is that the authors had a great deal of difficulty getting it accepted for publication: it would be sent to reviewers who worked on stress in monkeys, and they would recommend rejection. This went on for some years: the abstract was first published in 2003, but the full paper did not appear until 2012.

The second, disappointing conclusion comes from looking at citations of the original studies reviewed by Petticrew and Davey Smith in the human health literature since their review appeared. The systematic review garnered 4 citations in the period 2013-2015 and just one during 2016-2018. The mean citations for the 14 articles covered in their meta-analysis was 2.36 for 2013-2015, and 3.00 for 2016-2018. The article that was the source of the Figure above had six citations in the human health literature in 2013-2015 and four in 2016-2018. These numbers aren’t sufficient for more than impressionistic interpretation, and I only did a superficial trawl through abstracts of citing papers, so I am not in a position to determine if all of these articles accepted the study authors’ conclusions. However, the pattern of citations fits with past experience in other fields showing that when cherry-picked facts fit a nice story, they will continue to be cited, without regard to subsequent corrections,  criticism or even retraction.

The reason why this worries me is that the stark conclusion would appear to be that we can’t trust citations of the research literature unless they are based on well-conducted systematic reviews. Iain Chalmers has been saying this for years, and in his field of clinical trials these are more common than in other disciplines. But there are still many fields where it is seen as entirely appropriate to write an introduction to a paper that only cites supportive evidence and ignores a swathe of literature that shows null or opposite results. Most postgraduates have an initial thesis chapter that reviews the literature, but it's rare, at least in psychology, to see a systematic review - perhaps because this is so time-consuming and can be soul-destroying. But if we continue to cherry-pick evidence that suits us, then we are not so much standing on the shoulders of giants as slithering around on jellyfish, and science will not progress.

How to become a celebrity scientific expert

Maybe you’re tired of grotting away at the lab bench. Or finding it hard to get a tenured job. Perhaps your last paper was rejected and you haven’t the spirit to fight back. Do not despair. There is an alternative. The media are always on the look-out for a scientist who will fearlessly speak out and generate newsworthy stories. You can gain kudos as an expert, even if if you haven't got much of a track record in the subject, by following a few simple rules.

Rule #1. Establish your credentials. You need to have lots of letters after your name. It doesn’t really matter what they mean, so long as they sound impressive. It’s also good to be a fellow of some kind of Royal Society. Some of these are rather snooty and appoint fellows by an exclusive election process, but it’s a little known fact that others require little more than a minimal indication of academic standing and will admit you to the fellowship provided you fill in a form and agree to pay an annual subscription. So sign up as a Fellow of the Royal Society of Medicine, and keep good company with a range of naturopaths, homeopaths and chiropracters who have discovered this easy route to eminence. The really nice thing is that even academics can be hoodwinked by this one.

Rule #2. Find a controversial topic. This is key. You have to be willing to take a definite position on something that people have strong views about. A scare story is good - we’ve all been doing X for years but it could damage us. Finding someone to blame is also good - people who do Y are feckless. And the buzz word of the decade is neuroscience, so if you can work that in, success is guaranteed. If you're short of ideas, the list of the right might help inspire you. A recent article in the Biologist hits the spot with “The biological effects of day care”, managing to get us worried about an everyday activity, blame working mothers, and get in a neuro message all at once. It's even spiced up with a bit of conspiracy theory: experts know that day care is bad for children’s brains but nobody is allowed to speak out because it is too politically sensitive. This presses so many buttons that few journalists could resist the story.

Rule #3. Specify a causal chain. As we shall see when it comes to assembling evidence, it is particularly useful to have a causal chain with several steps. For instance:

The point here is that if you can usually find at least some studies that provide evidence for bits of the causal chain. Although it may be inconvenient if, as in this case, studies looking for a link between A and D fail to come up with clear evidence (Lucas-Thompson et al, 2010), you can rely on two things: first, few readers will be familiar with the research literature, so they will only know as much as you tell them. And second, step C, brain abnormality, is highly salient and once you start talking about that, it will distract attention from the other levels of description.

Rule #4. Avoid rigorous peer review. You don’t want to have your views critiqued by someone who knows the literature, or checks your sources. Writing books is a safe bet for avoiding pre-publication scientific critique. As far as journals go, the Biologist is ideal. This publication for the members of the Society of Biology claims to be peer-reviewed, but, as we shall see, the review process is far from rigorous.

Rule #5. Assemble supportive evidence. Note, it is important not to present all relevant studies, just those with findings that can be fitted into the causal chain.

The author of the paper in question, Dr Aric Sigman, presents us with so much positive evidence that he manages to give the impression that the whole casual chain has been validated. He starts by mentioning studies that investigated the link between A and B and find that salivary cortisol is increased in the afternoons in children attending day care. This is a product of the hypothalamo-pituitary-adrenal (HPA) axis, which is elevated in response to stress. This result is well-established, both from studies comparing groups of children who do and don’t attend day care, and from comparing the same children on days when they stay at home or go to day care. This is a potentially concerning finding, if it can be shown that there are consequences for children’s learning and behaviour. As Sigman notes “Of central concern is that the routine stress experienced at day care could cause permanent changes in the child’s neuroendocrine networks, with long-term consequences for their mental and physical health as adults.” (p. 30). The article that he cites does discuss this issue, but notes the complexity of causal relationships and cautions against assuming that the cortisol elevation is harmful. In fact the authors draw attention to an animal study suggesting a very different conclusion:

“As in the work on cortisol responses to fullday child care, these separations in squirrel monkey infants produced marked and repeated activations of the HPA axis. However, followed into the late juvenile and early adult age, animals exposed to this form of early life stress were found to be less fearful, to produce lower rather than higher cortisol responses to stressors, and to show more optimal development of prefrontal regulatory brain circuits; consistent with these findings, they also performed better on tests of executive functioning. Thus, at least for this animal model, repeated separation stress early in life fostered a form of resilience.” Gunnar et al (2010) (my emphasis)

Sigman avoids mentioning any of this and turns instead to look at the links between cortisol levels and ill health, starting with studies that show a link between cortisol and cardiovascular disease. He does not explain that these were done on people aged over 65 years, but rather implants in people’s minds the notion that this is relevant for his arguments about daycare in toddlers.  Next comes the serious stuff: research linking elevated cortisol to the brain. We are told that “Cortisol is considered neurotoxic and has a global impact on cerebral size (e.g. McEwen 2007; Sheline 2003).” (p. 29). Again, we are left with the distinct impression that children who attend day care will have small brains, but to find out what actually is meant here we need to read the cited articles. When we do, we find that McEwen (2007) is a thorough research review of the physiology and neurobiology of stress and adaptation that nowhere mentions the terms ‘neurotoxic’, ‘global’ or ‘cerebral size’. Rather, in this article McEwen develops a complex theory that considers both positive and negative impact of stress. It actually has a section subtitled “Protection and damage: the two sides of the response to stressors” which discusses animal studies demonstrating how elevated cortisol can either improve or interfere with brain function, depending on context. Neurotoxicity does feature in the other article, by Sheline (2003), but this is concerned with mood disorders in adults, and discusses effects of hypercortisolemia, a condition where there is chronic elevation of cortisol, rather than a temporary increase at specific times of day. Subsequent citations are to papers that considered the role of cortisol in psychiatric disorders such as anxiety and depression: note that now the evidence is focused on a link between cortisol and adult psychiatric disorders in the opposite direction (from D to B), yet it is presented in the context of discussing consequences of high cortisol in children who attended day care. Sigman further states: “a higher cortisol awakening curve may be a biological marker for an underlying disposition towards developing depressive and anxiety disorders” (p. 30), even though the studies of toddlers attending day care show a cortisol response that develops through the day, rather than a chronically raised level: see Vermeer and van IJzendoorn (2006) for a well-balanced discussion of such evidence.

It would be tedious to wade through every cited article, but it’s worth considering just one more example. We are told “In human grey matter, the quality of a mother’s care in early childhood is thought to alter the size of the hippocampus (Buss et al, 2007)” (p. 30). Erm no. Buss et al clearly stated there were no differences in left or right hippocampal volume between those categorised as having high or low maternal care. What they did find was a complicated interaction between birth weight, gender and maternal care, such that birth weight predicted hippocampal volume only in female subjects reporting low maternal care. But even this limited result doesn’t support Sigman’s case. We are talking about ‘low maternal care’, not ‘attendance at daycare’. And guess what? When we look at ‘low maternal care’ we find it measured from a self-report questionnaire, where low care is partly identified in terms of maternal overprotection. Those categorised this way were more likely to have endorsed items describing their mother in such terms as:

• Tended to baby me
• Tried to make me feel dependent on her
• Felt I could not look after myself unless she was around
• Was overprotective of me

I’m not an expert in the neurobiology of stress. I can track down articles that appear to have been cited by Sigman (the reference list is behind a paywall) and see where he's given a misleading account, but what I don’t know is how much relevant literature has been omitted. This, of course, is what the celebrity scientist can rely on: there's only a handful of people who both have the expertise in the area, and are obsessive enough to trawl through your writing (if they can access it) and challenge any misleading statements.

Rule #6. Anticipate criticism but don't let it worry you. People who actually do research in the area you are reviewing may get irritated, but most scientists in the field wouldn’t bother on the grounds that they don't know who you are, and aren’t interested in pursuing academic debates outside the domain of mainstream journals. The worst you may get is a few nerdy bloggers such as Gimpy or Mind Hacks criticising you for lack of scholarship, sensationalism and cherrypicking of evidence. Or, if you're really unlucky, you might be up against Ben Goldacre on Newsnight. But meanwhile, your purpose as celebrity scientist has been achieved: your views are all over the media.

References

Buss, C., Lord, C., Wadiwalla, M., Hellhammer, D. H., Lupien, S. J., Meaney, M. J., et al. (2007). Maternal care modulates the relationship between prenatal risk and hippocampal volume in women but not in men. Journal of Neuroscience, 27(10), 2592-2595. dx.doi.org/10.1523/JNEUROSCI.3252-06.2007

Gunnar MR, Kryzer E, Van Ryzin MJ, & Phillips DA (2010). The rise in cortisol in family day care: associations with aspects of care quality, child behavior, and child sex. Child Development, 81 (3), 851-69. PMID: 20573109

Lucas-Thompson, R. G., Goldberg, W. A., & Prause, J. A. (2010). Maternal work early in the lives of children and its distal associations with achievement and behavior problems: A meta-analysis. Psychological Bulletin, 136(6), 915-942. DOI: 10.1037/a0020875
McEwen, B. S. (2007). Physiology and neurobiology of stress and adaptation: Central role of the brain. Physiological Reviews, 87(3), 873-904.doi: 10.​1152/​physrev.​00041.​2006 (Open Access)

Sigman, A. (2011). Mother superior? The biological effects of day care. The Biologist, 5 (3), 29-32.


Vermeer, H. J. & van IJzendoorn, M. H. (2006). Children's elevated cortisol levels at daycare: A review and meta-analysis. Early Childhood Research Quarterly, 21(3), 390-401.doi 10.1016/j.ecresq.2006.07.004