Saturday 26 February 2011

Academic publishing: why isn't psychology like physics?


Copyright: www.CartoonStock.com

My job is to conduct research and publish the results, so I’ve become interested in discussions on the internet about models of publishing. There are three tensions in the field: (1) who pays to publish research? (2) who decides what gets published? and (3) who takes any profits?
In the traditional model, based on publishing on paper, the answers have been (1) readers pay, via journal subscriptions, (2) editors decide, advised by reviewers, (3) publishers take the profits.
But publishing on paper is in decline. Most of us read papers on the internet taken from electronic versions of journals; either directly on computer, or printed from a pdf. This has led to some re-evaluation of how publishing is done. Most publishers, with an eye to their profits, have attempted to just continue with the old system, charging subscriptions for electronic journals much as they have done for paper journals. These are largely paid for by institutional libraries rather than individuals. If your institution doesn’t subscribe, or you don’t belong to an institution, there is an option to pay for individual articles, but the charges are typically prohibitively high.
Psychologist Stevan Harnad was among the first to point out that the internet changes the publishing game completely. Way back in 1994, he suggested that academics could leave publishers out of the research communication cycle altogether. For instance, when I write an article, I could post it on my website, and anyone who wanted to read it could download it. Instead of institutions paying thousands of pounds in journal subscriptions, and individual readers being clobbered for $30 for a five-page article, academic exchange would be virtually free. What would be wrong with that? This appealing idea has over the past sixteen years fostered a huge amount of debate. What is surprising to me, though, is that there appears to be a massive discrepancy between disciplines in how far Harnad's ideas have been taken up. And Harnad's own discipline, psychology, is one that has been very slow to respond, compared, say, to physics - see http://arxiv.org/. Because many of my readers are psychologists, and many psychologists are wedded to the traditional printed journal, I'll briefly rehearse some of the arguments that have been made, before commenting on the possibilities opened up by e-publishing.
There are two obvious snags to bypassing journals. The first just concerns presentation. Traditionally, publishers have ensured that journal articles look nice: properly formatted, intelligible figures, grammatical writing, and so on. Their role in this aspect of publishing has, however, dwindled over the years. This does not mean that any academic can produce a nicely-formatted article: some skill is required. And many academics don’t write very well and benefit from the services of professional editors. But increasingly, publishers are cutting down on professional editors and graphic designers - indeed, increasingly they hold authors responsible for getting their figures formatted for direct web publication (see my post).
The second point has to do with quality control. Traditionally, journals have acted as gatekeepers. If an article appears in a peer-reviewed journal, it can usually be assumed that it doesn’t have serious flaws: the higher the quality of the journal, the safer this assumption should be. And for the top-notch journals with the highest impacts, publication of paper generally gives some guarantee that it is not only methodologically sound but also particularly important in terms of advancing the field.
Nevertheless, some have queried whether we need this traditional function of publishers, pointing out flaws in the peer review system and idiosyncratic behaviour by editors. Indeed, gatekeeping may not just be imperfect: In some fields it may damage science, by inducing a publication bias in favour of ‘significant’ (both statistically and theoretically) findings, and biasing against publication of replications (or non-replications).It's even been argued that publishing in high-impact journals may be unethical, because such journals are particularly likely to publish papers that over-state their conclusions. And unless there is a massive media stink, they resist accepting papers that fail to replicate exciting findings that they've previously published.
Such considerations have led some to argue that we should ditch not just publishers, but the whole editorial and reviewing process as well. In effect, everyone would just publish their own stuff, and it would be judged by the academic community. Although this solution appeals to the more anarchic side of my nature, I can see objections to it. The main is that we’d all be swamped by a tidal wave of information. The good stuff would be buried within a heap of rubbish. And because methodological flaws are often hard to detect unless you have done research in the field yourself, problematic but apparently important studies might have a large impact.
As Harnad noted, however, taking publishers out of the loop need not entail abandoning peer review. With the exception of a few top-notch journals, most academic publishers don’t employ editors. Instead, editors do the work for free, or for a small honorarium. It is very unusual for reviewers to be paid anything. So this gate-keeping role of publishers doesn’t need to reside with publishers. Any group of academics could decide to set up a journal, appoint editors and operate a reviewing process. Provided the journal was electronic, with no paper copies, this should be achievable at relatively modest cost.
But has it worked? Well, sadly, not for psychology. I've brooded about this issue for some years, but was stimulated to revisit it by my recent discovery of Kindle direct publishing. It’s different from an Open Access model, because the reader pays for content, but the sums involved are trivially small compared to current journal charges. The Kindle operation is designed for publishing books. Harnad has always distinguished commercial book publishing from what he terms ‘esoteric’ publishing, i.e. publication of academic articles of interest only to a small group of experts. But my Kindle experience made me wonder whether the distinction between the two types of publishing need be as sharp.
I came across Kindle e-publishing when I’d had my first novel turned down by two agents, both of whom spoke of a ‘saturated’ market for crime novels, the genre I’d written in. I’m not a typical author: though I’d be very pleased if people enjoyed reading my book as much as I enjoyed writing it, I don’t have any high expectations of a successful second career as a novelist. I write just for fun, and, being an academic, I’m used to not getting paid for what I write. And being an academic also makes me used to rejection, while at the same time recognising that I shouldn’t let it discourage me if I have a belief my work is good.
After my experience with open access journals, I’d expected that any business that offered a self-publishing option would charge authors, especially for ‘esoteric’ material. As a first novel by an unknown author and with neither agent nor publisher, my book was definitely in that category. So I was pleasantly surprised to find that not only did I not have to pay anything, but I could set my own price and get a 70 per cent royalty. For a book with low-volume sales, this is going to be negligible, especially as a great chunk of tax is taken off any US sales. (This can be avoided if you fill in a lot of forms, send away your passport to the US, and so on, but since I was not expecting more than about two US sales, this clearly was not a sensible option). Anyhow, you are walked through the process with easy instructions, which include downloading a couple of bits of software to convert your manuscript into the right format, and then you press a button and upload your book. The whole process took about the same amount of time than it typically takes to submit a journal article through an electronic portal. And  you don't need a Kindle, though the reading experience is better than on a PC or iPhone. I keep wondering if there’s a catch; no doubt someone will tell me if there is. In common with some others, I am a bit dubious about accuracy of the sales figures provided (there's a rather surreal exchange about this on the Kindle forum). But when I go to Kindle store, there it is: the Case of the Fremantle Fingers by Deevy Bishop at $2.99. This gives me a small glow of satisfaction.
So I started to think, couldn’t research papers adopt this model, but with even lower charges? I’m not saying the articles need be in Kindle format: I don’t think it’s ideal for scientific papers. But rather that the method of author download to a central site, from which others download at a minimal cost, would be an interesting one for psychology (and other discipline) journals to adopt. The difference from current practice is that the material would be affordable; instead of costing 20 to 30 dollars, individual articles would be priced at 20 to 30 cents. The Kindle store does sell some books, often out-of-print classics, for ridiculously low prices, (or even free), and the charging mechanism doesn’t seem to be a problem. Authors of academic articles don’t expect royalties, so these could be waived.
But what about the dangers of the free-for-all and lack of peer review to identify quality? I suspect that even a notional charge per download might be a help here, by identifying material that people would be willing to pay for. And the Amazon system also suggests a more explicit way in which refereeing be adapted to such a system, by using a star rating with optional comment. This is especially useful in the academic context if the reviewers themselves also have ratings. For instance, a 5 star reviewer would be one who’d published significant papers in the same area, a 3-star reviewer would be someone familiar with this or a related area but without significant publications in that area, and those with no expertise would be 1-star reviewers. It shouldn’t then be difficult for readers to see which papers had high ratings from knowledgeable people.
But would people make ratings? There has been disappointing take-up of Comments options for journals that offer this, such as the PLOS series. I don’t like anonymity, but  to encourage frank comments on papers,  there would have to be some way of registering reviewers that made it possible to check their credentials and assign them an anonymous ID. Otherwise, junior people might be scared to make adverse comments on senior figures.
The downside? For me, omitting the usual pre-publication reviewing stage would have disadvantages. As I’ve noted before, I find peer review painful but often helpful in improving papers. Sometimes it has saved me from revealing an embarrassing amount of ignorance. I’d be happier with a system that allowed an option for pre-publication review, and it would be interesting then to see whether papers that were identified as having undergone such review would attract better post-publication ratings than other papers. Learned societies, for instance, which currently produce paper journals, might instead use their existing reviewing systems for papers which are then deposited on the electronic store, with a quality kitemark.
Another drawback is that for some people, particularly those in low-income countries, even a nominal charge per download would be a deterrent. But it should be feasible to devise a system that subsidised those from resource-poor countries. Or  the charge could be waived once a certain number of downloads had been achieved, so that the most popular material becomes automatically freely available.
Since Harnad’s ground-breaking formulation of the Subversive Proposal much has been written about models of science publishing. Many changes have occurred, with increasing pressure for open access, and growing criticism of the peer review system. But there remains a massive difference between disciplines: physics embraced self-archiving rapidly, whereas scientists in other disciplines often don't even realise it's an option. Maybe this has to do with the pace of change and degree of competition in a field: physicists don't want to wait months before their work is published because it could be scooped. Psychology experiments are seldom so time-sensitive. Nevertheless, I think the pace of research in our field could be improved enormously if we broke free from the stranglehold of the traditional commercial publishers. 

btw, Tom Webb has drawn attention to a Commons Select Committee that is seeking evidence on the peer review process. This strikes many people, including me, as a bit strange, in that it would seem to fall outside the remit of government. If you want to comment, you need to do so by Thurs 10th March, 2011.


16 comments:

  1. This is a great post. I'm sad to say that in my own area, linguistics, there is relatively little in Open Access. Also, as a co-founder of a small Open Access journal I can attest that the costs are really negligible - even to the extent that charging for content seems excessive - there is plenty of volunteer labour on the one hand and the returns of a little-read journal would not contribute enough to the running of it to offset the downside of putting up a block to access.

    I think there needs to be an equivalent to Arxiv.org in the humanities. It's the universities (or even the government) who have to say no to the publishers. They should say, you're using the time of our scholars to produce journals with increasingly diminishing added value and then you resell this to us at exorbitant prices which means that many people who should will not be able to access our research. They could then form a consortium that would provide a platform that would make the workflows easy to manage, ask their scholars to contribute there and stop paying for new journals. The combined savings to libraries would have to be massive. Having all the peer reviewers a part of the system, it would be much easier to manage the process you describe.

    And then the next question is: do there really even need to be journals? There are now so many it's hard to distinguish between them. Why not think about different methods of curation similar to those on social networks like Digg or Wikipedia. Maybe we should do a study and ask scholars what proportion of their knowledge comes from the journals they read regularly and what from recommendations in social networks (like citations, conference recommendations, research blogs, articles they get for peer review, work their students uncover, etc.) And then we could replicate some of the social discovery in a central platform.

    Instead of journals, each academic would have a collection of papers they think are essential and people with similar groups could get together to create a recommendation feed. Then there would be no need for waiting for issues, etc. This could encourage interdisciplinary and a broader perspective. Maybe even the knowledge creation process could be more open and social (I've outlined some ideas on http://researchity.net).

    There are plenty of funding and organisational models in the web community: Kickstarter, Sourceforge, Digg, Archive.org, etc.

    ReplyDelete
  2. Nice article, thanks for the information.

    ReplyDelete
  3. Most of this was very new to me and made me look at Amazon Kindle for the first time.

    One problem for Learned Societies and open access publishing is that their income often largely depends on what they get from publishers. Will they turn themselves into publishers and cut out the middlemen?

    ReplyDelete
  4. Seems that your blog belongs next to this other story that appeared today. http://read.bi/f0RIjo

    If we were to publish our work on Kindle, how would we inform other scientists? I receive eTOCs every day, for some 30 or so journals, which alert me not just to work in my field, but also lots of other interesting papers I'd otherwise miss. I don't quite see a mechanism for this broad distribution of information if we each go it alone, whether by Kindle or something equivalent.

    ReplyDelete
  5. Thanks to all for comments.
    Stephen - Hashtags my dear, #hashtags!

    ReplyDelete
  6. People have been asking me whether you can read a Kindle book without a Kindle. Answer is yes.
    there's an iPhone app.
    And there's free software for Kindle reading on PC
    http://tinyurl.com/35y5yed

    ReplyDelete
  7. And one more thing I forgot to say to all those of you interested in publishing your novel.
    I got awesomely good advice from:
    http://www.nailyournovel.com/
    Author tweets as @dirtywhitecandy

    ReplyDelete
  8. Updates to the Subversive Proposal

    Open Access Self-Archiving ("Green OA") is not about posting unrefereed preprints, like physicists, but about posting refereed final drafts (postprints), immediately upon acceptance for publication.

    Open Access is about freeing peer-reviewed research from access-tolls, not about freeing it from peer review. Postpublication feedback and tagging is not peer review, and would need a lot of testing to see whether it can yield quality comparable with peer review. (It would also have to be tested to see whether it was sustainable and scalable.)

    Books are different from journal articles because -- without exception -- journal articles are written exclusively for research uptake, usage and impact, not for royalties. Not so for all or even most books. For this reason, Green OA self-archiving of refereed journal articles is being mandated by universities and research funders, but book OA is not.
    (see ROARMAP)


    "The 1994 Subversive Proposal at 15: A Critique"
    "The invisible hand of peer review" (Nature)
    "Learned Inquiry and the Net: The Role of Peer Review, Peer Commentary and Copyright."
    "Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals"
    "PostGutenberg Peer Review. the invariant essentials"
    "Peer Review Reform Hypothesis-Testing"
    "What About Open Access to Books? "

    [sorry for not providing links but the blog's spam filter rejects my commentary otherwise -- all citations can be googled]

    ReplyDelete
  9. Traditional publishing is on its way out. Or so I keep reading on dozens and dozens of blogs and news articles. We need to act accordingly.

    ReplyDelete
  10. Perhaps, and somewhat ironically, this post is missing the obvious alternative: blogs.
    Let's just imagine, in the present situation, I establish a blog for a particular niche or sub-discipline in psychology, or whatever field. Regardless of the rest of the content, readers or subscribers are encouraged or allowed to submit pre-publication copies or drafts of papers they intend to submit for publication elsewhere to the blog for commentary or 'review' by other subscribers or members, who would comment, anonymously or otherwise. The idea would be to help improve the paper before the author submits it elsewhere, after a certain time period of feedback. Once the feedback is over, the author sends it on to the journal, and sends an updated copy to the blog.

    I know this would possibly disrupt the traditional mechanisms, but that would be.the whole point. If blogs like this were established, and they would be much easier to set up than most of the suggestions above, authors would probably stop submitting to journals.

    ReplyDelete
  11. As usual and interesting and thought provoking post. Still there is no obvious solution for learned societies and the income streams that they derived from publishing. It's not clear how they might survive.

    For an interesting take on threats to 'traditional publishing' Michael Nielsen post is a must.

    http://michaelnielsen.org/blog/is-scientific-publishing-about-to-be-disrupted/

    ReplyDelete
  12. RockstarPhilosopher27 July 2011 at 13:13

    Hi :) I followed you here from TheConversation

    My degrees was in Philosophy and I specialised in Philosophy of Mind. As a result I had to read a lot of neurosci and psych stuff; for this I would often be depending on the quality of the journal to indicate the quality of the article. This was especially so for things that involve statistical analysis, obviously something a degree in Philosophy doesn't equip you for :P I had to depend that there had been smart brains looking at the papers before me to ensure that the methodology and the interpretation of the results were kosher, so the lack of peer reviewing would concern me. The idea of posting it blog style has merit, but I've seen arguments over methodology surrounding this kind of publishing and I don't know who to trust. The idea of a group of interested academics doing it on their own seems worthy, but I would have concerns that this would result in little clusters of navel gazing and siloed research. Perhaps it is something that universities themselves could be facilitating, allowing for another level of oversight.

    ReplyDelete
  13. Psychology is undoubtedly the study of the mind and the study of behaviours. It is also very specific because of environmental factors so it is hard to make it a universal science. Also there are many sub sciences maybe psychology could be a smaller part of a science, or bigger than a science. Thanks a lot.

    ReplyDelete
  14. I've also experimented with selling academic-related books for Kindle: http://neurodojo.blogspot.com/2012/06/presentation-tips-for-kindle.html

    How we might validate such efforts: http://neurodojo.blogspot.com/2012/10/upload-universe-validating-self.html

    ReplyDelete
  15. because such journals are particularly likely to publish papers that over-state their conclusions. And unless there is a massive media stink, they resist accepting papers that fail to replicate exciting findings that they've previously published. technology in the classroom

    ReplyDelete
  16. Most budgetary guides will encourage understudies to search with the expectation of complimentary cash first. Bodes well to apply for grants, spare cash or get from family or companions before applying for credits with premium. Payday Loans

    ReplyDelete