There's been a fair bit of discussion about Clarivate's decision to pause inclusion of eLife publications on the Science Citation Index (e.g. on Research Professional). What I find exasperating is that most of the discussion focuses on a single consequence - loss of eLife's impact factor. For authors, there are graver consequences.
I've reviewed for eLife but never published there; however, I have published a lot in Wellcome Open Research, which is another journal that aimed to disrupt the traditional publishing model, and has some similarities with eLife. Wellcome Open Research has never been included in Science Citation Index, despite the fact that it uses peer review. Wellcome Open Research has an unconventional model whereby submitted papers are immediately published, as a kind of pre-print prior to peer review, and then updated after peer review. It is true that some papers don't get sufficient approval to proceed to the peer-reviewed stage; the distinction between those that do and do not pass peer review is clearly flagged on the article. In addition to peer review, Wellcome Open Research maintains some quality control by limiting eligibility to researchers funded by Wellcome.
When Wellcome Open Research started up, all Wellcome-funded researchers were encouraged to publish there. As someone committed to Open Research, this seemed a great idea. There were no publication charges, and everything was open: access to the publication, data, and peer review. Peer reviewers even get DOIs for their reviews, some of which are worth citing in their own right. I was increasingly adopting open practices, and I think some of my best peer-reviewed work is published there.
I was shocked when I discovered that the journal wasn't included in Web of Science. I remember preparing a progress report for Wellcome and using Web of Science to check I hadn't omitted any publications. I was puzzled that I seemed to have published far less than I remembered. Then it became clear: everything in Wellcome Open Research was missing.
I was on the Advisory Board for Wellcome Open Research at the time, and raised this with them. They were shocked that I was upset. "We thought you of all people didn't care about impact factors", they said. This, of course, was true. But I did care a lot about my work being visible. I was also aware that any WOS-based H-index would exclude all the papers listed below: not a big deal for me, but potentially harmful to junior authors.
The reply I got was similar to the argument being made by eLife - well, the articles are indexed in Google Scholar and PubMed. That was really little consolation to me, given that I had relied heavily on Web of Science in my own literature searches, believing that it screened out dodgy journals. (This belief turns out to be false - there are many journals featured in WoS that are very low quality, which just rubs salt into the wound).
I have some criticisms of eLife's publishing model, but I would like them to succeed. We urgently need alternatives to the traditional journal model operated by the big commercial publishers. Their response to the open access movement has been to monetise it, with catastrophic consequences for science, as an unlimited supply of shoddy and fake work gets published - often in journals that are indexed in Web of Science.
I agree that we need an index of published academic work that has some quality control. Whether alternatives like OpenAlex will do the job remains to be seen.
Papers that aren't indexed on Web of Science
Bishop, D. V. M., & Bates, T. C. (2020). Heritability of language laterality assessed by functional transcranial Doppler ultrasound: A twin study. Wellcome Open Research, 4, 161. https://doi.org/10.12688/wellcomeopenres.15524.3
Bishop, D. V. M., Brookman-Byrne, A., Gratton, N., Gray, E., Holt, G., Morgan, L., Morris, S., Paine, E., Thornton, H., & Thompson, P. A. (2019). Language phenotypes in children with sex chromosome trisomies. Wellcome Open Research, 3, 143. https://doi.org/10.12688/wellcomeopenres.14904.2
Bishop, D. V. M., Grabitz, C. R., Harte, S. C., Watkins, K. E., Sasaki, M., Gutierrez-Sigut, E., MacSweeney, M., Woodhead, Z. V. J., & Payne, H. (2021). Cerebral lateralisation of first and second languages in bilinguals assessed using functional transcranial Doppler ultrasound. Wellcome Open Research, 1, 15. https://doi.org/10.12688/wellcomeopenres.9869.2
Frizelle, P., Thompson, P. A., Duta, M., & Bishop, D. V. M. (2019). The understanding of complex syntax in children with Down syndrome. Wellcome Open Research, 3, 140. https://doi.org/10.12688/wellcomeopenres.14861.2
Newbury, D. F., Simpson, N. H., Thompson, P. A., & Bishop, D. V. M. (2018). Stage 1 Registered Report: Variation in neurodevelopmental outcomes in children with sex chromosome trisomies: protocol for a test of the double hit hypothesis. Wellcome Open Research, 3, 10. https://doi.org/10.12688/wellcomeopenres.13828.2
Newbury, D. F., Simpson, N. H., Thompson, P. A., & Bishop, D. V. M. (2021). Stage 2 Registered Report: Variation in neurodevelopmental outcomes in children with sex chromosome trisomies: testing the double hit hypothesis. Wellcome Open Research, 3, 85.
https://doi.org/10.12688/wellcomeopenres.14677.4
Pritchard, V. E., Malone, S. A., Burgoyne, K., Heron-Delaney, M., Bishop, D. V. M., & Hulme, C. (2019). Stage 2 Registered Report: There is no appreciable relationship between strength of hand preference and language ability in 6- to 7-year-old children. Wellcome Open Research, 4, 81. https://doi.org/10.12688/wellcomeopenres.15254.1
Thompson, P. A., Bishop, D. V. M., Eising, E., Fisher, S. E., & Newbury, D. F. (2020). Generalized Structured Component Analysis in candidate gene association studies: Applications and limitations. Wellcome Open Research, 4, 142. https://doi.org/10.12688/wellcomeopenres.15396.2
Wilson, A. C., & Bishop, D. V. M. (2019). ‘If you catch my drift...’: Ability to infer implied meaning is distinct from vocabulary and grammar skills. Wellcome Open Research, 4, 68. https://doi.org/10.12688/wellcomeopenres.15210.3
Wilson, A. C., King, J., & Bishop, D. V. M. (2019). Autism and social anxiety in children with sex chromosome trisomies: An observational study. Wellcome Open Research, 4, 32. https://doi.org/10.12688/wellcomeopenres.15095.2
Woodhead, Z. V. J., Rutherford, H. A., & Bishop, D. V. M. (2020). Measurement of language laterality using functional transcranial Doppler ultrasound: A comparison of different tasks. Wellcome Open Research, 3, 104. https://doi.org/10.12688/wellcomeopenres.14720.3
I've never considered doing a literature search using WoS, as our engineering college does not subscribe. So I teach my students which databases are useful for computing, which ones have a lot of dodgy stuff in them, and how to look up the references in a relevant paper to find more interesting papers. I can only use the free access to WoS to see that they only have 10 of my publications indexed. Even dblp has 37 publications listed, Google Scholar has 87. I count 145 on my literature list (but I published on a variety of topics over the years).
ReplyDeleteSo WoS does not show one the breadth of research anyway.
Thanks for this blog and the comment. I never thought to check WOS for publications, I’m in biomedicine so it is never the case that PubMed has not been the right index. However, as RRIDs, my project, moves into chemistry and other fields we are starting to see gaps in citation gathering. Does anyone know a good database, not index, of papers that are in chemistry, earth sciences or computer science? Our team struggles with google scholar because it is so limited and not a database.
ReplyDeleteNot the main point you were making, but: "In addition to peer review, Wellcome Open Research maintains some quality control by limiting eligibility to researchers funded by Wellcome."
ReplyDeleteYou say that like it's a good thing. To me, it's a disaster. Same with the Gates Foundation platform. Siloing by funder is just rebuilding the walls we already have with siloing by publishers. And if this catches on and becomes the main way we publish, it will leave nowhere for unfunded publishers such as myself.
The truly radical thing for Wellcome (and Gates and others) to do is accept and publish legitimate research from ANYONE — their mission is to advance science, after all.
I get it that discovery and access points are crucial. Take a look at Curtis Brundy's pricing dataset resulting from public information requests and tell me that the $300,000 annually that the University of Oklahoma or Texas Tech University pay for metadata and biased gatekeeping wouldn't be better spent on, well, anything else (monographs, full-text databases, diamond OA)? https://doi.org/10.7191/jeslib.959
ReplyDeleteLibraries serve researchers. You have the power to say no to propping up the company that feeds and profits off of the systemic incentives that harm research and researchers.
The Journal of Open Source Software (JOSS) is not indexed too.
ReplyDelete