tag:blogger.com,1999:blog-58419107680790155342024-03-19T08:47:24.741+00:00BishopBlogRamblings on academic-related matters.
For information on my research see
https://www.psy.ox.ac.uk/research/oxford-study-of-children-s-communication-impairments.
Twin analysis blog: http://dbtemp.blogspot.com/ .
ERP time-frequency analysis blog: bishoptechbits.blogspot.com/ .
For tweets, follow @deevybee.deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.comBlogger281125tag:blogger.com,1999:blog-5841910768079015534.post-69804362460089666302024-02-09T10:16:00.002+00:002024-02-09T10:26:56.934+00:00The world of Poor Things at MDPI journals<p><br /></p>
<div class="separator" style="clear: both;"><a href="https://www.dailyinfo.co.uk/media/cache/resolve/legacy_sitewide_eight-column/image/dc99d2c22941ea049a0200c85664fd19" style="display: block; padding: 1em 0px; text-align: center;"><img alt="" border="0" data-original-height="429" data-original-width="711" src="https://www.dailyinfo.co.uk/media/cache/resolve/legacy_sitewide_eight-column/image/dc99d2c22941ea049a0200c85664fd19" width="320" /></a></div>
At the weekend, the Observer ran a piece by Robin McKie entitled "‘<a href="https://www.theguardian.com/science/2024/feb/03/the-situation-has-become-appalling-fake-scientific-papers-push-research-credibility-to-crisis-point">The situation has become appalling’: fake scientific papers push research credibility to crisis point</a>". I was one of those interviewed for the article, describing my concerns about a flood of dodgy papers that was polluting the scientific literature.
<p>
Two days later I received an email from the editorial office of MDPI publishers with the header "[Children] (IF: 2.4, ISSN 2227-9067): Good Paper Sharing on the Topic of" (sic) that began:
</p><p></p><blockquote><p>
<i>Greetings from the Children Editorial Office!
</i></p><p>
<i>We recently collected 10 highly cited papers in our journal related to Childhood Autism. And we sincerely invite you to visit and read these papers, because you are an excellent expert in this field of study.</i></p></blockquote>
<p></p><p>
Who could resist such a flattering invitation? MDPI is one of those publishers that appears to be encouraging publication of low quality work, with a<a href="https://arxiv.org/abs/2309.15884" target="_blank"> massive growth in special issues</a> where papers are published with remarkably rapid turnaround times. Only last week it was revealed that the journal is affected by <a href="https://www.chemistryworld.com/news/review-mills-identified-as-a-new-form-of-peer-review-fraud/4018888.article">fake peer review</a> that appears to be generated by AI. So I was curious to take a look.
</p><p>
The first article, by <a href="https://www.mdpi.com/2227-9067/9/2/181" target="_blank">Frolli et al (2022a) </a>was weird. It reported a comparison of two types of intervention designed to improve emotion recognition in children with autism, one of which used virtual reality. The first red flag was the sample size: two groups each of 30 children, all originally from the city of Caserta. I checked Wikipedia, which told me the population of Caserta was around 76,000 in 2017. <a href="http://deevybee.blogspot.com/2011/07/how-to-survive-in-psychological.html" target="_blank">Recruiting participants for intervention studies is typically slow and laborious</a> and this is a remarkable sample size to recruit from such a small region. But credibility is then stretched to breaking point on hearing that the selection criteria required that the children were all aged between 9 and 10 years and had IQs of 97 or above. No researcher in their right mind would impose unnecessary constraints on recruitment, and both the age and IQ criteria are far tighter than would usually be adopted. I wondered whether there might be a typo in this account, but we then hear that the IQ range of the sample is indeed remarkably narrow: </p><p></p><blockquote>"<i>The first experimental group (Gr1) was composed of 30 individuals with a mean age of 9.3 (SD 0.63) and a mean IQ of 103.00 (SD 1.70). ...... The second experimental group (Gr2) was composed of 30 individuals with a mean age of 9.4 (SD 0.49) and mean IQ of 103.13 (SD 2.04)....</i>"
</blockquote><p></p><p>Most samples for studies using Wechsler IQ scales have SD of at least 8, even if cutoffs are applied as selection criteria, so this is unbelievably low.</p><p>
This dubious paper prompted me to look at others by the first author. It was rather like pulling a thread on a hole in a sweater - things started to unravel fast. A paper published by <a href="https://doi.org/10.3390/bs13120994 " target="_blank">Frolli et al (2023a)</a> in the MDPI journal Behavioral Sciences claimed to have studied eighty 18-year-olds recruited from four different high schools. The selection criteria were again unbelievably stringent: IQ assessed on the WAIS-IV fell between 95-105 "<i>to ensure that participants fell within the average range of intellectual functioning, minimizing the impact of extreme cognitive variations on our analyses</i>". The lower IQ range selected here corresponds to z-score of -0.33 or 37th percentile. If the population of students covered the full range of IQ, then only around 25% would meet the criterion (between 37th and 63rd centile), so to obtain a sample of 80 it would be necessary to test over 300 potential participants. Furthermore, there are IQ screening tests that can be used in this circumstance that are relatively quick to administer, but the WAIS-IV is not one of them. We are told all participants were given the full test, which requires individual administration by a qualified psychologist and takes around one hour to complete. So who did all this testing, and where? The article states: "<i>The data were collected and analyzed at the FINDS Neuropsychiatry Outpatient Clinic by licensed psychologists in collaboration with the University of International Studies of Rome (UNINT).</i>" So we are supposed to believe that hundreds of 18-year-olds trekked to a neuropsychiatry outpatient clinic for a full IQ screening which most of them would not have passed. I cannot imagine a less efficient way of conducting such a study. I could not find any mention of compensation for participants, which is perhaps unsurprising as the research received no external funding. All of this is described as happening remarkably fast, with ethics approval in January 2023, and submission of the article in October 2023.
</p><p>
Another paper in Children in 2023 focused on ADHD, and again reported recruiting two groups of 30 children for an intervention that lasted 5 months (<a href="https://doi.org/10.3390/children10081350" target="_blank">Frolli et al., 2023b</a>). The narrow IQ selection criteria were again used, with WISC-IV IQs in the range 95-105, and the mean IQs were 96.48 (SD =1.09) and 98.44 (SD = 1.12) for groups 1 and 2 respectively. Again, the research received no external funding. The report of ethics approval is scanty "<i>The study was conducted in accordance with the Declaration of Helsinki. The study was approved by the Ethics Committee and the Academic Senate of the University of International Studies of Rome</i>."
</p><p>
The same first author published a paper on the impact of COVID-19 on cognitive development and executive functioning in adolescents in 2021 (<a href="https://doi.org/10.3390/brainsci11091222 " target="_blank">Frolli et al, 2021</a>). I have not gone over it in detail, but a quick scan revealed some very odd statistical reporting. There were numerous F-ratios, but they were all negative, which is impossible, as F is a ratio between two positive numbers. Furthermore, the reported p-values and degrees of freedom didn't always correspond to the F-ratio, even if the sign was ignored. </p><p>
At this point I was running out of steam, but a quick look at <a href="https://doi.org/10.3390/pediatric14040053 " target="_blank">Frolli et al (2022a)</a> on Executive Functions and Foreign Language Learning suggested yet more problems, with the sentence "<i>Significance at the level of 5%
<span style="font-size: small;">
<span face=""Calibri",sans-serif" style="mso-ansi-language: EN-GB; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">(α < 0.001)</span></span><span style="font-size: x-small;"><span face=""Calibri",sans-serif" style="mso-ansi-language: EN-GB; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></span>
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style>has been accepted</i>" featuring at least twice. It is hard to believe that a human being wrote this sentence, or that any human author, editor or reviewer read it without comment.
</p><p>
If anyone is interested in pulling at other related threads, I suspect it would be of interest to look at articles accepted for a <a href=" https://www.mdpi.com/journal/disabilities/special_issues/Technology_Disabilities" target="_blank">Special Issue of the MDPI journal Disabilities</a> co-edited by Frolli.
</p><p>
In his brilliant film Poor Things, Yorgos Lanthimos distorts familiar objects and places just enough to be disturbing. Lisbon looks like what I imagine Lisbon would be in the Victorian age, except that the colours are unusually vivid, there are strange flying cars in the sky, and nobody seems concerned at the central character wandering around only partially clothed (see, e.g., <a href="https://www.latimes.com/entertainment-arts/awards/story/2024-02-05/poor-things-production-design-holds-nothing-back" target="_blank">this review</a>). The combined impression is that MDPI publishes papers from that universe, where everything looks superficially like genuine science but with jarring features that tell you something is amiss. The difference is that Poor Things has a happy ending.
</p><p><b>References <br /></b></p><p>Frolli, A.; Ricci, M.C.; Di Carmine, F.; Lombardi, A.; Bosco, A.; Saviano, E.; Franzese, L. The Impact of COVID-19 on Cognitive Development and Executive Functioning in Adolescents: A First Exploratory Investigation. Brain Sci. 2021, 11, 1222. https://doi.org/10.3390/brainsci11091222
</p><p>
Frolli, A.; Savarese, G.; Di Carmine, F.; Bosco, A.; Saviano, E.; Rega, A.; Carotenuto, M.; Ricci, M.C. Children on the Autism Spectrum and the Use of Virtual Reality for Supporting Social Skills. Children 2022a, 9, 181. https://doi.org/10.3390/children9020181
</p><p>
Frolli, A.; Cerciello, F.; Esposito, C.; Ciotola, S.; De Candia, G.; Ricci, M.C.; Russo, M.G. Executive Functions and Foreign Language Learning. Pediatr. Rep. 2022b, 14, 450-456. https://doi.org/10.3390/pediatric14040053
</p><p>
Frolli, A.; Cerciello, F.; Ciotola, S.; Ricci, M.C.; Esposito, C.; Sica, L.S. Narrative Approach and Mentalization. Behav. Sci. 2023a, 13, 994. https://doi.org/10.3390/bs13120994
</p><p>
Frolli, A.; Cerciello, F.; Esposito, C.; Ricci, M.C.; Laccone, R.P.; Bisogni, F. Universal Design for Learning for Children with ADHD. Children 2023b, 10, 1350. https://doi.org/10.3390/children10081350
</p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com3tag:blogger.com,1999:blog-5841910768079015534.post-17997335665423661722024-02-02T08:23:00.002+00:002024-02-03T08:07:27.401+00:00An (intellectually?) enriching opportunity for affiliation<p><span style="font-size: large;"><span style="color: #351c75;">Guest Post by Nick Wise</span> </span></p><p><span style="font-size: large;"></span></p><div class="separator" style="clear: both; text-align: center;"><span style="font-size: large;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAWQqbO29xezBNVXkSWd6Gz-XYBwJTOpO5zgZ6qJgt7dAVu3m9ih7VsRYo2fvUhY5irRQCBrqmftjZlfBQvOPRdT0qJImy8aUZdSIEF1k6uj4YNNiYVf3Or51iutpSoz-6jQTGxY1owlDuxlcHC8lCq9XQ2bt1Ynjtzxe0TyrwXTjYGc4ABu5XPUSswTxr/s1690/CartoonStock_599579_CS207350.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1690" data-original-width="1500" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAWQqbO29xezBNVXkSWd6Gz-XYBwJTOpO5zgZ6qJgt7dAVu3m9ih7VsRYo2fvUhY5irRQCBrqmftjZlfBQvOPRdT0qJImy8aUZdSIEF1k6uj4YNNiYVf3Or51iutpSoz-6jQTGxY1owlDuxlcHC8lCq9XQ2bt1Ynjtzxe0TyrwXTjYGc4ABu5XPUSswTxr/s320/CartoonStock_599579_CS207350.jpeg" width="284" /></a></span></div><span style="font-size: large;"><br /> </span>
<p></p><p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">A couple of months ago a professor received the following email, which
they forwarded to me.</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;"> </span></p>
<p class="MsoNormal"><i style="mso-bidi-font-style: normal;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">"Dear
esteemed colleagues,<br />
<br />
We are delighted to extend an invitation
to apply for our prestigious remote research fellowships at the University of
Religions and Denominations (URD). These fellowships offer substantial
financial support to researchers with papers currently in press, accepted or
under review by Scopus-indexed journals. We welcome scholars from diverse
academic disciplines to seize this intellectually enriching opportunity.<br />
<br />
<b style="mso-bidi-font-weight: normal;">Fellowship
Details:<br />
</b>Fellowship Type: Remote Short-term
Research Fellowship.<br />
Research Focus: Diverse fields, spanning
humanities, social sciences, interdisciplinary studies, and more.<br />
Research Output: Publication of research
articles in Scopus-indexed journals.<br />
Affiliation: Encouragement for
researchers to acknowledge URD as their additional affiliation in published
articles.<br />
Remuneration: Project-based compensation
for each research article.<br />
Payment Range: Up to $1000 USD per
article (based on SJR journal ranking).<br />
Eligibility: Papers in press, accepted,
or under review by Scopus-indexed journals.</span></i></p>
<p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><i style="mso-bidi-font-style: normal;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">Preference:
Priority for indexing before December 30, 2023.<br />
<br />
<b style="mso-bidi-font-weight: normal;">Application
Process: </b><span style="mso-spacerun: yes;"></span> </span></i></p><p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><i style="mso-bidi-font-style: normal;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">To express your interest in securing
a fellowship, kindly submit your curriculum vitae to<span style="mso-spacerun: yes;"> </span>Ahmad Moghri at moghri.urd@gmail.com. When
emailing your application, please use the subject line: "Research
Fellowship, FULL NAME."<br />
<br />
<b style="mso-bidi-font-weight: normal;">Upon
Selection:<br />
</b>Successful applicants will receive
formal invitations to join our esteemed fellowship program. Invitation letters
and collaboration contracts will be dispatched within a maximum of 5 days.<br />
<br />
We firmly believe that this fellowship
program provides an invaluable platform for scholars to make substantial
contributions to their fields while collaborating with the distinguished
University of Religions and Denominations. We encourage all eligible
individuals to seize this exceptional opportunity.<br />
<br />
For inquiries or further information,
please do not hesitate to contact moghri.urd@gmail.com.<br />
<br />
Warmest Regards,”</span></i></p>
<p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">Why would the institution pay researchers to
say that they are affiliated with them? It could be that funding for the
university is related to the number of papers published in indexed journals.
More articles associated with the university can also improve their placing in
national or international university rankings, which could lead directly to
more funding, or to more students wanting to attend and bringing in more money.</span></p>
<p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">The University of Religions and Denominations
is a private Iranian university specialising, as the name suggests, in the
study of different religions and movements. Until recently the institution had
very few published papers associated with it </span><a href="https://app.dimensions.ai/discover/publication?and_facet_research_org=grid.449872.4"><span face=""Calibri",sans-serif" style="color: #1155cc; mso-fareast-font-family: Calibri;">according to Dimensions</span></a><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;"> and their subject matter was all related to
religion. However, last year there was a substantial increase to 103 published
papers, and so far this year there are already 35. This suggests that some
academics have taken them up on the offer in the advert to include URD as an
affiliation.</span></p>
<p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">Surbhi Bhatia Khan is a </span><a href="https://dsai-hub.salford.ac.uk/meet-the-team/"><span face=""Calibri",sans-serif" style="color: #1155cc; mso-fareast-font-family: Calibri;">lecturer in
data science at the University of Salford</span></a><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;"> in the UK since March
2023 and a top 2% scientist in the world according to Stanford University’s
rankings. She published 29 research articles last year according to Dimensions,
an impressive output, in which she was primarily affiliated to the University
of Salford. In addition though, 5 of those submitted in the 2nd half of last
year had an additional affiliation at the Department of Engineering and
Environment at URD, which is not listed as one of the departments on </span><a href="https://urd.ac.ir/en/faculties/"><span face=""Calibri",sans-serif" style="color: #1155cc; mso-fareast-font-family: Calibri;">the university website</span></a><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">.
Additionally, 19 of the 29 state that she’s affiliated to the Lebanese American
University in Beirut, which she was not affiliated with before 2023. She is yet
to mention her role at either of these additional affiliations on her LinkedIn
profile.</span></p>
<p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">Looking at the Lebanese American University,
another private university, its publication numbers have shot up from 201 in
2015 to 503 in 2021 and 2,842 in 2023, </span><a href="https://app.dimensions.ai/discover/publication?and_facet_research_org=grid.411323.6"><span face=""Calibri",sans-serif" style="color: #1155cc; mso-fareast-font-family: Calibri;">according to Dimensions</span></a><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">. So far in 2024 they have published 525, on
track for over 6,000 publications for the year. By contrast, according to the
university website, the faculty consisted of </span><a href="https://web.archive.org/web/20220120120757/https:/www.lau.edu.lb/about/facts.php"><span face=""Calibri",sans-serif" style="color: #1155cc; mso-fareast-font-family: Calibri;">547 full-time staff members in 2021</span></a><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;"> but
had shrunk to </span><a href="https://www.lau.edu.lb/about/facts.php"><span face=""Calibri",sans-serif" style="color: #1155cc; mso-fareast-font-family: Calibri;">423 in 2023</span></a><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">.<span style="mso-spacerun: yes;"> </span>It is
hard to imagine how such growth in publication numbers could occur without a
similar growth in the faculty, let alone with a reduction.</span></p>
<p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">How many other institutions are seeing
incredible increases in publication numbers? Last year we saw gaming of the
system on a grand scale </span><a href="https://www.nature.com/articles/d41586-023-01523-x"><span face=""Calibri",sans-serif" style="color: #1155cc; mso-fareast-font-family: Calibri;">by various Saudi Arabian universities</span></a><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">, but
how many offers like the one above are going around, whether by email or sent
through Whatsapp groups or similar?</span></p>
<p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">The Committee On Publication Ethics held </span><a href="https://publicationethics.org/resources/forum-discussion-topics/claiming-university-affiliations"><span face=""Calibri",sans-serif" style="color: #1155cc; mso-fareast-font-family: Calibri;">a forum on claiming institutional affiliations</span></a><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;"> in
December 2023, in recognition of the fact that guidance for what merits
affiliation to an institution is lacking and there are no accepted standards
for how many affiliations an author should give. It looks like such guidance
can’t come soon enough.</span></p><p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><span face=""Calibri",sans-serif" style="font-size: small; mso-fareast-font-family: Calibri;"><span><b>Nick Wise is a researcher at the University of Cambridge, UK.</b></span><span><b><br /></b></span></span></p><p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;"><span style="font-size: x-small;"><b>Note:</b> Comments are moderated to prevent spam and abuse, so please be patient if you post a comment and it does not appear immediately</span></span></p><p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;"><b>P.S. 3rd Feb 2024</b></span></p><p class="MsoNormal" style="margin-bottom: 12.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 12.0pt; margin: 12pt 0cm;"><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;">Someone on social media queried the "top 2% rating" for Khan. Nick tells me this is based on an Elsevier ranking for 2022: </span><span face=""Calibri",sans-serif" style="mso-fareast-font-family: Calibri;"><a href="https://elsevier.digitalcommonsdata.com/datasets/btchxktzyw/6" target="_blank">https://elsevier.digitalcommonsdata.com/datasets/btchxktzyw/6</a></span></p>
<p><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Arial",sans-serif;
mso-fareast-font-family:Arial;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-size:11.0pt;
mso-ansi-font-size:11.0pt;
mso-bidi-font-size:11.0pt;
mso-ascii-font-family:Arial;
mso-fareast-font-family:Arial;
mso-hansi-font-family:Arial;
mso-bidi-font-family:Arial;
mso-font-kerning:0pt;
mso-ligatures:none;}.MsoPapDefault
{mso-style-type:export-only;
line-height:115%;}div.WordSection1
{page:WordSection1;}</style></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com3tag:blogger.com,1999:blog-5841910768079015534.post-72671469509317848072023-12-05T11:17:00.001+00:002023-12-05T11:17:10.319+00:00Low-level lasers. Part 2. Erchonia and the universal panacea <p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgE4khWX-yqP49xGulR36ArPMW2IuzeInK_mBL1xcwQ3w8S90C4gOMktsdFhF6w0uFv4tZwQlo1E2-NcW7ncj1b0UEi8mdBkUDYn-r9N-XqZs5toc6NfVgQtOSQRXHVn1yX0WBWRVNrI4c9BG2Ixh14DOSCRbKYoYOt2TO0jCwLFtVDiNrPjq78vyWTY3kB/s435/davinci%20man.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="394" data-original-width="435" height="363" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgE4khWX-yqP49xGulR36ArPMW2IuzeInK_mBL1xcwQ3w8S90C4gOMktsdFhF6w0uFv4tZwQlo1E2-NcW7ncj1b0UEi8mdBkUDYn-r9N-XqZs5toc6NfVgQtOSQRXHVn1yX0WBWRVNrI4c9BG2Ixh14DOSCRbKYoYOt2TO0jCwLFtVDiNrPjq78vyWTY3kB/w400-h363/davinci%20man.png" width="400" /></a></div><br /><p></p><p>In my <a href="http://deevybee.blogspot.com/2023/11/low-level-lasers-part-1-shining-light.html" target="_blank">last blogpost</a>, I looked at a study that claimed continuing improvements of symptoms of autism after eight 5-minute sessions where a low-level laser was pointed at the head. The data were so extreme that I became interested in the company, Erchonia, who sponsored the study and in Regulatory Insight, Inc, whose statistician failed to notice anything odd. In exploring Erchonia's research corpus, I found that they have investigated the use of their low-laser products for a remarkable range of conditions. A search of <i>clinicaltrials.com</i> with the keyword Erchonia produced 47 records, describing studies of pain (chronic back pain, post-surgical pain, and foot pain), body contouring (circumference reduction, cellulite treatment), sensorineural hearing loss, Alzheimer's disease, hair loss, acne and toenail fungus. After excluding the trials on autism described in my previous post, fourteen of the records described randomised controlled trials in which an active laser was compared with a placebo device that looked the same, with both patient and researcher being kept in the dark about which device was which until the data were analysed. As with the autism study, the research designs for these RCTs specified on <i>clinicaltrials.com</i> looked strong, with statistician Elvira Cawthon from Regulatory Insight involved in data analysis. <br /></p><p>As shown in Figure 1, where results are reported for RCTs, they have been spectacular in virtually all cases. The raw data are mostly not available, and in general the plotted data look less extreme than in the autism trial covered in last week's post, but nonetheless, the pattern is a consistent one, where over half the active group meet the cutoff for improvement, whereas less than half (typically 25% or less) of the placebo group do so. </p>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCGMSrcQ6nTyz8tJQSE_2G8wbeQgQHAcx9nvG-LONUJxzYx7x4kZN1o7mu4Sj1rM8g9ing5QYRv1vO5xTv_T2VncAHvdH5D9pWsNe17M1R1acJXgfreQ5HjEa6r3tsZH1T_hjO5-hhB4tyJtyehqrAK_gD2Fn53GFajToI1tAtdtIv1YhoHvKbvqxM9wYn/s688/Screenshot%202023-12-05%20at%2010.41.40.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="455" data-original-width="688" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCGMSrcQ6nTyz8tJQSE_2G8wbeQgQHAcx9nvG-LONUJxzYx7x4kZN1o7mu4Sj1rM8g9ing5QYRv1vO5xTv_T2VncAHvdH5D9pWsNe17M1R1acJXgfreQ5HjEa6r3tsZH1T_hjO5-hhB4tyJtyehqrAK_gD2Fn53GFajToI1tAtdtIv1YhoHvKbvqxM9wYn/w400-h265/Screenshot%202023-12-05%20at%2010.41.40.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><i>FIGURE 1: Proportions in active treated group vs placebo group meeting preregistered criterion for improvement (Error bars show SE)*</i><br /></td></tr></tbody></table><p></p><p>I looked for results from mainstream science
against which to benchmark the Erchonia findings. I found a big review
of behavioural and pharmaceutical interventions for obesity by the US
Agency for Healthcare Research and Quality (<a href="https://www.ncbi.nlm.nih.gov/books/NBK532375/" target="_blank">LeBlanc et al, 2018</a>).
Figures 7 and 13 show results for binary outcomes - relative risk of
losing 5% or more of body weight over a 12 month period; i.e. the
proportion of treated individuals who met this criterion divided by the
proportion of controls. In 38 trials of behavioural interventions, the
mean RR was 1.94 [95% CI, 1.70 to 2.22]. For 31 pharmaeutical
interventions, the effect varied with the specific medication, with RR
ranging from 1.18 to 3.86. Only two pharmaceutical comparisons had RR in
excess of 3.0. By contrast, for five trials of body contouring or
cellulite reduction from Erchonia, the RRs ranged from 3.6 to 18.0.
Now, it is important to note that this is not comparing like with like:
the people in the Erchonia trials were typically not clinically obese:
they were mostly women seeking cosmetic improvements to their
appearance. So you could, and I am sure many would, argue it's an
unfair comparison. If anyone knows of another literature that might
provide a better benchmark, please let me know. The point is that the
effect sizes reported by Erchonia are enormous relative to the kinds of
effects typically seen with other treatments focused on weight
reduction.<br /></p><p>If we look more generally at the other results
obtained with low-level lasers, we can compare them to an overview of
effectiveness of common medications (<a href="https://doi.org/10.1186/s12916-015-0494-1" target="_blank">Leucht et al, 2015</a>).
These authors presented results from a huge review of different
therapies, with effect sizes represented as standardized mean
differences (SMD - familiar to psychologists as Cohen's d). I converted
Erchonia results into this metric*, and found that across all the
studies of pain relief shown in Figure 1, the average SMD was 1.30, with a range from 0.87 to
1.77. This contrasts with Leucht et al's estimated effect size of 1.06
for oxycodone plus paracetamol, and 0.83 for Sumatriptan for migraine.
So if we are to believe the results, they indicate that the effect of
Erchonia low-level lasers is as good or better than the most effective
pharmaceutical medications that we have for pain relief or weight loss.
I'm afraid I remain highly sceptical. <br /></p><p>I would not have
dreamed of looking at Erchonia's track
record if it were not for their impossibly good results in the Leisman
et al autism trial that I discussed in the previous blogpost. When I
looked in more detail, I was reminded of the kinds of claims made for
alternative treatments for children's learning difficulties, where
parents are drawn in with slick websites promising scientifically proven
interventions, and glowing testimonials from satisfied customers. Back
in <a href="http://deevybee.blogspot.com/2012/02/neuroscientific-interventions-for.html" target="_blank">2012 I blogged</a>
about how to evaluate "neuroscientific" interventions for dyslexia.
Most of the points I made there apply to the world of "photomodulation"
therapies, including the need to be wary when a provider claims that a
single method is effective for a whole host of different conditions. </p><p>Erchonia products are sold worldwide and seem popular with alternative
health practitioners. For instance, in Stockport, Manchester, you can
attend a <a href="https://www.chiropractichealthsolutions.co.uk/zerona-laser-treatment/" target="_blank">chiropractic clinic</a> where Zerona laser treatment will remove "stubborn body fat". In London there is <a href="https://www.londonpodiatry.com/treatments/treatments/cold-laser-injury-treatment/ " target="_blank">a podiatry centre</a> that reassures you: "<i>There
are numerous papers which show that cold laser affects the activity of
cells and chemicals within the cell. It has been shown that cold laser
can encourage the formation of stem cells which are key building blocks
in tissue reparation. It also affects chemicals such as cytochrome c and
causes a cascade of reactions which stimulates the healing. There is
much research to show that cold laser affects healing and there are now
several very good class 1 studies to show that laser can be effective</i>."
But when I looked for details of these "very good class 1 studies" they
were nowhere to be found. In particular, it was hard to find research by
scientists without vested interests in the technology. <br /></p><p>Of all the RCTs that I found, there were just two that were conducted at reputable universities. One of them, on hearing loss (<a href="https://classic.clinicaltrials.gov/ct2/show/NCT01820416" target="_blank">NCT01820416</a>)
was conducted at the University of Iowa, but terminated prematurely
because intermediate analysis showed no clinically or statistically
significant effects (<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3658388/" target="_blank">Goodman et al., 2013</a>). This contrasts sharply with <a href="https://clinicaltrials.gov/study/NCT00787189" target="_blank">NCT00787189</a>, which had the dramatic results reported in Figure 1 (not, as far as I know, published outside of <i>clinicaltrials.gov</i>).
The other university-based study was the autism study based in Boston
described in my previous post: again, with unpublished, unimpressive
results posted on <i>clinicaltrials.gov</i>.</p><p>This suggests it is
important when evaluating novel therapies to have results from studies
that are independent of those promoting the therapy. But, sadly, this is
easier to recommend than to achieve. Running a trial takes a lot of
time and effort: why would anyone do this if they thought it likely that
the intervention would not work and the postulated mechanism of action
was unproven? There would be a strong risk that you'd end up putting in
effort that would end in a null result, which would be hard to publish.
And you'd be unlikely to convince those who believed in the therapy -
they would no doubt say you had the wrong wavelength of light, or
insufficient duration of therapy, and so on. </p><p>I suspect the
response by those who believe in the power of low-level lasers will be
that I am demonstrating prejudice, in my reluctance to accept the
evidence that they provide of dramatic benefits. But, quite simply, if
low-level laser treatment was so remarkably effective
in melting fat and decreasing pain, surely it would have quickly been
publicised through word of mouth from satisfied customers. Many of us
are willing to subject our bodies to all kinds of punishments in a quest
to be thin and/or pain-free. If this could be done simply and
efficiently without the need for drugs, wouldn't this method have taken
over the world?</p>
*<span style="font-size: x-small;">Summary files (Erchonia_proportions4.csv) and script (Erchonia_proportions_for_blog.R) are on Github, <a href="https://github.com/oscci/miscellaneous" target="_blank">here</a>. </span>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com1tag:blogger.com,1999:blog-5841910768079015534.post-33842213543717338222023-11-25T11:21:00.001+00:002023-11-25T11:21:43.320+00:00Low-level lasers. Part 1. Shining a light on an unconventional treatment for autism<br /><p><span style="background-color: #fcff01;"></span> </p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHtDOmBqU6xfydtcNrcgbYlBi6m0hWZCSZ74wsfvx91jFGe-F-y2dA_kC8Sq90nQlMvNzi_OBUI4L5do9S9DN60RvFePs1gL_7HyEuIkaXLowgj5PJJBE4ntt1aJWv0CEFO-yXYwn6-llk7Vcghr66hSPBDzLm-54PmF2BXL-yIUAe6h1AeSD9p4THlNPS/s592/Screenshot%202023-11-25%20at%2010.58.37.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="431" data-original-width="592" height="233" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHtDOmBqU6xfydtcNrcgbYlBi6m0hWZCSZ74wsfvx91jFGe-F-y2dA_kC8Sq90nQlMvNzi_OBUI4L5do9S9DN60RvFePs1gL_7HyEuIkaXLowgj5PJJBE4ntt1aJWv0CEFO-yXYwn6-llk7Vcghr66hSPBDzLm-54PmF2BXL-yIUAe6h1AeSD9p4THlNPS/s320/Screenshot%202023-11-25%20at%2010.58.37.png" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><i>'Light enters, then a miracle happens, and good things come out!' (Quirk & Whelan, 2011*)<br /></i></td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td></tr></tbody></table><p></p><p>
I'm occasionally asked to investigate weird interventions for children's neurodevelopmental conditions, and recently I've found myself immersed in the world of low-level laser treatments. The material I've dug up is not new - it's been around for some years, but has not been on my radar until now. </p><p>A starting point is this <a href="https://www.erchonia.com/blog-news/erchonia-submits-data-to-us-fda-to-support-low-level-laser-510k-market-clearance-for-autism/" target="_blank">2018 press statement</a> by Erchonia, a firm that makes low-level laser devices for quasi-medical interventions.
</p><p>
They had tested a device that was supposed to reduce irritability in autistic children by applying low-level laser light to the temporal and posterior regions of the head (see Figure 1) for 5 minute sessions twice a week for 4 weeks.</p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBdz2BuUB62pzTqe9vrpVdJMX9leBYPe1T4JkXxOlqtdI0-FLmg226aODw3r8webBf8oS8ZRN81BhejvkA_3v_b6KeCuyVdBeZmoC-bE1Mqyi5YdhOF0c93fhepebxncW95e5EcD4DtqQeTHBPcSgCc1h3HAI_8XskqP8M7s_6N562ouaeL3AgQYG838vP/s253/Picture%201.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="125" data-original-width="253" height="125" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBdz2BuUB62pzTqe9vrpVdJMX9leBYPe1T4JkXxOlqtdI0-FLmg226aODw3r8webBf8oS8ZRN81BhejvkA_3v_b6KeCuyVdBeZmoC-bE1Mqyi5YdhOF0c93fhepebxncW95e5EcD4DtqQeTHBPcSgCc1h3HAI_8XskqP8M7s_6N562ouaeL3AgQYG838vP/s1600/Picture%201.png" width="253" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="font-size: x-small;">Figure 1: sites of stimulation by low-level laser</span></td><td class="tr-caption" style="text-align: center;"><span style="font-size: x-small;"><br /></span></td><td class="tr-caption" style="text-align: center;"><span style="font-size: x-small;"><br /></span></td></tr></tbody></table><p> The study, which was reported <a href="https://link.springer.com/chapter/10.1007/5584_2018_234," target="_blank">here</a>, was carefully designed as a randomized controlled trial. Half the children received a placebo intervention. Placebo and active laser devices were designed to look identical and both emitted light, and neither the child nor the person administering the treatment knew whether the active or placebo light was being used. </p><p></p><p>
According to Erchonia “<i>The results are so strong, nobody can argue them</i>.” (sic). Alas, their confidence turned out to be misplaced.
</p><p>
The rationale given by Leisman et al (with my annotations in yellow in square brackets) is as follows:
"LLLT promotes cell and neuronal repair (Dawood and Salman 2013) [<span style="background-color: #fcff01;">This article is about wound healing, not neurons</span>] and brain network rearrangement (Erlicher et al. 2002) [<span style="background-color: #fcff01;">This is a study of rat cells in a dish</span>] in many neurologic disorders identified with lesions in the hubs of default mode networks (Buckner et al. 2008)[<span style="background-color: #fcff01;">This paper does not mention lasers</span>]. LLLT facilitates a fast-track wound-healing (Dawood and Salman 2013) as mitochondria respond to light in the red and near-infrared spectrum (Quirk and Whelan 2011*)[<span style="background-color: #fcff01;">review of near-infrared irradiation photobiomodulation that notes inadequate knowledge of mechanisma - see cartoon</span>]. On the other hand, Erlicher et al. (2002) have demonstrated that weak light directs the leading edge of growth cones of a nerve [<span style="background-color: #fcff01;">cells in a dish</span>]. Therefore, when a light beam is positioned in front of a nerve’s leading edge, the neuron will move in the direction of the light and grow in length (Black et al. 2013 [<span style="background-color: #fcff01;">rat cells in a dish</span>]; Quirk and Whelan 2011). Nerve cells appear to thrive and grow in the presence of low-energy light, and we think that the effect seen here is associated with the rearrangement of connectivity."
</p><p>
I started out looking at the <a href=" https://clinicaltrials.gov/study/NCT03379662" target="_blank">registration of the trial on ClinicalTrials.gov</a>. This included a very thorough document that detailed a protocol and analysis plan, but there were some puzzling inconsistencies; I documented them<a href="https://pubpeer.com/publications/1F857AEAD85D6B96C9370D801937B3" target="_blank"> here on PubPeer</a>, and subsequently a much more detailed critique was posted there by Florian Naudet and André Gillibert.
Among other things, there was confusion about where the study was done. The registration document said it was done in Nazareth, Israel, which is where the first author, Gerry Leisman was based. But it also said that the PI was Calixto Machado, who is based in Havana, Cuba.
</p><p>
Elvira Cawthon, from Regulatory Insight, Inc, Tennessee was mentioned on the protocol as clinical consultant and study monitor. The role of the study monitor is specified as follows: </p><p><i>"The study Monitor will assure that the investigator is executing the protocol as outlined and intended. This includes insuring that a signed informed consent form has been attained from each subject’s caregiver prior to commencing the protocol, that the study procedure protocol is administered as specified, and that all study evaluations and measurements are taken using the specified methods and correctly and fully recorded on the appropriate clinical case report forms."
</i></p><p>
This does not seem ideal, given that the study monitor was in Tennessee, and the study was conducted in either Nazareth or Havana. Accordingly, I contacted Ms Cawthon, who replied: </p><p><i>"I can confirm that I performed statistical analysis on data from the clinical study you reference that was received from paper CRFs from Dr. Machado following completion of the trial. I was not directly involved in the recruitment, treatment, or outcomes assessment of the subjects whose data was recorded on those CRFs. I have not reviewed any of the articles you referenced below so I cannot attest to whether the data included was based on the analyses that I performed or not or comment on any of the discrepancies without further evaluation at this time."</i>
</p><p>
I had copied Drs Leisman and Machado into my query, and Dr Leisman also replied.
He stated:
</p><p>
"<i>I am the senior author of the paper pertaining to a trial of low-level laser therapy in autism spectrum disorder.... I take full responsibility for the publication indicated above and vouch for having personally supervised the implementation of the project whose results were published under the following citation:
</i></p><p><i>
Leisman, G. Machado, C., Machado, Y, Chinchilla-Acosta, M. Effects of Low-Level Laser Therapy in Autism Spectrum Disorder. Advances in Experimental Medicine and Biology 2018:1116:111-130. DOI:10.1007/5584_2018_234. The publication is referenced in PubMed as: PMID: 29956199.
</i></p><p><i>
I hold a dual appointment at the University of Haifa and at the University of the Medical Sciences of Havana with the latter being "Professor Invitado" by the Ministry of Health of the Republic of Cuba. Ms. Elvira Walls served as the statistical consultant on this project.</i>"
</p><p>
However, Dr Leisman denied any knowledge of subsequent publications of follow-up data by Dr Machado. I asked if I could see the data from the Leisman et al study, and he provided a link to a data file on ResearchGate, the details of which I have put on PubPeer.
</p><p>
Alas, the data were amazing, but not in a good way. The main data came from five subscales of the Aberrant Behavior Checklist (ABC)**, which can be combined into a Global score. (There were a handful of typos in the dataset for the Global score, which I have corrected in the following analysis). For the placebo group, 15 of 19 children obtained exactly the same global score on all 4 sessions. Note that there is no restriction of range for this scale: reported scores range from 9 to 154. This pattern was also seen in the five individual subscales. You might think that is to be expected if the placebo intervention is ineffective, but that's not the case. Questionnaire measures such as that used here are never totally stable. In part this is because children's behaviour fluctuates. But even if the behaviour is constant, you expect to see some variability in responses, depending on how the rater interprets the scale of measurement. Furthermore, when study participants are selected because they have extreme scores on a measure, the tendency is for scores to improve on later testing - a phenomenon known as <a href="http://deevybee.blogspot.com/2010/08/three-ways-to-improve-cognitive-test.html" target="_blank">regression to the mean</a>, Such unchanging scores are out of line with anything I have ever come across in the intervention literature. If we turn to the treated group, we see that 20 of 21 children showed a progressive decline in global scores (i.e. improvement), with each measurement improving from the previous one over 4 sessions. This again is just not credible because we'd expect some fluctuation in children's behaviour as well as variable ratings due to error of measurement. These results were judged to be abnormal in a further commentary by Gillibert and Naudet on PubPeer. They also noted that the statistical distribution of scores was highly improbable, with far more even than odd numbers.
</p><p>
Although Dr Machado has been copied into my correspondence, he has not responded to queries. Remember, he was PI for the study in Cuba, and he is first author on a follow-up study from which Dr Leisman dissociated himself. Indeed, I subsequently found that there were no fewer than three follow-up reports, all appearing in a strange journal whose DOIs did not appear to be genuine: </p><p><span style="font-size: x-small;">Machado, C., Machado, Y., Chinchilla, M., & Machado, Yazmina. (2019a). Follow-up assessment of autistic children 6 months after finishing low lever (<span style="background-color: #fcff01;">sic)</span> laser therapy. Internet Journal of Neurology, 21(1). https://doi.org/10.5580/IJN.54101 (available from https://ispub.com/IJN/21/1/54101).
</span></p><p><span style="font-size: x-small;">
Machado, C., Machado, Y., Chinchilla, M., & Machado, Yazmina. (2019b). Twelve months follow-up comparison between autistic children vs. Initial placebo (treated) groups. Internet Journal of Neurology, 21(2). https://doi.org/10.5580/IJN.54812 (available from https://ispub.com/IJN/21/2/54812).
</span></p><p><span style="font-size: x-small;">
Machado, C., Machado, Y., Chinchilla, M., & Machado, Yazmina. (2020). Follow-up assessment of autistic children 12 months after finishing low lever (<span style="background-color: #fcff01;">sic</span>) laser therapy. Internet Journal of Neurology, 21(2). https://doi.org/10.5580/IJN.54809 (available from available from https://ispub.com/IJN/21/2/54809)
</span></p><p>
The 2019a paper starts by talking of a study of anatomic and functional brain connectivity in 21 children, but then segues to an extended follow-up (6 months) of the 21 treated and 19 placebo children from the Leisman et al study. The Leisman et al study is mentioned but not adequately referenced. Remarkably, all the original participants participated in the follow-up. The same trend as before continued: the placebo group stagnated, whereas the treated group continue to improve up to 6 months later, even though they received no further active treatment after the initial 4 week period. The 2020 Abstract reported a further follow-up to 12 months. The huge group difference was sustained (see Figure 2). Three of the treated group were now reported as scoring in the normal range on a measure of clinical impairment. </p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-lBsrkn1SQjSfrgvTGGld5cmC_4XZSKLSlP2M78nA1MRbJy-LN6ZI5cTDAx6wyujrB8QAtLjUg5bFZwyH-uj3IWTRdNzKWaRgcy9wAoa9wStBIeEZH1WIOFilRvzUMDdN78duS2ebLDKKnwQO4ghjqA730kjQswJjsBOBJhMbo8-FPrZT-6RKuz5agSFn/s428/machado2019b_12%20mo%20followup.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="253" data-original-width="428" height="236" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-lBsrkn1SQjSfrgvTGGld5cmC_4XZSKLSlP2M78nA1MRbJy-LN6ZI5cTDAx6wyujrB8QAtLjUg5bFZwyH-uj3IWTRdNzKWaRgcy9wAoa9wStBIeEZH1WIOFilRvzUMDdN78duS2ebLDKKnwQO4ghjqA730kjQswJjsBOBJhMbo8-FPrZT-6RKuz5agSFn/w400-h236/machado2019b_12%20mo%20followup.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="font-size: x-small;">Figure 2. Chart 1 from Machado et al 2020</span><br /></td></tr></tbody></table> <p></p><p>In the 2019b paper, it is reported that, after the stunning success of the initial phase of the study, the placebo group were offered the intervention, and all took part, whereupon they proceeded to make an almost identical amount of remarkable progress on all five subscales, as well as the global scale (see Figure 3). We might expect the 'baseline' scores of the cross-over group to correspond to the scores reported at the final follow-up (as placebo group prior to cross-over) but they don't. </p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOn76kjOY3hJv8LG9TfNH7E0rDxBhXpZLSS_2b40dRVw7LfV2Ud5bW-IrRIXhzfEqZSeDGhBrnipUjqgtiy0mjEhBbGTkU6gW9_MLN4mydn4uzqPZN4V6ixjiVn6GmnDauZ4ZP5I8Vf5Rjm8TwM8giBc-6qYQuHqbq03X_6C-f9WOvNV-puafYqyyFnw3n/s409/machado2019b_chart2_crossover_irritability.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="212" data-original-width="409" height="208" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOn76kjOY3hJv8LG9TfNH7E0rDxBhXpZLSS_2b40dRVw7LfV2Ud5bW-IrRIXhzfEqZSeDGhBrnipUjqgtiy0mjEhBbGTkU6gW9_MLN4mydn4uzqPZN4V6ixjiVn6GmnDauZ4ZP5I8Vf5Rjm8TwM8giBc-6qYQuHqbq03X_6C-f9WOvNV-puafYqyyFnw3n/w400-h208/machado2019b_chart2_crossover_irritability.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="font-size: x-small;">Figure 3: Chart 2 of Machado et al 2019b</span><br /></td></tr></tbody></table><p>I checked for other Erchonia studies on clinicaltrials.gov. Another study, virtually identical except for the age range, was registered in 2020 with Dr Leon Morales-Quezada of Spaulding Rehabilitation Hospital, Boston as Principal Investigator. Comments in the documents suggest this was conducted after Erchonia failed to get the desired FDA approval. Although I have not found a published report of this second trial, I found a<a href="https://rally.massgeneralbrigham.org/study/asd__lllt," target="_blank"> recruitment advertisement</a>, which confusingly cites the NCT registration number of the 2013 study. Some <a href="https://www.clinicaltrials.gov/study/NCT04895605" target="_blank">summary results are included on clinicaltrials.gov</a>, and they are strikingly different from the Leisman et al trial, with no indication of any meaningful difference between active and placebo groups in the final outcome measure, and both groups showing some improvement. I have requested fuller data from Elvira Cawthon (listed as results point of contact) with cc. to Dr Morales-Quezada and will update this post if I hear back.<br /></p><p>
It would appear that at one level this is a positive story, because it shows the regulatory system working. We do not know why FDA rejected Erchonia's request for 510k Market Clearance, but the fact that they did so might indicate that they were unimpressed by the data provided by Leisman and Machado. The fact that Machado et al reported their three follow-up studies in what appears to be an unregistered journal suggests they had difficulty persuading regular journals that the findings were legitimate. If eight 5-minute sessions with a low-level laser pointed at the head really could dramatically improve the function of children with autism 12 months later, one would imagine that Nature, Cell and Science would be scrambling to publish the articles. On the other hand, any device that has the potential to stimulate neuronal growth might also ring alarm bells in terms of potential for harm.<br /></p><p>
Use of low-level lasers to treat autism is only part of the story. Questions remain about the role of Regulatory Insight, Inc., whose statistician apparently failed to notice anything strange about the data from the first autism study. In another post, I plan to look at cases where the same organisation was involved in monitoring and analysing trials of Erchonia laser devices for other conditions such as cellulite, pain, and hearing loss.
</p><p>
<b>Notes
</b></p><p>
* Quirk, B. J., & Whelan, H. T. (2011). Near-infrared irradiation photobiomodulation: The need for basic science. Photomedicine and Laser Surgery, 29(3), 143–144. https://doi.org/10.1089/pho.2011.3014. This article states "<i>clinical uses of NIR-PBM have been studied in such diverse areas as wound healing, oral mucositis, and retinal toxicity. In addition, NIR-PBM is being considered for study in connection with areas such as aging and neural degenerative diseases (Parkinson's disease in particular). One thing that is missing in all of these pre-clinical and clinical studies is a proper investigation into the basic science of the NIR-PBM phenomenon. Although there is much discussion of the uses of NIR, there is very little on how it actually works. As far as explaining what really happens, we are basically left to resort to saying 'light enters, then a miracle happens, and good things come out!' Clearly, this is insufficient, if for no other reason than our own intellectual curiosity</i>." </p><p>**Aman, M. G., Singh, N. N., Stewart, A. W., & Field, C. J. (1985).
The aberrant behavior checklist: A behavior rating scale for the
assessment of treatment effects. American Journal of Mental Deficiency,
89(5), 485–491. N. B. this is different from the Autism Behavior
Checklist which is a commonly used autism assessment. <br /></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com5tag:blogger.com,1999:blog-5841910768079015534.post-77972140503699089452023-11-19T13:49:00.001+00:002023-11-20T09:00:00.652+00:00Defence against the dark arts: a proposal for a new MSc course<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzDagfyYk9vEGCYpLfc73L_fdNzUsaxbfWhpc81ET2n3b6RoGWsqfwoxwcERdLMUVrynpV_spc7u-8e4ZZ4CsH7unbJXpvhl9gfb2S5rwlRsHikOpRAk0KA4_pkOsR6OWwCn2DbnWPmzk1dNzotVCdQXraM58NUR1NH2TWoJepVMBvO5GHHi-G_w-FZU0H/s333/Screenshot%202023-11-19%20at%2009.25.11.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="333" data-original-width="312" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzDagfyYk9vEGCYpLfc73L_fdNzUsaxbfWhpc81ET2n3b6RoGWsqfwoxwcERdLMUVrynpV_spc7u-8e4ZZ4CsH7unbJXpvhl9gfb2S5rwlRsHikOpRAk0KA4_pkOsR6OWwCn2DbnWPmzk1dNzotVCdQXraM58NUR1NH2TWoJepVMBvO5GHHi-G_w-FZU0H/s320/Screenshot%202023-11-19%20at%2009.25.11.png" width="300" /></a></div><br /><p></p><p>
Since I retired, an increasing amount of my time has been taken up with investigating scientific fraud. In recent months, I've become convinced of two things: first, fraud is a far more serious problem than most scientists recognise, and second, we cannot continue to leave the task of tackling it to volunteer sleuths. </p><p>
If you ask a typical scientist about fraud, <a href="https://blogs.bmj.com/bmj/2021/07/05/time-to-assume-that-health-research-is-fraudulent-until-proved-otherwise/#:~:text=We%20have%20now%20reached%20a,and%20having%20ethics%20committee%20approval." target="_blank">they will usually tell you it is extremely rare</a>, and that it would be a mistake to damage confidence in science because of the activities of a few unprincipled individuals. Asked to name fraudsters they may, depending on their age and discipline, mention <a href="https://www.bmj.com/content/381/bmj.p1442" target="_blank">Paolo Macchiarini,</a> <a href="https://www.science.org/doi/10.1126/science.6828878" target="_blank">John Darsee</a>, <a href="https://www.waterstones.com/book/bad-blood/john-carreyrou/9781035006779" target="_blank">Elizabeth Holmes</a> or <a href="http://nick.brown.free.fr/stapel">Diederik Stapel, </a>all high profile, successful individuals, who were brought down when unambiguous evidence of fraud was uncovered. Fraud has been around for years, as documented in an excellent book by Horace Judson (2004), and yet, we are reassured, science is self-correcting, and has prospered despite the activities of the occasional "bad apple". The problem with this argument is that, on the one hand, we only know about the fraudsters who get caught, and on the other hand, science is not prospering particularly well - numerous published papers produce results that fail to replicate and major discoveries are few and far between (Harris, 2017). We are <a href="https://arxiv.org/abs/2309.15884" target="_blank">swamped with scientific publications</a>, but it is increasingly hard to distinguish the signal from the noise. In my view, it is getting to the point where in many fields it is impossible to build a cumulative science, because we lack a solid foundation of trustworthy findings. And it's getting worse and worse.</p><p>
My gloomy prognosis is partly engendered by a consideration of a very different kind of fraud: the <a href="https://publicationethics.org/resources/research/paper-mills-research" target="_blank">academic paper mill</a>. In contrast to the lone fraudulent scientist who fakes data to achieve career advancement, the paper mill is an industrial-scale operation, where vast numbers of fraudulent papers are generated, and placed in peer-reviewed journals with authorship slots being sold to willing customers. This process is facilitated in some cases by <a href="https://osf.io/preprints/psyarxiv/6mbgv" target="_blank">publishers who encourage special issues</a>, which are then taken over by "guest editors" who work for a paper mill. Some <a href="https://febs.onlinelibrary.wiley.com/doi/10.1002/1873-3468.13747" target="_blank">paper mill products are very hard to detect</a>: they may be created from a convincing template with just a few details altered to make the article original. Others are incoherent nonsense, with spectacularly strange prose emerging when "<a href="https://arxiv.org/abs/2107.06751" target="_blank">tortured phrases</a>" are inserted to evade plagiarism detectors. </p><p>
You may wonder whether it matters if a proportion of the published literature is nonsense: surely any credible scientist will just ignore such material? Unfortunately, it's not so simple. First, it is likely that the paper mill products that are detected are just the tip of the iceberg - a clever fraudster will modify their methods to evade detection. Second, many fields of science attempt to synthesise findings using big data approaches, automatically combing the literature for studies with specific keywords and then creating databases, e.g. of genotypes and phenotypes. If these contain a large proportion of fictional findings, then attempts to use these databases to generate new knowledge will be frustrated. Similarly, in clinical areas, there is <a href="https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.ED000152/full" target="_blank">growing concern that systematic reviews</a> that are supposed to synthesise evidence to get at the truth instead lead to confusion because a high proportion of studies are fraudulent. A third and more indirect negative consequence of the explosion in published fraud is that those who have committed fraud can rise to positions of influence and eminence on the back of their misdeeds. They may become editors, with the power to publish further fraudulent papers in return for money, and if promoted to professorships they will train a whole new generation of fraudsters, while being careful to sideline any honest young scientists who want to do things properly. I fear in some institutions this has already happened.</p><p>
To date, the response of the scientific establishment has been wholly inadequate. There is <a href="https://journals.sagepub.com/doi/10.1177/1745691612460687?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%20%200pubmed" target="_blank">little attempt to proactively check for fraud</a>: science is still regarded as a gentlemanly pursuit where we should assume everyone has honourable intentions. Even when evidence of misconduct is strong, <a href="https://scholarlykitchen.sspnet.org/2022/08/18/guest-post-who-cares-about-publication-integrity/" target="_blank">it can take months or years for a paper to be retracted</a>. As whistleblower Raphaël Levy asked on his blog: <a href="https://raphazlab.wordpress.com/2022/12/15/is-it-somebody-elses-problem-to-correct-the-scientific-literature/" target="_blank">Is it somebody else's problem to correct the scientific literature</a>? There is dawning <a href="https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000737" target="_blank">awareness that our methods for hiring and promotion might encourage misconduct</a>, but getting institutions to change is a very slow business, not least because those in positions of power succeeded in the current system, and so think it must be optimal. </p><p>
The task of unmasking fraud is largely left to hobbyists and volunteers, a self-styled army of "data sleuths", who are mostly motivated by anger at seeing science corrupted and the bad guys getting away with it. They have developed expertise in spotting certain kinds of fraud, such as image manipulation and improbable patterns in data, and they have also uncovered webs of bad actors who have infiltrated many corners of science. One might imagine that the scientific establishment would be grateful that someone is doing this work, but the usual response to a sleuth who finds evidence of malpractice is to ignore them, brush the evidence under the carpet, or accuse them of vexatious behaviour. Publishers and academic institutions are both at fault in this regard.
</p><p>
If I'm right, this relaxed attitude to the fraud epidemic is a disaster-in-waiting. There are a number of things that need to be done urgently. One is to change research culture so that rewards go to those whose work is characterised by openness and integrity, rather than those who get large grants and flashy publications. Another is for publishers to act far more promptly to investigate complaints of malpractice and issue retractions where appropriate. Both of these things are beginning to happen, slowly. But there is a third measure that I think should be taken as soon as possible, and that is to train a generation of researchers in fraud busting. We owe a huge debt of gratitude to the data sleuths, but the scale of the problem is such that we need the equivalent of a police force rather than a volunteer band. Here are some of the topics that an MSc course could cover:</p><ul style="text-align: left;"><li>
How to spot dodgy datasets</li><li>How to spot manipulated figures</li><li>Textual characteristics of fraudulent articles</li><li>Checking scientific credentials</li><li>Checking publisher credentials/identifying predatory publishers <br /></li><li>How to raise a complaint when fraud is suspected</li><li>How to protect yourself from legal attacks</li><li>Cognitive processes that lead individuals to commit fraud</li><li>Institutional practices that create perverse incentives</li><li>The other side of the coin: "Merchants of doubt" whose goal is to discredit science</li></ul><p>
I'm sure there's much more that could be added and would be glad of suggestions. </p><p>Now, of course, the question is what could you do with such a qualification. If my predictions are right, then individuals with such expertise will increasingly be in demand in academic institutions and publishing houses, to help ensure the integrity of work they produce and publish. I also hope that there will be growing recognition of the need for more formal structures to be set up to investigate scientific fraud and take action when it is discovered: graduates of such a course would be exactly the kind of employees needed in such an organisation.</p><p>
It might be argued that this is a hopeless endeavour. In Harry Potter and the Half-Blood Prince (Rowling, 2005) Professor Snape tells his pupils:</p><p></p><blockquote><i> "The Dark Arts, are many, varied, ever-changing, and eternal. Fighting them is like fighting a many-headed monster, which, each time a neck is severed, sprouts a head even fiercer and cleverer than before. You are fighting that which is unfixed, mutating, indestructible."</i></blockquote><p></p><p>
This is a pretty accurate description of what is involved in tackling scientific fraud. But Snape does not therefore conclude that action is pointless. On the contrary, he says: </p><p></p><blockquote><i>"Your defences must therefore be as flexible and inventive as the arts you seek to undo."
</i></blockquote><p></p><p>
I would argue that any university that wants to be ahead of the field in this enterprise could should flexibility and inventiveness in starting up a postgraduate course to train the next generation of fraud-busting wizards. </p><p><b>Bibliography</b></p><div class="csl-bib-body" style="line-height: 2; margin-left: 2em; text-indent: -2em;">
<div class="csl-entry">Bishop, D. V. M. (2023). <i>Red flags for papermills need to go beyond the level of individual articles: A case study of Hindawi special issues</i>. https://osf.io/preprints/psyarxiv/6mbgv</div><div class="csl-entry">Boughton, S. L., Wilkinson, J., & Bero, L. (2021). When beauty is but skin deep: Dealing with problematic studies in systematic reviews | Cochrane Library. <i>Cochrane Database of Systematic Reviews</i>, <i>5</i>. Retrieved 4 June 2021, from <a href="https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.ED000152/full">https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.ED000152/full</a>
<span class="Z3988" title="url_ver=Z39.88-2004&ctx_ver=Z39.88-2004&rfr_id=info%3Asid%2Fzotero.org%3A2&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=When%20beauty%20is%20but%20skin%20deep%3A%20dealing%20with%20problematic%20studies%20in%20systematic%20reviews%20%7C%20Cochrane%20Library&rft.jtitle=Cochrane%20Database%20of%20Systematic%20Reviews&rft.volume=5&rft.aufirst=Stephanie%20L&rft.aulast=Boughton&rft.au=Stephanie%20L%20Boughton&rft.au=Jack%20Wilkinson&rft.au=Lisa%20Bero"></span>
</div><div class="csl-entry"> Byrne, J. A., & Christopher, J. (2020). Digital magic, or the dark arts of the 21st century—How can journals and peer reviewers detect manuscripts and publications from paper mills? <i>FEBS Letters</i>, <i>594</i>(4), 583–589. <a href="https://doi.org/10.1002/1873-3468.13747">https://doi.org/10.1002/1873-3468.13747</a>
</div></div><div class="csl-bib-body" style="line-height: 2; margin-left: 2em; text-indent: -2em;">
<div class="csl-entry">Cabanac, G., Labbé, C., & Magazinov, A. (2021). <i>Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals</i> (arXiv:2107.06751). arXiv. <a href="https://doi.org/10.48550/arXiv.2107.06751">https://doi.org/10.48550/arXiv.2107.06751</a></div><div class="csl-entry">Carreyrou, J. (2019). <i>Bad Blood: Secrets and Lies in a Silicon Valley Startup</i>. Pan Macmillan.</div><div class="csl-entry">COPE & STM. (2022). <i>Paper mills: Research report from COPE & STM</i>. Committee on Publication Ethics and STM. <a href="https://doi.org/10.24318/jtbG8IHL">https://doi.org/10.24318/jtbG8IHL</a> </div><div class="csl-entry">Culliton, B. J. (1983). Coping with fraud: The Darsee Case. <i>Science (New York, N.Y.)</i>, <i>220</i>(4592), 31–35. <a href="https://doi.org/10.1126/science.6828878">https://doi.org/10.1126/science.6828878</a> </div><div class="csl-entry">Grey, S., & Bolland, M. (2022, August 18). <i>Guest Post—Who Cares About Publication Integrity?</i> The Scholarly Kitchen. <a href="https://scholarlykitchen.sspnet.org/2022/08/18/guest-post-who-cares-about-publication-integrity/">https://scholarlykitchen.sspnet.org/2022/08/18/guest-post-who-cares-about-publication-integrity/</a> </div><div class="csl-entry">Hanson, M., Gómez Barreiro, P., Crosetto, P., & Brockington, D. (2023). <i>The strain on scientific publishing</i> (2309; p. 33343265 Bytes). arXiv. <a href="https://arxiv.org/ftp/arxiv/papers/2309/2309.15884.pdf">https://arxiv.org/ftp/arxiv/papers/2309/2309.15884.pdf</a> </div><div class="csl-entry">Harris, R. (2017). <i>Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions</i> (1st edition). Basic Books.
</div></div><p></p><p>Judson, H. F. (2004). <i>The Great Betrayal</i>. Orlando.</p><div class="csl-bib-body" style="line-height: 2; margin-left: 2em; text-indent: -2em;"><div class="csl-entry">Lévy, R. (2022, December 15). Is it somebody else’s problem to correct the scientific literature? <i>Rapha-z-Lab</i>. <a href="https://raphazlab.wordpress.com/2022/12/15/is-it-somebody-elses-problem-to-correct-the-scientific-literature/">https://raphazlab.wordpress.com/2022/12/15/is-it-somebody-elses-problem-to-correct-the-scientific-literature/</a></div><div class="csl-entry"> Moher, D., Bouter, L., Kleinert, S., Glasziou, P., Sham, M. H., Barbour, V., Coriat, A.-M., Foeger, N., & Dirnagl, U. (2020). The Hong Kong Principles for assessing researchers: Fostering research integrity. <i>PLOS Biology</i>, <i>18</i>(7), e3000737. <a href="https://doi.org/10.1371/journal.pbio.3000737">https://doi.org/10.1371/journal.pbio.3000737</a></div><div class="csl-entry"> Oreskes, N., & Conway, E. M. (2010). <i>Merchants of Doubt: How a handful of scientists obscured the truth on issues from tobacco smoke to global warming</i>. Bloomsbury Press.
<span class="Z3988" title="url_ver=Z39.88-2004&ctx_ver=Z39.88-2004&rfr_id=info%3Asid%2Fzotero.org%3A2&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Merchants%20of%20Doubt%3A%20How%20a%20handful%20of%20scientists%20obscured%20the%20truth%20on%20issues%20from%20tobacco%20smoke%20to%20global%20warming&rft.publisher=Bloomsbury%20Press&rft.aufirst=Naomi&rft.aulast=Oreskes&rft.au=Naomi%20Oreskes&rft.au=Eric%20M%20Conway&rft.date=2010"></span>
</div><div class="csl-entry"> Paterlini, M. (2023). Paolo Macchiarini: Disgraced surgeon is sentenced to 30 months in prison. <i>BMJ</i>, <i>381</i>, p1442. <a href="https://doi.org/10.1136/bmj.p1442">https://doi.org/10.1136/bmj.p1442</a>
<span class="Z3988" title="url_ver=Z39.88-2004&ctx_ver=Z39.88-2004&rfr_id=info%3Asid%2Fzotero.org%3A2&rft_id=info%3Adoi%2F10.1136%2Fbmj.p1442&rft_id=info%3Apmid%2F37348897&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Paolo%20Macchiarini%3A%20Disgraced%20surgeon%20is%20sentenced%20to%2030%20months%20in%20prison&rft.jtitle=BMJ&rft.stitle=BMJ&rft.volume=381&rft.aufirst=Marta&rft.aulast=Paterlini&rft.au=Marta%20Paterlini&rft.date=2023-06-22&rft.pages=p1442&rft.issn=1756-1833&rft.language=en"></span> </div><div class="csl-entry">Rowling, J. K. (2005) <i>Harry Potter and the Half-Blood Prince</i>. Bloomsbury, London. ISBN: 9780747581086 </div></div><div class="csl-bib-body" style="line-height: 2; margin-left: 2em; text-indent: -2em;">
<div class="csl-entry">Smith, R. (2021, July 5). <i>Time to assume that health research is fraudulent until proven otherwise?</i> The BMJ. <a href="https://blogs.bmj.com/bmj/2021/07/05/time-to-assume-that-health-research-is-fraudulent-until-proved-otherwise/">https://blogs.bmj.com/bmj/2021/07/05/time-to-assume-that-health-research-is-fraudulent-until-proved-otherwise/</a></div><div class="csl-entry">Stapel, D. (2016). <i>Faking science: A true story of academic fraud. Translated by Nicholas J. Brown. </i><span style="-webkit-text-stroke-width: 0px; background-color: #eff3f4; caret-color: rgb(15, 20, 25); color: #0f1419; display: inline !important; float: none; font-family: TwitterChirp, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; white-space: pre-wrap; widows: auto; word-spacing: 0px;"><a href="http://nick.brown.free.fr/stapel" target="_blank">http:// nick.brown.free.fr/stapel.</a></span></div><div class="csl-entry">Stroebe, W., Postmes, T., & Spears, R. (2012). Scientific misconduct and the myth of self-correction in science. <i>Perspectives on Psychological Science</i>, <i>7</i>(6), 670–688. <a href="https://doi.org/10.1177/1745691612460687">https://doi.org/10.1177/1745691612460687</a>
<span class="Z3988" title="url_ver=Z39.88-2004&ctx_ver=Z39.88-2004&rfr_id=info%3Asid%2Fzotero.org%3A2&rft_id=info%3Adoi%2F10.1177%2F1745691612460687&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Scientific%20misconduct%20and%20the%20myth%20of%20self-correction%20in%20science&rft.jtitle=Perspectives%20on%20Psychological%20Science&rft.stitle=Perspect%20Psychol%20Sci&rft.volume=7&rft.issue=6&rft.aufirst=Wolfgang&rft.aulast=Stroebe&rft.au=Wolfgang%20Stroebe&rft.au=Tom%20Postmes&rft.au=Russell%20Spears&rft.date=2012-11-01&rft.pages=670-688&rft.spage=670&rft.epage=688&rft.issn=1745-6916&rft.language=en"></span>
</div><div class="csl-entry"> </div></div><p><span style="font-size: x-small;"><b>Note</b>: On-topic comments are welcome but are moderated to avoid spam, so there may be a delay before they appear. </span><br /></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com9tag:blogger.com,1999:blog-5841910768079015534.post-27825872441930549052023-10-12T17:24:00.003+01:002023-10-13T09:27:15.023+01:00When privacy rules protect fraudsters<p> </p><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgm22K5dAMdkA_4rR_HDnSzxlhyphenhypheneyjkaC_mtn5FxPzN8v_zLWeqh1PLRLctYPOGCEK1kfwbTbJjIkSEroMWcWLTNikM0uby5yilLphP7bhLmX-eLYo5OTOVDbiUkwciPuHdjvua839bIsMbvx7NJW55n15aIYimyXw4GSflIoF8kuL6wdA21kBySPL5VT5D/s2480/CartoonStock_mod%20588118_CS175084.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="2480" data-original-width="1645" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgm22K5dAMdkA_4rR_HDnSzxlhyphenhypheneyjkaC_mtn5FxPzN8v_zLWeqh1PLRLctYPOGCEK1kfwbTbJjIkSEroMWcWLTNikM0uby5yilLphP7bhLmX-eLYo5OTOVDbiUkwciPuHdjvua839bIsMbvx7NJW55n15aIYimyXw4GSflIoF8kuL6wdA21kBySPL5VT5D/s320/CartoonStock_mod%20588118_CS175084.jpeg" width="212" /></a></div> </div><div class="separator" style="clear: both; text-align: left;">I was recently contacted with what I thought was a simple request: could I check the Oxford University Gazette to confirm that a person, X, had undergone an oral examination (viva) for a doctorate a few years ago. The request came indirectly from a third party, Y, via a colleague who knew that on the one hand I was interested in scientific fraud, and on the other hand, that I was based at Oxford. </div><p></p><p>
</p><p>
My first thought was that this was a rather cumbersome way of checking someone's credentials. For a start, as Y had discovered, you can consult the on-line University Gazette only if you have an official affiliation with the university. In theory, when someone has a viva, the internal examiner notifies the University Gazette, which announces details in advance so that members of the university can attend if they so wish. In practice, it is vanishingly rare for an audience to turn up, and the formal notification to the Gazette may get overlooked. </p><p>
But why, I wondered, didn't Y just check the official records of Oxford University listing names and dates of degrees? Well, to my surprise, it turned out that you can't do that. The university website is clear that to verify someone's qualifications you need to meet two conditions. First, the request can only be made by "<i>employers, prospective employers, other educational institutions, funding bodies or recognised voluntary organisations</i>". Second, "<i>the student's permission ... should be acquired prior to making any verification request</i>".</p><p>
</p><p>
Anyhow, I found evidence online that X had been a graduate student at the university, but when I checked the Gazette I could find no mention of X having had an oral examination. The other source of evidence would be the University Library where there should be a copy of the thesis for all higher degrees. I couldn't find it in the catalogue. I suggested that Y might check further but they were already ahead of me, and had confirmed with the librarian that no thesis had been deposited in that name. </p><p>
</p><p>
Now, I have no idea whether X is fraudulently claiming to have an Oxford doctorate, but I'm concerned that it is so hard for a private individual to validate someone's credentials. As far as I can tell, the justification comes from data protection regulations, which control what information organisations can hold about individuals. This is not an Oxford-specific interpretation of rules - I checked a few other UK universities, and the same processes apply. </p><p>
</p><p>
Having said that, Y pointed out to me that there is a precedent for Oxford University to provide information when there is media interest in a high-profile case: in response to a freedom of information request, they <a href="https://www.theguardian.com/education/2022/may/16/ferdinand-marcos-jr-urged-to-stop-pretending-he-has-an-oxford-degree" target="_blank">confirmed that Ferdinand Marcus Jr did not have the degree</a> he was claiming.<br /></p><p>
</p><p>
There will always be tension between openness and the individual's right to privacy, but the way the rules are interpreted mean that anyone could claim they had a degree from a UK university and it would be impossible to check this. Is there a solution? I'm no lawyer, but I would have thought it should be trivial to require that on receipt of a degree, the student is asked to give signed permission for their name, degree and date of degree to be recorded on a publicly searchable database. I can't see a downside to this, and going forward it would save a lot of administrative time dealing with verification requests. </p><p>
</p><p>
Something like this does seem to work outside Europe. I only did a couple of spot checks, but found<a href="https://registrar.yorku.ca/graduation/yuverify" target="_blank"> this</a> for York University (Ontario):<br /></p><p></p><blockquote>
"<i>It is the University's policy to make information about the degrees or credentials conferred by the University and the dates of conferral routinely available. In order to protect our alumni information as much as possible, YU Verify will give users a result only if the search criteria entered matches a unique record. The service will not display a list of names which may match criteria and allow you to select.</i>"</blockquote><p></p><p>
</p><p>
And for Macquarie University, Australia, there is exactly the <a href="https://students.mq.edu.au/study/graduation/graduate-register/pre-2002-graduate-register" target="_blank">kind of searchable website</a> that I'd assumed Oxford would have. </p><p>
</p><p>
I'd be interested if anyone can think of unintended bad consequences of this approach. I had a bit of to-and-fro on Twitter about this with someone who argued that it was best to keep as much information as possible out of the public domain. I remain unconvinced: academic qualifications are important for providing someone with credentials as an expert, and if we make it easy for anyone to pretend to have a degree from a prestigious institution, I think the potential for harm is far greater than any harms caused by lack of privacy. Or have I missed something? </p><p><i> N.B. Comments on the blog are moderated so may only appear after a delay. </i></p><p><i><br /></i></p><p><i>P.S. Some thoughts via Mastodon from Martin Vueilleme on potential drawback of directory: </i></p><p><span style="background-color: white; font-family: mastodon-font-sans-serif, sans-serif; white-space: pre-wrap;"><span style="font-size: x-small;"><i>Far fetched, but I could see the following reasons:</i></span></span></p><p><i><span style="background-color: white; font-family: mastodon-font-sans-serif, sans-serif; font-size: 15px; white-space: pre-wrap;">- You live in an oppressive country that targets academics, intellectuals</span><br style="font-family: mastodon-font-sans-serif, sans-serif; font-size: 15px; white-space: pre-wrap;" /><span style="background-color: white; font-family: mastodon-font-sans-serif, sans-serif; font-size: 15px; white-space: pre-wrap;">- Hiding your university helps prevent stalkers (or other predators) from getting further information on you</span><br style="font-family: mastodon-font-sans-serif, sans-serif; font-size: 15px; white-space: pre-wrap;" /><span style="background-color: white; font-family: mastodon-font-sans-serif, sans-serif; font-size: 15px; white-space: pre-wrap;">- Hiding your university background to fit in a group</span><br style="font-family: mastodon-font-sans-serif, sans-serif; font-size: 15px; white-space: pre-wrap;" /><span style="background-color: white; font-family: mastodon-font-sans-serif, sans-serif; font-size: 15px; white-space: pre-wrap;">- Your thesis is on a sensitive topic or a topic forbidden from being studied where you live</span><br style="font-family: mastodon-font-sans-serif, sans-serif; font-size: 15px; white-space: pre-wrap;" /><span style="background-color: white; font-family: mastodon-font-sans-serif, sans-serif; font-size: 15px; white-space: pre-wrap;">- Hiding your university degree because you were technically not allowed to get it (eg women)</span></i></p><p><span style="background-color: white; font-family: mastodon-font-sans-serif, sans-serif; font-size: 15px; white-space: pre-wrap;">My (DB) response is that I think that in terms of balancing probabilities of risks against the risk of fraudsters benefiting from lack of checking, the case for the open directory is strengthened, as these risks seem very slight for UK universities (at least for now!). And the other cost/benefit analysis is of finances, where an open directory would seem superior; i.e. it costs to maintain the directory, but that has to be done anyhow, Currently there are extra costs for people who are employed to respond to requests for validation. </span></p><p>
</p><p>
</p><p>
</p><p>
</p><p>
</p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com4tag:blogger.com,1999:blog-5841910768079015534.post-3840897936439726602023-10-03T09:00:00.000+01:002023-10-03T17:16:32.285+01:00Bishopblog catalogue (updated 4 October 2023)<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMSDhXGOrkm47TjPz8xs3wHtgYzkydZNn5z_Mdue5TgNAK7XDjBnSaqxsKs4evAgnpR2FlcHKLvUDmRXNqPOZC_IUzyitAJOq35ZCygMyHKI4WX78fEQILFNcP4OQyO7_xmwaYjne2H9J4/s1600/sifting-through-ideas.gif" style="margin-left: auto; margin-right: auto;"><img border="0" height="260" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMSDhXGOrkm47TjPz8xs3wHtgYzkydZNn5z_Mdue5TgNAK7XDjBnSaqxsKs4evAgnpR2FlcHKLvUDmRXNqPOZC_IUzyitAJOq35ZCygMyHKI4WX78fEQILFNcP4OQyO7_xmwaYjne2H9J4/s320/sifting-through-ideas.gif" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Source: http://www.weblogcartoons.com/2008/11/23/ideas/</td></tr>
</tbody></table>
<br />
<div class="MsoNormal">
<div style="font-family: inherit;">
<span style="font-size: 10.5pt; line-height: 115%;">Those of you who follow this blog may have noticed a lack of
thematic coherence. I write about whatever is exercising my mind at the time,
which can range from technical aspects of statistics to the design of bathroom
taps. I decided it might be helpful to introduce a bit of order into this
chaotic melange, so here is a catalogue of posts by topic.</span></div>
<br />
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Language impairment, dyslexia and related disorders </span></b></div>
</div>
<a href="http://www.guardian.co.uk/science/2010/dec/01/sli-autism-childhood-developmental-disorders">The common childhood disorders that have been left out in the cold</a> (1 Dec 2010)
<a href="http://deevybee.blogspot.co.uk/2010/12/whats-in-name.html">What's in a name?</a> (18 Dec 2010)
<a href="http://deevybee.blogspot.co.uk/2010/12/neuroprognosis-in-dyslexia.html">Neuroprognosis in dyslexia</a> (22 Dec 2010)
<a href="http://deevybee.blogspot.co.uk/2011/03/where-commercial-and-clinical-interests.html">Where commercial and clinical interests collide: Auditory processing disorder</a> (6 Mar 2011)
<a href="http://wellcometrust.wordpress.com/2011/03/30/auditory-processing-disorder-a-cause-of-language-problems-or-an-incidental-finding/">Auditory processing disorder</a> (30 Mar 2011)
<a href="http://deevybee.blogspot.co.uk/2011/04/special-educational-needs-will-they-be.html">Special educational needs: will they be met by the Green paper proposals?</a> (9 Apr 2011)
<a href="http://deevybee.blogspot.co.uk/2011/06/its-tough-being-parent-of-child-with.html">Is poor parenting really to blame for children's school problems?</a> (3 Jun 2011)
<a href="http://deevybee.blogspot.co.uk/2011/09/early-intervention-whats-not-to-like.html">Early intervention: what's not to like?</a> (1 Sep 2011)
<a href="http://deevybee.blogspot.co.uk/2011/10/lies-damned-lies-and-spin.html">Lies, damned lies and spin</a> (15 Oct 2011)
<a href="http://deevybee.blogspot.co.uk/2011/10/message-to-world.html">A message to the world</a> (31 Oct 2011)
<a href="http://deevybee.blogspot.co.uk/2011/11/vitamins-genes-and-language.html">Vitamins, genes and language</a> (13 Nov 2011)
<a href="http://deevybee.blogspot.co.uk/2012/02/neuroscientific-interventions-for.html">Neuroscientific interventions for dyslexia: red flags</a> (24 Feb 2012)
<a href="http://deevybee.blogspot.co.uk/2012/04/phonics-screening-sense-and-sensibility.html">Phonics screening: sense and sensibility</a> (3 Apr 2012) <a href="http://deevybee.blogspot.co.uk/2012/09/what-chomsky-didnt-get-about-child.html">What Chomsky doesn't get about child language</a> (3 Sept 2012) <a href="http://deevybee.blogspot.co.uk/2012/10/data-from-phonics-screen-worryingly.html">Data from the phonics screen</a> (1 Oct 2012)
<a href="http://deevybee.blogspot.co.uk/2012/10/auditory-processing-disorder-apd.html">Auditory processing disorder: schisms and skirmishes</a> (27 Oct 2012)
<a href="http://deevybee.blogspot.co.uk/2013/03/high-impact-journals-where.html">High-impact journals (Action video games and dyslexia: critique) </a> (10 Mar 2013) <a href="http://deevybee.blogspot.co.uk/2013/06/overhyped-genetic-findings-case-of.html">Overhyped genetic findings: the case of dyslexia</a> (16 Jun 2013) <a href="http://deevybee.blogspot.co.uk/2013/08/the-arcuate-fasciculus-and-word.html">The arcuate fasciculus and word learning</a> (11 Aug 2013) <a href="http://deevybee.blogspot.co.uk/2013/08/changing-childrens-brains.html">Changing children's brains</a> (17 Aug 2013)
<a href="http://deevybee.blogspot.co.uk/2013/09/raising-awareness-of-language-learning.html">Raising awareness of language learning impairments</a> (26 Sep 2013) <a href="http://deevybee.blogspot.co.uk/2013/10/good-and-bad-news-on-phonics-screen.html">Good and bad news on the phonics screen</a> (5 Oct 2013)
<a href="http://deevybee.blogspot.co.uk/2014/01/what-is-educational-neuroscience.html">What is educational neuroscience?</a> (25 Jan 2014)
<a href="http://deevybee.blogspot.co.uk/2014/02/parent-talk-and-child-language.html">Parent talk and child language</a> (17 Feb 2014)
<a href="http://deevybee.blogspot.co.uk/2014/03/my-thoughts-on-dyslexia-debate.html">My thoughts on the dyslexia debate</a> (20 Mar 2014)
<a href="http://deevybee.blogspot.co.uk/2014/08/labels-for-unexplained-language.html">Labels for unexplained language difficulties in children</a> (23 Aug 2014)
<a href="http://deevybee.blogspot.co.uk/2014/09/international-reading-comparisons-is.html">International reading comparisons: Is England really do so poorly?</a> (14 Sep 2014)
<a href="http://deevybee.blogspot.co.uk/2015/05/great-expectations-our-early.html">Our early assessments of schoolchildren are misleading and damaging</a> (4 May 2015)
<a href="http://deevybee.blogspot.co.uk/2015/08/opportunity-cost-new-red-flag-for.html">Opportunity cost: a new red flag for evaluating interventions</a> (30 Aug 2015)
<a href="https://deevybee.blogspot.co.uk/2017/07/the-step-physical-literacy-programme.html">The STEP Physical Literacy programme: have we been here before?</a> (2 Jul 2017)
<a href="http://deevybee.blogspot.com/2017/11/prisons-developmental-language-disorder.html">Prisons, developmental language disorder, and base rates</a> (3 Nov 2017)
<a href="http://deevybee.blogspot.com/2017/11/reproducibility-and-phonics-necessary.html">Reproducibility and phonics: necessary but not sufficient</a> (27 Nov 2017)
<a href="http://deevybee.blogspot.com/2018/06/developmental-language-disorder-need.html">Developmental language disorder: the need for a clinically relevant definition</a> (9 Jun 2018)
<a href="http://deevybee.blogspot.com/2020/02/changing-terminology-for-childrens.html">Changing terminology for children's language disorders</a> (23 Feb 2020)
<a href="http://deevybee.blogspot.com/2020/02/the-tldr-too-long-didnt-read-message-in.html">Developmental Language Disorder (DLD) in relaton to DSM5</a> (29 Feb 2020)
<a href="http://deevybee.blogspot.com/2022/01/why-i-am-not-engaging-with-reading-wars.html">Why I am not engaging with the Reading Wars</a> (30 Jan 2022)
<br />
<br />
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Autism </span></b></div>
<a href="http://deevybee.blogspot.co.uk/2011/05/autism-diagnosis-in-cultural-context.html">Autism diagnosis in cultural context</a> (16 May 2011)
<a href="http://deevybee.blogspot.co.uk/2011/05/are-our-gold-standard-autism-diagnostic.html">Are our ‘gold standard’ autism diagnostic instruments fit for purpose?</a> (30 May 2011)
<a href="http://www.guardian.co.uk/science/blog/2011/jun/07/how-common-autism-diagnosis">How common is autism?</a> (7 Jun 2011)
<a href="http://deevybee.blogspot.co.uk/2011/06/autism-diagnosis-and-hyper-systemizing.html">Autism and hypersystematising parents</a> (21 Jun 2011) <a href="http://deevybee.blogspot.co.uk/2011/08/open-letter-to-baroness-susan.html">An open letter to Baroness Susan Greenfield</a> (4 Aug 2011)
<a href="http://deevybee.blogspot.co.uk/2011/08/defence-of-susan-greenfield.html">Susan Greenfield and autistic spectrum disorder: was she misrepresented?</a> (12 Aug 2011)
<a href="http://deevybee.blogspot.co.uk/2012/01/psychoanalytic-treatment-for-autism.html">Psychoanalytic treatment for autism: Interviews with French analysts</a> (23 Jan 2012)
<a href="http://deevybee.blogspot.co.uk/2012/06/autism-epidemic-and-diagnostic.html">The ‘autism epidemic’ and diagnostic substitution</a> (4 Jun 2012)
<a href="http://deevybee.blogspot.co.uk/2014/06/how-wishful-thinking-is-damaging-petas.html">How wishful thinking is damaging Peta's cause</a> (9 June 2014)<a href="http://deevybee.blogspot.com/2019/01/neuropointdxs-blood-test-for-autism.html"> NeuroPointDX's blood test for Autism Spectrum Disorder</a> ( 12 Jan 2019) <a href="http://deevybee.blogspot.com/2022/12/biomarkers-to-screen-for-autism-again.html">Biomarkers to screen for autism (again)</a> (6 Dec 2022) <br />
<br />
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Developmental disorders/paediatrics </span></b></div>
<a href="http://www.guardian.co.uk/science/blog/2010/nov/25/neglected-tropical-diseases">The hidden cost of neglected tropical diseases</a> (25 Nov 2010)
<a href="http://deevybee.blogspot.co.uk/2011/06/national-childrens-study-view-from.html">The National Children's Study: a view from across the pond</a> (25 Jun 2011)
<a href="http://www.guardian.co.uk/commentisfree/2011/sep/14/daycare-cortisol-levels-children">The kids are all right in daycare</a> (14 Sep 2011) <a href="http://deevybee.blogspot.co.uk/2012/11/moderate-drinking-in-pregnancy-toxic-or.html">Moderate drinking in pregnancy: toxic or benign?</a> (21 Nov 2012)
<a href="http://deevybee.blogspot.co.uk/2014/05/changing-landscape-of-psychiatric.html">Changing the landscape of psychiatric research</a> (11 May 2014)
<a href="http://deevybee.blogspot.com/2019/10/the-sinister-side-of-french.html">The sinister side of French psychoanalysis revealed</a> (15 Oct 2019) <a href="http://deevybee.blogspot.com/2022/10/a-desire-for-clickbait-can-hinder.html">A desire for clickbait can hinder an academic journal's reputation</a> (4 Oct 2022) <a href="http://deevybee.blogspot.com/2023/09/polyunsaturated-fatty-acids-and.html">Polyunsaturated fatty acids and children's cognition: p-hacking and the canonisation of false facts</a> (4 Sep 2023)
<br />
<br />
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Genetics </span></b></div>
<a href="http://www.guardian.co.uk/science/blog/2010/sep/09/gene-intelligence-genetic-testing">Where does the myth of a gene for things like intelligence come from?</a> (9 Sep 2010)
<a href="http://deevybee.blogspot.co.uk/2010/09/genes-for-optimism-dyslexia-and-obesity.html">Genes for optimism, dyslexia and obesity and other mythical beasts</a> (10 Sep 2010)
<a href="http://deevybee.blogspot.co.uk/2011/05/x-and-y-of-sex-differences.html">The X and Y of sex differences</a> (11 May 2011)
<a href="http://deevybee.blogspot.co.uk/2011/06/review-of-how-genes-influence-behavior.html">Review of How Genes Influence Behaviour</a> (5 Jun 2011)
<a href="http://deevybee.blogspot.co.uk/2012/04/getting-genetic-effect-sizes-in.html">Getting genetic effect sizes in perspective</a> (20 Apr 2012) <a href="http://deevybee.blogspot.co.uk/2012/11/moderate-drinking-in-pregnancy-toxic-or.html">Moderate drinking in pregnancy: toxic or benign?</a> (21 Nov 2012) <a href="http://deevybee.blogspot.co.uk/2012/12/genes-brains-and-lateralisation-how.html">Genes, brains and lateralisation</a> (22 Dec 2012)<a href="http://deevybee.blogspot.co.uk/2013/01/genetic-variation-and-neuroimaging-some.html"> Genetic variation and neuroimaging</a> (11 Jan 2013) <a href="http://deevybee.blogspot.co.uk/2013/05/have-we-become-slower-and-dumber.html">Have we become slower and dumber?</a> (15 May 2013) <a href="http://deevybee.blogspot.co.uk/2013/06/overhyped-genetic-findings-case-of.html">Overhyped genetic findings: the case of dyslexia</a> (16 Jun 2013)
<a href="http://deevybee.blogspot.co.uk/2016/10/on-incomprehensibility-of-much.html">Incomprehensibility of much neurogenetics research</a> ( 1 Oct 2016)
<a href="http://deevybee.blogspot.co.uk/2017/01/a-common-misunderstanding-of-natural.html">A common misunderstanding of natural selection</a> (8 Jan 2017)
<a href="https://deevybee.blogspot.co.uk/2017/04/sample-selection-in-genetic-studies.html">Sample selection in genetic studies: impact of restricted range </a> (23 Apr 2017)
<a href="http://deevybee.blogspot.com/2017/10/why-we-need-to-stop-publishing.html">Pre-registration or replication: the need for new standards in neurogenetic studies</a> (1 Oct 2017)
<a href="http://deevybee.blogspot.com/2019/04/review-of-innate-by-kevin-mitchell.html">Review of 'Innate' by Kevin Mitchell</a> ( 15 Apr 2019)
<a href="http://deevybee.blogspot.com/2020/02/why-eugenics-is-wrong.html">Why eugenics is wrong</a> (18 Feb 2020)
<br />
<br />
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Neuroscience </span></b></div>
<a href="http://deevybee.blogspot.co.uk/2010/12/neuroprognosis-in-dyslexia.html">Neuroprognosis in dyslexia</a> (22 Dec 2010) <a href="http://deevybee.blogspot.co.uk/2011/06/brain-scans-show-that.html">Brain scans show that…</a> (11 Jun 2011)
<a href="http://deevybee.blogspot.co.uk/2012/03/time-for-neuroimaging-to-clean-up-its.html">Time for neuroimaging (and PNAS) to clean up its act</a> (5 Mar 2012)
<a href="http://deevybee.blogspot.co.uk/2012/05/neuronal-migration-in-language-learning.html">Neuronal migration in language learning impairments</a> (2 May 2012)
<a href="http://deevybee.blogspot.co.uk/2012/05/sharing-of-mri-dyslexia-datasets.html">Sharing of MRI datasets</a> (6 May 2012)
<a href="http://deevybee.blogspot.co.uk/2013/01/genetic-variation-and-neuroimaging-some.html">Genetic variation and neuroimaging</a> (1 Jan 2013) <a href="http://deevybee.blogspot.co.uk/2013/08/the-arcuate-fasciculus-and-word.html">The arcuate fasciculus and word learning</a> (11 Aug 2013) <a href="http://deevybee.blogspot.co.uk/2013/08/changing-childrens-brains.html">Changing children's brains</a> (17 Aug 2013)
<a href="http://deevybee.blogspot.co.uk/2014/01/what-is-educational-neuroscience.html">What is educational neuroscience?</a> ( 25 Jan 2014) <a href="http://deevybee.blogspot.co.uk/2014/05/changing-landscape-of-psychiatric.html">Changing the landscape of psychiatric research</a> (11 May 2014)
<a href="http://deevybee.blogspot.co.uk/2016/10/on-incomprehensibility-of-much.html">Incomprehensibility of much neurogenetics research</a> ( 1 Oct 2016)
<br />
<br />
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Reproducibility</span></b><br />
<a href="http://deevybee.blogspot.co.uk/2011/10/accentuate-negative.html">Accentuate the negative</a> (26 Oct 2011) <a href="http://deevybee.blogspot.co.uk/2012/01/novelty-interest-and-replicability.html">Novelty, interest and replicability</a> (19 Jan 2012)
<a href="http://deevybee.blogspot.co.uk/2013/03/high-impact-journals-where.html">High-impact journals: where newsworthiness trumps methodology</a> (10 Mar 2013) <a href="http://deevybee.blogspot.co.uk/2015/11/whos-afraid-of-open-data.html">Who's afraid of open data?</a> (15 Nov 2015)
<a href="http://deevybee.blogspot.co.uk/2013/03/blogging-as-post-publication-peer.html">Blogging as post-publication peer review</a> (21 Mar 2013) <a href="http://deevybee.blogspot.co.uk/2013/06/research-fraud-more-scrutiny-by.html">Research fraud: More scrutiny by administrators is not the answer</a> (17 Jun 2013) <a href="http://deevybee.blogspot.co.uk/2014/01/off-with-old-and-on-with-new-pressures.html">Pressures against cumulative research</a> (9 Jan 2014)
<a href="http://deevybee.blogspot.co.uk/2014/01/why-does-so-much-research-go-unpublished.html">Why does so much research go unpublished?</a> (12 Jan 2014) <a href="http://deevybee.blogspot.co.uk/2014/08/replication-and-reputation-whose-career.html">Replication and reputation: Whose career matters?</a> (29 Aug 2014) <a href="http://deevybee.blogspot.co.uk/2015/12/open-code-not-just-data-and-publications.html">Open code: note just data and publications</a> (6 Dec 2015)
<a href="https://deevybee.blogspot.co.uk/2016/01/the-amazing-significo-why-researchers.html">Why researchers need to understand poker</a> ( 26 Jan 2016)
<a href="https://deevybee.blogspot.co.uk/2016/03/there-is-reproducibility-crisis-in.html">Reproducibility crisis in psychology</a> ( 5 Mar 2016)
<a href="https://deevybee.blogspot.co.uk/2016/03/better-control-of-publication-time-line.html">Further benefit of registered reports</a> ( 22 Mar 2016)
<a href="https://deevybee.blogspot.co.uk/2016/05/would-paying-by-results-improve.html">Would paying by results improve reproducibility?</a> ( 7 May 2016)
<a href="https://deevybee.blogspot.co.uk/2016/05/ten-serendipitous-findings-in-psychology.html">Serendipitous findings in psychology</a> ( 29 May 2016)
<a href="http://deevybee.blogspot.co.uk/2016/09/some-thoughts-on-statcheck-project.html">Thoughts on the Statcheck project</a> ( 3 Sep 2016)
<a href="http://deevybee.blogspot.co.uk/2016/12/when-is-replication-not-replication.html">When is a replication not a replication?</a> (16 Dec 2016)
<a href="https://deevybee.blogspot.co.uk/2017/05/reproducible-practices-are-future-for.html">Reproducible practices are the future for early career researchers</a> (1 May 2017)
<a href="https://deevybee.blogspot.co.uk/2017/05/which-neuroimaging-measures-are-useful.html">Which neuroimaging measures are useful for individual differences research?</a> (28 May 2017)
<a href="https://deevybee.blogspot.co.uk/2017/06/prospecting-for-kryptonite-value-of.html">Prospecting for kryptonite: the value of null results</a> (17 Jun 2017)
<a href="http://deevybee.blogspot.com/2017/10/why-we-need-to-stop-publishing.html">Pre-registration or replication: the need for new standards in neurogenetic studies</a> (1 Oct 2017)
<a href="http://deevybee.blogspot.com/2017/10/citing-research-literature-distorting.html">Citing the research literature: the distorting lens of memory</a> (17 Oct 2017)
<a href="http://deevybee.blogspot.com/2017/11/reproducibility-and-phonics-necessary.html">Reproducibility and phonics: necessary but not sufficient</a> (27 Nov 2017)
<a href="http://deevybee.blogspot.com/2018/02/improving-reproducibility-future-is.html">Improving reproducibility: the future is with the young</a> (9 Feb 2018)
<a href="http://deevybee.blogspot.com/2018/05/sowing-seeds-of-doubt-how-gilbert-et.html">Sowing seeds of doubt: how Gilbert et al's critique of the reproducibility project has played out</a> (27 May 2018)
<a href="http://deevybee.blogspot.com/2018/06/preprint-publication-as-karaoke.html">Preprint publication as karaoke</a> ( 26 Jun 2018)
<a href="http://deevybee.blogspot.com/2018/07/standing-on-shoulders-of-giants-or.html">Standing on the shoulders of giants, or slithering around on jellyfish: Why reviews need to be systematic</a> ( 20 Jul 2018)
<a href="http://deevybee.blogspot.com/2018/08/matlab-vs-open-source-costs-and.html">Matlab vs open source: costs and benefits to scientists and society</a> ( 20 Aug 2018)
<a href="http://deevybee.blogspot.com/2019/09/responding-to-replication-crisis.html">Responding to the replication crisis: reflections on Metascience 2019</a> (15 Sep 2019)
<a href="http://deevybee.blogspot.com/2020/05/manipulated-images-hiding-in-plain-sight.html">Manipulated images: hiding in plain sight</a> (13 May 2020)
<a href="http://deevybee.blogspot.com/2020/06/frogs-or-termites-gunshot-or-cumulative.html">Frogs or termites: gunshot or cumulative science?</a> ( 6 Jun 2020)
<a href="http://deevybee.blogspot.com/2021/03/open-data-we-know-whats-needed-now-lets.html">Open data: We know what's needed - now let's make it happen</a> (27 Mar 2021) <a href="http://deevybee.blogspot.com/2022/06/a-proposal-for-data-sharing-that.html">A proposal for data-sharing the discourages p-hacking</a> (29 Jun 2022) <a href="http://deevybee.blogspot.com/2022/08/can-systematic-reviews-help-clean-up.html">Can systematic reviews help clean up science</a> (9 Aug 2022)<i></i><a href="http://deevybee.blogspot.com/2023/09/polyunsaturated-fatty-acids-and.html">Polyunsaturated fatty acids and children's cognition: p-hacking and the canonisation of false facts</a> (4 Sep 2023)
<b><span style="font-size: 10.5pt; line-height: 115%;"> </span></b><br />
<br />
<b><span style="font-size: 10.5pt; line-height: 115%;">Statistics </span></b></div>
<a href="http://deevybee.blogspot.co.uk/2010/06/book-review-biography-of-richard-doll.html">Book review: biography of Richard Doll</a> (5 Jun 2010)
<a href="http://deevybee.blogspot.co.uk/2010/06/book-review-invisible-gorilla.html">Book review: the Invisible Gorilla</a> (30 Jun 2010)
<a href="http://deevybee.blogspot.co.uk/2010/07/difference-between-p-05-and-screening.html">The difference between p < .05 and a screening test</a> (23 Jul 2010)
<a href="http://deevybee.blogspot.co.uk/2010/08/three-ways-to-improve-cognitive-test.html">Three ways to improve cognitive test scores without intervention</a> (14 Aug 2010)
<a href="http://deevybee.blogspot.co.uk/2011/04/short-nerdy-post-about-use-of.html">A short nerdy post about the use of percentiles</a> (13 Apr 2011)
<a href="http://deevybee.blogspot.co.uk/2011/10/joys-of-inventing-data.html">The joys of inventing data</a> (5 Oct 2011)
<a href="http://deevybee.blogspot.co.uk/2012/04/getting-genetic-effect-sizes-in.html">Getting genetic effect sizes in perspective</a> (20 Apr 2012) <a href="http://deevybee.blogspot.co.uk/2012/06/causal-models-of-developmental.html">Causal models of developmental disorders: the perils of correlational data</a> (24 Jun 2012) <a href="http://deevybee.blogspot.co.uk/2012_10_01_archive.html">Data from the phonics screen</a> (1 Oct 2012)<a href="http://deevybee.blogspot.co.uk/2012/11/moderate-drinking-in-pregnancy-toxic-or.html">Moderate drinking in pregnancy: toxic or benign?</a> (1 Nov 2012) <a href="http://deevybee.blogspot.co.uk/2012/11/flaky-chocolate-and-new-england-journal.html">Flaky chocolate and the New England Journal of Medicine</a> (13 Nov 2012) <a href="http://deevybee.blogspot.co.uk/2013/06/interpreting-unexpected-significant.html">Interpreting unexpected significant results</a> (7 June 2013) <a href="http://deevybee.blogspot.co.uk/2014/04/data-analysis-ten-tips-i-wish-id-known.html">Data analysis: Ten tips I wish I'd known earlier</a> (18 Apr 2014) <a href="http://deevybee.blogspot.co.uk/2014/05/data-sharing-exciting-but-scary.html">Data sharing: exciting but scary</a> (26 May 2014)
<a href="http://deevybee.blogspot.co.uk/2014/07/percentages-quasi-statistics-and-bad.html">Percentages, quasi-statistics and bad arguments</a> (21 July 2014)
<a href="http://deevybee.blogspot.co.uk/2016/09/in-defence-of-excel.html">Why I still use Excel</a> ( 1 Sep 2016)
<a href="https://deevybee.blogspot.co.uk/2017/04/sample-selection-in-genetic-studies.html">Sample selection in genetic studies: impact of restricted range </a> (23 Apr 2017)
<a href="https://deevybee.blogspot.co.uk/2017/06/prospecting-for-kryptonite-value-of.html">Prospecting for kryptonite: the value of null results</a> (17 Jun 2017)
<a href="http://deevybee.blogspot.com/2017/11/prisons-developmental-language-disorder.html">Prisons, developmental language disorder, and base rates</a> (3 Nov 2017)
<a href="http://deevybee.blogspot.com/2017/11/how-analysis-of-variance-works.html">How Analysis of Variance Works</a> (20 Nov 2017)
<a href="http://deevybee.blogspot.com/2017/11/anova-t-tests-and-regression-different.html">ANOVA, t-tests and regression: different ways of showing the same thing</a> (24 Nov 2017)
<a href="http://deevybee.blogspot.com/2017/12/using-simulations-to-understand.html">Using simulations to understand the importance of sample size</a> (21 Dec 2017)
<a href="http://deevybee.blogspot.com/2017/12/using-simulations-to-understand-p-values.html">Using simulations to understand p-values</a> (26 Dec 2017)
<a href="http://deevybee.blogspot.com/2018/07/one-big-study-or-two-small-studies.html">One big study or two small studies?</a> ( 12 Jul 2018)
<a href="http://deevybee.blogspot.com/2021/01/time-to-ditch-relative-risk-in-media.html">Time to ditch relative risk in media reports</a> (23 Jan 2020)
<br />
<div style="font-family: inherit;">
<b><br /></b></div>
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Journalism/science communication </span></b></div>
<a href="http://deevybee.blogspot.co.uk/2010/06/orwellian-prize-for-journalistic.html">Orwellian prize for scientific misrepresentation</a> (1 Jun 2010)
<a href="http://deevybee.blogspot.co.uk/2010/06/journalists-and-scientific-breakthrough.html">Journalists and the 'scientific breakthrough'</a> (13 Jun 2010)
<a href="http://deevybee.blogspot.co.uk/2010/09/science-journal-editors-taxonomy.html">Science journal editors: a taxonomy</a> (28 Sep 2010)
<a href="http://deevybee.blogspot.co.uk/2011/01/orwellian-prize-for-journalistic.html">Orwellian prize for journalistic misrepresentation: an update</a> (29 Jan 2011)
<a href="http://deevybee.blogspot.co.uk/2011/02/academic-publishing-why-isnt-psychology.html">Academic publishing: why isn't psychology like physics?</a> (26 Feb 2011)
<a href="http://deevybee.blogspot.co.uk/2011/05/scientific-communication-comment-option.html">Scientific communication: the Comment option</a> (25 May 2011)
<a href="http://deevybee.blogspot.co.uk/2011/12/publishers-psychological-tests-and.html">Publishers, psychological tests and greed</a> (30 Dec 2011)
<a href="http://deevybee.blogspot.co.uk/2012/01/time-for-academics-to-withdraw-free.html">Time for academics to withdraw free labour</a> (7 Jan 2012) <a href="http://deevybee.blogspot.co.uk/2012/01/2011-orwellian-prize-for-journalistic.html">2011 Orwellian Prize for Journalistic Misrepresentation</a> (29 Jan 2012)
<a href="http://deevybee.blogspot.co.uk/2012/03/time-for-neuroimaging-to-clean-up-its.html">Time for neuroimaging (and PNAS) to clean up its act</a> (5 Mar 2012)
<a href="http://deevybee.blogspot.co.uk/2012/07/communicating-science-in-age-of.html">Communicating science in the age of the internet</a> (13 Jul 2012) <a href="http://deevybee.blogspot.co.uk/2012_08_01_archive.html">How to bury your academic writing</a> (26 Aug 2012)
<a href="http://deevybee.blogspot.co.uk/2013/03/high-impact-journals-where.html">High-impact journals: where newsworthiness trumps methodology</a> (10 Mar 2013) <a href="http://deevybee.blogspot.co.uk/2013/04/a-short-rant-about-numbered-journal.html">A short rant about numbered journal references</a> (5 Apr 2013)<a href="http://deevybee.blogspot.co.uk/2013/05/schizophrenia-and-child-abuse-in-media.html"> Schizophrenia and child abuse in the media</a> (26 May 2013) <a href="http://deevybee.blogspot.co.uk/2013/07/why-we-need-pre-registration.html">Why we need pre-registration</a> (6 Jul 2013)
<a href="http://deevybee.blogspot.co.uk/2013/10/on-need-for-responsible-reporting-by.html">On the need for responsible reporting of research</a> (10 Oct 2013)
<a href="http://deevybee.blogspot.co.uk/2014/01/a-new-years-letter-to-academic.html">A New Year's letter to academic publishers</a> (4 Jan 2014)
<a href="http://deevybee.blogspot.co.uk/2015/02/journals-without-editors-what-is-going.html">Journals without editors: What is going on?</a> (1 Feb 2015)
<a href="http://deevybee.blogspot.co.uk/2015/02/editors-behaving-badly.html">Editors behaving badly?</a> (24 Feb 2015) <a href="http://deevybee.blogspot.co.uk/2015/03/will-elsevier-say-sorry.html">Will Elsevier say sorry?</a> (21 Mar 2015)
<a href="http://deevybee.blogspot.co.uk/2015/04/how-long-does-scientific-paper-need-to.html">How long does a scientific paper need to be?</a> (20 Apr 2015)
<a href="http://deevybee.blogspot.co.uk/2015/05/will-traditional-science-journals.html">Will traditional science journals disappear?</a> (17 May 2015)
<a href="http://deevybee.blogspot.co.uk/2015/06/my-collapse-of-confidence-in-frontiers.html">My collapse of confidence in Frontiers journals</a> (7 Jun 2015)
<a href="http://deevybee.blogspot.co.uk/2015/07/publishing-replication-failures-some.html">Publishing replication failures</a> (11 Jul 2015)
<a href="http://www.theguardian.com/science/head-quarters/2015/aug/28/reproducibility-project-psychology-research-hopeless-case-or-pioneering-field">Psychology research: hopeless case or pioneering field?</a> (28 Aug 2015) <a href="https://deevybee.blogspot.co.uk/2016/02/desperate-marketing-from-journal-of.html">Desperate marketing from J. Neuroscience</a> ( 18 Feb 2016)
<a href="https://deevybee.blogspot.co.uk/2016/06/editorial-integrity-publishers-on-front.html">Editorial integrity: publishers on the front line</a> ( 11 Jun 2016)
<a href="http://deevybee.blogspot.co.uk/2016/12/when-scientific-communication-is-one.html">When scientific communication is a one-way street</a> (13 Dec 2016)
<a href="https://deevybee.blogspot.co.uk/2017/07/breaking-ice-with-buxom-grapefruits.html">Breaking the ice with buxom grapefruits: Pratiques de publication and predatory publishing</a> (25 Jul 2017)
<a href="http://deevybee.blogspot.com/2018/08/should-editors-edit-reviewers.html">Should editors edit reviewers?</a> ( 26 Aug 2018)
<a href="http://deevybee.blogspot.com/2019/08/corrigendum-word-you-may-hope-never-to.html">Corrigendum: a word you may hope never to encounter</a> (3 Aug 2019)
<a href="http://deevybee.blogspot.com/2020/07/percent-by-most-prolific-author-score.html">Percent by most prolific author score and editorial bias</a> (12 Jul 2020)
<a href="http://deevybee.blogspot.com/2020/08/pepiops-prolific-editors-who-publish-in.html">PEPIOPs – prolific editors who publish in their own publications</a> (16 Aug 2020)
<a href="http://deevybee.blogspot.com/2020/12/faux-peer-reviewed-journals-threat-to.html">Faux peer-reviewed journals: a threat to research integrity</a> (6 Dec 2020)
<a href="http://deevybee.blogspot.com/2021/01/time-to-ditch-relative-risk-in-media.html">Time to ditch relative risk in media reports</a> (23 Jan 2020)
<a href="http://deevybee.blogspot.com/2021/03/time-for-publishers-to-consider-rights.html">Time for publishers to consider the rights of readers as well as authors</a> (13 Mar 2021)
<a href="http://deevybee.blogspot.com/2021/11/universities-vs-elsevier-who-has-upper.html">Universities vs Elsevier: who has the upper hand?</a> (14 Nov 2021)
<a href="http://deevybee.blogspot.com/2022/04/book-review-fiona-fox-beyond-hype.html">Book Review. Fiona Fox: Beyond the Hype</a> (12 Apr 2022) <a href="http://deevybee.blogspot.com/2022/09/we-need-to-talk-about-editors.html">We need to talk about editors</a> (6 Sep 2022)
<a href="http://deevybee.blogspot.com/2022/09/so-do-we-need-editors.html">So do we need editors?</a> (11 Sep 2022)
<a href="http://deevybee.blogspot.com/2022/09/reviewer-finding-algorithms-dangers-for.html">Reviewer-finding algorithms: the dangers for peer review</a> (30 Sep 2022)
<a href="http://deevybee.blogspot.com/2022/10/a-desire-for-clickbait-can-hinder.html">A desire for clickbait can hinder an academic journal's reputation</a> (4 Oct 2022)
<a href="http://deevybee.blogspot.com/2022/10/what-is-going-on-in-hindawi-special.html">What is going on in Hindawi special issues?</a> (12 Oct 2022) <a href="http://deevybee.blogspot.com/2022/12/new-years-eve-quiz-2022.html">New Year's Eve Quiz: Dodgy journals special</a> (31 Dec 2022) <a href="http://deevybee.blogspot.com/2023/03/a-suggestion-for-elife.html">A suggestion for e-Life</a> (20 Mar 2023) <a href="http://deevybee.blogspot.com/2023/04/papers-affected-by-misconduct-erratum.html">Papers affected by misconduct: Erratum, correction or retraction?</a> (11 Apr 2023) <a href="http://deevybee.blogspot.com/2023/07/is-hindawi-well-positioned-for.html">Is Hindawi “well-positioned for revitalization?”</a> (23 Jul 2023)
<a href="http://deevybee.blogspot.com/2023/08/the-discussion-section-kill-it-or.html">The discussion section: Kill it or reform it?</a> (14 Aug 2023)
<a href="http://deevybee.blogspot.com/2023/10/spitting-out-ai-gobbledegook-sandwich.html">Spitting out the AI Gobbledegook sandwich: a suggestion for publishers</a> (2 Oct 2023)
<br />
<br />
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Social Media </span></b></div>
<a href="http://deevybee.blogspot.co.uk/2011/06/gentle-introduction-to-twitter-for.html">A gentle introduction to Twitter for the apprehensive academic</a> (14 Jun 2011)
<a href="http://deevybee.blogspot.co.uk/2011/11/importance-of-not-being-earnest-on.html">Your Twitter Profile: The Importance of Not Being Earnest</a> (19 Nov 2011)
<a href="http://deevybee.blogspot.co.uk/2012/01/will-i-still-be-tweeting-in-2013.html">Will I still be tweeting in 2013?</a> (2 Jan 2012)
<a href="http://deevybee.blogspot.co.uk/2012/03/blogging-in-service-of-science.html">Blogging in the service of science</a> (10 Mar 2012) <a href="http://deevybee.blogspot.co.uk/2013/03/blogging-as-post-publication-peer.html">Blogging as post-publication peer review</a> (21 Mar 2013)
<a href="http://deevybee.blogspot.co.uk/2013/12/the-impact-of-blogging-on-reputation.html">The impact of blogging on reputation</a> ( 27 Dec 2013) <a href="http://deevybee.blogspot.co.uk/2014/04/wespeechies-meeting-point-on-twitter.html">WeSpeechies: A meeting point on Twitter</a> (12 Apr 2014)
<a href="https://deevybee.blogspot.co.uk/2016/04/to-world-from-deevybee-re-we-have.html">Email overload</a> ( 12 Apr 2016)
<a href="http://deevybee.blogspot.com/2018/05/how-to-survive-on-twitter-simple-rule.html">How to survive on Twitter - a simple rule to reduce stress</a> (13 May 2018)
<br />
<br />
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Academic life </span></b></div>
<a href="http://deevybee.blogspot.co.uk/2010/06/exciting-day-in-life-of-scientist.html">An exciting day in the life of a scientist</a> (24 Jun 2010)
<a href="http://deevybee.blogspot.co.uk/2010/08/how-our-current-reward-structures-have.html">How our current reward structures have distorted and damaged science</a> (6 Aug 2010)
<a href="http://deevybee.blogspot.co.uk/2010/10/challenge-for-science-speech-by-colin.html">The challenge for science: speech by Colin Blakemore</a> (14 Oct 2010)
<a href="http://deevybee.blogspot.co.uk/2010/12/when-ethics-regulations-have-unethical.html">When ethics regulations have unethical consequences</a> (14 Dec 2010)
<a href="http://deevybee.blogspot.co.uk/2010/12/day-working-from-home.html">A day working from home</a> (23 Dec 2010)
<a href="http://deevybee.blogspot.co.uk/2011/01/should-we-ration-research-grant.html">Should we ration research grant applications?</a> (8 Jan 2011)
<a href="http://deevybee.blogspot.co.uk/2011/03/one-hour-lecture-how-to-captivate-your.html">The one hour lecture</a> (11 Mar 2011)
<a href="http://deevybee.blogspot.co.uk/2011/03/expansion-of-research-regulators.html">The expansion of research regulators</a> (20 Mar 2011)
<a href="http://deevybee.blogspot.co.uk/2011/06/should-we-ever-fight-lies-with-lies.html">Should we ever fight lies with lies?</a> (19 Jun 2011)
<a href="http://deevybee.blogspot.co.uk/2011/07/how-to-survive-in-psychological.html">How to survive in psychological research</a> (13 Jul 2011)
<a href="http://deevybee.blogspot.co.uk/2011/08/so-you-want-to-be-research-assistant.html">So you want to be a research assistant?</a> (25 Aug 2011)
<a href="http://deevybee.blogspot.co.uk/2011/12/nhs-research-ethics-procedures-modern.html">NHS research ethics procedures: a modern-day Circumlocution Office</a> (18 Dec 2011)
<a href="http://deevybee.blogspot.co.uk/2012/03/ref-monster-that-sucks-time-and-money.html">The REF: a monster that sucks time and money from academic institutions</a> (20 Mar 2012)
<a href="http://deevybee.blogspot.co.uk/2012/04/ultimate-email-auto-response.html">The ultimate email auto-response</a> (12 Apr 2012)
<a href="http://deevybee.blogspot.co.uk/2012/05/well-this-should-be-easy.html">Well, this should be easy….</a> (21 May 2012)<a href="http://deevybee.blogspot.co.uk/2013/01/journal-impact-factors-and-ref-2014.html"> Journal impact factors and REF2014</a> (19 Jan 2013) <a href="http://deevybee.blogspot.co.uk/2013/01/an-alternative-to-ref2014.html">An alternative to REF2014</a> (26 Jan 2013) <a href="http://deevybee.blogspot.co.uk/2013/02/postgraduate-education-time-for-rethink.html">Postgraduate education: time for a rethink</a> (9 Feb 2013)
<a href="http://deevybee.blogspot.co.uk/2013/03/ten-things-than-can-sink-grant-proposal.html">Ten things that can sink a grant proposal</a> (19 Mar 2013)<a href="http://deevybee.blogspot.co.uk/2013/03/blogging-as-post-publication-peer.html">Blogging as post-publication peer review</a> (21 Mar 2013) <a href="http://deevybee.blogspot.co.uk/2013/05/the-academic-backlog.html">The academic backlog</a> (9 May 2013) <a href="http://deevybee.blogspot.co.uk/2013/06/discussion-meeting-vs-conference-in.html">Discussion meeting vs conference: in praise of slower science</a> (21 Jun 2013) <a href="http://deevybee.blogspot.co.uk/2013/07/why-we-need-pre-registration.html">Why we need pre-registration</a> (6 Jul 2013)
<a href="http://deevybee.blogspot.co.uk/2013/09/evaluate-evaluate-evaluate.html">Evaluate, evaluate, evaluate</a> (12 Sep 2013)
<a href="http://deevybee.blogspot.co.uk/2013/10/high-time-to-revise-phd-thesis-format.html">High time to revise the PhD thesis format</a> (9 Oct 2013)
<a href="http://deevybee.blogspot.co.uk/2013/10/the-matthew-effect-and-ref2014.html">The Matthew effect and REF2014</a> (15 Oct 2013) <a href="http://deevybee.blogspot.co.uk/2014/06/the-university-as-big-business.html">The University as big business: the case of King's College London</a> (18 June 2014)
<a href="http://deevybee.blogspot.co.uk/2014/07/should-vice-chancellors-earn-more-than.html">Should vice-chancellors earn more than the prime minister?</a> (12 July 2014)
<a href="http://deevybee.blogspot.co.uk/2014/10/some-thoughts-on-use-of-metrics-in.html">Some thoughts on use of metrics in university research assessment</a> (12 Oct 2014)
<a href="http://cdbu.org.uk/tuition-fees-must-be-high-on-the-agenda-before-the-election/">Tuition fees must be high on the agenda before the next election</a> (22 Oct 2014)
<a href="http://deevybee.blogspot.co.uk/2014/10/blaming-universities-for-our-nations.html">Blaming universities for our nation's woes</a> (24 Oct 2014)
<a href="http://cdbu.org.uk/staff-satisfaction-is-as-important/">Staff satisfaction is as important as student satisfaction</a> (13 Nov 2014)
<a href="http://deevybee.blogspot.co.uk/2014/11/metricophobia-among-academics.html">Metricophobia among academics</a> (28 Nov 2014)
<a href="http://deevybee.blogspot.co.uk/2014/12/why-evaluating-scientists-by-grant.html">Why evaluating scientists by grant income is stupid</a> (8 Dec 2014)
<a href="http://deevybee.blogspot.co.uk/2014/12/dividing-up-pie-in-psychology-in.html">Dividing up the pie in relation to REF2014</a> (18 Dec 2014)
<a href="http://cdbu.org.uk/shaky-foundations-of-the-tef/">Shaky foundations of the TEF</a> (7 Dec 2015)
<a href="http://deevybee.blogspot.co.uk/2015/12/a-lamentable-performance-by-jo-johnson.html">A lamentable performance by Jo Johnson</a> (12 Dec 2015)
<a href="http://cdbu.org.uk/more-misrepresentation-in-the-green-paper/">More misrepresentation in the Green Paper</a> (17 Dec 2015)
<a href="http://deevybee.blogspot.co.uk/2015/12/the-green-papers-level-playing-field.html">The Green Paper’s level playing field risks becoming a morass</a> (24 Dec 2015)
<a href="https://www.timeshighereducation.com/blog/nss-and-teaching-excellence-wrong-measure-wrongly-analysed">NSS and teaching excellence: wrong measure, wrongly analysed</a> (4 Jan 2016)
<a href="https://deevybee.blogspot.co.uk/2016/03/on-need-for-clarity-of-purpose-in-ref.html">Lack of clarity of purpose in REF and TEF</a> ( 2 Mar 2016)
<a href="https://deevybee.blogspot.co.uk/2016/05/who-wants-tef.html">Who wants the TEF?</a> ( 24 May 2016)
<a href="https://deevybee.blogspot.co.uk/2016/07/cost-benefit-analysis-of-teaching.html">Cost benefit analysis of the TEF</a> ( 17 Jul 2016)
<a href="http://deevybee.blogspot.co.uk/2016/08/alternative-providers-and-alternative.html">Alternative providers and alternative medicine</a> ( 6 Aug 2016)
<a href="http://deevybee.blogspot.co.uk/2017/02/we-know-whats-best-for-you-politicians.html">We know what's best for you: politicians vs. experts</a> (17 Feb 2017)
<a href="https://deevybee.blogspot.co.uk/2017/03/advice-for-early-career-researchers-re.html">Advice for early career researchers re job applications: Work 'in preparation'</a> (5 Mar 2017)
<a href="http://deevybee.blogspot.com/2018/04/should-research-funding-be-allocated-at.html">Should research funding be allocated at random?</a> (7 Apr 2018)
<a href="http://deevybee.blogspot.com/2018/05/power-responsibility-and-role-models-in.html">Power, responsibility and role models in academia</a> (3 May 2018)
<a href="http://deevybee.blogspot.com/2018/05/my-response-to-epas-strengthening.html">My response to the EPA's 'Strengthening Transparency in Regulatory Science'</a> (9 May 2018)
<a href="http://deevybee.blogspot.com/2018/08/more-haste-less-speed-in-calls-for.html">More haste less speed in calls for grant proposals</a> ( 11 Aug 2018)
<a href="http://deevybee.blogspot.com/2018/10/has-society-for-neuroscience-lost-its.html">Has the Society for Neuroscience lost its way?</a> ( 24 Oct 2018)
<a href="http://deevybee.blogspot.com/2019/02/the-paper-in-day-approach.html">The Paper-in-a-Day Approach</a> ( 9 Feb 2019)
<a href="http://deevybee.blogspot.com/2019/03/benchmarking-in-tef-something-doesnt.html">Benchmarking in the TEF: Something doesn't add up</a> ( 3 Mar 2019)
<a href="http://deevybee.blogspot.com/2019/05/the-do-it-yourself-diy-conference.html">The Do It Yourself conference</a> ( 26 May 2019)<a href="http://deevybee.blogspot.com/2019/07/a-call-for-funders-to-ban-institutions.html"> A call for funders to ban institutions that use grant capture targets</a> (20 Jul 2019)
<a href="http://deevybee.blogspot.com/2020/01/research-funders-need-to-embrace-slow.html">Research funders need to embrace slow science</a> (1 Jan 2020)
<a href="http://deevybee.blogspot.com/2020/01/should-i-stay-or-should-i-go-when.html">Should I stay or should I go: When debate with opponents should be avoided</a> (12 Jan 2020)
<a href="http://deevybee.blogspot.com/2020/02/stemming-flood-of-illegal-external.html">Stemming the flood of illegal external examiners</a> (9 Feb 2020)
<a href="http://deevybee.blogspot.com/2020/03/what-can-scientists-do-in-emergency.html">What can scientists do in an emergency shutdown?</a> (11 Mar 2020)
<a href="http://deevybee.blogspot.com/2020/05/stepping-back-level-stress-management.html">Stepping back a level: Stress management for academics in the pandemic</a> (2 May 2020)<br />
<a href="http://deevybee.blogspot.com/2020/07/tef-in-time-of-pandemic.html">TEF in the time of pandemic</a> (27 Jul 2020)
<a href="http://deevybee.blogspot.com/2021/03/university-staff-cuts-under-cover-of.html">University staff cuts under the cover of a pandemic: the cases of Liverpool and Leicester</a> (3 Mar 2021)
<a href="http://deevybee.blogspot.com/2022/03/some-quick-thoughts-on-academic.html">Some quick thoughts on academic boycotts of Russia</a> (6 Mar 2022)
<a href="http://deevybee.blogspot.com/2022/12/when-there-are-no-consequences-for.html">When there are no consequences for misconduct</a> (16 Dec 2022) <a href="http://deevybee.blogspot.com/2023/02/open-letter-to-cnrs.html">Open letter to CNRS</a> (30 Mar 2023)
<br />
<br />
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Celebrity scientists/quackery </span></b></div>
<a href="http://deevybee.blogspot.co.uk/2010/08/three-ways-to-improve-cognitive-test.html">Three ways to improve cognitive test scores without intervention</a> (14 Aug 2010) <a href="http://deevybee.blogspot.co.uk/2011/07/what-does-it-take-to-become-fellow-of.html">What does it take to become a Fellow of the RSM?</a> (24 Jul 2011)
<a href="http://deevybee.blogspot.co.uk/2011/08/open-letter-to-baroness-susan.html">An open letter to Baroness Susan Greenfield</a> (4 Aug 2011)
<a href="http://deevybee.blogspot.co.uk/2011/08/defence-of-susan-greenfield.html">Susan Greenfield and autistic spectrum disorder: was she misrepresented?</a> (12 Aug 2011)
<a href="http://deevybee.blogspot.co.uk/2011/09/how-to-become-celebrity-scientific.html">How to become a celebrity scientific expert</a> (12 Sep 2011)<a href="http://www.guardian.co.uk/commentisfree/2011/sep/14/daycare-cortisol-levels-children"> The kids are all right in daycare</a> (14 Sep 2011)
<a href="http://deevybee.blogspot.co.uk/2011/11/weird-world-of-us-ethics-regulation.html">The weird world of US ethics regulation</a> (25 Nov 2011)
<a href="http://deevybee.blogspot.co.uk/2011/12/pioneering-treatment-or-quackery-how-to.html">Pioneering treatment or quackery? How to decide</a> (4 Dec 2011) <a href="http://deevybee.blogspot.co.uk/2012/01/psychoanalytic-treatment-for-autism.html">Psychoanalytic treatment for autism: Interviews with French analysts</a> (23 Jan 2012) <a href="http://deevybee.blogspot.co.uk/2012/02/neuroscientific-interventions-for.html">Neuroscientific interventions for dyslexia: red flags</a> (24 Feb 2012)
<a href="http://deevybee.blogspot.co.uk/2014/09/why-most-scientists-dont-take-susan.html">Why most scientists don't take Susan Greenfield seriously</a> (26 Sept 2014)
<a href="http://deevybee.blogspot.com/2019/01/neuropointdxs-blood-test-for-autism.html">NeuroPointDX's blood test for Autism Spectrum Disorder</a> ( 12 Jan 2019)
<br />
<br />
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Women </span></b></div>
<a href="http://deevybee.blogspot.co.uk/2010/05/academic-mobbing-in-cyberspace.html">Academic mobbing in cyberspace</a> (30 May 2010)
<a href="http://deevybee.blogspot.co.uk/2011/01/what-works-for-women-some-useful-links.html">What works for women: some useful links</a> (12 Jan 2011)
<a href="http://deevybee.blogspot.co.uk/2011/04/burqa-ban-whats-liberal-response.html">The burqua ban: what's a liberal response</a> (21 Apr 2011) <a href="http://deevybee.blogspot.co.uk/2012/03/cmon-sisters-speak-out.html">C'mon sisters! Speak out!</a> (28 Mar 2012)
<a href="http://deevybee.blogspot.co.uk/2012/12/psychology-where-are-all-men.html">Psychology: where are all the men?</a> (5 Nov 2012)
<a href="http://deevybee.blogspot.co.uk/2014/06/should-rennard-be-reinstated.html">Should Rennard be reinstated?</a> (1 June 2014)
<a href="http://deevybee.blogspot.co.uk/2015/06/how-media-spun-tim-hunt-story.html">How the media spun the Tim Hunt story</a> (24 Jun 2015)
<br />
<br />
<div style="font-family: inherit;">
<b><span style="font-size: 10.5pt; line-height: 115%;">Politics and Religion </span></b></div>
<a href="http://deevybee.blogspot.co.uk/2011/10/lies-damned-lies-and-spin.html">Lies, damned lies and spin</a> (15 Oct 2011) <a href="http://deevybee.blogspot.co.uk/2012/03/letter-to-nick-clegg-from-ex-liberal.html">A letter to Nick Clegg from an ex liberal democrat</a> (11 Mar 2012)
<a href="http://deevybee.blogspot.co.uk/2012/04/bbcs-extensive-coverage-of-nhs-bill.html">BBC's 'extensive coverage' of the NHS bill</a> (9 Apr 2012)
<a href="http://deevybee.blogspot.co.uk/2012/06/schoolgirls-who-have-sex-deserve-to-die.html">Schoolgirls' health put at risk by Catholic view on vaccination</a> (30 Jun 2012)
<a href="http://deevybee.blogspot.co.uk/2013/11/a-letter-to-boris-johnson.html"> A letter to Boris Johnson </a>(30 Nov 2013)
<a href="http://deevybee.blogspot.co.uk/2014/01/how-government-spins-crisis-blame-game.html">How the government spins a crisis (floods)</a> (1 Jan 2014)
<a href="http://deevybee.blogspot.co.uk/2017/02/the-alt-right-guide-to-fielding.html">The alt-right guide to fielding conference questions</a> (18 Feb 2017)
<a href="http://deevybee.blogspot.co.uk/2017/02/we-know-whats-best-for-you-politicians.html">We know what's best for you: politicians vs. experts</a> (17 Feb 2017)
<a href="https://deevybee.blogspot.co.uk/2017/02/barely-good-word-for-donald-trump-in.html">Barely a good word for Donald Trump in Houses of Parliament</a> (23 Feb 2017)
<a href="http://deevybee.blogspot.com/2018/01/do-you-really-want-another-referendum.html">Do you really want another referendum? Be careful what you wish for</a> (12 Jan 2018)
<a href="http://deevybee.blogspot.com/2018/05/my-response-to-epas-strengthening.html">My response to the EPA's 'Strengthening Transparency in Regulatory Science'</a> (9 May 2018)
<a href="http://deevybee.blogspot.com/2019/03/what-is-driving-theresa-may.html">What is driving Theresa May?</a> ( 27 Mar 2019)
<a href="http://deevybee.blogspot.com/2019/08/a-day-out-at-10-downing-street.html">A day out at 10 Downing St</a> (10 Aug 2019)
<a href="http://deevybee.blogspot.com/2019/09/voting-in-eu-referendum-ignorance.html">Voting in the EU referendum: Ignorance, deceit and folly</a> ( 8 Sep 2019)
<a href="http://deevybee.blogspot.com/2019/10/harry-potter-and-beast-of-brexit.html">Harry Potter and the Beast of Brexit</a> (20 Oct 2019)
<a href="http://deevybee.blogspot.com/2019/10/attempting-to-communicate-with-bbc-its.html">Attempting to communicate with the BBC</a> (8 May 2020)
<a href="http://deevybee.blogspot.com/2020/05/boris-bingo-strategies-for-not.html">Boris bingo: strategies for (not) answering questions</a> (29 May 2020)
<a href="http://deevybee.blogspot.com/2021/11/linking-responsibility-for-climate.html">Linking responsibility for climate refugees to emissions</a> (23 Nov 2021)
<a href="http://deevybee.blogspot.com/2022/01/response-to-philip-balls-critique-of.html">Response to Philip Ball's critique of scientific advisors</a> (16 Jan 2022)
<a href="http://deevybee.blogspot.com/2022/01/boris-johnson-leads-world-in-number-of.html">Boris Johnson leads the world ....in the number of false facts he can squeeze into a session of PMQs</a> (20 Jan 2022)
<a href="http://deevybee.blogspot.com/2022/03/some-quick-thoughts-on-academic.html">Some quick thoughts on academic boycotts of Russia</a> (6 Mar 2022) <a href="http://deevybee.blogspot.com/2022/08/contagion-of-political-system.html">Contagion of the political system</a> (3 Apr 2022)<a href="http://deevybee.blogspot.com/2022/12/when-there-are-no-consequences-for.html">When there are no consequences for misconduct</a> (16 Dec 2022)
<br />
<br />
<b><span style="font-size: 10.5pt; line-height: 115%;">Humour and miscellaneous </span></b>
<a a="" href="http://deevybee.blogspot.co.uk/2013/11/a-letter-to-boris-johnson.html">
</a><a href="http://deevybee.blogspot.co.uk/2010/06/orwellian-prize-for-journalistic.html">Orwellian prize for scientific misrepresentation</a> (1 Jun 2010)
<a href="http://deevybee.blogspot.co.uk/2010/06/exciting-day-in-life-of-scientist.html">An exciting day in the life of a scientist</a> (24 Jun 2010)
<a href="http://deevybee.blogspot.co.uk/2010/09/science-journal-editors-taxonomy.html">Science journal editors: a taxonomy</a> (28 Sep 2010)
<a href="http://deevybee.blogspot.co.uk/2010/11/parasites-pangolins-peer-review.html">Parasites, pangolins and peer review</a> (26 Nov 2010)
<a href="http://deevybee.blogspot.co.uk/2010/12/day-working-from-home.html">A day working from home</a> (23 Dec 2010)
<a href="http://deevybee.blogspot.co.uk/2011/03/one-hour-lecture-how-to-captivate-your.html">The one hour lecture</a> (11 Mar 2011)
<a href="http://deevybee.blogspot.co.uk/2011/03/expansion-of-research-regulators.html">The expansion of research regulators</a> (20 Mar 2011)
<a href="http://deevybee.blogspot.co.uk/2011/05/scientific-communication-comment-option.html">Scientific communication: the Comment option</a> (25 May 2011)
<a href="http://deevybee.blogspot.co.uk/2011/07/how-to-survive-in-psychological.html">How to survive in psychological research</a> (13 Jul 2011)
<a href="http://deevybee.blogspot.co.uk/2011/11/importance-of-not-being-earnest-on.html">Your Twitter Profile: The Importance of Not Being Earnest</a> (19 Nov 2011)
<a href="http://deevybee.blogspot.co.uk/2012/01/2011-orwellian-prize-for-journalistic.html">2011 Orwellian Prize for Journalistic Misrepresentation</a> (29 Jan 2012)
<a href="http://deevybee.blogspot.co.uk/2012/04/ultimate-email-auto-response.html">The ultimate email auto-response</a> (12 Apr 2012)
<a href="http://deevybee.blogspot.co.uk/2012/05/well-this-should-be-easy.html">Well, this should be easy….</a> (21 May 2012)
<a href="http://deevybee.blogspot.co.uk/2012/07/the-bewildering-bathroom-challenge.html">The bewildering bathroom challenge</a> (19 Jul 2012) <a href="http://deevybee.blogspot.co.uk/2012/11/are-starbucks-hiding-their-profits-on.html">Are Starbucks hiding their profits on the planet Vulcan?</a> (15 Nov 2012) <a href="http://deevybee.blogspot.co.uk/2013/04/forget-tower-of-hanoi-new-ecologically.html">Forget the Tower of Hanoi</a> (11 Apr 2013) <a href="http://deevybee.blogspot.co.uk/2014/03/how-do-you-communicate-with.html">How do you communicate with a communications company?</a> ( 30 Mar 2014)
<a href="http://deevybee.blogspot.co.uk/2014/07/a-film-review-from-32000-feet-noah.html">Noah: A film review from 32,000 ft</a> (28 July 2014)
<a href="http://deevybee.blogspot.co.uk/2015/09/the-rationalist-spa-unmet-need.html">The rationalist spa</a> (11 Sep 2015)
<a href="https://deevybee.blogspot.co.uk/2016/04/talking-about-tax-avoidance-weasel-words.html">Talking about tax: weasel words</a> ( 19 Apr 2016)
<a href="http://deevybee.blogspot.co.uk/2016/12/controversial-statues-remove-or-revise.html">Controversial statues: remove or revise?</a> (22 Dec 2016)
<a href="http://deevybee.blogspot.co.uk/2017/02/the-alt-right-guide-to-fielding.html">The alt-right guide to fielding conference questions</a> (18 Feb 2017)
<a href="http://deevybee.blogspot.co.uk/2017/01/my-most-popular-posts-of-2016.html">My most popular posts of 2016</a> (2 Jan 2017)
<a href="http://deevybee.blogspot.com/2018/09/an-index-of-neighbourhood-advantage.html">An index of neighbourhood advantage from English postcode data</a> ( 15 Sep 2018)
<a href="http://deevybee.blogspot.com/2018/10/working-memories-brief-review-of-alan.html">Working memories: A brief review of Alan Baddeley's memoir</a> ( 13 Oct 2018) <a href="http://deevybee.blogspot.com/2022/12/new-years-eve-quiz-2022.html">New Year's Eve Quiz: Dodgy journals special</a> (31 Dec 2022)
deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-85086241382410287512023-10-02T16:45:00.000+01:002023-10-02T16:45:03.233+01:00Spitting out the AI Gobbledegook sandwich: a suggestion for publishers<p>
</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9Etaxc0GTa2GfhWaV9kghPTYIYAGgmsrwBkfREDHWZTMFskwCVJrivGc32oEdJwtt2Dpo2fORpJJFdadypWlhMksJhOHeUxmfO98bUTAzlKrTYf9aArlz6vPTujWE7qGIxPZdvOwup0Z9L5kZylh1SlfkFa7zmtTDDtCPgv2Y8sCKXeTmCM8ctd1lcUMg/s3064/CartoonStock_587103_CS470512.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1945" data-original-width="3064" height="203" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9Etaxc0GTa2GfhWaV9kghPTYIYAGgmsrwBkfREDHWZTMFskwCVJrivGc32oEdJwtt2Dpo2fORpJJFdadypWlhMksJhOHeUxmfO98bUTAzlKrTYf9aArlz6vPTujWE7qGIxPZdvOwup0Z9L5kZylh1SlfkFa7zmtTDDtCPgv2Y8sCKXeTmCM8ctd1lcUMg/s320/CartoonStock_587103_CS470512.jpg" width="320" /></a></div><br /><p></p><p class="MsoNormal">The past couple of years have been momentous for some academic
publishers. As documented in <a href="https://arxiv.org/abs/2309.15884" target="_blank">a preprint this week</a>, after rapid growth, largely via "special issues" of journals,
they have dramatically increased the number of published articles, and at the
same time made enormous profits. A recent guest post by <a href="http://deevybee.blogspot.com/2023/07/is-hindawi-well-positioned-for.html" target="_blank">Huanzi Zhang</a>,
however, showed this has not been without problems. Unscrupulous operators of so-called
"papermills" saw an opportunity to boost their own profits by selling
authorship slots and then placing fraudulent articles in special issues that
were controlled by complicit editors. Gradually, publishers realised they had a
problem and started to retract fraudulent articles. To date, Hindawi has
retracted over 5000 articles since 2021*.<span style="mso-spacerun: yes;">
</span>As described in Huanzi's blogpost, this has made shareholders nervous
and dented the profits of parent company Wiley. <br /></p><p class="MsoNormal"> </p><p class="MsoNormal">
</p><p class="MsoNormal">There are numerous papermills, and we only know about the
less competent ones whose dodgy articles are relatively easy to detect. For a
deep dive into papermills in Hindawi journals see t<a href="https://forbetterscience.com/2023/01/03/hindawi-garbage-sorting-system-based-on-citations/" target="_blank">his blogpost</a> by the
anonymous sleuth Parashorea tomentella.
At least one papermill is the source of a series of articles that follow a
template that I have termed the "AI gobbledegook sandwich".<span style="mso-spacerun: yes;"> </span>See for instance <a href="https://pubpeer.com/publications/307EC0C6A697775858DB5DB2D8DF22" target="_blank">my comments here</a> on an
article that has yet to be retracted.
For further examples, search the website PubPeer with the search term
"gobbledegook sandwich". </p><p class="MsoNormal"><span style="mso-spacerun: yes;"> <br /></span></p><p class="MsoNormal">
</p><p class="MsoNormal">After studying a number of these articles, my impression is
that they are created as follows. You start with a genuine article. Most of
these look like student projects. The topics are various, but in general they
are weak on scientific content. They may be a review of an area, or if data is
gathered, it is likely to be some kind of simple survey.<span style="mso-spacerun: yes;"> </span>In some cases, reference is made to a public
dataset. To create a paper for submission, the following steps are taken:</p><p class="MsoNormal"> </p>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span>The title is changed to include terms that
relate to the topic of a special issue, such as "Internet of Things"
or "Big data". </p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span>Phrases are scattered in the Abstract and Introduction
mentioning these terms.</p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span>A technical section is embedded in the middle of
the original piece describing the method to be used. <span style="mso-spacerun: yes;"> </span>Typically this is full of technical equations.
I suspect these are usually correct, in that they use standard formulae from
areas such as machine learning, and in some cases can be traced to Wikipedia or
another source. <span style="mso-spacerun: yes;"> </span>It is not uncommon to
see very basic definitions, e.g. formulae for sensitivity and specificity of
prediction. </p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span>A results section is created showing figures
that purport to demonstrate how the AI method has been applied to the data. This
often reveals that the paper is problematic, as plots are at best unclear and
at worst bear no relationship to anything that has gone before. <span style="mso-spacerun: yes;"> </span>Labels for figures and axes tend to be vague. A
typical claim is that the prediction from the AI model is better than results
from other, competing models. It is usually hard to work out what is being
predicted from what. </p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span>The original essay resumes for a Conclusions
section, but with a sentence added to say how AI methods have been useful in
improving our understanding.</p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span>An optional additional step is to sprinkle
irrelevant citations in the text: we know that papermills collect further
income by selling citations, and new papers can act as vehicles for these. </p>
<p class="MsoNormal"><style>@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;
mso-font-charset:77;
mso-generic-font-family:decorative;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 -2147483647 0;}@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}ol
{margin-bottom:0cm;}ul
{margin-bottom:0cm;}</style><span style="mso-spacerun: yes;"> <br /></span></p><p class="MsoNormal"><style>@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;
mso-font-charset:77;
mso-generic-font-family:decorative;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 -2147483647 0;}@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}ol
{margin-bottom:0cm;}ul
{margin-bottom:0cm;}</style>
</p><p class="MsoNormal">Papermills have got away with this, because the content of
these articles is sufficiently technical and complex that the fraud may only be
detectable on close reading. Where I am confident there is fraud, I will use
the term "Gobbledegook sandwich" in my report on PubPeer, but there
are many, many papers where my suspicions are raised but it would take more
time than it is worth for me to comb through the article to find compelling evidence.</p><p class="MsoNormal"> </p><p class="MsoNormal">
</p><p class="MsoNormal">For a papermill, the beauty of the AI gobbledegook sandwich
is that you can apply AI methods to almost any topic, and there are so many
different algorithms that can be used that there is a potentially infinite
number of papers that can be written according to this template.<span style="mso-spacerun: yes;"> </span>The ones I have documented include topics
ranging from educational methods, hotel management, sports, art, archaeology, Chinese
medicine, music, building design, mental health and promotion of Marxist
ideology. In none of these papers did the application of AI methods make any
sense, and they would not get past a competent editor or reviewers, but once a
complicit editor is planted in a journal, they can accept numerous articles. </p><p class="MsoNormal"> </p><p class="MsoNormal">
</p><p class="MsoNormal">Recently, Hindawi has ramped up its integrity operations and
is employing many more staff to try and shut this particular stable door. <span style="mso-spacerun: yes;"> </span>But Hindawi is surely not the only publisher infected
by this kind of fraud, and we need a solution that can be used by all journals.
My simple suggestion is to focus on prevention rather than cure, by requiring
that all articles that report work using AI/ML methods adopt reporting standards
that are being developed for machine-learning based science, as described on
<a href=" https://reporting-standards.cs.princeton.edu/" target="_blank">this website</a>.<span style="mso-spacerun: yes;"> </span>This requires computational reproducibility, i.e.,
data and scripts must be provided so that all results can be reproduced.<span style="mso-spacerun: yes;"> </span>This would be a logical impossibility for AI gobbledegook
sandwiches.</p><p class="MsoNormal"> </p><p class="MsoNormal">
</p><p class="MsoNormal">Open science practices were developed with the aim of
improving reproducibility and credibility of science, but, as<a href="https://www.slideshare.net/deevybishop/open-research-practices-in-the-age-of-a-papermill-pandemic" target="_blank"> I've argued elsewhere</a>, they could be highly effective in preventing fraud. <span style="mso-spacerun: yes;"></span>Mandating reporting standards could be an
important step, which, if accompanied also by open peer review, will make life
of the papermillers much harder.</p><p class="MsoNormal"><br /></p><p class="MsoNormal"><br /></p><p class="MsoNormal">
</p><p class="MsoNormal">*<span style="font-size: x-small;">Source is <a href="https://docs.google.com/spreadsheets/d/1CWG84ncHW7-7mgzq9EZBoPC6Sw5n9mS6pInStNs78gA/edit#gid=0" target="_blank">spreadsheet</a> maintained by the anonymous sleuth
Parashorea tomentella</span></p><p class="MsoNormal"><span style="font-size: x-small;"> </span></p><p class="MsoNormal"><span style="font-size: x-small;">N.B. Comments on this blog are moderated, so there may be a delay before they appear. <br /></span></p>
<p class="MsoNormal"><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}a:link, span.MsoHyperlink
{mso-style-noshow:yes;
mso-style-priority:99;
color:blue;
text-decoration:underline;
text-underline:single;}a:visited, span.MsoHyperlinkFollowed
{mso-style-noshow:yes;
mso-style-priority:99;
color:#954F72;
mso-themecolor:followedhyperlink;
text-decoration:underline;
text-underline:single;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style> <br /></p>
<p class="MsoNormal"><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style> <br /></p>
<p class="MsoNormal"><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style> <br /></p>
<p class="MsoNormal"><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style> <br /></p>
<p class="MsoNormal"><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style><span style="mso-spacerun: yes;"> </span></p>
<p class="MsoNormal"><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style> <br /></p>
<p><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-font-kerning:1.0pt;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com3tag:blogger.com,1999:blog-5841910768079015534.post-91102324372963504522023-09-04T13:51:00.003+01:002023-09-04T13:51:27.667+01:00Polyunsaturated fatty acids and children's cognition: p-hacking and the canonisation of false facts<p>One of my favourite articles is a piece by Nissen et al (2016) called "<a href="https://elifesciences.org/articles/21451" target="_blank">Publication bias and the canonization of false facts</a>". In it, the authors model how false information can masquerade as overwhelming evidence, if, over cycles of experimentation, positive results are more likely to be published than null ones. But their article is not just about publication bias: they go on to show how p-hacking magnifies this effect, because it leads to a false positive rate that is much higher than the nominal rate (typically .05).</p><p>I was reminded of this when looking at some literature on polyunsaturated fatty acids and children's cognition. This was a topic I'd had a passing interest in years ago when fish oil was being promoted for children with dyslexia and ADHD. I reviewed the literature back in 2008 for a talk at the British Dyslexia Association (<a href=" https://www.slideshare.net/deevybishop/exercisefish-oil-intervention-for-dyslexia" target="_blank">slides here</a>). What was striking then was that, whilst there were studies claiming positive effects of dietary supplements, they all obtained different findings. It looked suspicious to me, as if authors would keep looking in their data, and divide it up every way possible, in order to find something positive to report – in other words, p-hacking seemed rife in this field. </p><p>My interest in this area was piqued more recently simply because I was looking at articles that had been flagged up because they contained "tortured phrases". These are verbal expressions that seem to have been selected to avoid plagiarism detectors: they are often unintentionally humorous, because attempts to generate synonyms misfire. For instance, in <a href="https://www.tandfonline.com/doi/full/10.1080/10942912.2022.2070642" target="_blank">this article</a> by Khalid et al, published in Taylor and Francis' International Journal of Food Properties we are told: </p><p></p><blockquote>"Parkinson’s infection is a typical neurodegenerative sickness. The mix of hereditary and natural variables might be significant in delivering unusual protein inside explicit neuronal gatherings, prompting cell brokenness and later demise" </blockquote><p></p><p>And, regarding autism: </p><p></p><blockquote>"Chemical imbalance range problem is a term used to portray various beginning stage social correspondence issues and tedious sensorimotor practices identified with a solid hereditary part and different reasons." </blockquote><p></p><p>The paper was interesting, though, for another reason. It contained a table summarising results from ten randomized controlled trials of polyunsaturated fatty acid supplementation in pregnant women and young children. This was not a systematic review, and it was unclear how the studies had been selected. As <a href="https://pubpeer.com/publications/1807C87BF4CBA5077BEB4023D7207A#3" target="_blank">I documented on PubPeer</a>, there were errors in the descriptions of some of the studies, and the interpretation was superficial. But as I checked over the studies, I was also struck by the fact that all studies concluded with a claim of a positive finding, even when the planned analyses gave null results. But, as with the studies I'd looked at in 2008, no two studies found the same thing. All the indicators were that this field is characterised by a mixture of p-hacking and hype, which creates the impression that the benefits of dietary supplementation are well-established, when a more dispassionate look at the evidence suggests considerable scepticism is warranted. </p><p>There were three questionable research practices that were prominent. First, testing a large number of 'primary research outcomes' without any correction for multiple comparisons. Three of the papers cited by Khalid did this, and they are marked in Table 1 below with "hmm" in the main analysis column. Two of them argued against using a method such as Bonferroni correction:</p><p></p><blockquote>"Owing to the exploratory nature of this study, we did not wish to
exclude any important relationships by using stringent correction
factors for multiple analyses, and we recognised the potential for a
type 1 error." (<a href="https://pubmed.ncbi.nlm.nih.gov/17185423/" target="_blank">Dunstan et al, 2008</a>)<br /></blockquote><p></p><blockquote>"Although multiple comparisons are inevitable in studies of this nature,
the statistical corrections that are often employed to address this
(e.g. Bonferroni correction) infer that multiple relationships (even if
consistent and significant) detract from each other, and deal with this
by adjustments that abolish any findings without extremely significant
levels (<i class="italic">P</i> values). However, it has been validly argued that where there are consistent, repeated, coherent and biologically
plausible patterns, the results ‘reinforce’ rather than detract from
each other (even if <i class="italic">P</i> values are significant but not very large)" (<a href="https://www.cambridge.org/core/journals/british-journal-of-nutrition/article/effects-of-highdose-fish-oil-supplementation-during-early-infancy-on-neurodevelopment-and-language-a-randomised-controlled-trial/77B688BFADFC17933DD943D87A140EB1" target="_blank">Meldrum et al, 2012</a>)<br /></blockquote> While it is correct that Bonferroni correction is overconservative with correlated outcome measures, there are other methods for protecting the analysis from inflated type I error that should be applied in such cases (<a href="https://f1000research.com/articles/10-991" target="_blank">Bishop, 2023</a>).<br /><p></p> <p></p><p>The second practice is conducting subgroup analyses: the initial analysis finds nothing, so a way is found to divide up the sample to find a subgroup that does show the effect. There is a <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3068511/" target="_blank">nice paper by Peto</a> that explains the dangers of doing this. The third practice, looking for correlations between variables rather than main effects of intervention: with sufficient variables, it is always possible to find something 'significant' if you don't employ any correction for multiple comparisons. This inflation of false positives by correlational analysis is a well-recognised problem in the field of neuroscience (e.g. <a href="https://cogns.northwestern.edu/cbmg/Vul2009PsycholSci.pdf" target="_blank">Vul et al., 2008)</a>.<br /></p><p>Given that such practices were normative in my own field of psychology for many years, I suspect that those who adopt them here are unaware of how serious a risk they run of finding spurious positive results. For instance, if you compare two groups on ten unrelated outcome measures, then the probability that something will give you a 'significant' p-value below .05 is not 5% but 40%. (The probability that none of the 10 results is significant is .95^10, which is .6. So the probability that at least one is below .05 is 1-.6 = .4). Dividing a sample into subgroups in the hope of finding something 'significant' is another way to multiply the rate of false positive findings. <br /></p><p>In many fields, p-hacking is virtually impossible to detect because authors will selectively report their 'significant' findings, so the true false positive rate can't be estimated. In randomised controlled trials, the situation is a bit better, provided the study has been registered on a trial registry – this is now standard practice, precisely because it's recognised as an important way to avoid, or at least increase detection of, analytic flexibility and outcome switching. Accordingly, I catalogued, for the 10 studies reviewed by Khalid et al, how many found a significant effect of intervention on their planned, primary outcome measure, and how many focused on other results. The results are depressing. Flexible analyses are universal. Some authors emphasised the provisional nature of findings from exploratory analyses, but many did not. And my suspicion is that, even if the authors add a word of caution, those citing the work will ignore it. </p><p><br />
<b>Table 1: Reporting outcomes for 10 studies cited by Khalid et al (2022)</b>
</p><p>
</p><table>
<tbody><tr>
<th><span style="font-size: x-small;">Khalid #</span></th>
<th><span style="font-size: x-small;">Register </span></th>
<th><span style="font-size: x-small;">N </span></th>
<th><span style="font-size: x-small;">Main result* </span></th>
<th><span style="font-size: x-small;">Subgrp </span></th>
<th><span style="font-size: x-small;">Correlatn </span></th>
<th><span style="font-size: x-small;">Abs -ve</span></th>
<th><span style="font-size: x-small;">Abs +ve</span></th>
</tr>
<tr><td><span style="font-size: x-small;">41</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">86 </span></td><td><span style="font-size: x-small;">NS</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">yes</span></td> </tr>
<tr><td><span style="font-size: x-small;">42</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">72</span></td> <td><span style="font-size: x-small;">hmm</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">yes</span></td></tr>
<tr><td><span style="font-size: x-small;">43</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">420</span></td> <td><span style="font-size: x-small;">hmm</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">yes</span></td></tr>
<tr><td><span style="font-size: x-small;">44</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">90</span></td> <td><span style="font-size: x-small;">NS</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">yes</span></td></tr>
<tr><td><span style="font-size: x-small;">45</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">90</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">NA</span><br /></td> <td><span style="font-size: x-small;">yes</span></td> </tr>
<tr><td><span style="font-size: x-small;">46</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">150</span></td> <td><span style="font-size: x-small;">hmm</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">yes</span></td> </tr>
<tr><td><span style="font-size: x-small;">47</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">175</span></td> <td><span style="font-size: x-small;">NS</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">yes</span></td> </tr>
<tr><td><span style="font-size: x-small;">48</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">107</span></td> <td><span style="font-size: x-small;">NS</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">yes</span></td> </tr>
<tr><td><span style="font-size: x-small;">49</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">1094</span></td> <td><span style="font-size: x-small;">NS</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">yes</span></td> </tr>
<tr><td><span style="font-size: x-small;">50</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">27</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">no</span></td> <td><span style="font-size: x-small;">yes</span></td> <td><span style="font-size: x-small;">yes</span></td> </tr>
</tbody></table><p><span style="font-size: x-small;"><b>Key</b>: Main result coded as NS (nonsignificant), yes (significant) or hmm (not significant if Bonferroni corrected); Subgrp and Correlatn coded yes or no depending on whether post hoc subgroup or correlational analyses conducted. Abs -ve coded yes if negative results reported in abstract, no if not, and NA if no negative results obtained. Abs +ve coded yes if positive results mentioned in abstract.
</span></p><p>I don't know if the Khalid et al review will have any effect – it is so evidently flawed that I hope it will be retracted. But the problems it reveals are not just a feature of the odd rogue review: there is a systemic problem with this area of science, whereby the desire to find positive results, coupled with questionable research practices and publication bias, have led to the construction of a huge edifice of evidence based on extremely shaky foundations. The resulting waste in researcher time and funding that comes from pursuing phantom findings is a scandal that can only be addressed by researchers prioritising rigour, honesty and scholarship over fast and flashy science.
</p><p></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com0tag:blogger.com,1999:blog-5841910768079015534.post-20044089468824371122023-08-14T12:13:00.004+01:002023-08-14T12:13:35.311+01:00The Discussion section: kill it or reform it?<p> </p><p>
I’m impelled to write a short piece about Discussion sections, after a bit of to and fro on Twitter, which started with @SchoeneggerPhil tweeting about <a href="https://doi.org/10.1007/s11229-023-04267-3" target="_blank">a new paper by Schoenegger and Pils</a>: </p><p></p><blockquote> @SchoeneggerPhil: New paper out with @PilsRaimund! We propose a new solution to the crises facing the social sciences: removing the discussion sections. We argue that they harm honest scientific reporting and would provide epistemic benefits if outsourced from the standard article.
</blockquote><p></p><p>
Michael C Frank then remarked: </p><p></p><blockquote>@mcxfrank: Hot take: no one reads discussions so removing them will improve efficiency but not solve interpretive crises (which are driven by titles and abstracts).
</blockquote><p></p><p>
Well, as someone who enjoys reading and writing discussion sections, I found this all very depressing. I argued that we already have a solution to the problems Schoenegger and Pils were trying to fix – Registered Reports. In addition, I said:</p><p></p><blockquote>@deevybee: It seems you are so worried about people HARKing and otherwise misrepresenting data, that you end up preventing exploratory research and speculation. I see them as crucial for science; it's just a case of clarifying what is hypothesis-testing and what is not.
</blockquote><p></p><p>
Russ Poldrack started further discussion by weighing in on the side of Schoenegger/Pils: </p><p></p><blockquote>@russpoldrack: I see this as a separate issue from the utility of the discussion section, which is what I liked about the original post. I agree that we need exploration - but I'd rather read speculation in the intro (as motivation for new work) rather than in the discussion (as ad hockery)
</blockquote><p></p><blockquote>
@deevybee: not sure i get this. Are you saying introduction should anticipate results before they’ve been reported; or that nobody should ever report a new idea that was stimulated by having seen the results?
</blockquote><blockquote>
@russpoldrack: what I mean is that if you have a new idea stimulated by your results, you should go do some additional work to test the idea, and then write a paper about that. if you want to speculate in your discussion that's fine, I just don't usually feel like it's worth my time to read it
</blockquote><p>
For those who don’t know me, I should start by explaining that I am fully convinced of the problems with how the research literature currently works. I have various talks and slide-decks online on this topic (see <a href="https://www.slideshare.net/deevybishop/what-is-the-reproducibility-crisis-in-science-and-what-can-we-do-about-it" target="_blank">here</a>) as well as <a href="https://journals.sagepub.com/doi/full/10.1177/1747021819886519" target="_blank">a long article on cognitive biases</a>. So I agree with the problem that Schoenegger and Pils want to fix: the Discussion section of a paper is often the provenance of over-hyped findings that are damaging to science as a cumulative process, because they encourage people to waste time following seductive but ultimately false leads. But it’s a bit of a leap from saying that many people misuse the Discussion section to concluding it should be banned, particularly when we do have other solutions to the problems.</p><p>What troubles me about the views expressed by Phil and Russ is that it sounds as if they are opposed to anyone reporting a new idea that emerges from consideration of the data. I spend a lot of my time arguing for solutions to the reproducibility crisis, and am familiar with the push-back from those that say “You’ll kill creativity”, and “You are forbidding exploratory research”. My response has always been to say I thoroughly approve of people reporting creative insights that come from observing data. The only thing you should <b>not</b> do is to formulate and test a hypothesis from the same data - The Registered Reports model, of which I am a fan, deals very nicely with what I’ve called <a href="https://www.nature.com/articles/d41586-019-01307-2" target="_blank">the four horsemen of the reproducibility apocalypse</a> – p-hacking, HARKing, low power and publication bias – but it does not preclude the researcher coming up with novel ideas in the Discussion: instead, it draws a very clear boundary between what is hypothesis-testing and what is exploratory, and does not allow someone to include hypothesis-testing analyses that were not preregistered.
</p><p>
I found Russ’s comments depressing, because it implies you shouldn’t report a new idea without first doing further work to test it. If you're making a bold new claim, then of course you need to do that. But I see scientific progress as incremental, and some insights could be valuable for others working on the topic to take into consideration. If we could only report on ideas that had been shored up by more experimentation, it could slow down discovery, because people just wouldn't bother. Also, it would make research a sadly solitary activity, where instead of exchanging ideas, we all plod on in our own narrow furrow. </p><p>I should come clean and explain that I am currently finalising the write-up of an analysis of a dataset focused on language lateralisation, where I am comparing methods of deriving a laterality index in a purely exploratory fashion. This has generated new ideas about factors that may drive individual differences in laterality – something that has intrigued me for 50 years. I think the appropriate way to handle this in the Discussion is to end with specific predictions that follow from my ideas. Maybe others would be interested in reanalysing existing data, or doing new studies that build on this work, maybe not. I am aiming to do more myself with existing datasets, but not in a position to gather new data. I am explicit in the write-up that my current study is exploratory and I do not present statistical tests. But it seems to me it would be a bit perverse if I didn’t mention how my analyses had changed my thinking to generate new predictions. If it’s true that nobody reads discussion sections, then I’ll have cast my seed on barren ground, but that still feels better than doing nothing with it - and it's useful for me to have a clear account that can be the basis for subsequent preregistered analyses. <br /></p><p>
If we adopted the Schoenegger and Pils model, I’d just have to hope that (a) someone else would be interested enough to write a Discussion paper based on my results, and (b) they would have useful insights into what it all means. I have all the cognitive biases of a typical human, so of course I think my own ideas are likely to be better than those of others (despite years of negative feedback). But the underlying nature of laterality is a topic that has intrigued me since I started in neuropsychology, so I don’t think it’s unreasonable for me to think that my insights not be obvious to others and may also have some value. As Schoenegger and Pils noted: </p><p></p><blockquote>“… one central cost with our proposal is that we may lose the epistemic advantages of authors discussing their own data in some instances. Particularly, the authors of a study often have unique insights into their data that may not be immediately apparent to third-party researchers. This is especially true for studies that involve complex datasets. By outsourcing the discussion section to third-party authors, we may miss out on important nuances and insights that only the original authors can provide.”
</blockquote><p></p>
In sum, my view is that, when used properly, the Discussion section serves two purposes. It communicates succinctly the import of the reported results in relation to a priori hypotheses, and it provides an opportunity to consider new ideas stimulated by the results. In practice, the Discussion is often misused. People play down results they don’t like, over-interpret those that accord with their preferred theory, and engage in HARKing. But to say you should get rid of the Discussion because it is misused is like saying we should all give up cars because some people drive too fast and cause accidents.
<p> </p><p><span style="font-size: x-small;"><b>Note re comments</b>. To avoid spam, comments are moderated. In general they will be approved if on-topic and non-abusive, but approval may not be immediate. </span><br /></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com0tag:blogger.com,1999:blog-5841910768079015534.post-16305137274845071952023-07-23T12:48:00.003+01:002023-07-24T09:20:12.048+01:00Is Hindawi “well-positioned for revitalization?”<p><br /></p><h3 style="text-align: left;">
<i>Guest post by Huanzi Zhang* <br /></i></h3><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGgbY6dr4_SOyvPvy-7xTEBUwGciNROtPRbFvKD_6HmYWjZIWwZWOh21sGmPcezRILgrnNnxqBbUB86P3iDW4zHVwkvQGQX0VTbkGGN0MZRohhkXkyxNupUNz9k13L4-NDaXuZ2hvY82Sz8aiYUCwQdLafXWOB0WfJrXes8or28WxBMvJc2YfzkU4qsMI6/s403/Picture%201.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="227" data-original-width="403" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGgbY6dr4_SOyvPvy-7xTEBUwGciNROtPRbFvKD_6HmYWjZIWwZWOh21sGmPcezRILgrnNnxqBbUB86P3iDW4zHVwkvQGQX0VTbkGGN0MZRohhkXkyxNupUNz9k13L4-NDaXuZ2hvY82Sz8aiYUCwQdLafXWOB0WfJrXes8or28WxBMvJc2YfzkU4qsMI6/s320/Picture%201.png" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="font-size: x-small;"> Screenshot from
https://www.wiley.com/en-us/network/publishing/research-publishing/open-access/wiley-acquires-hindawi-qa-with-liz-ferguson</span></td></tr></tbody></table> <p></p><p>Over the past year, special issues of dozens of Hindawi journals have been exposed as being systematically manipulated, resulting in the delisting of more than 20 Hindawi journals from major journal databases, as well as the retraction of more than 2,700 papers by the publisher. This "unexpected event" at Hindawi also led to a slump in profits for the parent company, John Wiley & Sons. However, in a recent statement, the president, CEO & director of Wiley, Brian Napack, stated that Hindawi was now ready for revitalization and reinstatement of the special issue program. In my opinion, Wiley has not dealt adequately with the integrity issues that led to the problem, but appears focused on growth through the medium of special issues. This raises questions as to whether Hindawi’s operation is sustainable in the long term. </p><p>Napack’s statement can be read <a href="https://investors.wiley.com/events-and-presentations/events/event-details/2023/Q4-2023-Earnings-Call/default.aspx" target="_blank">here</a>. He stated:</p><p></p><blockquote> “Fiscal '24 (starts on May 1, 2023) will be a year of revitalization for Hindawi with positive signs already emerging. We've now named a new leader of Hindawi, a talented Wiley veteran with deep expertise in the area. We've restarted the special issues program and we will be ramping it up throughout the year. We're working through the large article backlog, and we are executing our journal growth plans.”</blockquote><p></p><p>One might, however, be forgiven for being a bit sceptical about this upbeat message, since just a year before, <a href="https://investors.wiley.com/events-and-presentations/events/event-details/2022/Q4-2022-Earnings-Call/default.aspx " target="_blank">Napack said</a>:</p><p></p><blockquote> “Hindawi performed at a very high level this year, delivering strong double-digit organic revenue growth and 36% article output growth on a pro forma basis, and has achieved this with exceptional margins. We have now completed the integration and we are benefiting significantly from Hindawi's industry leading open publishing practices and its highly efficient systems.” </blockquote><p></p><p>So what is the problem with Hindawi journals that suddenly got Wiley into trouble? Is Hindawi really well-positioned for revitalization? </p><h2 style="text-align: left;">Wiley’s acquisition of Hindawi</h2><p>Wiley <a href="https://newsroom.wiley.com/press-releases/press-release-details/2021/Wiley-Announces-the-Acquisition-of-Hindawi/default.aspx" target="_blank">announced the acquisition of Hindawi</a> on January 5, 2021.
The person who pushed Wiley to buy Hindawi was Judy Verses, Wiley's Executive Vice President, <a href="https://www.elsevier.com/about/press-releases/corporate/elsevier-appoints-judy-verses-as-president-academic-and-government-markets" target="_blank">who left Wiley to join Elsevier</a> a few months after the acquisition was completed. On January 11, 2021, in an interview with Verses and Wiley's Senior Vice President Liz Ferguson, we learned that Wiley expected that Hindawi journals would publish many articles by Chinese authors, <a href="https://scholarlykitchen.sspnet.org/2021/01/11/wiley-acquires-hindawi-interview" target="_blank">expanding their market in China</a>:</p><p></p><blockquote> “It has surpassed the US in recent years, and in an increasing number of disciplines is undoubtedly the global leader. Hindawi had the foresight to launch and develop journals that reflect strengths in the China research space. Similar to Wiley, Hindawi identified early on how important it was to serve the needs of China-based researchers. Bringing together our two teams now means we have an even stronger position to be able to work to the needs of those researchers.”</blockquote><p></p><p>The <a href="https://publishingperspectives.com/2021/01/exclusive-wiley-hindawi-acquisition-jay-flynn-interview-covid19/" target="_blank">Publishing Perspective's interview with Jay Flynn</a>, <span class="ContentPaneDiv"><span class="ContentPaneDiv1">Senior Vice President and Chief Product Offer of Wiley Research, </span></span>published on January 5, 2021, and Wiley's <a href="https://www.wiley.com/en-us/network/publishing/research-publishing/open-access/wiley-acquires-hindawi-qa-with-liz-ferguson" target="_blank">internal interview with Ferguson</a>, published on February 23, 2021, mentioned the importance of China.
The new niche market for Hindawi journals seemed to respond soundly to this appointment, and the number of submissions began to increase significantly (Figure 1), while no such increase had been seen in the preceding months. In September 2021,<a href="https://newsroom.wiley.com/press-releases/press-release-details/2021/WileyNamesJay-Flynnas-Head-ofResearch/default.aspx" target="_blank"> Jay Flynn was promoted to <span class="ContentPaneDiv"><span class="ContentPaneDiv1">Executive Vice President of Wiley Research to replace</span></span> Judy Verses.</a></p><p> </p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirjJHhYoPQaL42ER_RgQVftDOmw4PL_Rzo3wjr2tefEoR9GsTtsZ8waTQV9uXkNU-gT4uuyNGq4B38_fXAZwtcCVkLhmEtOuyJLQvP7QMVODCbENooexEewhYg5mGIFzrd3N47Dg3YYjAZNzuETiUhNagJypRs4dxrWlNBYUUSpXPNz8WS80dWubIEOgGW/s1237/Fig2_Nsubmissions.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="696" data-original-width="1237" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirjJHhYoPQaL42ER_RgQVftDOmw4PL_Rzo3wjr2tefEoR9GsTtsZ8waTQV9uXkNU-gT4uuyNGq4B38_fXAZwtcCVkLhmEtOuyJLQvP7QMVODCbENooexEewhYg5mGIFzrd3N47Dg3YYjAZNzuETiUhNagJypRs4dxrWlNBYUUSpXPNz8WS80dWubIEOgGW/w400-h225/Fig2_Nsubmissions.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="font-size: x-small;">Figure 1. Number of submissions received per month for 24 journals that
subsequently had papers retracted by Hindawi. Data from Hindawi Journal
Report. </span><br /></td></tr></tbody></table><br /><p></p><h2 style="text-align: left;">The papermill problem </h2><p>The first concerns were raised by research integrity communities and individuals, who since 2021 posted thousands of comments on the PubPeer journal club relating to papers published in hundreds of special issues of Hindawi journals. Many comments were made by anonymous sleuths <i>Rhipidura albiventris</i>, <i>Hoya camphorifolia</i> and <i>Parashorea tomentella</i>, whose contributions sometimes exceeded 100 per day.
Problems were not limited to individual special issues or even individual journals. There appeared to be systematic manipulation of the publishing process, especially affecting special issues, indicating activities of so-called “<a href="https://forbetterscience.com/2021/05/26/the-chinese-paper-mill-industry-interview-with-smut-clyde-and-tiger-bb8/" target="_blank">paper mills</a>” – fraudulent organisations that will sell authorship and/or citations of papers, often faked, for a fee. <a href="https://forbetterscience.com/about/" target="_blank">Leonid Schneider</a>, who runs the blog For Better Science, assisted <a href="https://www.lemonde.fr/series-d-ete/article/2023/07/18/smut-clyde-denicheur-de-champignons-et-de-science-bidon_6182449_3451060.html" target="_blank">David Bimler</a> and others in posting their findings from Hindawi journals. Nick Wise discussed “<a href="http://deevybee.blogspot.com/2022/10/what-is-going-on-in-hindawi-special.html" target="_blank">What is going on in Hindawi special issues?</a>” on October 12, 2022. <a href="https://forbetterscience.com/2022/09/05/cyclotron-branch-before-the-fall/" target="_blank">These sleuths noted</a> that many supposedly ‘peer-reviewed’ manuscripts had incoherent or unintelligible content, and corresponding authors used email addresses from other institutions; furthermore, <a href="https://forbetterscience.com/2023/01/03/hindawi-garbage-sorting-system-based-on-citations/" target="_blank">a large number of papers cited irrelevant references</a>, presumably to boost citation counts; some paper mills <a href="https://forbetterscience.com/2023/05/15/eligible-for-a-full-waiver/ " target="_blank">used the article processing charge (APC) waiver policy of Hindawi</a> to make more profit. The pattern of abnormal citations confirmed that the fraud was not bounded by journals, and so publication-based investigations made it difficult to expose specific paper mills. </p><p>It soon became clear that the special issues that were such a lucrative source of income for Hindawi were wide open to corrupt "guest editors" who, once appointed, could use fake peer review and flood the journal with fraudulent papers and irrelevant citations.
In the gold open access model, Wiley earns APCs for every article published in Hindawi journals, whether in a special issue or not, so their incentive to exert quality control is compromised. Some examples are so extreme that nobody could take them seriously, such as an <a href="https://www.hindawi.com/journals/cin/2022/4176595/ " target="_blank">article on the ideological and political education of the Chinese Communist Party</a> in a special issue "Exploration of Human Cognition using Artificial Intelligence in Healthcare", which was submitted, peer-reviewed and received on the same day. There are thousands of other papers which may be genuine but whose subject falls well outside the scope of the special issue where they are published, indicating that the journal is out of editorial control. </p><p>Many of the authors and guest editors of the problematic papers mentioned by Bimler came from Asia. <a href="https://orcid.org/0000-0001-9042-1762" target="_blank">Ruihang Huang</a>, <a href="https://orcid.org/0000-0002-6711-3486" target="_blank">Chunjiong Zhang</a>, and <a href="https://orcid.org/0000-0003-4080-8373" target="_blank">Hanlie Cheng</a>, PhD students at Donghua University, Tongji University and China University of Geosciences, Beijing, respectively, were <a href="(https://forbetterscience.com/2023/01/03/hindawi-garbage-sorting-system-based-on-citations/" target="_blank">beneficiaries of the citation manipulation and participants in the manipulation of special issues</a>. Another example is Kaifa Zhao, who approved many nonsense manuscripts for publication as a guest editor at two journals: Computational Intelligence and Neuroscience and Journal of Environmental and Public Health. <i>TigerBB8</i> identified Zhao as a PhD student at the Hong Kong Polytechnic University, and Dorothy Bishop requested an investigation by the university. <a href="https://retractionwatch.com/2023/02/16/exclusive-prof-stole-former-students-identity-to-edit-two-journal-special-issues/" target="_blank">As reported in Retraction Watch</a>, their report claimed that Zhao's identity had been stolen by Yizhang Jiang, Zhao's master's program advisor. </p><p></p><blockquote>"According to Mr Zhao, he was not aware of relevant emails from Hindawi and has never responded to emails that are related to the two special issues"</blockquote><p></p><h2 style="text-align: left;"> Hindawi takes action (slowly)</h2><p>After a year of rapid growth in the Chinese market (Figure 2), Wiley acted. On September 28, 2022, <a href="https://www.hindawi.com/post/advancing-research-integrity-collaboratively-and-vigour/ " target="_blank">Ferguson announced</a> that 511 papers would be retracted from Hindawi journals. Intrestingly, no mention was made of the comments on PubPeer and Bimler's <a href="https://forbetterscience.com/2022/09/05/cyclotron-branch-before-the-fall/ " target="_blank">blog post</a>; instead it was stated that these retractions were based on the findings of the Hindawi Research Integrity Team. The first retractions were seen in mid-November with concentrated releases during the Lunar New Year.</p><p> </p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5-tQRfdXOekITlsf5ae8VF_ZIIDXMPqFyvw5Fzqabc8LG6ajUCkCwabW7-tphyRsFf7DUspgG79-GWa7KBe5oJHnAeX_A4LKTAdcW9H8yx2DJFVDFxgT-sTMAoNfkcbJG0bjVrEmDO3FERKHb_ytTsDqDALM5J8rKmaKEf781ox8d4ZrJiScnw4v7q3OH/s1237/Picture%202_rapid%20growth.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="696" data-original-width="1237" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5-tQRfdXOekITlsf5ae8VF_ZIIDXMPqFyvw5Fzqabc8LG6ajUCkCwabW7-tphyRsFf7DUspgG79-GWa7KBe5oJHnAeX_A4LKTAdcW9H8yx2DJFVDFxgT-sTMAoNfkcbJG0bjVrEmDO3FERKHb_ytTsDqDALM5J8rKmaKEf781ox8d4ZrJiScnw4v7q3OH/w400-h225/Picture%202_rapid%20growth.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="font-size: x-small;">Figure 2. Number of articles and reviews published in 14 Hindawi journals 2019-2022. Data from Scopus. </span></td></tr></tbody></table><br /><p></p><p>It is possible that mass retractions were delayed because Wiley did not want to disrupt their agenda at the 74th Frankfurter Buchmesse (October 19 to 23, 2022). There was no indication that Wiley shared Hindawi's problems at the book fair. Instead, they were busy with other things. Flynn <a href="https://publishingperspectives.com/2022/10/frankfurt-week-wiley-opens-its-partner-solutions-division/ " target="_blank">announced </a>the creation of Wiley Partners Solution to meet the "scholarly publishing needs at scale" on October 17. Ferguson participated in a forum entitled "<a href="https://web.archive.org/web/20230511011309/https://publishingperspectives.com/pp-forum/" target="_blank">How the Article-Based Economy is Transforming Research Publishing</a>" on October 19. Intriguingly, <a href="https://www.chemistryworld.com/news/sanctioning-of-50-journals-raises-concerns-over-special-issues-in-mega-journals/4017315.article" target="_blank">an essay posted by Chemistry World </a>on April 24, 2023, citing Flynn, noted that Wiley had convened a meeting of publishers at that book fair, invited Clarivate, the owner of major journal database Web of Science (WoS) Core Collection, and disclosed to them the problems with Hindawi journals. We do not know the outcome of this meeting, but change did occur. On the one hand, Hindawi began issuing retractions on November 16, 2022. Nevertheless, in October 2022, special issues of Hindawi journals were still being published with many compromised articles, though from December 2022 onwards, the number of papers published in special issues decreased.
</p><h2 style="text-align: left;">Delisting of Hindawi journals</h2><p>The public information prompted journal databases to re-evaluate whether Hindawi journals should continue to be indexed. In February 2023, <a href="https://twitter.com/deevybee/status/1628058351086014465" target="_blank">DOAJ (Directory of Open Access Journals) delisted thirteen Hindawi journals</a>. Then <a href="https://www.scopus.com/sources.uri" target="_blank">Scopus discontinued the indexing</a> of six Hindawi journals. On March 20, 2023, <a href="https://clarivate.com/blog/supporting-integrity-of-the-scholarly-record-our-commitment-to-curation-and-selectivity-in-the-web-of-science/" target="_blank">Clarivate delisted nineteen Hindawi journals</a> from WoS Core Collection. The fact that <a href="https://www.hindawi.com/journals/edri/" target="_blank">Education Research International</a> was delisted suggested Clarivate conducted an independent investigation, as this journal had not been mentioned in relevant sources. </p><p>As shown in Figure 3, the actions of the publisher and journal databases did not always involve the same journals. </p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghIsE1LsyGzuPDdu1Qhs9Y8RMZ9qeINf8RUFamMSAEh-b97IIJajaxVLW94bIRmCwR7h5dPZ3BVk2rA4tlbwmLYHjU98HL2R49jHK-AAUftLwucGGWyQoG9XrNm77gI1-wlYX61OsA224NjyYdxMzcjrmYkV0YmlQlYJ1VyQm9RYLqtaOhNjfOvPVrxraB/s1237/fig3%2026%20problematic.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="696" data-original-width="1237" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghIsE1LsyGzuPDdu1Qhs9Y8RMZ9qeINf8RUFamMSAEh-b97IIJajaxVLW94bIRmCwR7h5dPZ3BVk2rA4tlbwmLYHjU98HL2R49jHK-AAUftLwucGGWyQoG9XrNm77gI1-wlYX61OsA224NjyYdxMzcjrmYkV0YmlQlYJ1VyQm9RYLqtaOhNjfOvPVrxraB/w400-h225/fig3%2026%20problematic.png" width="400" /></a></div><br /><i><span style="font-size: x-small;">Figure 3. Twenty-six problematic Hindawi journals. WoS Core Collection: The journal was delisted from WoS Core Collection in March 2023. DOAJ: The journal was delisted from DOAJ in February 2023. Scopus: The journal was delisted from Scopus in the first half of 2023. Ferguson 500+: Papers in the journal were retracted and Liz Ferguson's statement was cited in the retraction statement. Flynn 2200+: Papers in the journal were retracted by Wiley using similar statements after May 2023. </span></i><p></p><p>Clarivate did not publish the reasons for the delisting of each journal, nor did they delist more Hindawi journals before the release of 2023 Journal Citation Reports on June 28, 2023. Compared to <a href="https://www.mdpi.com/journal/ijerph/announcements/5536" target="_blank">MDPI</a>, whose mega-journal the International Journal of Environmental Research and Public Health was delisted, Wiley's public response was subdued. In a mildly worded <a href="https://mp.weixin.qq.com/s/0Jk7dElYOi2wvP_QvJe1hA" target="_blank">statement</a> on March 22, 2023, on their WeChat Official Account, Hindawi said they were "disappointed" that their journals were delisted by Clarivate but did not offer any defence. In <a href="https://mp.weixin.qq.com/s/RmHWXSqZLbn4TcozNTcojg" target="_blank">another post on April 5, 2023</a>, Hindawi stated they would not appeal the delisting and suggested that the authors submit their manuscripts to Wiley journals. A <a href="https://scholarlykitchen.sspnet.org/2023/04/04/guest-post-addressing-paper-mills-and-a-way-forward-for-journal-security/" target="_blank">guest post</a> by Flynn in the Scholarly Kitchen on April 4, 2023, mentioned that:</p><p></p><blockquote> “At Wiley we take full responsibility for the quality of the content we publish across our portfolio.” </blockquote><p></p><p>He also announced a further 1,200 retractions to be issued by Hindawi journals. Flynn <a href="https://www.chemistryworld.com/news/sanctioning-of-50-journals-raises-concerns-over-special-issues-in-mega-journals/4017315.article " target="_blank">disclosed to Chemistry World</a> how they selected which publications to retract, specifically that he deployed 200 people from his editorial staff to conduct "a manual review of every single paper that we thought may have been compromised'". </p><h2 style="text-align: left;">Impact on authors</h2><p>Many authors published in Hindawi journals because they had the cachet of being listed in scholarly databases. One author of an article published in February 2023 in Oxidative Medicine and Cellular Longevity distributed email templates she drafted to others via an instant messaging software, encouraging them to ask Hindawi to work with Clarivate to index papers with publication dates before March 19, 2023. Anonymous sources described the chaos of Hindawi's customer service in late March 2023. Many people complained that Hindawi never responded to their emails. On the other hand, one author received a response from Oxidative Medicine and Cellular Longevity (omcl@hindawi.com), even though his complaint was about an article in another journal. Some authors of accepted manuscripts complained that Hindawi delayed their requests to withdraw their submissions, and feared that manuscripts would be accidentally published. </p><p>Other authors turned on the sleuths who had exposed paper mill activity on PubPeer, describing their activities as "social media-related PubPeer extortion". Jincheng Wang from the Nanjing Drum Tower Hospital, who had published in <a href="https://www.hindawi.com/journals/cmmm/2022/5334095/" target="_blank">a compromised special issue</a>, <a href="https://www.dxy.cn/bbs/newweb/pc/post/48422573" target="_blank">suggested</a> that the intent of those who posted comments was to blackmail the authors, under the threat of translating publicly available comments into Chinese and then posting them on social media in China. He <a href="https://www.dxy.cn/bbs/newweb/pc/post/48427024" target="_blank">encouraged authors to report these comments to the moderators</a>. </p><h2 style="text-align: left;">Impact of delisting on Hindawi’s business </h2><p>Wiley did not inform investors about the retractions in Hindawi journals in their <a href="https://investors.wiley.com/events-and-presentations/events/event-details/2022/Q2-2023-Earnings-Call/default.aspx" target="_blank">2nd quarter report </a>(August 1 to October 31, 2022) published on December 7, 2022. In the <a href="https://investors.wiley.com/events-and-presentations/events/event-details/2023/Q3-2023-Earnings-Call/default.aspx" target="_blank">3rd quarter report</a> (November 1, 2022, to January 31, 2023) on March 9, 2023, Napack confronted the issue head-on: </p><p></p><blockquote>“Upon discovery, the Wiley team responded quickly, suspending the Hindawi special issues program and fixing the source of the problem by purging the external bad actors and by implementing measures to prevent this from happening again. To date, these actions include increasing editorial controls and introducing new AI-based screening tools into the editorial process. We've also been scrubbing the archive and publicly retracting any compromised articles.”</blockquote><p></p><p> And, </p><p></p><blockquote>“We put the fixes in place. We feel very good about what we've done. We are reopening the programs. And we are moving forward to clear the backlog and drive forward with our publishing program.” </blockquote><p></p><p>However, the statistics on publications showed that publications in special issues continued through November 2022 to January 2023. Despite the claim that the problem had been resolved, more than 200 articles published in special issues of 34 Hindawi journals in 2023 received comments on PubPeer relating to concerns about the publishing process.
The release of retraction statements was also delayed. As of <a href="https://twitter.com/SmutClyde/status/1675464635879944194" target="_blank">July 20, 2023</a>, retraction statements were issued on six dates, on May 24, June 21, June 28, June 29, July 12, and July 19, with 112, 559, 514, 1, 521, and 510 retractions, respectively. The total number of retracted papers, approximately 2700 including the initial 500 from 2022, substantially exceeds the <a href="https://www.science.org/content/article/fast-growing-open-access-journals-stripped-coveted-impact-factors" target="_blank">1200 mentioned by Flynn</a>. </p><p>An interesting development has been the involvement of law firms who specialize in shareholder rights litigation, such as <a href="https://www.businesswire.com/news/home/20230615790426/en/WLY-WLYB-INVESTOR-NEWS-ROSEN-TRUSTED-INVESTOR-COUNSEL-Encourages-John-Wiley-Sons-Inc.-Investors-With-Losses-in-Excess-of-100K-to-Inquire-About-Securities-Class-Action-Investigation-%E2%80%93-WLY-WLYB " target="_blank">Rosen Law Firm</a>, <a href="https://www.businesswire.com/news/home/20230616436356/en/Kirby-McInerney-LLP-Continues-Investigation-of-Shareholder-Claims-Against-John-Wiley-Sons-Inc" target="_blank">Kirby McInerney LLP</a>, <a href="https://www.businesswire.com/news/home/20230616029623/en/INVESTIGATION-REMINDER-The-Schall-Law-Firm-Announces-it-is-Investigating-Claims-Against-John-Wiley-Sons-Inc.-and-Encourages-Investors-with-Losses-to-Contact-the-Firm" target="_blank">Schall Law Firm</a>, <a href="https://www.businesswire.com/news/home/20230616165565/en/Glancy-Prongay-Murray-LLP-a-Leading-Securities-Fraud-Law-Firm-Announces-Investigation-of-John-Wiley-Sons-Inc.-WLY-on-Behalf-of-Investors" target="_blank">Glancy Prongay & Murray LLP</a>, the <a href="https://www.businesswire.com/news/home/20230616627063/en/The-Law-Offices-of-Frank-R.-Cruz-Announces-Investigation-of-John-Wiley-Sons-Inc.-WLY-on-Behalf-of-Investors" target="_blank">Law Offices of Frank R. Cruz</a> and the <a href="https://www.businesswire.com/news/home/20230616316688/en/INVESTOR-ALERT-Law-Offices-of-Howard-G.-Smith-Announces-Investigation-of-John-Wiley-Sons-Inc.-WLY-on-Behalf-of-Investors " target="_blank">Law Offices of Howard G. Smith</a>. All of these firms recently advertised that they are investigating whether Wiley issued misleading information about Hindawi to the investing public. </p><p>In the 4th quarterly report for 2023, Napack stated that they had remedied all the problems, and were ready for “revitalization” </p><p></p><blockquote>“As discussed in Q3, we suspended the fast-growing special issues program after identifying a research integrity issue. This issue was the result of external misconduct by non-Wiley editors and reviewers. Essentially, Wiley decided to take a short-term hit to preserve the integrity of our journals and the value of our highly respected Wiley brand. This industry-wide issue has been widely reported on, and we believe that we now have it fully remediated in Wiley.” </blockquote><p></p><p>In the same report, Napack was still expressing satisfaction with Hindawi's performance since the acquisition: </p><p></p><blockquote>“Our expectation for Hindawi was a couple-fold. One, it would accelerate our position in that market, which it has; and that it would provide significant growth, which it has, and it will provide the ability to provide significant cascade across our portfolio that we could find homes for the many hundreds of thousands of articles that we get every year that are not published. Our expectations are the same going forward. We expect that over the next 12 to 18 months, we will be fully ramping back up. So, by '25, we're back on course with our volume growth and it should drop to the bottom line at/or we're close to the margin -- very healthy margin that it always has in our -- across all of our Open Access, but certainly across the Hindawi asset. So, the -- relative to our initial expectations, this acquisition has outperformed if you just can look aside for a minute against a very short-term thing that happened to us. But we're going to lead our way -- lead the industry out of it, and we feel very, very good about the future of our overall Open Access program.” </blockquote><p></p><p>So the paper mill debacle, which led to thousands of fraudulent papers being published in Hindawi journals over at least two years, is described as “a very short-term thing that happened to us” and is now “fully remediated”, was blamed on “external misconduct by non-Wiley editors and reviewers”. This leaves hanging the question of how those non-Wiley editors and reviewers not only achieved powerful positions determining what was published in Hindawi journals, but also continued to do so long after attention had been drawn to the problem by sleuths.</p><h2 style="text-align: left;">Regaining the market? </h2><p>Wiley is a mighty, major international publisher, and they have the potential to achieve a Hindawi revitalization, if revitalization is defined as a significant rebound in the number of papers published in Hindawi journals. The question is, which niche market does Hindawi intend to regain? Are they well-positioned to do so? Do they have any appreciation of the tension between their goal of publishing as much as possible, and the reputational costs of publishing papers that are low-quality at best and fraudulent at worst?</p><p>The sleuth <i>Parashorea tomentella </i><a href="https://forbetterscience.com/2022/10/26/papermill-your-local-partner-for-special-issues-in-china/" target="_blank">described the evolution of the niche of special issues</a> of Hindawi journals after its acquisition in China. There is a highly competitive "first-tier" niche of authors from research universities and institutions, who have many manuscripts. China also has an extended, "second-tier" niche of authors from community colleges, polytechnics, and non-teaching hospitals. There are many, many of these potential customers, but they lack manuscripts on the one hand, and desire publications for promotion on the other. Paper mills fill the need of authors in this niche to publish and the needs of publishers to make money. It would be difficult for Hindawi to regain the first-tier niche, because most authors and institutions care about the reputation of the publisher, and even if they aren’t concerned about integrity they are spooked by unprecedented large-scale retractions. </p><p>Hindawi is, however, in a strong position to regain the second-tier niche. First, despite all the problems, they still have many journals that are recognized by the Chinese authorities (typically those listed in databases such as WoS Core Collection and Engineering Village). Wiley has close links with those who maintain databases and may have an advantage in avoiding their journals being blacklisted.
Wiley's <a href="https://scholarlykitchen.sspnet.org/2021/01/11/wiley-acquires-hindawi-interview/" target="_blank">longstanding commitment and partnership with Chinese research institutes and government stakeholders</a> has brought them particularly close to the National Science Library, Chinese Academy of Sciences (<a href="http://english.las.cas.cn/About/about/" target="_blank">NSL/CAS)</a>, a bureaucracy that compiles a blacklist which is recognized by many Chinese institutions. On June 16, 2023, <a href="https://news.sciencenet.cn/htmlnews/2023/6/503299.shtm" target="_blank">Wiley and NSL/CAS announced</a> the establishment of a Joint Laboratory on Scientific and Technical Journal Innovation. The press release mentioned that an important topic for the joint lab is research integrity, and Liying Yang, director of journal evaluation in NSL/CAS, <a href="https://www.science.org/content/article/fake-scientific-papers-are-alarmingly-common" target="_blank">introduced what NSL/CAS can do</a>, including updating their Early Warning Journal List, which aimed to target paper mills. </p><p>For reasons unknown, NSL/CAS has been particularly kind to Hindawi journals. NSL/CAS released their controversial Early Warning Journal List in <a href="https://earlywarning.fenqubiao.com/#/en/early-warning-journal-list-2020 " target="_blank">December 2020</a>, <a href="https://earlywarning.fenqubiao.com/#/en/early-warning-journal-list-2021" target="_blank">December 2021</a> and <a href="https://earlywarning.fenqubiao.com/#/en/early-warning-journal-list-2023" target="_blank">January 2023</a>. In the latest version, the Hindawi journals Biomed Research International, Complexity, Advances in Civil Engineering, Shock and Vibration, Scientific Programming and Journal of Mathematics, which had been on the blacklist, were reinstated. Other problematic Hindawi journals were never on the blacklist. In contrast, the blacklist compiled by another Chinese bureaucracy, the <a href="https://www.istic.ac.cn/html/1/529/558/560/561/index.html" target="_blank">Institute of Scientific and Technical Information of China</a> (ISTIC), does not show undue goodwill toward Hindawi journals. In January and February 2023, some Chinese institutions, such as <a href="https://mp.weixin.qq.com/s/NO5By3PtF0XPwNxyKl8j1A" target="_blank">Zhejiang Gongshang University</a> and <a href="https://www.dxy.cn/bbs/newweb/pc/post/47757958" target="_blank">Anhui Provincial Hospital</a> (The First Affiliated Hospital of University of Science and Technology of China), told their employees not to submit to Hindawi journals. NSL/CAS was sending a different signal from other Chinese institutions and encouraged authors to continue submitting to Hindawi journals, although this effect was offset by Clarivate's delisting of nineteen Hindawi journals three months later. </p><p>Hindawi’s determination to retain the second-tier niche may explain why they have continued to publish their customers' manuscripts from known paper mills. For instance, <a href="https://www.hindawi.com/journals/cin/2023/3320547/ " target="_blank">an article</a> (now retracted) was published on March 11, 2023, long after the guest editor Kaifa Zhao had been proven to be an impostor. To take another example, the Hindawi Research Integrity Team retracted nine articles from a special issue of BioMed Research International on “Minimally Invasive Treatment Protocols in Clinical Dentistry” between November 22, 2022, and February 14, 2023, but subsequently published new articles in the same special issue on topics out of scope.</p><p> </p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8ednzsyxnawgXAla4FO-AKCXAyrfhr2cgAM-xNPLVfpvzrXMSDQutH_5QeDuNRP8LeEoh-1KvdW5-QfAJAid20UTbsiN1RbhXfa5V80Qu-lykZrWLPyWu3yE968g9uiifmUs2q_zqEIrnUlK5QmjicCaTqMFKP2Wd8N1ho4hF0v5RGEKJA5TgE5dqbnov/s1909/picture%204%20screenshot.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1909" data-original-width="1274" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8ednzsyxnawgXAla4FO-AKCXAyrfhr2cgAM-xNPLVfpvzrXMSDQutH_5QeDuNRP8LeEoh-1KvdW5-QfAJAid20UTbsiN1RbhXfa5V80Qu-lykZrWLPyWu3yE968g9uiifmUs2q_zqEIrnUlK5QmjicCaTqMFKP2Wd8N1ho4hF0v5RGEKJA5TgE5dqbnov/w268-h400/picture%204%20screenshot.png" width="268" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="font-size: x-small;">Figure 6. A special issue
(https://www.hindawi.com/journals/bmri/si/652179/ ) of BioMed Research
International published articles on a topic out of scope in between
retraction statements</span></td></tr></tbody></table> <p></p><p>Even more alarmingly, special issues of four journals (Computational and Mathematical Methods in Medicine, Journal of Healthcare Engineering, Journal of Environmental and Public Health, and Computational Intelligence and Neuroscience) continued to publish questionable articles <a href="https://www.hindawi.com/post/evolving-our-portfolio-response-integrity-challenges/ " target="_blank">after Hindawi announced that the journals had closed</a> on May 2, 2023, see e.g., <a href="https://pubpeer.com/publications/1F1263D5537A0EF96588929A60D15B" target="_blank">https://pubpeer.com/publications/1F1263D5537A0EF96588929A60D15B</a> ). In one weird case, <a href="https://www.hindawi.com/journals/jhe/2023/3794886/ " target="_blank">an article that had been accepted 604 days</a> previously was published in Journal of Healthcare Engineering on July 7, 2023. Perhaps this manuscript had been blocked by production processes or the Hindawi Research Integrity Team, but it was eventually published after the journal was closed. </p><h2 style="text-align: left;">Reasons for pessimism </h2><p>The reason I am pessimistic is that so far Wiley's proposals to improve the publishing process for Hindawi journals have focused on the use of AI-based screening tools. Wiley has not committed to hiring more editors for Hindawi journals. As the number of submissions increases, the situation will only get worse if peer-review processes hosted by guest editors are assigned to overworked in-house editors to oversee. </p><p>Wiley still has a chance to fix things. The first thing they should do is stop manuscripts received by external bad actors from continuing to be published. The second is to issue more retractions. I'm glad to see that in June 2023, Catriona MacCallum, the director of Open Science of Hindawi <a href="https://twitter.com/EASEeditors/status/1664611133381066753" target="_blank">shared their approach to scaling up retractions</a>, including focusing on manipulation of the process rather than author wrongdoing. Publisher retractions are painful for the publisher, journals, and authors, but they are necessary, and there are not enough of them. The third is to investigate the internal bad actors in an open and transparent manner. I would also encourage them to recruit more in-house editors, release the identities of the external bad actors they have found, and document the details of how internal controls failed so lessons can be learned. </p><p>Most of us value our reputation for its own sake. As Shakespeare said in Othello: </p><p></p><blockquote><p>“Good name in man and woman, dear my lord, </p><p>Is the immediate jewel of their souls: </p><p>Who steals my purse steals trash; ’tis something, nothing; ’twas mine, ’tis his, and has been slave to thousands; </p><p>But he that filches from me my good name </p><p>Robs me of that which not enriches him, </p><p>And makes me poor indeed” </p></blockquote><p></p><p>Indeed, as the Hindawi story shows, for a commercial organization, reputation is not just a desirable feel-good factor – it has huge financial implications. If an academic publisher like Wiley becomes known for boosting their profits by publishing screeds of arrant nonsense, their bottom line will ultimately suffer. Reputable researchers will not want their name associated with a publisher who behaves this way. If Wiley are not willing to control fraud because it is the right thing to do, they should at least recognize the importance of integrity for retaining the confidence of the academic institutions on whom they depend. </p><p> </p><h4 style="text-align: left;">Footnote </h4><p>* The author declares that there is no potential conflict of interest. The author uses a pseudonym because he/she lives in an authoritarian state and fears facing unpredictable political reprisals.</p><br />deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com2tag:blogger.com,1999:blog-5841910768079015534.post-40913628667930972862023-04-11T18:31:00.008+01:002023-04-11T18:35:40.963+01:00Papers affected by misconduct: Erratum, correction or retraction?<p> </p><p>This week, <a href="https://mailchi.mp/retractionwatch/the-rw-daily-yale-faked-data-ori-chatbot-references-peer-review-predatory-publishers" target="_blank">Retraction Watch</a> drew attention to a <a href="https://ori.hhs.gov/content/case-summary-spirli-carlo" target="_blank">case summary</a> of a misconduct investigation by the Office of Research Integrity (ORI) into grants and publications by Carlo Spirli, an Assistant Professor of Medicine, Department of Digestive Diseases, Yale University. This was based on an investigation conducted by Yale University plus analysis by ORI, which is reported with commendable transparency.<br /></p>
<p></p>
<p>The conclusions were stark: </p>
<p><i>“ORI found that Respondent engaged in research misconduct by knowingly, intentionally, or recklessly falsifying and/or fabricating data included in the following four published papers, two presentations, and three grant applications submitted for PHS funds”. </i>Details of the fabricated material in each of these sources were listed. </p>
<p></p>
<p>I suspect this investigation has been going on for a while; I could find no publications by Dr Spirli since 2019. In response to this report, he will "<i>exclude himself voluntarily for a period of four years beginning on March 28, 2023</i>” from contracting or subcontracting (presumably applying for grants) or serving on US Public Health Service committees. Compared to <a href="http://deevybee.blogspot.com/2023/02/open-letter-to-cnrs.html" target="_blank">a French case that I blogged about</a> recently this is a rather more serious outcome, though it nevertheless attracted <a href="https://twitter.com/sTeamTraen/status/1645502990160392192?s=20" target="_blank">critical comment on Twitter</a>, and it is less severe than the measures that respondents thought appropriate for misconduct in a recently published survey of Fellows of the National Science Foundation. See <a href="https://www.nature.com/articles/s41598-023-32445-3" target="_blank">Table 5, here</a>.<br /></p>
<p></p>
<p></p>
<p>My focus, here, however, is on another feature, which is similar to the French case. The report concluded that “<i>Respondent will request that the following papers be corrected or retracted</i>”, and then listed three articles published in Hepatology, two from 2012, and another from 2013. </p>
<p></p>
<p>Two of these have already had an ‘erratum’ published in 2022 (more details in Appendix below).</p>
<p></p>
<p>This seems inappropriate for two reasons. </p>
<p>First, according to <a href="https://www.elsevier.com/editors/perk/policy-and-best-practice-errata-And-corrigenda" target="_blank">Elsevier best practice guidelines</a>, ‘an erratum refers to a correction of errors <b>introduced to the article by the publisher</b>’, as opposed to a ‘corrigendum’, which is a correction made on request by the author. </p><p>Dr Spirli has an <a href="http://drsbm.unipv.it/wp-content/uploads/2017/12/Spirli-CV.pdf" target="_blank">old CV online</a> dating from 2017, in which he states he is a member of the Editorial Board of Hepatology. One wonders if this influenced the Editor who agreed to listing these two corrections as ‘Erratum’. </p>
<p>Second, though, the other category of ‘Corrigendum’ (i.e. Correction) also seems inappropriate here. We all make mistakes – I’ve got corrections to some of my papers, even though I try to be careful. It is all too easy to upload the wrong figure or miscompute some values when submitting a paper. If the conclusions are not affected by the error, a Correction is appropriate. But where there is a repeated pattern of falsification of data, or evidence that figures have deliberately been manipulated to fit a narrative, then a correction is not appropriate. The accompanying statements for Spirli’s ‘errata’ (see Appendix below) state that the conclusions are not affected. But the ORI report states that there was ‘reckless falsification or fabrication’ of data. Why, we ask ourselves, would an author falsify or fabricate data? The answer is obvious – to make inconclusive, inconsistent or null findings publishable. If the findings were solid in showing a desired result, there would be no need to engage in fraud. And if an author has shown a repeated tendency to engage in fraud, how can we trust the other data in their papers?</p>
<p>So this is a plea to ORI, CNRS, and other institutions, as well as editors, to start being more robust about the need for retraction of articles when misconduct has been demonstrated. Trying to ‘correct’ fraudulent articles is like trying to cut out a bad section from a rotting fish. The whole thing needs to be thrown away if you want to get rid of the stink. </p>
<p><span style="font-size: medium;"><b>Appendix</b></span></p>
<p><b>May 4 2022, Erratum to Spirli et al (2012a),</b> Hepatology 2012;56:2363-74. doi: 10.1002/hep.25872</p>
<p><i>In reference to Spirli et al.,[1] we have become aware of possible errors in Figures 4C, and 5 A, B, and C. Forensic analysis concluded that in Figure 4C, the Actin blot appears to have been spliced and replicated. Therefore, the readings of CC3 as an index of apoptosis induced by Sorafenib are inconclusive. In Figure 5A, splicing is also present in Figure 5A (lane 1 and 12) and 5B (lane 12). These figures intend to show the paradoxical effect of Sorafenib on B-Raf and Raf-1 activity in WT and PC2-defective cells. The phenomenon remains valid, as shown in supplementary Figure 5, where exposure to Raf265, a Raf inhibitor with similar mechanism of Sorafenib generated a similar paradoxical effect. In Figure 5C there is a splice between lines 4 and 6 (effect of the higher concentration-10 μM- sorafenib in PKI treated cells). However, the observation that inhibition of cAMP/PKA with PKI prevents the paradoxical effect of Sorafenib on pERK and proliferation as shown in Figure 6 remains valid and is consistent with the in vivo finding. We believe that within the above limitations, the results and interpretation of the paper remain valid. </i></p>
<p>In addition to the four problematic figures (‘possible errors’) noted here, the ORI report mentions problems with Figures 3 and 6. </p>
<p><b>April 17 2022, Erratum to Spirli et al (2012b),</b> Hepatology 2012;55(3):856-68. doi:10.1002/hep.24723</p>
<p><i>In reference to Spirli et al.,[1] we have become aware of an error in Figure 6A. This figure is intended to show that ER Calcium depletion (in this case using thapsigargin, an inhibitor of SERCA, the pump that allows ER Calcium entry) results in activation of the ERK pathway. The blot shows an example of Western blots from which the averages between phosphorylated ERK and total ERK shown in the bar graphs are then calculated. Forensic analysis concluded that Figure 6A contains lines seemingly duplicated for re-use in separate groups, as the bottom line 1–3 appears the same as lines 4–6. As such this figure should be considered erroneous (or falsified). However, reducing ER Calcium by another mean (chelation by TPEN) still increases ERK phosphorylation, and thus the results and interpretation of the paper remain valid.</i></p>
<p><b>17 June 2022, Retraction of Spirli et al (2015),</b> Hepatology 2015 Dec;62(6):1828-39. doi: 10.1002/hep.28138. </p>
<p><i>The retraction has been agreed upon due to recently verified concerns regarding data authenticity rendering the conclusions uncertain. Several figures included in the article were found to have been falsified.</i></p>
<p></p>
<p>One can see from the ORI report that this one had so many figure manipulations that it was beyond help. It is the only paper in the report that had been<a href="https://pubpeer.com/publications/9393A884782AB6F0804B0BD8456DB3" target="_blank"> flagged (by an anonymous commenter) on PubPeer</a>.</p><p> </p><p><i><b>Finally - please note that I welcome civil and on-topic comments, but they may take a while to appear, as comments are moderated to prevent spam.</b></i></p><p> <br /></p>
deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com2tag:blogger.com,1999:blog-5841910768079015534.post-21604388117116583522023-03-30T10:30:00.001+01:002023-03-30T10:30:00.171+01:00Open letter to CNRS<h4 style="text-align: left;">Need
for transparent and robust response when research misconduct is found</h4><h4 style="text-align: left;"><i>Scroll down for update on correspondence with CNRS Scientific Integrity Officer, 30th March 2023.
</i></h4><h4 style="text-align: left;"><span style="font-weight: normal;"><i>(French translation available in Appendix 3 of <a href="https://docs.google.com/document/d/1di9gvmdMDi81PPgX-7jK8QDD_nREHDVKpuNvOi3rjbM/edit?usp=sharing" target="_blank">this document</a>) </i></span><br /></h4><p class="MsoNormal"><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Arial",sans-serif;
mso-fareast-font-family:Arial;
mso-ansi-language:EN;}p.MsoTitle, li.MsoTitle, div.MsoTitle
{mso-style-priority:10;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-link:"Title Char";
mso-style-next:Normal;
margin-top:0cm;
margin-right:0cm;
margin-bottom:3.0pt;
margin-left:0cm;
line-height:115%;
mso-pagination:widow-orphan lines-together;
page-break-after:avoid;
font-size:26.0pt;
font-family:"Arial",sans-serif;
mso-fareast-font-family:Arial;
mso-ansi-language:EN;}span.TitleChar
{mso-style-name:"Title Char";
mso-style-priority:10;
mso-style-unhide:no;
mso-style-locked:yes;
mso-style-link:Title;
mso-ansi-font-size:26.0pt;
mso-bidi-font-size:26.0pt;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-size:11.0pt;
mso-ansi-font-size:11.0pt;
mso-bidi-font-size:11.0pt;
font-family:"Arial",sans-serif;
mso-ascii-font-family:Arial;
mso-fareast-font-family:Arial;
mso-hansi-font-family:Arial;
mso-bidi-font-family:Arial;
mso-ansi-language:EN;}.MsoPapDefault
{mso-style-type:export-only;
line-height:115%;}div.WordSection1
{page:WordSection1;}</style><span lang="EN"> </span></p><p class="MsoNormal"><span lang="EN">This Open Letter is prompted by <a href="https://www.lemonde.fr/sciences/article/2022/12/05/une-affaire-d-inconduite-scientifique-agite-un-laboratoire-de-recherche-en-chimie_6153035_1650684.html">an
article in Le Monde</a> describing an investigation into alleged malpractice at
a chemistry lab in CNRS-Université Sorbonne Paris Nord and the subsequent
report into the case by CNRS. The signatories are individuals from different
institutions who have been involved in investigations of research misconduct in
different disciplines, all concerned that the same story is repeated over and
over when someone identifies unambiguous evidence of data manipulation.<span style="mso-spacerun: yes;"> </span>Quite simply, the response by institutions,
publishers and funders is typically slow, opaque and inadequate, and is biased
in favour of the accused, paying scant attention to the impact on those who use
research, and placing whistleblowers in a difficult position.</span></p>
<p class="MsoNormal"><span lang="EN"><span style="mso-spacerun: yes;"> </span></span></p>
<p class="MsoNormal"><span lang="EN">The facts in this case are clear. More than 20
scientific articles from the lab of one principal investigator </span><span lang="EN" style="font-size: 10pt; line-height: 115%;"><span style="mso-spacerun: yes;"> </span>h</span><span lang="EN">ave been shown to
contain recycled and doctored graphs and electron microscopy images. That is,
results from different experiments that should have distinctive results are
illustrated by identical figures, with changes made to the axis legends by
copying and pasting numbers on top of previous numbers.</span></p>
<p class="MsoNormal"><span lang="EN"><span style="mso-spacerun: yes;"> </span></span></p>
<p class="MsoNormal"><span lang="EN">Everyone is fallible, and no scientist should
be accused of malpractice when honest errors are committed. We need also to be
aware of the possibility of accusations made in bad faith by those with an axe
to grind. However, there comes a point when there is a repeated pattern of
errors for a prolonged period for which there is no innocent explanation. This
point is surely reached here: the problematic data are well-documented in a
number of PubPeer comments on the articles (see links in Appendix 1 of <a href="https://docs.google.com/document/d/1di9gvmdMDi81PPgX-7jK8QDD_nREHDVKpuNvOi3rjbM/edit?usp=sharing" target="_blank">this document</a>).</span></p>
<p class="MsoNormal"><span lang="EN"><span style="mso-spacerun: yes;"> </span></span></p>
<p class="MsoNormal"><span lang="EN">The response by CNRS to this case, as
explained in their report (see Appendix 2 of <a href="https://docs.google.com/document/d/1di9gvmdMDi81PPgX-7jK8QDD_nREHDVKpuNvOi3rjbM/edit?usp=sharing" target="_blank">this document</a>), was to request correction rather
than retraction of what were described as “shortcomings and errors”, to accept
the scientist’s account that there was no intentionality, despite clear
evidence of a remarkable amount of manipulation and reuse of figures; a
disciplinary sanction of exclusion from duties was imposed for just one
month.<span style="mso-spacerun: yes;"> </span></span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">So what should happen when fraud is
suspected?<span style="mso-spacerun: yes;"> </span>We propose that there should
be a prompt investigation, with all results transparently reported. Where there
are serious errors in the scientific record, then the research articles should immediately
be retracted, any research funding used for fraudulent research should be
returned to the funder, and the person responsible for the fraud should not be
allowed to run a research lab or supervise students. The whistleblower should
be protected from repercussions.</span></p>
<p class="MsoNormal"><span lang="EN"><span style="mso-spacerun: yes;"> </span></span></p>
<p class="MsoNormal"><span lang="EN">In practice, this seldom happens. Instead, we
typically see, as in this case, prolonged and secret investigations by
institutions, journals and/or funders. There is a strong bias to minimize the
severity of malpractice, and to recommend that published work be “corrected”
rather than retracted.</span></p>
<p class="MsoNormal"><span lang="EN"><span style="mso-spacerun: yes;"> </span></span></p>
<p class="MsoNormal"><span lang="EN">One can see why this happens. First, all of
those concerned are reluctant to believe that researchers are dishonest, and
are more willing to assume that the concerns have been exaggerated. It is easy
to dismiss whistleblowers as deluded, overzealous or jealous of another’s
success. Second, there are concerns about reputational risk to an institution
if accounts of fraudulent research are publicised. And third, there is a
genuine risk of litigation from those who are accused of data manipulation. So
in practice, research misconduct tends to be played down. </span></p>
<p class="MsoNormal"><span lang="EN"><span style="mso-spacerun: yes;"> </span></span></p>
<p class="MsoNormal"><span lang="EN">However, this failure to act effectively has
serious consequences:</span></p>
<p class="MsoNormal" style="margin-left: 54pt; text-indent: -18pt;"><span lang="EN">1.<span style="mso-spacerun: yes;"> </span>It gives credibility to fictitious
results, slowing down the progress of science by encouraging others to pursue
false leads. This can be particularly damaging for junior researchers who may
waste years trying to build on invented findings. And in the age of big data,
where results in fields such as genetics and pharmaceuticals are harvested to
contribute to databases of knowledge, erroneous data pollutes the databases on
which we depend.</span></p>
<p class="MsoNormal" style="margin-left: 54pt; text-indent: -18pt;"><span lang="EN">2.<span style="mso-spacerun: yes;"> </span>Where the research has
potential for clinical or commercial application, there can be direct damage to
patients or businesses.</span></p>
<p class="MsoNormal" style="margin-left: 54pt; text-indent: -18pt;"><span lang="EN">3.<span style="mso-spacerun: yes;"> </span>It allows those who are
prepared to cheat to compete with other scientists to gain positions of
influence, and so perpetuate further misconduct, while damaging the prospects
of honest scientists who obtain less striking results.</span></p>
<p class="MsoNormal" style="margin-left: 54pt; text-indent: -18pt;"><span lang="EN">4.<span style="mso-spacerun: yes;"> </span>It is particularly
destructive when data manipulation involves the Principal Investigator of a
lab. This creates challenges for honest early-career scientists based in the
lab where malpractice occurs – they usually have the stark options of damaging
their career prospects by whistleblowing, or leaving science. Those with
integrity are thus removed from the pool of active researchers. Those who
remain are those who are prepared to overlook integrity in return for career
security.<span style="mso-spacerun: yes;"> </span>CNRS has a mission to support
research training: it is hard to see how this can be achieved if trainees are
placed in a lab where misconduct occurs.</span></p>
<p class="MsoNormal" style="margin-left: 54pt; text-indent: -18pt;"><span lang="EN">5.<span style="mso-spacerun: yes;"> </span>It wastes public money from
research grants.</span></p>
<p class="MsoNormal" style="margin-left: 54pt; text-indent: -18pt;"><span lang="EN">6.<span style="mso-spacerun: yes;"> </span>It damages public trust in
science and trust between scientists.</span></p>
<p class="MsoNormal" style="margin-left: 54pt; text-indent: -18pt;"><span lang="EN">7.<span style="mso-spacerun: yes;"> </span>It damages the reputation of
the institutions, funders, journals and publishers associated with the
fraudulent work.</span></p>
<p class="MsoNormal" style="margin-left: 54pt; text-indent: -18pt;"><span lang="EN">8.<span style="mso-spacerun: yes;"> </span>Whistleblowers, who should
be praised by their institution for doing the right thing, are often made to
feel that they are somehow letting the side down by drawing attention to
something unpleasant. They are placed at high risk of career damage and stress,
and without adequate protection by their institution, may be at risk of
litigation. Some institutions have codes of conduct where <b style="mso-bidi-font-weight: normal;">failure</b> to report an incident that gives reasonable suspicion of
research misconduct is itself regarded as misconduct, yet the motivation to
adhere to that code will be low if the institution is known to brush such
reports under the carpet.</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">The point of this letter is not to revisit the
rights and wrongs of this specific case or to promote a campaign against the
scientist involved. Rather, we use this case to illustrate what we see as an
institutional malaise that is widespread in scientific organisations. <span style="mso-spacerun: yes;"> </span>We write to CNRS to express our frustration at
their inadequate response to this case, and to ask that they review their
disciplinary processes and consider adopting a more robust, timely and
transparent process that treats data manipulation with the seriousness it
deserves, and serves the needs not just of their researchers, but also of other
scientists, and of the public who ultimately provide the research funding.</span></p>
<p class="MsoNormal"><span lang="EN"><span style="mso-spacerun: yes;"> </span></span></p>
<p class="MsoNormal"><span lang="EN">Signed by:</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Dorothy Bishop, FRS, FBA, FMedSci, Professor
of Developmental Neuropsychology (Emeritus), University of Oxford, UK.</span></p>
<p class="MsoNormal"><span lang="EN"><span style="mso-spacerun: yes;"> </span></span></p>
<p class="MsoNormal"><span lang="EN">Patricia Murray, Professor of Stem Cell
Biology and Regenerative Medicine, University of Liverpool, UK.</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Elisabeth Bik, PhD, Science Integrity
Consultant</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Florian Naudet, Professor of Therapeutics,
Université de Rennes and Institut Universitaire de France, Paris</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">David Vaux, AO FAA, FAHMS, Honorary Fellow
WEHI, & Emeritus Professor University of Melbourne, Australia</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">David A. Sanders, Department of Biological
Sciences, Purdue University, USA.</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Ben W. Mol, Professor of Obstetrics and
Gynecology, Melbourne, Australia</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Timothy D. Clark, PhD, School of Life &
Environmental Sciences, Deakin University, Geelong, Australia</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">David Robert Grimes, PhD, School of Medicine,
Trinity College Dublin, Ireland </span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Fredrik Jutfelt, Professor of Animal
Physiology, Norwegian University of Science and Technology, Trondheim, Norway </span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Nicholas J. L. Brown, PhD, Linnaeus
University, Sweden</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Dominique Roche, Marie Skłodowska-Curie Global
FellowD, Institut de biologie, Université de Neuchâtel, Switzerland</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Lex M. Bouter, Professor Emeritus of
Methodology and Integrity, Amsterdam University Medical Center and Vrije
Universiteit, Amsterdam, The Netherlands</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Josefin Sundin, PhD, Department of Aquatic
Resources, Swedish University of Agricultural Sciences, Sweden</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Nick Wise, PhD, Engineering Department,
University of Cambridge, UK</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Guillaume Cabanac, Professor of Computer
Science, Université Toulouse 3 – Paul Sabatier and Institut Universitaire de
France</span></p>
<p class="MsoNormal"><span lang="EN"> </span></p>
<p class="MsoNormal"><span lang="EN">Iain Chalmers, DSc, MD, FRCPE, Centre for
Evidence-Based Medicine, University of Oxford. </span></p>
<p class="MsoNormal"><span lang="EN"> </span></p><p>
<b><span style="font-size: large;">Response from CNRS, received 28th Feb 2023. </span></b></p><p> French version below. Version en français plus bas.
======================================== </p><p>Dear Colleagues,
I have read the open letter you sent me by email on February 22, entitled "Need for transparent and robust response when research misconduct is found". </p><p>I am very surprised that you did not think it necessary to contact the CNRS before publishing this open letter. You are obviously not familiar, or at least very unfamiliar, with CNRS policy and procedures regarding scientific integrity. </p><p>The CNRS deals with these essential issues without any complacency, but tries to be fair and to ensure that the sanctions are proportional to the misconduct committed, while respecting the rules of the French civil service. </p><p> Your letter mixes generalities about the so-called actions of scientific institutions with paragraphs that apply, perhaps, to the CNRS. If you wish to know how scientific misconduct is handled at the CNRS, I invite you to contact our scientific integrity officer, Rémy Mosseri </p><p>Kind regards, </p><p>Antoine Petit
================== </p><p>Professer Antoine Petit
CNRS
CEO </p><p>======================================== </p><p>Chers et chères collègues,
J’ai pris connaissance de la lettre ouverte que vous m’avez adressée par courriel le 22 février dernier dont le titre est « Nécessité d'une réponse transparente et robuste en cas de découverte de manquements à l’intégrité scientifique ». </p><p>Je suis très étonné que vous n’ayez pas jugé utile de prendre contact avec le CNRS avant de publier cette lettre ouverte. Vous ne connaissez visiblement pas, ou au minimum très mal, la politique et les procédures du CNRS en ce qui concerne l’intégrité scientifique. </p><p> Le CNRS traite ces questions essentielles sans aucune complaisance mais en essayant d’être justes et que les sanctions soient proportionnelles aux fautes commises, tout en respectant les règles de la fonction publique française. </p><p>Votre lettre mélange des généralités sur les soi-disant agissement des institutions scientifiques et des paragraphes qui s’appliquent, peut-être, au CNRS. Si vous souhaitez savoir comment les méconduites scientifiques sont traitées au CNRS, je vous invite à prendre contact avec notre référent intégrité scientifique, Rémy Mosseri </p><p> Bien à vous,
================ </p><p>Antoine Petit
CNRS
Président - Directeur général </p><p> </p><p> </p><p><span style="font-size: large;"><b>
<a id="Update2">Update: March 30th 2023</a>
</b></span><br /></p><p>
As recommended by Prof Petit, we contacted Dr Rémy Mosseri, Scientific Integrity Officer, with some specific questions about how research integrity is handled at CNRS. The ensuing correspondence is provided here: <br /></p>
<h3 style="text-align: left;">13th March 2023 </h3><p> Dear Dr Mosseri</p>
<p>As you will have seen, Prof Antoine Petit replied to our previous open letter (which you were copied into) concerning the case of research misconduct at Université Sorbonne Paris Nord, featured in Le Monde.
We can add that since drawing attention to this case, additional serious concerns have been raised about papers of this group: </p><p><a href="https://pubpeer.com/publications/0FA5031C555737851A865644B55B66" target="_blank">https://pubpeer.com/publications/0FA5031C555737851A865644B55B66</a>. (comments #2 and #3) </p><p><a href="https://pubpeer.com/publications/67AC8D60812782300BB58D6D32E67D " target="_blank">https://pubpeer.com/publications/67AC8D60812782300BB58D6D32E67D </a> </p><p><a href=" https://pubpeer.com/publications/274206B58670596FD557A1E71D41FF " target="_blank"> https://pubpeer.com/publications/274206B58670596FD557A1E71D41FF </a></p><p><a href=" https://pubpeer.com/publications/E1BEDDC613F4DE1F0DBF68F2CE6C57" target="_blank"> https://pubpeer.com/publications/E1BEDDC613F4DE1F0DBF68F2CE6C57</a></p>
<p>At the suggestion of Prof Petit, we are writing now to request further information about the processes used to evaluate research integrity by CNRS. </p>
<p>The specific points where it would be helpful to have clarification are: </p><p>1. When problems are repeated across many papers, what are the criteria for concluding that there are “shortcomings and errors” rather than misconduct or fraud. Are specific definitions used by CNRS? </p><p>2. When an investigation concludes that a publication contains material that is fabricated, falsified or plagiarised, what criteria are used to determine a recommendation that the paper be corrected, retracted, or other? </p><p> 3. Where it is concluded that a paper should be corrected or retracted, does CNRS require that the notice of retraction/correction mention the reason for this action? </p><p>4. We note that some CNRS reports into research misconduct have been published (https://mis.cnrs.fr/rapports/). What criteria are used to determine whether reports are confidential or public? </p><p>5. What training do CNRS staff and students have in research integrity, and are specific training measures implemented in cases where misconduct has been confirmed? </p><p>6. Do CNRS rules specify that a failure to report suspected research misconduct is itself misconduct? </p><p>7. What measures does CNRS take to protect whistleblowers? </p><p> (signed by Dorothy Bishop + signatories of original open letter) </p><p><br /></p>
<h3 style="text-align: left;">
15th March 2023 </h3><p>Dear collegues,</p>
<p>I will be pleased to try to answer (as best as possible, some questions are more complicate than others) your questions (I guess in english). Due to overbusiness, please forgive me if this is not done immediately. But I expect being able to answer within 2 weeks max.</p>
<p>I would prefer, if you can agree with that, that these answers stay informal. In other word, this would not be considered as an interview or an official document from me, from which I may find in the future selected part reproduced on the internet, without possibly (once it is in the net) the precise context in which they have been written. Would you agree on that?</p>
<p>There are some points in your open letters with which I may disagree, as far as CNRS is concerned. The difficulty is that you wrote an open letter to CNRS, but included general criticisms addressed apparently to the general academic IS treatment (I guess not only focused on CNRS, and even not only to France). If you are interested by my remarks, beside your own questions, I may formulate them. If interested, we could also have a more open and reactive discussion on that, by zoom.</p>
<p>In the meantime, please find enclosed a recent summary (in english) of the MIS activity, which may already get you interested.</p>
<p>Yours sincerely </p><p>Rémy Mosseri </p><p><br /></p>
<h3 style="text-align: left;">17th March </h3><p>Dear Dr Mosseri
</p><p>Thanks for your prompt reply, and the interesting MIS summary. We do of course understand that you need time to reply.
We would prefer to have a formal response from you, in your role as integrity officer, relating to the specific questions we have raised. The reasons we are writing to you is because of concerns about how CNRS has responded to the case reported in Le Monde. These are of particular interest to the signatories because of our prior experience with institutional responses to cases of fraud. There is considerable international public interest in these matters. I hope you would be able to respond to our questions in a way that we could share publicly. I am happy to give an undertaking that I would not knowingly misrepresent anything you say, or present it out of context. <br /></p>
<p>Yours sincerely </p><p>Dorothy Bishop, FRS, FMedSci, FBA </p><p><br /></p>
<p></p>
<h3 style="text-align: left;">
18th March </h3><p>Dear Mrs Bishop,</p>
<p>I return to you about two points</p>
<p>1) You may know that (i) I must apply a strict confidentiality about the cases we treat, (ii) I cannot start (decide alone) an investigation without having a documented allegation that I can then send to the targetted persons asking for a reply. You mention in your letters 4 pubpeer new posts concerning the case discussed in a french newspaper last december. It is not clear for me whether you considered that mentioning these posts was a formal allegation or just an information. In the first case, I must tell you that just sending to a pubpeer post is not considered by us as a formal allegation. If you ask for an investigation to be opened on new elements, you are invited write and send us a detailed allegation.</p>
<p>2) I have a problem with your answer. I am always very interested to discuss and present the rules underlying our practice (and my impression is that you miss informations about them), and even to listen to propositions to improve them. I proposed an unformal open discussion with your group, even by zoom, in which I could expose the coherence underlying our action, and the rules themselves. Notice that we claimed from the start (2018 for the MIS) that these rules are certainly perfectible; I also had in mind to explain why I may object to some statements of your letter. You do not seem interested by all this. By the way, I find quite questionable that your questions (which are certainly interesting, and do not cover the full subject) are sent to us after you opened your public campaign, and not before (as far as I know, but I may be wrong, no prior contact has been taken by your group with CNRS). I therefore do not think that presenting the coherence of our action can rely on your future decisions.</p>
<p>So we will probably proceed differently. Although some of these informations are already present (in french) in our website, we will write a public document, posted on our web site, in french and english, detailing our rules and the principles guiding our action. We already had this in mind, but did not find time to dot it (in particular having informations written in english). Most of your questions (and many others) should be answered in this more global document, and you will therefore be free to use this information (by citing the whole text).</p>
<p>Sincerely yours</p>
<p>Rémy Mosseri </p><p><br /></p>
<h3 style="text-align: left;">
28th March</h3>
<p>Dear Dr Mosseri
Your suggestion of creating a global document in response to the questions we raised is most welcome. Thank you. </p><p>The information about MIS is also very welcome.
Thank you also for explaining the situation with regard to allegations of malpractice.
This does make clear the distinctive characteristics of the CNRS procedures in investigating integrity. It is understandable that a formal allegation might be needed to initiate new investigation, to avoid CNRS being overwhelmed by information or by trivial complaints, and to protect employees from malicious actors; it was rather surprising, though, to hear that you would ignore additional evidence relating to an existing case, especially when brought to you by serious integrity experts.
Given that the research that is the topic of the case is clinically relevant, the malpractice has potential to be damaging to public health, as well as to the research community, to junior scientists, to whistleblowers, and not least to the reputation of CNRS. It would seem a matter of some urgency to remedy matters if a CNRS-funded research group is publishing manipulated data in multiple papers.
</p><p> To avoid complications of co-ordinating numerous people, I hereby make a formal request in my own name, specifically asking you to investigate a number of new issues that have arisen since your original investigation. I am ccing to the Research Integrity officer at Université Sorbonne Paris Nord, who I assume would also need to be involved in any investigation. </p><p>Here are specific concerns regarding publications from the Laboratoire de Réactivité de Surfaces, UMR CNRS 7197 and CNRS, UMR 7244, CSPBAT, Laboratoire de Chimie, Structures et Propriétés de Biomateriaux et d'Agents Therapeutiques. The evidence of data fabrication and questionable methods is evident in the published papers and is described in the linked PubPeer comments, which I briefly summarise here:
</p><p><a href="https://pubpeer.com/publications/684C7691DAAD7FCD6B7E9BBCE5346C" target="_blank">https://pubpeer.com/publications/684C7691DAAD7FCD6B7E9BBCE5346C</a>. Rectangles placed over images showing data, obscuring some regions. </p><p>In <a href="https://pubpeer.com/publications/99DFA69EC0222D3C40477DE9B8F8D6">https://pubpeer.com/publications/99DFA69EC0222D3C40477DE9B8F8D6</a> Concerns expressed about inadequate corrections of earlier work. This suggests that where CNRS has proposed correction of problematic work, it has not confirmed that this is satisfactory. </p><p><a href="https://pubpeer.com/publications/E1BEDDC613F4DE1F0DBF68F2CE6C57">https://pubpeer.com/publications/E1BEDDC613F4DE1F0DBF68F2CE6C57</a> An expert, Elisabeth Bik, has identified evidence of cut-and-paste of areas in photos of tumours. </p><p><a href="https://pubpeer.com/publications/274206B58670596FD557A1E71D41FF">https://pubpeer.com/publications/274206B58670596FD557A1E71D41FF</a> Repeated plot in different publications. </p><p><a href="https://pubpeer.com/publications/1076593A614D44E5019C69C642282B">https://pubpeer.com/publications/1076593A614D44E5019C69C642282B</a> Another unsatisfactory correction, where inconsistencies remain in the paper </p><p><a href="https://pubpeer.com/publications/0FA5031C555737851A865644B55B66">https://pubpeer.com/publications/0FA5031C555737851A865644B55B66</a>. In addition to reuse of the same histograms across multiple papers, already noted by Raphael Levy, further comments have been added by Elisabeth Bik noting evidence of duplication of regions of plots within figures </p><p><a href="https://pubpeer.com/publications/EA48A476C8B55E382AFD4BD56BDEC6">https://pubpeer.com/publications/EA48A476C8B55E382AFD4BD56BDEC6</a> Yet another correction that does not satisfactorily deal with concerns. </p><p><a href="https://pubpeer.com/publications/C9081BBA3DCD96D61FC7E1C22274FA">https://pubpeer.com/publications/C9081BBA3DCD96D61FC7E1C22274FA</a> And another correction that seems to raise more questions. </p><p><a href="https://pubpeer.com/publications/36885F09E68EA7D5E881C625BFD998">https://pubpeer.com/publications/36885F09E68EA7D5E881C625BFD998</a>. Curves that should describe experimental data appear to be generated by formula, and have identical noise patterns. </p><p><a href="https://pubpeer.com/publications/FA4ABD243E8518B6C72024EDB98DFA#">https://pubpeer.com/publications/FA4ABD243E8518B6C72024EDB98DFA#</a>. Curves that should describe experimental data appear to be generated by formula, and have identical noise patterns. </p><p><a href="https://pubpeer.com/publications/DE9875DC8BA22466DB129179506638 " target="_blank">https://pubpeer.com/publications/DE9875DC8BA22466DB129179506638 </a>A retracted paper appears to have been republished with only minor changes. </p><p><a href="https://pubpeer.com/publications/5569A968DD6668A7FBCDD3A355507E" target="_blank">https://pubpeer.com/publications/5569A968DD6668A7FBCDD3A355507E</a>
Inconsistencies in reported size of nanoparticles and the figures.
</p><p>Please note that this list is likely to grow, as I have been told of concerns regarding other publications that are still being compiled. It would be helpful if your committee could monitor these proactively on PubPeer, rather than relying on sleuths to bring them to your attention with a formal allegation. </p><p>I am sorry we disagree about the benefits of confidentiality vs. transparency. I appreciate that you may not wish to communicate further with me, because I do intend to make correspondence with CNRS public, as I think this is in the public interest. This is not a comfortable situation, but I hope that in the long term further scrutiny of cases of misconduct and institutional responses to them might help us reach a rapprochement about the appropriate methods to adopt in such cases. </p><p>Yours sincerely </p><p>Dorothy Bishop </p>
<h3 style="text-align: left;">28th March 2023 </h3><p>Dear Mrs Bishop, </p><p>I understand that you do not agree with our imperative rules of
confidentiality, and with the form under which an allegation should be
sent to us in order to possibly open an investigation.
It seems that, as a general principle, emails have the same status as
private correspondance, and should therefore not be tranferred to third
parties without the consent of the author of the email.
I politely answered to your emails, but had not in mind that these
answers would be made public without my consent.
Knowing that, do what your personal ethics tells you... </p><p>Yours sincerely
</p><p> Rémy Mosseri <br /></p>
<h3 style="text-align: left;">Afterword </h3><p>My personal ethics tell me to publish this correspondence, even though Dr Mosseri feels this is inappropriate.
There are situations when confidentiality is important, especially early in an investigation when allegations are made and information is discussed that could affect a scientist’s reputation, before the validity of the allegations is established. However, none of the matters discussed with Dr Mosseri are of this nature. Our questions to him were general ones about CNRS procedures. We rejected his suggestion that we should discuss these informally, and asked instead for a formal response by him in his role as Scientific Integrity Officer.
Insofar as evidence of scientific misconduct is mentioned in our correspondence, this relates to a case that has already been discussed in a report that is in the public domain, and all the PubPeer comments are also in the public domain. </p>
<p>Ethical judgements involve weighing up conflicting interests. As noted in my last email, in this case, research malpractice has the potential to be damaging to public health, as well as to the research community, to junior scientists, to whistleblowers, and not least to the reputation of CNRS. I think it is more important that we have transparency about the response when data manipulation has been demonstrated by scientists funded by CNRS, than that I take into account Dr Mosseri’s sensitivities.
</p>
<p>
<i><b>Note re comments on this blog</b>. Comments are moderated to protect against spam. There may be some delay before they appear; if this is a concern, please email me. I generally publish comments provided they are on topic, coherent and not libellous.</i>
</p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com14tag:blogger.com,1999:blog-5841910768079015534.post-36875595096299043482023-03-20T18:09:00.007+00:002023-03-20T18:09:37.091+00:00A suggestion for eLife<p class="MsoNormal">According to a <a href="https://www.nature.com/articles/d41586-023-00831-6" target="_blank">piece today in Nature</a>, there’s uproar at eLife,
where a new publishing model has been introduced by Editor-in-Chief, Michael
Eisen. The idea is that authors first submit their paper as a preprint, and
then a decision is made by editors as to whether it is sent out for review – at
a cost of $2000 to the author. <span style="mso-spacerun: yes;"> </span>Papers
that are reviewed are published, with the reviews, and an editorial comment, regardless
of any criticisms in the review. <span style="mso-spacerun: yes;"> </span>Authors
have an opportunity to update the article to take into account reviewer
comments if they wish, but once reviewed, cannot be rejected.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Of course, this does not really remove a “quality control”
filter by the journal – it just moves it to the stage where a decision is made
on whether or not to send the paper out for review.</p>
<p class="MsoNormal">The guidance given to editors in making that judgement is “can
you generate high-quality and broadly useful public reviews of this paper?”<span style="mso-spacerun: yes;"> </span>Concerns have been expressed over whether
this would disadvantage less well-known authors, if editors preferred to play
it safe and only send papers for review if the authors had a strong track
record.<span style="mso-spacerun: yes;"> </span>But the main concern is that there
will be a drop in quality of papers in eLife, which will lose its reputation as
a high-prestige outlet.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I have a simple suggestion for how to counteract such a
concern, and that is that the journal should adopt a different criterion for deciding
which papers to review – this should be done solely on the basis of the
introduction and methods, without any knowledge of the results. Editors could
also be kept unaware of the identity of authors. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">If eLife wants to achieve a distinctive reputation for
quality, it could do so by only taking forward to review those articles that
have identified an interesting question and tackled it with robust methodology.
It’s well-known that editors and reviewers tend to be strongly swayed by novel
and unexpected results, and will disregard methodological weaknesses if the findings
look exciting. If authors had to submit a results-blind version of the manuscript
in the first instance, then I predict that the initial triage by editors would
look rather different.<span style="mso-spacerun: yes;"> </span>The question for
the editor would no longer be one about the kind of review the paper would
generate, but would focus rather on whether this was a well-conducted study
that made the editor curious to know what the results would look like. <span style="mso-spacerun: yes;"> </span>The papers that subsequently appeared in eLife
would look different to those in its high-profile competitors, such as Nature
and Science, but in a good way.<span style="mso-spacerun: yes;"> </span>Those ultra-exciting
but ultimately implausible papers would get filtered out, leaving behind only those
that could survive being triaged solely on rationale and methods. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 1 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com0tag:blogger.com,1999:blog-5841910768079015534.post-6512204787220586412022-12-31T13:23:00.016+00:002022-12-31T13:27:10.200+00:00New Year's Eve Quiz 2022<p>
</p>
<p class="MsoNormal"><span style="font-family: inherit;"><b><span style="font-size: medium;">Dodgy journals special</span></b></span></p><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">With
so much happening in the world this year, it’s easy to miss some recent
developments in the world of academic publishing. <span style="mso-spacerun: yes;"> </span>Test your knowledge here, to see how alert you
are to news from the dark underbelly of research communication. </span>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">1.
Which of these is part of a paper mill<sup>1</sup>?</span></p><p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaTP-Zf1oLJqUEpUdDfLsTezqsXmEpcH62IZGh7D7-hBeqD6qx3cbXaNTy2orfRusgTLV64V_kHneTPJ7yVmQKvesJa2Mdoa75_pRCXIYbwd8vMJ9rpIDX-aJ6cfI5utCGxSnNtqu0g8vZsndo_8ocyzAdu1TTxYiS4yFl2VmaFau8sMhOEmf4APo0ZQ/s720/figs%20for%20item%201.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="405" data-original-width="720" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaTP-Zf1oLJqUEpUdDfLsTezqsXmEpcH62IZGh7D7-hBeqD6qx3cbXaNTy2orfRusgTLV64V_kHneTPJ7yVmQKvesJa2Mdoa75_pRCXIYbwd8vMJ9rpIDX-aJ6cfI5utCGxSnNtqu0g8vZsndo_8ocyzAdu1TTxYiS4yFl2VmaFau8sMhOEmf4APo0ZQ/w400-h225/figs%20for%20item%201.png" width="400" /></a></div><p></p><br /><p class="MsoNormal"></p><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span><p></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"><br /></span>
</p><p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">2.
How many of these tortured phrases<sup>2</sup> can you decode?</span></p><p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 1 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-fareast-font-family:"Times New Roman";}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">a)
In context of chemistry experiment: “watery arrangements”</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">b)
in context of pharmaceuticals: “medication conveyance”</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">c)
in context of statistics: “irregular esteem”</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">d)
in context of medicine: “bosom peril”</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">e)
in context of optical sensors: “wellspring of blunder”</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">3.
Which journal published a paper beginning with the sentence:</span></p>
<p class="MsoNormal"><span style="font-size: small;"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">“<i><span style="font-family: georgia;">Persistent
harassment is a major source of inefficiency and your</span></i></span><i><span style="font-family: georgia;"> growth will likely
increase over the next several years.</span></i>”</span></p><p class="MsoNormal"> </p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">and
ending with:</span></p>
<p class="MsoNormal"><span style="font-size: small;"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">“<i><span style="font-family: georgia;">The
method out</span></i></span><i><span style="font-family: georgia;"><span role="presentation">lined here can be used to easily
illuminate clinical beginnings</span><span role="presentation" style="transform: scaleX(0.975159);"> about confinement
in appropriate treatment, sensitivity and</span> the number of treatment
sessions, and provides an incentive to investigate the brain regions of
two mice and humans</span></i>”</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18pt;"><span face=""Arial",sans-serif" style="font-size: 12.5pt; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">a)<span style="font: 7pt "Times New Roman";"> </span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">a) Proceedings
of the National Academy of Science</span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18pt;"><span face=""Arial",sans-serif" style="font-size: 12.5pt; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">b)<span style="font: 7pt "Times New Roman";"> </span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">b) Acta
Scientifica</span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18pt;"><span face=""Arial",sans-serif" style="font-size: 12.5pt; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">c)<span style="font: 7pt "Times New Roman";"> </span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">c) Neurosciences
and Brain Imaging</span></p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -18pt;"><span face=""Arial",sans-serif" style="font-size: 12.5pt; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">d)<span style="font: 7pt "Times New Roman";"> </span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">d) Serbian
Journal of Management</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">4.
What have these authors got in common?<span style="mso-spacerun: yes;"> </span></span></p><p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"><span style="mso-spacerun: yes;"> </span></span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">Georges
Chastellain, Jean Bodel, Suzanne Lilar, Henri Michaux, and Pierre Mertens</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l1 level1 lfo2; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">a)<span style="font: 7pt "Times New Roman";"> </span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">a) They are all eminent French literary figures </span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo2; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">b)<span style="font: 7pt "Times New Roman";"> </span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">b) They all had a cat called Fifi</span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo2; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">c)<span style="font: 7pt "Times New Roman";"> </span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">c) They are authors of papers in the Research
Journal of Oncology, vol 6, issue 5</span></p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l1 level1 lfo2; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">d)<span style="font: 7pt "Times New Roman";"> </span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">d) They were born in November<span style="mso-spacerun: yes;"> </span></span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">5.
What kind of statistical test would be appropriate for these data?<span style="mso-spacerun: yes;"> <br /></span></span></p><p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"><span style="mso-spacerun: yes;"></span></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUx1biAelIdlF-Gh9VTRVZeA4sisLOu6q3Jh0XUrozG2FnehxqrghoMZJNH3gyXMbSDpGS0VbAYuitGgxzo638HJrpS4tWQvv3qSPgVknRU63Uf7E3Uc7PCBBeStgldpLeGvVWbRmKp81_i7KrSU34QcwDqEjya6nYWPbyctyMpf6aQQ0QZ8wGHZGhOQ/s606/T%20fig.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="405" data-original-width="606" height="214" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUx1biAelIdlF-Gh9VTRVZeA4sisLOu6q3Jh0XUrozG2FnehxqrghoMZJNH3gyXMbSDpGS0VbAYuitGgxzo638HJrpS4tWQvv3qSPgVknRU63Uf7E3Uc7PCBBeStgldpLeGvVWbRmKp81_i7KrSU34QcwDqEjya6nYWPbyctyMpf6aQQ0QZ8wGHZGhOQ/s320/T%20fig.png" width="320" /></a></div><p></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">a)
t-test</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">b)
no-way analysis of variance</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">c)
subterranean insect optimisation</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">d)
flag to commotion ratio</span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">6.
Many eminent authors have published in one of these Prime Scholars journals: </span></p>
<p class="MsoListParagraphCxSpFirst" style="margin-left: 54pt; mso-add-space: auto; mso-list: l2 level1 lfo4; text-indent: -36pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">i)<span style="font: 7pt "Times New Roman";">
</span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Polymer Sciences</span></p>
<p class="MsoListParagraphCxSpMiddle" style="margin-left: 54pt; mso-add-space: auto; mso-list: l2 level1 lfo4; text-indent: -36pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">ii)<span style="font: 7pt "Times New Roman";">
</span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Journal of Autacoids</span></p>
<p class="MsoListParagraphCxSpMiddle" style="margin-left: 54pt; mso-add-space: auto; mso-list: l2 level1 lfo4; text-indent: -36pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">iii)<span style="font: 7pt "Times New Roman";">
</span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Journal of HIV and Retrovirus</span></p>
<p class="MsoListParagraphCxSpMiddle" style="margin-left: 54pt; mso-add-space: auto; mso-list: l2 level1 lfo4; text-indent: -36pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">iv)<span style="font: 7pt "Times New Roman";">
</span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">British Journal of Research</span></p>
<p class="MsoListParagraphCxSpLast" style="margin-left: 54pt; mso-add-space: auto;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">Can
you match the author to the journal?</span></p>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l3 level1 lfo3; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">a)<span style="font: 7pt "Times New Roman";"> </span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Jane Austen </span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l3 level1 lfo3; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">b)<span style="font: 7pt "Times New Roman";">
</span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Kurt Vonnegut </span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l3 level1 lfo3; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">c)<span style="font: 7pt "Times New Roman";">
</span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Walt Whitman </span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l3 level1 lfo3; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">d)<span style="font: 7pt "Times New Roman";">
</span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Herman Hesse </span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l3 level1 lfo3; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">e)<span style="font: 7pt "Times New Roman";">
</span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Tennessee Williams </span></p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l3 level1 lfo3; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">f)<span style="font: 7pt "Times New Roman";">
</span></span></span><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;">Ayn Rand </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p><p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">7.
Some poor authors have their names badly mangled by those who use their name while attempting to avoid
plagiarism checks. <span style="mso-spacerun: yes;"> </span>Can you reconstruct
the correct versions of these two names (and affiliation for author 1)?<br /></span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p><p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"><span style="mso-spacerun: yes;"></span></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4nn_eijMADYjb_K_uFzJdezLgSfLaO2FbTsWCh5Fh-Ih1EcjyJ4VGs7LPTRM1y-qXOv0kYetI6h964XXMSKtm3INldKjqPHp1oFusOjF6c_ki4bz96eBd4JNeVxSx1CmAzYN5F8H0c1GFgaBzhfGBE_jO-HdNLWSduyoOW7v25zc2HmWFqv9h1r-PEg/s529/williams%20wt%20unit.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="176" data-original-width="529" height="106" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4nn_eijMADYjb_K_uFzJdezLgSfLaO2FbTsWCh5Fh-Ih1EcjyJ4VGs7LPTRM1y-qXOv0kYetI6h964XXMSKtm3INldKjqPHp1oFusOjF6c_ki4bz96eBd4JNeVxSx1CmAzYN5F8H0c1GFgaBzhfGBE_jO-HdNLWSduyoOW7v25zc2HmWFqv9h1r-PEg/s320/williams%20wt%20unit.png" width="320" /></a></div><p></p><p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"><span style="mso-spacerun: yes;"></span></span></p><p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"><span style="mso-spacerun: yes;">---------------------------------------------------------------------------------------<br /></span></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjl-94b9yQ2BSWls9rfQT9W2VnNuB6PRLVPQEXAj5PR4CRtEmwcAVnV7qqbAHpcbtGM5RF_LTRQ_OnWGQ4wDEBvqOket3-yHHvBxtDGQPAX-RB33LnUW-GQw76gIUjrn9nVHmjg28aTJ4U4zkA6HNNYLdsnyIAq_5gVIZJWt1DKrun2Dy-R4XlOEtmgQQ/s534/Falsehood%20Lu.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="162" data-original-width="534" height="97" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjl-94b9yQ2BSWls9rfQT9W2VnNuB6PRLVPQEXAj5PR4CRtEmwcAVnV7qqbAHpcbtGM5RF_LTRQ_OnWGQ4wDEBvqOket3-yHHvBxtDGQPAX-RB33LnUW-GQw76gIUjrn9nVHmjg28aTJ4U4zkA6HNNYLdsnyIAq_5gVIZJWt1DKrun2Dy-R4XlOEtmgQQ/s320/Falsehood%20Lu.png" width="320" /></a></div><br /> <br /><p></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><b><span style="font-size: medium;"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">Final
thoughts</span></span></b></p><p class="MsoNormal"><b><span style="font-size: medium;"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></span></b></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">While
the absurdity of dodgy journals can make us laugh, there is, of course, a dark
side to all of this that cannot be ignored. The huge demand for places to publish
has not only led to obviously predatory publishers, who will publish anything
for money, but also has infiltrated supposedly reputable publishers.<span style="mso-spacerun: yes;"> </span>Papermills are seen as a growing problem, and
all kinds of fraud abound, even among some of the upper echelons of academia. As
I argued in my last <a href="http://deevybee.blogspot.com/2022/12/when-there-are-no-consequences-for.html" target="_blank">blogpost</a>, it’s far too easy to get away with academic misconduct,
and the incentives on researchers to fake data and publications are growing all
the time. My New Year’s wish is that funders, academic societies and
universities start to grapple with this problem more urgently, so that there won’t
be material for such a quiz in 2023. </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><b><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">References</span></b></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal" style="layout-grid-mode: char;"><sup><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">1 </span></sup><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">COPE &
STM. (2022). <i>Paper mills: Research report from COPE & STM</i>. Committee
on Publication Ethics and STM. <a href="https://doi.org/10.24318/jtbG8IHL">https://doi.org/10.24318/jtbG8IHL</a></span></p>
<p class="MsoNormal" style="layout-grid-mode: char;"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"></span> </p>
<span title="url_ver=Z39.88-2004&ctx_ver=Z39.88-2004&rfr_id=info%3Asid%2Fzotero.org%3A2&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=report&rft.btitle=Paper%20mills%3A%20research%20report%20from%20COPE%20%26%20STM&rft.aulast=COPE%20%26%20STM&rft.au=COPE%20%26%20STM&rft.date=2022-06-20&rft.language=en">
<p class="MsoNormal" style="layout-grid-mode: char;"><sup><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">2 </span></sup><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">Cabanac, G.,
Labbé, C., & Magazinov, A. (2021). <i>Tortured phrases: A dubious writing
style emerging in science. Evidence of critical issues affecting established
journals</i> (arXiv:2107.06751). arXiv. <a href="https://doi.org/10.48550/arXiv.2107.06751">https://doi.org/10.48550/arXiv.2107.06751</a></span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span face=""Calibri",sans-serif" style="mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"> </span></p>
<p class="MsoNormal"><span style="font-family: verdana;"><b>ANSWERS</b></span></p><p class="MsoNormal"><span style="font-family: verdana;"><b> </b></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">1. B is a solicitation for an academic paper mill. A is a flour mill and C is a paper mill of the more regular kind. B was discussed <a href="https://pubpeer.com/publications/53852496271CE4AA49D75A4EA3EF98" target="_blank">here</a>.</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"> </span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">2.
Who knows? Best guesses are:</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">a)
aqueous solutions</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">b)
drug delivery</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">c)
random value</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">d)
breast cancer</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">e)
source of error</span></p><p class="MsoNormal"><span style="font-family: verdana;"> </span></p><p class="MsoNormal"><span style="font-family: verdana;">If you enjoy this sort of word game, you can help by typing "tortured phrases" into PubPeer and checking out the papers that have been detected by the <a href="https://dbrech.irit.fr/pls/apex/f?p=9999:1::::::" target="_blank">Problematic Paper Screener</a>. <br /></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"> </span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">3.
c) <a href="https://www.primescholars.com/articles/a-short-note-on-mechanism-of-brain-in-animals-and-humans.pdf">https://www.primescholars.com/articles/a-short-note-on-mechanism-of-brain-in-animals-and-humans.pdf</a><span style="mso-spacerun: yes;"> </span></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"> </span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">4.
c) see <a href="https://www.primescholars.com/archive/iprjo-volume-6-issue-5-year-2022.html">https://www.primescholars.com/archive/iprjo-volume-6-issue-5-year-2022.html</a><span style="mso-spacerun: yes;"> </span></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">If
you answered (a) you are misled by the Poirot fallacy – all of them except Bodel
are Belgian.<span style="mso-spacerun: yes;"> </span></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"> </span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">5.
Fortunately the paper has been retracted and so no answer is required. For further details see <a href="https://retractionwatch.com/2022/12/05/a-paper-used-capital-ts-instead-of-error-bars-but-wait-theres-more/" target="_blank">here</a>. <br /></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">c)
is a reference to tortured phrase version of “ant colony optimisation” (which is a real thing!) and d)
is reference to “signal-to-noise” ratio. </span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"> </span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"> </span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">6.
<span style="mso-tab-count: 1;"> </span></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">Jane
Austen<span style="mso-spacerun: yes;"> </span>(ii) and (iv)</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">Kurt
Vonnegut (ii) and (iii)</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">Walt
Whitman (i) (ii) and (iv)</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">Herman
Hesse (iv)</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">Tennessee
Williams (ii)</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">Ayn
Rand (iii)</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"> </span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">See:
<a href="https://www.primescholars.com/archive/jac-volume-3-issue-2-year-2022.html">https://www.primescholars.com/archive/jac-volume-3-issue-2-year-2022.html</a><span style="mso-spacerun: yes;"> </span></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"><a href="https://www.primescholars.com/archive/ipbjr-volume-9-issue-7-year-2022.html">https://www.primescholars.com/archive/ipbjr-volume-9-issue-7-year-2022.html</a><span style="mso-spacerun: yes;"> </span></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"><a href="https://www.primescholars.com/archive/ipps-volume-7-issue-4-year-2022.html">https://www.primescholars.com/archive/ipps-volume-7-issue-4-year-2022.html</a><span style="mso-spacerun: yes;"> </span></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"><a href="https://www.primescholars.com/archive/ipps-volume-7-issue-2-year-2022.html">https://www.primescholars.com/archive/ipps-volume-7-issue-2-year-2022.html</a><span style="mso-spacerun: yes;"> </span></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"><a href="https://www.primescholars.com/archive/ipbjr-volume-9-issue-9-year-2022.html">https://www.primescholars.com/archive/ipbjr-volume-9-issue-9-year-2022.html</a><span style="mso-spacerun: yes;"> </span></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"><a href="https://www.primescholars.com/archive/ipbjr-volume-9-issue-7-year-2022.html">https://www.primescholars.com/archive/ipbjr-volume-9-issue-7-year-2022.html</a><span style="mso-spacerun: yes;"> </span></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;"> </span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">7.
</span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">This
article is available here: <a href="https://www.primescholars.com/archive/jac-volume-2-issue-3-year-2020.html">https://www.primescholars.com/archive/jac-volume-2-issue-3-year-2020.html</a>.<span style="mso-spacerun: yes;"> A genuine email has been added to the paper</span> and is the clue to the person whose identity was used for this paper: Williams, GM with address at New York Medical College, Valhalla campus. Given the mangling of his name, I suspect he is no more aware of his involvement in the paper than Jane Austen or Kurt Vonnegut.<br /></span></p><span style="font-family: verdana;">
</span><p class="MsoNormal"><span style="font-family: verdana;">For
2<sup>nd</sup> e.g. see <a href="https://pubpeer.com/publications/B7E65FDF7565448A0507B32123E4D8">https://pubpeer.com/publications/B7E65FDF7565448A0507B32123E4D8</a><span style="mso-spacerun: yes;"> </span></span></p><span style="font-family: verdana;">
</span></span><p class="MsoNormal"><style><span style="font-family: verdana;">@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 1 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-fareast-font-family:"Times New Roman";}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;
text-underline:single;}a:visited, span.MsoHyperlinkFollowed
{mso-style-noshow:yes;
mso-style-priority:99;
color:#954F72;
mso-themecolor:followedhyperlink;
text-decoration:underline;
text-underline:single;}p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}ol
{margin-bottom:0cm;}ul
{margin-bottom:0cm;}</span></style></p><span style="font-family: verdana;"><br />
</span><br /><br />deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com0tag:blogger.com,1999:blog-5841910768079015534.post-31528432163383027722022-12-16T16:26:00.013+00:002022-12-20T11:39:33.905+00:00When there are no consequences for misconduct: Parallels between politics and science<div><p> </p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><img border="0" data-original-height="1009" data-original-width="800" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtH0YX3idoOv-F-lFdbX9ZuL0PtRpgQ1qHUibbHQaJrLwYKZsXFEw4VTclGPTOM-h15F48L5iTIm0aBGqfD0m5cgofMtF3sVoEQjffIUF7i8mO-Lf8I41n7HioR-QN02OCXT-Z7om-4b2WV9046VP9PR7R_AgMkqmu06jU9avLvwrzbThl_cYeJkv8Hg/s320/Dore%20Paradise_Lost_1.jpg" style="margin-left: auto; margin-right: auto;" width="254" /></td></tr><tr><td class="tr-caption" style="text-align: center;">Gustave Doré: Illustration for Paradise Lost<br /></td></tr></tbody></table><p></p><div class="separator" style="clear: both; text-align: center;"></div>
<p></p><p class="MsoNormal">(Updated 17 Dec 2022) <br /></p><p class="MsoNormal">As children, we grow up with stories of the battle between
good and evil, but good ultimately triumphs. In adulthood, we know things can
be more complicated: bad people can get into positions of power and make
everyone suffer.<span style="mso-spacerun: yes;"> </span>And yet, we tell
ourselves, we have a strong legal framework, there are checks and balances, and
a political system aspires to be free and fair. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">During the last decade, I started for the first time to have
serious doubts about those assumptions. In both the UK and the US, the same
pattern is seen repeatedly: the media report on a scandal involving the
government or a public figure, there is a brief period of public outrage, but
then things continue as before.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">In the UK we have become accustomed to <a href="https://boris-johnson-lies.com/" target="_blank">politicians lying to Parliament and failing to correct the record</a>, to <a href="https://www.theguardian.com/politics/2020/nov/20/priti-patel-bullying-inquiry-why-was-it-held-and-what-did-it-find" target="_blank">bullying by senior politicians</a>,
and <a href="https://www.insidehousing.co.uk/news/news/grenfell-tower-smoke-system-failed-to-comply-with-requirements-of-regulations-expert-says-75980" target="_blank">to safety regulations being ignored</a>.<span style="mso-spacerun: yes;"> </span>The current scandal is a case of disaster capitalism where <a href="https://www.bmj.com/content/372/bmj.n399" target="_blank">government cronies made vast fortunes </a>from the Covid pandemic by gaining contracts for personal protective equipment – which was
not only provided at inflated prices, but then could not be used as it was
substandard. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">These are all shocking stories, but even more shocking is
the lack of any serious consequences for those who are guilty. In the past,
politicians would have resigned for minor peccadilloes, with pressure from the
Prime Minister if need be. During Boris Johnson’s premiership, however, the
Prime Minister was part of the problem.<span style="mso-spacerun: yes;"> </span></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">During the Trump presidency in the US, Sarah Kendzior wrote
about <a href="https://www.theglobeandmail.com/opinion/article-americans-are-the-witnesses-to-democracys-demise/" target="_blank">“saviour syndrome”</a><span style="mso-spacerun: yes;"><a href="https://www.theglobeandmail.com/opinion/article-americans-are-the-witnesses-to-democracys-demise/" target="_blank"> </a> </span>- the belief
people had that someone would come along and put things right.
As she noted: “<i>Mr. Trump has openly committed crimes and even confessed to
crimes: What is at stake is whether anyone would hold him accountable</i>.” And,
sadly, the answer has been no.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><b>No consequences for scientific fraud</b></p>
<p class="MsoNormal">So what has this got to do with science?<span style="mso-spacerun: yes;"> </span>Well, I get the same sinking feeling that
there is a major problem, everyone can see there's a problem, but nobody is going to rescue us. Researchers who
engage in obvious malpractice repeatedly get away with no consequences.<span style="mso-spacerun: yes;"> </span>This has been a recurring theme from those
who have exposed academic papermills (<a href="https://link.springer.com/article/10.1007/s11192-021-03871-9" target="_blank">Byrne et al., 2021</a>) and/or reported
manipulation of figures in journal articles (<a href="https://journals.asm.org/doi/pdf/10.1128/mBio.00809-16" target="_blank">Bik et al., 2016</a>).<span style="mso-spacerun: yes;"> </span>For instance, when <a href="https://www.nature.com/articles/d41586-020-01363-z" target="_blank">Bik was interviewed by Nature</a>, she noted that 60-70% of the 800 papers she had reported to journals
had not been dealt with within 5 years.
That matches my more limited experience; if one points out academic malpractice
to publishers or institutions, there is often no reply. Those who do reply typically
say they will investigate, but then you hear no more. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">At a recent symposium on Research Integrity at Liverpool
Medical Institution*, David Sanders (Purdue University) told of repeated experiences of being given
the brush-off by journals and institutions when reporting suspect papers. For
instance, he reported an article that had simply recycled a table from a
previous paper on a different topic. The response was “<i>We will look into it</i>”.
“<i>What</i>”, said David incredulously, “<i>is there to look into?</i>”. This is the concern
– that there can be blatant evidence of malpractice within a paper, yet the
complainant is ignored. In this case, nothing happened. There are honorable
exceptions, but it seems shocking that serious and obvious errors in work are
not dealt with in a prompt and professional manner. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">At the same seminar, there was a searing presentation by
Peter Wilmshurst, whose experiences of exposing medical fraud by powerful
individuals and organisations have led him to be the subject of numerous libel
complaints. <span style="mso-spacerun: yes;"> </span>Here are a few details of two
of the cases he presented:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><b>Paolo Macchiarini:</b> <span style="mso-spacerun: yes;"> </span>Convicted
in 2022 of causing bodily harm with an experimental transplant of a synthetic
windpipe that he performed between 2011-2012.<span style="mso-spacerun: yes;">
</span>Wilmshurst noted that the descriptions of the experimental surgery in
journals were incorrect. For a summary see <a href="https://www.bmj.com/content/377/bmj.o1516" target="_blank">this BMJ article</a>. <span style="mso-spacerun: yes;"> </span>A 2008 paper by Macchiarini and colleagues is
still published in the Lancet, despite <a href="https://www.bmj.com/content/376/bmj.o498" target="_blank">demands for it to be retracted.</a><span style="mso-spacerun: yes;"> </span></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><b>Don Poldermans</b>: An eminent cardiologist who conducted a
series of studies on perioperative betablockers, leading them to be recommended
in guidelines from the European Society of Cardiology, <span style="mso-spacerun: yes;"> </span>whose task force he chaired. A meta-analysis challenged
that conclusion, showing mortality increased; an investigation found that work
by Poldermans had serious integrity problems, and he was fired. Nevertheless,
<a href="https://retractionwatch.com/category/by-author/don-poldermans/" target="_blank">the papers have not been retracted</a>. Wilmshurst estimated that thousands of deaths
would have resulted from physicians following the guidelines recommending
betablockers. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">The week before the Liverpool meeting, there was a session
on <a href="https://www.youtube.com/watch?v=tPhoPTk49JE" target="_blank">Correcting the Record</a> at AIMOS2022.<span style="mso-spacerun: yes;"> </span>The four speakers, John Loadsman
(anaesthesiology), Ben Mol (Obstetrics and Gynecology), Lisa Parker (Oncology)
and Jana Christopher (image integrity) covered the topic from a range of
different angles, but in every single talk, the message came through loud and
clear: it’s not enough to flag up cases of fraud – you have to then get someone
to act on them, and that is far more difficult than it should be. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">And then on the same day as the Liverpool meeting, Le Monde
ran <a href="https://www.lemonde.fr/sciences/article/2022/12/05/une-affaire-d-inconduite-scientifique-agite-un-laboratoire-de-recherche-en-chimie_6153035_1650684.html" target="_blank">a piece</a> about a researcher whose body of work contained numerous problems:
the same graphs were used across different articles that purported to show
different experiments, and other figures had signs of manipulation.<span style="mso-spacerun: yes;"> </span>There was an investigation by the institution
and by the funder, Centre National de la Recherche Scientifique (CNRS), which
concluded that there had been several breaches of scientific integrity.
However, it seems that the recommendation was simply that the papers should be
“corrected”.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><b>Why is scientific fraud not taken seriously?</b></p>
<p class="MsoNormal">There are several factors that conspire to get scientific
fraud brushed under the carpet.</p>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l2 level1 lfo1; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">1.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span><b>Accusations of fraud may be unfounded.</b> In
science, as in politics, there may be individuals or organisations who target
people unfairly – either for personal reasons, or because they don’t like their
message. Furthermore, everyone makes mistakes and it would be dangerous to
vilify researchers for honest errors. So it is vital to do due diligence and
establish the facts. In practice, however, this typically means giving the
accused the benefit of the doubt, even when the evidence of misconduct is
strong.<span style="mso-spacerun: yes;"> </span>While it is not always easy to
demonstrate intent, there are many cases, such as those noted above, where a
pattern of repeated transgressions is evident in published papers – and yet
nothing is done. <span style="mso-spacerun: yes;"> </span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l2 level1 lfo1; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">2.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span><b>Conflict of interest.</b> Institutions may be
reluctant to accept that someone is fraudulent if that person occupies a
high-ranking role in the organisation, especially if they bring in grant income.
Worries about reputational risk also create conflict of interest. The <a href="https://printeger.eu/wp-content/uploads/2019/01/D3_6-Report-integrating-case-studies.pdf" target="_blank">Printeger project</a>
is a set of case studies of individual research misconduct cases, which
illustrates just how inconsistently these are handled in different countries,
especially with regard to transparency vs confidentiality of process. It
concluded “<i>The reflex of research organisations to immediately contain and
preferably minimise misconduct<br />
cases is remarkable</i>”.</p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l2 level1 lfo1; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">3.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span><b>Passing the buck</b>. Publishers may be reluctant to
retract papers unless there is an institutional finding of misconduct, even if
there is clear evidence that the published work is wrong. I discussed this
<a href="http://deevybee.blogspot.com/2021/03/time-for-publishers-to-consider-rights.html" target="_blank">here</a>. My view is that leaving flawed research in
the public record is analogous to a store selling poisoned cookies to customers
– you have a responsibility to correct the record as soon as possible when the evidence is clear to avoid harm to consumers. Funders might be expected to also play a role in correcting the record when research they have funded is shown to be flawed. Where public money is
concerned, funders surely have a moral responsibility to ensure it is not
wasted on fraudulent or sloppy research. Yet in her introduction to the
Liverpool seminar, Patricia Murray noted that the new UK Committee on Research
Integrity (CORI) does not regard investigation of research misconduct as within
its purview.<span style="mso-spacerun: yes;"> </span></p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l2 level1 lfo1; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">4.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span><b>Concerns about litigation</b>. Organisations often
have concerns that they will be sued if they make investigations of misconduct
public, even if they are confident that misconduct occurred. These concerns are
justified, as can be seen from the lawsuits that most of the sleuths who spoke
at AIMOS and Liverpool have been subjected to. <span style="mso-spacerun: yes;"> </span>My impression is that, provided there is clear
evidence of misconduct, the fraudsters typically lose libel actions, but I’d be
interested in more information on that point. </p>
<p class="MsoNormal" style="margin-left: 18pt;"> </p>
<p class="MsoNormal"><b> </b></p>
<p class="MsoNormal"><b>Consequences when misconduct goes unpunished</b></p>
<p class="MsoNormal"><b> </b></p>
<p class="MsoNormal">The lack of consequences for misconduct has many corrosive
impacts on society.<span style="mso-spacerun: yes;"> </span></p>
<p class="MsoNormal"> </p>
<p class="MsoListParagraph" style="mso-list: l1 level1 lfo2; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">1.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span>Political and scientific institutions can only
operate properly if there is trust. If lack of integrity is seen to be
rewarded, this erodes public confidence.<span style="mso-spacerun: yes;">
</span></p>
<p class="MsoNormal"> </p>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l1 level1 lfo2; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">2.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;"> </span></span></span>People
depend on us getting things right. We are confronting major challenges to
health and to our environment. If we can’t trust researchers to be honest, then
we all suffer as scientific progress stalls.<span style="mso-spacerun: yes;">
</span>Over-hyped findings that make it into the literature can lead subsequent
generations of researchers to waste time pursuing false leads.<span style="mso-spacerun: yes;"> </span>Ultimately, people are harmed if we don’t fix
fraud.</p>
<p class="MsoListParagraphCxSpMiddle"> </p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l1 level1 lfo2; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">3.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span>Misconduct leads to waste of resources. It is
depressing to think of all the research that could have been supported by the
funds that have been spent on fraudulent studies.</p>
<p class="MsoNormal"> </p>
<p class="MsoListParagraph" style="mso-list: l1 level1 lfo2; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">4.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;"> </span></span></span>People
engage in misconduct because in a competitive system, it brings them personal
benefits, in terms of prestige, tenure, power and salary. If the fraudsters are
not tackled, they end up in positions of power, where they will perpetuate a
corrupt system; it is not in their interests to promote those who might challenge
them. </p>
<p class="MsoNormal"> </p>
<p class="MsoListParagraph" style="mso-list: l1 level1 lfo2; text-indent: -18pt;"><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">5.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span>The new generation entering the profession will
become cynical if they see that one needs to behave corruptly in order to
succeed. They are left with the stark choice of joining in the corruption or
leaving the field. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="mso-spacerun: yes;"> </span></p>
<p class="MsoNormal"><b>What can be done?</b></p><p class="MsoNormal"><b> </b></p>
<p class="MsoNormal">There’s no single solution, but I think
there are several actions that are needed to help clean up the mess.</p><p class="MsoNormal"> </p>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo3; text-indent: -18pt;"><b><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">1.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span></b><b>Appreciate the scale of the problem. </b></p>
<p class="MsoListParagraphCxSpMiddle">When fraud is talked about in scientific
circles, you typically get the response that “<i>fraud is rare</i>” and “<i>science is
self-correcting</i>”.<span style="mso-spacerun: yes;"> </span>A hole has been blown
in the first assumption by the emergence of industrial-scale fraud in the form
of academic paper-mills . The large publishers are now worried enough about
this to be taking concerted action to detect papermill activity, and some of
them have engaged in mass retractions of fraudulent work (see, e.g. the case of
IEEE retractions <a href="https://retractionwatch.com/2015/06/25/one-publisher-appears-to-have-retracted-thousands-of-meeting-abstracts-yes-thousands/" target="_blank">here</a>).
Yet, I have documented on PubPeer numerous new papermill articles in Hindawi special
issues appearing since September of this year, when the <a href="https://retractionwatch.com/2022/09/28/exclusive-hindawi-and-wiley-to-retract-over-500-papers-linked-to-peer-review-rings/" target="_blank">publisher announced it would be engaging in retraction of 500 papers</a>.
It’s as if the publisher is trying to clean up with a mop while a fire-hose is
spewing out fraudulent content.<span style="mso-spacerun: yes;"> </span>This
kind of fraud is different from that reported by Wilmshurst, but it illustrates
just how slow the business of correcting the scientific record can be – even
when the evidence for fraud is unambiguous. </p><p class="MsoListParagraphCxSpMiddle"></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi5rSbbfXKVDImSp7MaJRWriuj3p6kS8JgFIGZEzTOEg81PzncrV8cgN_nXq3ibPAlTdqbOjtnmjef-RgFe2bsoMD1i17MTgFMJq4Vqx98wu9juEiAvzIQ6gh7ZCX-kmVzE4pdhE2xvdBLBwxlshRyuKWII5bkyf0QX2_qpq14IHE3BSEq9LqRH0DNeg/s1872/fraud%20shitstorm.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1461" data-original-width="1872" height="313" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi5rSbbfXKVDImSp7MaJRWriuj3p6kS8JgFIGZEzTOEg81PzncrV8cgN_nXq3ibPAlTdqbOjtnmjef-RgFe2bsoMD1i17MTgFMJq4Vqx98wu9juEiAvzIQ6gh7ZCX-kmVzE4pdhE2xvdBLBwxlshRyuKWII5bkyf0QX2_qpq14IHE3BSEq9LqRH0DNeg/w400-h313/fraud%20shitstorm.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Publishers trying to mop up papermill outputs<br /></td></tr></tbody></table> <p></p><p class="MsoListParagraphCxSpMiddle">Yes, self-correction will ultimately happen
in science, when people find they cannot replicate the flawed research on which
they try to build. But the time-scale for such self-correction is often far
longer than it needs to be.<span style="mso-spacerun: yes;"> </span>We have to
understand just how much waste of time and money is caused by reliance on a
passive, natural evolution of self-correction, rather than a more proactive
system to root out fraud.<span style="mso-spacerun: yes;"> </span></p>
<p class="MsoListParagraphCxSpMiddle"> </p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo3; text-indent: -18pt;"><b><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">2.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span></b><b>Full transparency</b></p>
<p class="MsoListParagraphCxSpMiddle">There’s been a fair bit of debate about
open data, and now it is recognised that we also need open code (scripts to
generate figures etc.) to properly evaluate results. I would go further,
though, and say we also need open peer review. This need not mean that the peer
reviewer is identified, but just that their report is available for others to
read. I have found <a href="http://deevybee.blogspot.com/2022/09/so-do-we-need-editors.html" target="_blank">open peer reviews very useful in identifying papermill products</a>. </p>
<p class="MsoListParagraphCxSpMiddle"><b> </b></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo3; text-indent: -18pt;"><b><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">3.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span></b><b>Develop shared standards</b></p>
<p class="MsoListParagraphCxSpMiddle">Organisations such as the Committee on
Publication Ethics (COPE) give <a href=" https://publicationethics.org/resources/flowcharts/systematic-manipulation-publication-process" target="_blank">recommendations for editors</a> about how to respond when
an accusation of misconduct occurs. <span style="mso-spacerun: yes;"></span>Although this looks like a start in
specifying standards to which reputable journals should adhere, several
speakers at the AIMOS meeting suggested that COPE guidelines were not suited
for dealing with papermills and could actually delay and obfuscate
investigations. Furthermore, COPE has no regulatory power and publishers are
under no obligation to follow the guidelines (even if they state they will do
so). </p>
<p class="MsoListParagraphCxSpMiddle"> </p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo3; text-indent: -18pt;"><b><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">4.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span></b><b>National bodies for promoting scientific
integrity</b></p>
<p class="MsoListParagraphCxSpMiddle">The Printeger project (cited above) noted
that “<i>A typical reaction of a research organisation facing unfamiliar research
misconduct without appropriate procedures is to set up ad hoc investigative
committees, usually consisting of in-house senior researchers…. Generally, this
does not go well</i>.”</p>
<p class="MsoListParagraphCxSpMiddle">In response to some high-profile cases that
did not go well, some countries have set up national bodies for promoting scientific
integrity. These are growing in number, but those who report cases to them
often complain that they are not much help when fraud is discovered – sometimes
this is because they lack the funding to defend a legal challenge. But, as with
shared standards, this is at least a start, and they may help gather data on
the scale and nature of the problem. <span style="mso-spacerun: yes;"> </span></p>
<p class="MsoListParagraphCxSpMiddle"> </p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo3; text-indent: -18pt;"><b><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">5.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span></b><b>Transparent discussion of breaches of
research integrity</b></p>
<p class="MsoListParagraphCxSpMiddle">Perhaps the most effective way of
persuading institutions, publishers and funders to act is by publicising when
they have failed to respond adequately to complaints.<span style="mso-spacerun: yes;"> </span>David Sanders described a case where journals
and institutions took no action despite multiple examples of image manipulation
and plagiarism from one lab.<span style="mso-spacerun: yes;"> </span>He only got
a response when <a href="https://www.nytimes.com/2017/03/08/science/cancer-carlo-croce.html" target="_blank">the case was featured in the New York Times</a>.
</p>
<p class="MsoListParagraphCxSpMiddle">Nevertheless, as the Printeger project
noted, relying on the media to highlight fraud is far from ideal – there can a
tendency to sensationalise and simplify the story, with potential for disproportionate
damage to both accused and whistleblowers. <span style="mso-spacerun: yes;">If we had trustworthy and official channels to report suspected research misconduct, then whistleblowers would be less likely to seek publicity through other means. <br /></span></p>
<p class="MsoListParagraphCxSpMiddle"> </p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo3; text-indent: -18pt;"><b><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">6.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span></b><b>Protect whistleblowers</b></p>
<p class="MsoListParagraphCxSpMiddle">In her introduction to the Liverpool
Research Integrity seminar, Patricia Murray noted the lack of consistency in
institutional guidelines on research integrity. In some cases, the approach to
whistleblowers seemed hostile, with the guidelines emphasising that they would
be guilty of misconduct if they were found to have made frivolous, vexatious
and/or malicious allegations. This, of course, is fair enough, but it needs to
be countered by recommendations that allow for whistleblowers who are none of
these things, who are doing the institution a service by casting light on
serious problems. Indeed, Prof Murray noted that in her institution, <b>failure</b>
to report an incident that gives reasonable suspicion of research misconduct is
itself regarded as misconduct.<span style="mso-spacerun: yes;"> </span>At
present, whistleblowers are often treated as nuisances or cranks who need to be
shut down. As was evident from the cases of both Sanders and Wilmshurst, they
are at risk of litigation, and careers may be put in jeopardy if they challenge
senior figures. </p>
<p class="MsoListParagraphCxSpMiddle"> </p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo3; text-indent: -18pt;"><b><span style="mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">7.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-style: normal; font-variant-caps: normal; font-weight: normal; font: 7pt "Times New Roman"; line-height: normal;">
</span></span></span></b><b>Changing the incentive structure in
science</b></p>
<p class="MsoListParagraphCxSpLast">It’s well-appreciated that if you really want
to stop a problem, you should understand what causes it and stop it at source.
People do fraudulent research because the potential benefits are large and the
costs seem negligible.<span style="mso-spacerun: yes;"> </span>We can change
that balance by, on the one hand having serious and public sanctions for those
who commit fraud, and on the other hand, rewarding scientists who emphasise
integrity, transparency and accuracy in their work, rather than those that get flashy,
eyecatching results. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I'm developing my ideas on this topic and I welcome thoughts on these suggestions. Comments are
moderated and so do not appear immediately, but I will post any that are on
topic and constructive. </p><p class="MsoNormal"><br /></p><p class="MsoNormal"><br /></p>
<p class="MsoNormal"><i><b>Update 17th December 2022 </b></i></p><p class="MsoNormal"><i><b><br /></b></i></p><p class="MsoNormal">Jennifer Byrne suggested one further recommendation, as follows:<br /></p><p class="MsoNormal" style="-moz-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 11pt; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; margin: 0cm 0cm 0.0001pt; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">To change the incentive structure in scientific publishing. Journals are presently rewarded for publishing, as publishing drives both income (through subscriptions and/or open access charges) and the journal impact factor. In contrast, journals and publishers do not earn income and are not otherwise rewarded for correcting the literature that they publish. This means that the (seemingly rare) journals that work hard to correct, flag and retract erroneous papers are rewarded identically to journals that appear to do very little. Proactive journals appear to represent a minority, but while there are no incentives for journals to take a proactive approach to published errors and misinformation, it should not be surprising that few journals join their efforts. Until publication and correction are recognized as two sides of the same coin, and valued as such, it seems inevitable that we will see a continued drive towards publishing more and correcting very little, or continuing to value publication quantity over quality.</p><p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: Calibri, sans-serif; font-size: 11pt; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; margin: 0cm 0cm 0.0001pt; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"> </p><p class="MsoNormal"></p>
<p class="MsoNormal"><span style="font-size: large;"><b>Bibliography <br /></b></span></p><p class="MsoNormal">I'll also add here additional resources. I'm certainly not the first to have made the points in this post, and it may be useful to have other articles gathered together in one place. </p><p class="MsoNormal"> </p><p class="MsoNormal">
</p><p class="MsoNormal">Besançon, L., Bik, E., Heathers, J., & Meyerowitz-Katz,
G. (2022). Correction of scientific literature: Too little, too late! <i>PLOS
Biology</i>, <i>20</i>(3), e3001572. <a href="https://doi.org/10.1371/journal.pbio.3001572">https://doi.org/10.1371/journal.pbio.3001572</a> <br /></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Byrne, J. A., Park, Y., Richardson, R. A. K., Pathmendra,
P., Sun, M., & Stoeger, T. (2022). Protection of the human gene research
literature from contract cheating organizations known as research paper mills.
Nucleic Acids Research, gkac1139. <a href="https://doi.org/10.1093/nar/gkac1139">https://doi.org/10.1093/nar/gkac1139</a> <br /></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Christian, K., Larkins, J., & Doran, M. R. (2022). <i>The
Australian academic STEMM workplace post-COVID: a picture of disarray</i>.
BioRxiv. <a href="https://doi.org/10.1101/2022.12.06.519378">https://doi.org/10.1101/2022.12.06.519378</a> <br /></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Lévy, R. (2022, December 15). Is it somebody else’s problem
to correct the scientific literature? <i>Rapha-z-Lab</i>. <a href="https://raphazlab.wordpress.com/2022/12/15/is-it-somebody-elses-problem-to-correct-the-scientific-literature/">https://raphazlab.wordpress.com/2022/12/15/is-it-somebody-elses-problem-to-correct-the-scientific-literature/</a> <br /></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><i>Research misconduct: Theory & Pratico – For Better
Science</i>. (n.d.). Retrieved 17 December 2022, from <a href="https://forbetterscience.com/2022/08/31/research-misconduct-theory-pratico/">https://forbetterscience.com/2022/08/31/research-misconduct-theory-pratico/</a> <br /></p><p class="MsoNormal"><br /></p>
<p class="MsoNormal"><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
mso-themecolor:hyperlink;
text-decoration:underline;
text-underline:single;}a:visited, span.MsoHyperlinkFollowed
{mso-style-noshow:yes;
mso-style-priority:99;
color:#954F72;
mso-themecolor:followedhyperlink;
text-decoration:underline;
text-underline:single;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style> </p><p class="MsoNormal" style="font-family: "Times New Roman", serif; margin: 0cm;"><i><span face="Calibri, sans-serif">Star marine ecologist committed misconduct, university says</span></i><span face="Calibri, sans-serif">. (n.d.). Retrieved 17 December 2022, from <a href="https://www.science.org/content/article/star-marine-ecologist-committed-misconduct-university-says" style="color: #954f72;">https://www.science.org/content/article/star-marine-ecologist-committed-misconduct-university-says</a> <o:p></o:p></span></p><p class="MsoNormal" style="font-family: "Times New Roman", serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: "Times New Roman", serif; margin: 0cm;"><b>Additions on 18th December</b>: Yet more relevant stuff coming to my attention! <br /></p><p class="MsoNormal" style="font-family: "Times New Roman", serif; margin: 0cm;"> <br /></p><p class="MsoNormal" style="font-family: "Times New Roman", serif; margin: 0cm;">Naudet, Florian (2022) Lecture: <a href="https://www.youtube.com/watch?v=PYxE7ohDP-8" target="_blank">Busting two zombie trials in a post-COVID world</a>. </p><p class="MsoNormal" style="font-family: "Times New Roman", serif; margin: 0cm;"> </p><p class="MsoNormal" style="font-family: "Times New Roman", serif; margin: 0cm;">Wilmshurst, Peter (2022) Blog: <a href="https://drpeterwilmshurst.wordpress.com/2022/11/15/has-cope-membership-become-a-way-for-unprincipled-journals-to-buy-a-fake-badge-of-integrity/ " target="_blank">Has COPE membership become a way for unprincipled journals to buy a fake badge of integrity? </a><br /></p><p class="MsoNormal" style="font-family: "Times New Roman", serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: "Times New Roman", serif; margin: 0cm;"><b> *Addition on 20th December</b></p><p class="MsoNormal" style="font-family: "Times New Roman", serif; margin: 0cm;">Liverpool Medical Institution seminar on Research Integrity: The introduction by Patricia Murray, talk by Peter Wilmshurt, and Q&A are now available on <a href="https://www.youtube.com/watch?v=J6b2SLJyzb4" target="_blank">Youtube</a>. <br /></p><p class="MsoNormal" style="font-family: "Times New Roman", serif; margin: 0cm;"><br /></p><p class="MsoNormal"><span style="font-size: large;"><b>And finally.... <br /></b></span></p><p class="MsoNormal">A couple of sobering thoughts:</p><p class="MsoNormal">
</p><p class="MsoNormal">Alexander Trevelyan on Twitter noted <span style="mso-spacerun: yes;"> </span>a great quote from the anonymous @mumumouse <span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-GB;">(author
of Research misconduct blogpost above):<span style="mso-spacerun: yes;"> </span>“To imagine what it’s like to be
a whistleblower in the science community, imagine you are trying to report a
Ponzi scheme, but instead of receiving help you are told, nonchalantly, to call
Bernie Madoff, if you wish." <br /></span></p><p class="MsoNormal"><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-GB;"> </span></p><p class="MsoNormal"><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-GB;">Peter Wilmshurst started his talk
by relaying a conversation with Patricia Murray in the run-up to his talk. He
said he planned to talk about the 3 Fs, fabrication, falsification and honesty.</span></p><p class="MsoNormal"><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-GB;"> </span><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-GB;">To which Patricia replied, “There
is no F in honesty”.<span style="mso-spacerun: yes;"> </span></span><br /></p></div><div style="text-align: left;">
<span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-GB;">(This may take a few moments to
appreciate).</span>
</div><div style="text-align: left;"><p class="MsoNormal"><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-469750017 -1073683329 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style></p><p class="MsoNormal"> </p><p class="MsoNormal"> <br /></p>
<p><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 1 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
mso-themecolor:hyperlink;
text-decoration:underline;
text-underline:single;}a:visited, span.MsoHyperlinkFollowed
{mso-style-noshow:yes;
mso-style-priority:99;
color:#954F72;
mso-themecolor:followedhyperlink;
text-decoration:underline;
text-underline:single;}p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}ol
{margin-bottom:0cm;}ul
{margin-bottom:0cm;}</style></p></div>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com12tag:blogger.com,1999:blog-5841910768079015534.post-44137689294094844902022-12-06T12:43:00.003+00:002022-12-10T08:55:58.928+00:00Biomarkers to screen for autism (again)<p class="MsoNormal"><br />Diagnosis of autism from biomarkers is a holy grail for
biomedical researchers. The days when it was thought we would find “the autism
gene” are long gone, and it’s clear that both the biology and the psychology of
autism is highly complex and heterogeneous. One approach is to search for
individual genes where mutations are more likely in those with autism. Another
is to address the complexity head-on by looking for combinations of biomarkers
that could predict who has autism.<span style="mso-spacerun: yes;"> </span>The
latter approach is adopted in a paper by <a href="https://www.nature.com/articles/s41380-022-01826-x" target="_blank">Bao et al (2022)</a> who claimed that an
ensemble of gene expression measures taken from blood samples could accurately
predict which toddlers were autistic (ASD) and which were typically-developing (TD). An <a href=" https://pubpeer.com/publications/B693366B2B51D143C713359F151F7B" target="_blank">anonymous commenter on PubPeer</a>
queried whether the method was as robust as the authors claimed, arguing that
there was evidence for “overfitting”. I was asked for my thoughts by a
journalist, and they were complicated enough to merit a blogpost. <span style="mso-spacerun: yes;"> </span>The bottom line is that there are reasons to
be cautious about the conclusion of the authors that they have developed “an
innovative and accurate ASD gene expression classifier”.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Some of the points I raise here applied to a previous biomarker
study that <a href=" http://deevybee.blogspot.com/2019/01/neuropointdxs-blood-test-for-autism.html " target="_blank">I blogged about in 2019.</a><span style="mso-spacerun: yes;"> </span>These are general issues about the
mismatch between what is done in typical studies in this area and what is
needed for a clinically useful screening test. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><b>Base rates</b></p>
<p class="MsoNormal">Consider first how a screening test might be used. One
possibility is that there might be a move towards universal screening, allowing
early diagnosis that might help ensure intervention starts young. <span style="mso-spacerun: yes;"> </span>But for effective screening in that context,
you need extremely high diagnostic accuracy, and accuracy depends on the
frequency of autism in the population.<span style="mso-spacerun: yes;"> </span>I
discussed this <a href="http://deevybee.blogspot.com/2010/07/difference-between-p-05-and-screening.html." target="_blank">back in 2010</a>.<span style="mso-spacerun: yes;"> </span>The levels of accurate classification reported by Bao et al would be of no use for population screening because there would be an
extremely high rate of false positives, given that most children don’t have
autism. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><b>Diagnostic specificity</b></p>
<p class="MsoNormal">But, you may say, we aren’t talking about universal
screening. <span style="mso-spacerun: yes;"> </span>The test might be particularly
useful for those who either (a) already have an older child with autism, or (b)
are concerned about their child’s development. <span style="mso-spacerun: yes;"> </span>Here the probability of a positive autism
diagnosis is higher than in the general population.<span style="mso-spacerun: yes;"> </span>However, if that’s what we are interested in,
then we need a different comparison group – not typically-developing toddlers,
but unaffected siblings of children with autism, and/or children with other neurodevelopmental
disorders. <span style="mso-spacerun: yes;"> </span></p>
<p class="MsoNormal">When I had a look at <a href="https://github.com/LewisLabUCSD/autism_classifier/blob/main/1_arranging_the_datasets/Test_LD_long_data_arranger_completeDataset.R " target="_blank">the code that the authors deposited for data analysis</a>, it implied that they did have data on children with more general
developmental delays, and sibs of those with autism, but they are not reported
in this paper. </p><p class="MsoNormal"> </p><p class="MsoNormal"><span style="mso-spacerun: yes;"></span></p>
<p class="MsoNormal">The analyses done by the researchers are extremely complex
and time-consuming, and it is understandable that they may prefer to start out
with the clearest case of comparing autism with typically-developing children.
But the acid test of the suitability of the classifier for clinical use would
be a demonstration that it could distinguish children with autism from unaffected
siblings, and from nonautistic children with intellectual disability. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><b>Reliability of measures</b></p>
<p class="MsoNormal">If you run a diagnostic test, an obvious question is whether
you’d get the same result on a second test run.<span style="mso-spacerun: yes;">
</span>With biological and psychological measures the answer is almost always
no, but the key issue for a screener is just how much change there is. Gene
expression levels could vary from occasion to occasion depending on time of day
or what you’d eaten – I have no idea how important this might be, but it's not possible to evaluate in this paper, where measures come from a
single blood sample. My personal view is that the whole field of biomedical
research needs to wake up to the importance of reliability of measurement so
that researchers don’t waste time exploring the predictive power of measures
that may be too unreliable to be useful.<span style="mso-spacerun: yes;">
</span>Information about stability of measures over time is a basic requirement
for any diagnostic measure. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">A related issue concerns comparability of procedures for
autism and TD groups. Were blood samples collected by the same clinicians over
the same period and processed in the same lab for these two groups? Were the
blood analyses automated and/or done blind? It’s crucial to be confident that
minor differences in clinical or lab procedures do not bias results in this
kind of study.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><b>Overfitting</b></p>
<p class="MsoNormal">Overfitting is really just a polite way of saying that the
data may be noise. If you run enough analyses, something is bound to look
significant, just by chance.<span style="mso-spacerun: yes;"> </span>In the
first step of the analysis, the researchers ran 42,840 models on “training” data
from 93 autistic and 82 TD children and found 1,822 of them performed better
than .80 on a measure that reflects diagnostic accuracy (AUC-ROC – which roughly
corresponds to proportion correctly classified: .50 is chance, and 1.00 is
perfect classification). <span style="mso-spacerun: yes;"> </span>So we can see
that just over 4% of the models (1822/42840) performed this well. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">The researchers were aware of the possibility of
overfitting, and they addressed it head-on, saying: “To test this, we permuted
the sample labels (i.e., ASD and TD) for all subjects in our Training set
and ran the pipeline to test all feature engineering and classification
methods. Importantly, we tested all 42,840 candidate models and found the
median AUC-ROC score was 0.5101 with the 95th CI (0.42–0.65) on the randomized
samples. As expected, only rare chance instances of good 'classification'
occurred.”<span style="mso-spacerun: yes;"> </span>The distribution of scores is
shown in Figure 2b. </p><p class="MsoNormal"> </p>
<p class="MsoNormal"> </p><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIiPP5X2PwSovD2GSdUvPeT3BmuPOHJXnIdfy3qfAUs3q7Jub4nJV-_XsUEod4-5ZGPfngKMx2RHiqj6nHLyTXsFTIdmmAa9HU_OgOOe4Ugdjw3ZwTpWFbBMz-Hw0cuhBuHomyh6PyG3stmH9dq04bojE7RtGrKz6T_3hzBhfCluFZgPLLc8Ns6PD9DA/s495/Screenshot%202022-12-06%20at%2012.11.39.png" style="display: block; margin-left: auto; margin-right: auto; padding: 1em 0px; text-align: center;"><img alt="" border="0" data-original-height="343" data-original-width="495" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIiPP5X2PwSovD2GSdUvPeT3BmuPOHJXnIdfy3qfAUs3q7Jub4nJV-_XsUEod4-5ZGPfngKMx2RHiqj6nHLyTXsFTIdmmAa9HU_OgOOe4Ugdjw3ZwTpWFbBMz-Hw0cuhBuHomyh6PyG3stmH9dq04bojE7RtGrKz6T_3hzBhfCluFZgPLLc8Ns6PD9DA/s320/Screenshot%202022-12-06%20at%2012.11.39.png" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 2b from Bao et al (2022)<br /></td></tr></tbody></table>
<p class="MsoNormal"> </p>
<p class="MsoNormal">They then ran a further analysis on a “test set” of 34 autistic and
31 TD children who had been held out of the original analysis, and found that
742 of the 1822 models performed better than .80 in classification. That’s 40%
of the tested models.<span style="mso-spacerun: yes;"> </span>Assuming I have
understood the methods correctly, that does look meaningful and hard to explain
just in terms of statistical noise. <span style="mso-spacerun: yes;"> </span>In
effect, they have run a replication study and found that a substantial subset
of the identified models do continue to separate autism and TD groups when new
children are considered. The claim is that there is substantial overlap in the models that fall in the right-hand area under the curve for the red and pink distributions.<br /></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">The PubPeer commenter seems concerned that results look too
good to be true. In particular, Figure 2b suggests the models perform a bit better
in the test set than in the training set. But the figure shows the distribution
of scores for all the models (not just the selected models) and, given the
small sample sizes, the differences between distributions does not seem large
to me. I was more surprised by the relatively tight distribution of AUC-ROC
values obtained in the permutation analysis, as I would have anticipated some
models would have given high classification accuracy just by chance in a sample
of this size. </p><p class="MsoNormal">The researchers went on to present data for the set of
models that achieved .8 classification in both training and test sets. This
seemed a reasonable approach to me. The PubPeer commenter is correct in arguing that there will be some bias caused by selecting models this way, and that one would expect less good performance in a completely new sample, but the 2-stage selection of models would seem to ensure there is not "massive overfitting". I think there would be a problem if only 4% of the 1822 selected models had given accurate classification, but the good rate of agreement between the models selected in the training and test samples, coupled with the lack of good models in the permuted data, suggests there is a genuine effect here. </p><p class="MsoNormal"> </p>
<p class="MsoNormal"><b>Conclusion</b></p>
<p class="MsoNormal">So, in sum, I think that the results can’t just be
attributed to overfitting, but I nevertheless have reservations about whether
they would be useful for screening for autism. <span style="mso-spacerun: yes;"> </span>And one of the first things I’d check if I
were the researchers would be the reliability of the diagnostic classification
in repeated blood samples taken on different occasions, as that would need to
be high for the test to be of clinical use. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="font-size: x-small;"><b>Note: </b>I'd welcome comments or corrections on this post. Please note, comments are moderated to avoid spam, and so may not appear immediately. If you post a comment and it has not appeared in 24 hr, please email me and I'll ensure it gets posted. </span><br /></p>
<p class="MsoNormal"> PS. See comment from original PubPeer poster attached. <br /></p><p class="MsoNormal">Also, 8th Dec 2022, I added a further PubPeer comment asking authors to comment on Figure 2B, which does seem odd. <br /></p><p class="MsoNormal"><a href="https://pubpeer.com/publications/B693366B2B51D143C713359F151F7B#4 " target="_blank">https://pubpeer.com/publications/B693366B2B51D143C713359F151F7B#4 </a></p><p class="MsoNormal"><br /></p><p class="MsoNormal">PPS. 10th Dec 2022. <br /></p><p class="MsoNormal">Author Eric Courchesne has responded to several of the points made in this blogpost on Pubpeer: <a href="https://pubpeer.com/publications/B693366B2B51D143C713359F151F7B#5 " target="_blank">https://pubpeer.com/publications/B693366B2B51D143C713359F151F7B#5 </a> </p><p class="MsoNormal"><br /></p><p class="MsoNormal"><br /></p><p class="MsoNormal"> </p><p class="MsoNormal"><br /></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
mso-themecolor:hyperlink;
text-decoration:underline;
text-underline:single;}a:visited, span.MsoHyperlinkFollowed
{mso-style-noshow:yes;
mso-style-priority:99;
color:#954F72;
mso-themecolor:followedhyperlink;
text-decoration:underline;
text-underline:single;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com1tag:blogger.com,1999:blog-5841910768079015534.post-75620012105877483552022-10-12T12:32:00.005+01:002022-10-12T13:52:24.472+01:00What is going on in Hindawi special issues?<p><span style="font-size: small;"><b>A guest blogpost by Nick Wise</b> </span></p><p><i> http://www.eng.cam.ac.uk/profiles/nhw24 </i></p><p><br />
The Hindawi journal Wireless Communications and Mobile Computing is booming. Until a few years ago they published 100-200 papers a year, however they published 269 papers in 2019, 368 in 2020 and 1,212 in 2021. So far in 2022 they have published 2,429. This growth has been achieved primarily by the creation of special issues, which makes sense. It would be nearly impossible for a journal to increase its publication rate by an order of magnitude in 2 years without outsourcing the massive increase in workload to guest editors.</p><p>Recent special issues include ‘Machine Learning Enabled Signal Processing Techniques for Large Scale 5G and 5G Networks’ (182 articles), ‘Explorations in Pattern Recognition and Computer Vision for Industry 4.0’ (244) and ‘Fusion of Big Data Analytics, Machine Learning and Optimization Algorithms for Internet of Things’ (204). Each of these special issues contains as many papers as the journal published in a year until recently. They also contain many papers that are flagged on Pubpeer for irrelevant citations, tortured phrases and surprising choices of corresponding email addresses.</p><p>However, I am going to focus on one special issue that is still open for submissions, and so far contains a modest 62 papers: ‘<a href="https://www.hindawi.com/journals/wcmc/si/890259/page/1/" target="_blank">AI-Driven Wireless Energy Harvesting in Massive IoT for 5G and Beyond</a>’, edited by Hamurabi Gamboa Rosales, Danijela Milosevic and Dijana Capeska Bogatinoska. Given the title of the special issue, it is perhaps surprising that only two of the articles contain ‘wireless’ in the title and none contain ‘energy’. The authors of the other papers (or whoever submitted them) appear to have realised that as long as they included the buzzwords ‘AI’, ‘IoT’ (Internet of Things) or ‘5G’ in the title, the paper could be about anything at all. Hence, the special issue contains titles such as:</p><ul style="text-align: left;"><li>
Analysis Model of the Guiding Role of National Sportsmanship on the Consumer Market of Table Tennis and Related IoT Applications </li><li>Evaluation Method of the Metacognitive Ability of Chinese Reading Teaching for Junior Middle School Students Based on Dijkstra Algorithm and IoT Applications </li><li>The Construction of Shared Wisdom Teaching Practice through IoT Based on the Perspective of Industry-Education Integration<br /></li></ul><p>
Of the 62 papers, 60 give Hamurabi Gamboa Rosales as the academic editor and 2 give Danijela Milosevic. Why is the distribution of labour so lopsided? One can imagine an arrangement where the lead editor does the admin of waving through irrelevant papers and the other 2 guest editors get to say that they’ve guest-edited a special issue on their CV.</p><p>Of course, in addition to boosting publication numbers for the authors and providing CV points for the guest editors, every paper in the special issue has a references section. Each reference gives someone a citation, another academic brownie point on which careers can be built. An anonymous Pubpeer sleuth has trawled through the references section of every paper in this special issue and found that Malik Bader Alazzam of Amman Arab University in Jordan has been cited 139 times across the 62 papers. The chance that the authors of almost every article would independently decide to cite the same person seems small.</p><p>The most intriguing fact about the papers in the special issue however, is that only 4 authors give corresponding email addresses that match their affiliation. These 4 include the only 3 papers with non-Chinese authors. Of the other 58, 1 uses an email address from Guangzhou University, 6 use email addresses from Changzhou University, and 51 use email addresses from Ma’anshan University. All of the Ma’anshan addresses are of the form 1940XXXX@masu.edu.cn and many are nearly sequential, suggesting that someone somewhere purchased a block of sequential email addresses (you do not need to be at Ma’anshan University to have an @masu email address). The screenshot below shows a sample (the full dataset is linked <a href="https://docs.google.com/spreadsheets/d/1gSRiC2YnI0UAbZ-uapWcxQaDr5mk7GaUpSdCd_mIhiU/edit#gid=0" target="_blank">here</a>).</p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyKn-02vbgfD9hBaLvn-DJ2RtXhsdwkmJD3SKwwoXqwa93BvM5Eg_XXDjXywGxTTwe9rSvDxIf7wREG6Op7Xc04mnx0ihFKmrRi0MTqyMjugEX_IoUI6UibG9m01Y9gqhSf-1iY_tQTkUsBT3v29unWp5PpdQrxFdwB_cIc2v66wlCJafSUrJ4qLscvA/s624/Screenshot%202022-10-12%20at%2012.26.34.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="216" data-original-width="624" height="139" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyKn-02vbgfD9hBaLvn-DJ2RtXhsdwkmJD3SKwwoXqwa93BvM5Eg_XXDjXywGxTTwe9rSvDxIf7wREG6Op7Xc04mnx0ihFKmrRi0MTqyMjugEX_IoUI6UibG9m01Y9gqhSf-1iY_tQTkUsBT3v29unWp5PpdQrxFdwB_cIc2v66wlCJafSUrJ4qLscvA/w400-h139/Screenshot%202022-10-12%20at%2012.26.34.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><h4><i><span id="docs-internal-guid-d3dfae1b-7fff-4f1a-a068-c1ccb63de5c4" style="background-color: transparent; color: black; font-family: Arial; font-size: 11pt; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">A subset of the titles from the special issue with their corresponding email addresses, all of the form 1940XXXX@masu.edu.cn</span></i></h4></td></tr></tbody></table><p></p><p>The use and form of the email addresses suggests that all of these papers are the work of a paper mill. It is hard to imagine otherwise how 51 different authors could submit papers to the same special issue using the same institutional email domain and format. Indeed, before 2022 only 2 papers had ever used @masu.edu.cn as a corresponding address <a href="https://app.dimensions.ai/discover/publication?search_mode=content&search_text=%22%40masu.edu.cn%22&search_type=kws&search_field=full_search" target="_blank">according to Dimensions</a>. It is equally hard to imagine how Hamurabi Gamboa Rosales is unaware. How can you not notice that, of the 19 papers you receive for your special issue on the 12th of July, 18 use the same email domain that doesn’t match their affiliation? This may also explain why Hamurabi has dealt with almost all the papers himself. This special issue should be closed for submissions and an investigation begun.</p><p>Stepping back from this special issue, this is not an isolated problem. There are at least 40 other papers published in Wireless Communications and Mobile Computing with corresponding emails from Ma’anshan, and Dimensions finds there are 46 in Computational Intelligence and Neuroscience, 38 in Computational and Mathematical Methods in Medicine and 30 in Mobile Information Systems, all published in 2022 and all in Hindawi journals. What are the chances that 18404032@masu.edu.cn is used in a special issue in Computational Intelligence and Neuroscience, 18404038@masu.edu.cn in Disease Markers and 18404041@masu.edu.cn in Wireless Communications and Mobile Computing?</p><p>Finally, masu.edu.cn is only one example of a commonly used email domain that doesn’t match the author’s affiliation. It is conceivable that the entire growth in publications of Wireless Communications and Mobile Computing, Computational Intelligence and Neuroscience (163 articles in 2020, 3,079 in 2022) and Computational and Mathematical Methods in Medicine (225 in 2020, 1,488 in 2022) is from paper mills publishing in corrupted special issues.</p><p><br />
Nick Wise</p><p><br />
*All numbers accurate as of the 12th October 2022.
</p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p><p></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com7tag:blogger.com,1999:blog-5841910768079015534.post-88569163447159256722022-10-04T11:35:00.005+01:002022-10-04T11:39:17.922+01:00A desire for clickbait can hinder an academic journal's reputation<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixJ91GzEJUoi4P_e_6UsDX8ENKxzvgldJIJtxc4ZdbPdU92Xj3Y-GN5e2BOBL4dBMPDirg7A_Ldz95xLjkbLZS2ljrJCaZGxDOFGqVP-zN8pl5Bdz4u3MMnQd7-MYNTxaAzVJ0kCNUGqq22tUr4bLPG3_xpn3XB19jZMqI7H3hjz6aKHZYZzc3gQ-GrQ/s1296/Screenshot%202022-10-04%20at%2011.36.13.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="668" data-original-width="1296" height="165" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixJ91GzEJUoi4P_e_6UsDX8ENKxzvgldJIJtxc4ZdbPdU92Xj3Y-GN5e2BOBL4dBMPDirg7A_Ldz95xLjkbLZS2ljrJCaZGxDOFGqVP-zN8pl5Bdz4u3MMnQd7-MYNTxaAzVJ0kCNUGqq22tUr4bLPG3_xpn3XB19jZMqI7H3hjz6aKHZYZzc3gQ-GrQ/s320/Screenshot%202022-10-04%20at%2011.36.13.png" width="320" /></a></div><br /><p></p><p>On 28th September, I woke up to look at Twitter and find Pete Etchells fulminating about a piece in the Guardian. <br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2BGCBQBAbrVkrZPGTianR2JtoIa029ZEn9n9-O8t2QHhYBTDi6ywDDubatWw36yxWqLbI7H2vVrH7rYbJIC7L95ey_WlXFj4IxH4WbpkYzzh0TU7l23YX5DXwbju4Da_TTAR4T4_3NqiXiXe9vHTzKDcp2EwWfOWGD23KzK5nVsJNVEEPcnFzLCHy4g/s625/Screenshot%202022-10-04%20at%2010.18.38.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="541" data-original-width="625" height="277" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2BGCBQBAbrVkrZPGTianR2JtoIa029ZEn9n9-O8t2QHhYBTDi6ywDDubatWw36yxWqLbI7H2vVrH7rYbJIC7L95ey_WlXFj4IxH4WbpkYzzh0TU7l23YX5DXwbju4Da_TTAR4T4_3NqiXiXe9vHTzKDcp2EwWfOWGD23KzK5nVsJNVEEPcnFzLCHy4g/s320/Screenshot%202022-10-04%20at%2010.18.38.png" width="320" /></a></div><p></p><p></p><p>
It was particularly galling for him to read a piece that implied research studies had shown that voice-responsive devices were harming children’s development when he and Amy Orben had provided <a href=" https://www.sciencemediacentre.org/expert-reaction-to-an-opinion-piece-on-voice-controlled-devices-and-child-development/ " target="_blank">comments to the Science Media Centre</a> that were available to the journalist. They both noted that: </p><p>
a) This was a Viewpoint piece, not new research </p><p>b) Most of the evidence it provided consisted of anecdotes from newspaper articles </p><p>
I agreed with Pete’s criticism of the Guardian, but having read the original Viewpoint in the Archives of Disease in Childhood, I had another question, namely, why on earth was a reputable paediatrics journal doing a press release on a flimsy opinion piece written by two junior medics with no track record in the area? </p><p>
</p><p>
So I wrote to the Editor with my concerns, as follows: </p><p></p><p style="margin-left: 40px; text-align: left;">
Dear Dr Brown </p><p style="margin-left: 40px; text-align: left;"></p><p style="margin-left: 40px; text-align: left;">
<b>Viewpoint: Effects of smart voice control devices on children: current challenges and future perspectives doi 10.1136/archdischild-2022-323888 Journal: Archives of Disease in Childhood </b></p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
I am writing to enquire why this Viewpoint was sent out to the media under embargo as if it was a substantial piece of new research. I can understand that you might want to publish less formal opinion pieces from time to time, but what I cannot understand is the way this was done to attract maximum publicity by the media. </p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
The two people who commented about it for the Science Media Centre both noted this was an opinion piece with no new evidence, relying mainly on media reports. </p><p style="margin-left: 40px; text-align: left;">
https://www.sciencemediacentre.org/expert-reaction-to-an-opinion-piece-on-voice-controlled-devices-and-child-development/ </p><p style="margin-left: 40px; text-align: left;">
Unfortunately, despite this warning, it has been picked up by the mainstream media, where it is presented as ‘new research’, which will no doubt give parents of young children something new to worry about. </p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
I checked out the authors, and found these details: </p><p style="margin-left: 40px; text-align: left;">
https://orcid.org/0000-0003-4881-8293 </p><p style="margin-left: 40px; text-align: left;">
https://www.researchgate.net/profile/Ananya-Arora-3 </p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
These confirm that neither has a strong research track record, or any evidence of expertise in the topic of the Viewpoint. I can only assume that ADC is desperate for publicity at any cost, regardless of scientific evidence or impact on the public. </p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
As an Honorary Fellow of the Royal College of Paediatrics and Child Health, and someone who has previously published in ADC, I am very disappointed to see the journal sink so low. </p><p>
</p><p>
Yesterday I got a reply that did nothing to address my concerns. Here’s what the editor, Nick Brown*, said (in italic), with my reactions added: </p><p>
</p><p style="margin-left: 40px; text-align: left;">
<i>Thank you for making contact . My response reflects the thoughts of both the BMJ media and publication departments </i></p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
Given my reactions, below, this is more worrying than reassuring. It would be preferable to have heard that there had been some debate as to the wisdom of promoting this article to the press. </p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
<i>It is a key role of a scientific journal to raise awareness of, and stimulate debate on, live and emerging issues. Voice control devices are becoming increasingly common and their impact on children's development is a legitimate topic of discussion. </i> </p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
I have no quarrel with the idea that impact of voice control devices on children is a legitimate topic for the journal. But I wonder about how far its role is ‘raising awareness of, and stimulating debate’ when the topic is one on which we have very little evidence. A scientific journal might be expected to provide a balanced account of evidence, whereas the Viewpoint presented one side of the ‘debate’, mainly using anecdotes. I doubt it would have been published if it had concluded that there was no negative impact of voice control devices. </p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
<i>Opinion pieces are part of a very wide range of content that is selected for press release from among BMJ's portfolio of journals. They are subject to internal review in line with BMJ journals´overall editorial policy: the process (intentionally) doesn't discriminate against authors who don't have a strong research track record in a particular field </i></p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
I’ve been checking up on how frequently ADC promotes an article for press release. This information can be obtained <a href="https://www.bmj.com/company/newsroom/." target="_blank">here</a>. This year, they have published 219 papers, of which three other articles have merited a press release: an analysis of survey data on weight loss (July), a research definition of Long Covid in children (February) and a data-based analysis of promotional claims about baby food (February). Many of the papers that were not press-released are highly topical and of general interest – a quick scan found papers on vaping, monkey pox, transgender adolescents, unaccompanied minors as asylum seekers, as well as many papers relating to Covid. It’s frankly baffling why a weakly evidenced viewpoint on a topic with little evidence was selected as meriting special treatment with a press release. </p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
<i>As for the press release pathway itself, all potential pieces are sent out under embargo, irrespective of article type. This maximises the chances of balanced coverage: an embargo period enables journalists to contact the authors with any queries and to contact other relevant parties for comment. </i></p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
My wording may have been clumsy here and led to misunderstanding. My concern was more with the fact that the paper was press-released, which is, as established above, highly unusual, rather than with the embargo. </p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
<i>The press release clearly stated (3 times) this article was a viewpoint and not new research, and that it hadn't been externally peer reviewed. We also always include a direct URL link to the article in question in our press releases so that journalists can read the content in full for themselves. </i></p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
I agree that the press release included these details, and indeed, had journalists consulted the Science Media Centre’s commentaries, the lack of peer review and data would have been evident. But nevertheless, it’s well-known that (a) journalists seldom read original sources, and (b) some of the less reputable newspapers are looking for clickbait, so why provide them with the opportunity for sensationalising journal content?</p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
<i>While we do all we can to ensure that journalists cover our content responsibly, we aren't responsible for the manner in which they choose to do so. </i></p><p style="margin-left: 40px; text-align: left;">
</p><p style="margin-left: 40px; text-align: left;">
I agree that part of the blame for the media coverage lies with journalists. But I think the journal must bear some responsibility for the media uptake of the article. It’s a reasonable assumption that if a reputable journal issues a press release, it’s because the article in question is important and provides novel information from recognised experts in the field. It is unfortunate that that assumption was not justified in that case. </p><p style="margin-left: 40px; text-align: left;">
</p><p>
I just checked to see how far the media interest in the story had developed. The Guardian, confronted with criticism, changed the lede to say “Researchers suggest”, rather than “New research says”, but the genie was well out of the bottle by that time. The paper has an Altmetric ‘attention’ score of 1577, and been picked up by 209 news outlets. There’s no indication that the article has “stimulated debate”. Rather it has been interpreted as providing a warning about a new danger facing children. The headlines, which can be found <a href=" https://bmj.altmetric.com/details/136527876/news/page:1," target="_blank">here</a>, are variants of: </p><p style="margin-left: 40px; text-align: left;"> “Alexa and Siri make children rude” </p><p style="margin-left: 40px; text-align: left;">
“Siri, Alexa and Google Home could hinder children’s social and cognitive development” </p><p style="margin-left: 40px; text-align: left;">
“Voice-control devices may have an impact on children’s social, emotional development: Study” </p><p style="margin-left: 40px; text-align: left;">
“According to a study, voice-controlled electronic aides can impair children’s development” </p><p style="margin-left: 40px; text-align: left;">
“Experts warn that AI assistants affect children’s social development” </p><p style="margin-left: 40px; text-align: left;">
“Experts warn AI assistants affect social growth of children” </p><p style="margin-left: 40px; text-align: left;">
“Why Alexa and Siri may damage kids’ social and emotional development” </p><p style="margin-left: 40px; text-align: left;">
“Voice assistants harmful for your child’s development, claims study” </p><p style="margin-left: 40px; text-align: left;">
“Alexa, Siri, and Other Voice Assistants could negatively rewire your child’s brain” </p><p style="margin-left: 40px; text-align: left;">
“Experts warn using Alexa and Siri may be bad for children” </p><p style="margin-left: 40px; text-align: left;">
“Parents issued stark warning over kids using Amazon’s Alexa” </p><p style="margin-left: 40px; text-align: left;">
“Are Alexa and Siri making our children DUMB?” </p><p style="margin-left: 40px; text-align: left;">
“Use of voice-controlled devices ‘might have long-term consequences for children’” </p><p>
And most alarmingly, from the Sun: </p><p style="margin-left: 40px; text-align: left;">
“Urgent Amazon Alexa warning for ALL parents as new danger revealed” </p><p>Maybe the journal’s press office regards that as a success. I think it’s a disaster for the journal’s reputation as a serious academic journal. <br /></p><p>
</p><p>
<span style="font-size: x-small;">*Not the sleuth Nick Brown. Another one. </span></p><p>
</p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com1tag:blogger.com,1999:blog-5841910768079015534.post-9865239118004865162022-09-30T14:11:00.006+01:002022-09-30T14:35:21.069+01:00Reviewer-finding algorithms: the dangers for peer review<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgh_iipJSlOUdy-rjnrXzOnJfpMqVKhV6sz54Ilf0mrlrCA8DLMBqVjzjRNx5AiRIPXnJH_ZGtKsr4rq8eHni-qzwFpuFmA5WC8S3YgHIa3-9uJoUHkzVXPDkc65nJIut82Si4_nt5nWqwFtp1rF0_Zq8ZJ5qrw8x4jQprQW6IBvsZi-O5vBK4iP8rpNA/s3300/CartoonStock_551193_CS194183.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="2630" data-original-width="3300" height="255" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgh_iipJSlOUdy-rjnrXzOnJfpMqVKhV6sz54Ilf0mrlrCA8DLMBqVjzjRNx5AiRIPXnJH_ZGtKsr4rq8eHni-qzwFpuFmA5WC8S3YgHIa3-9uJoUHkzVXPDkc65nJIut82Si4_nt5nWqwFtp1rF0_Zq8ZJ5qrw8x4jQprQW6IBvsZi-O5vBK4iP8rpNA/s320/CartoonStock_551193_CS194183.jpeg" width="320" /></a></div><br /><p></p><p>Last week many words were written for <a href="https://peerreviewweek.wordpress.com/" target="_blank">Peer Review Week,</a> so you might wonder whether there is anything left to say. I may have missed something, but I think I do have a novel take on this, namely to point out that some recent developments in automation may be making journals vulnerable to fake peer review. </p><p>Finding peer reviewers is notoriously difficult these days. Editors are confronted with a barrage of submissions, many outside their area. They can ask authors to recommend peer reviewers, but this raises concerns of malpractice, if authors recommend friends, or even individuals tied up with <a href="https://blog.mdpi.com/2022/05/09/paper-mills/" target="_blank">paper mills</a>, who might write a positive review in return for payment.
</p><p>One way forward is to harness the power of big data to identify researchers who have a track record of publishing in a given area. Many publishers now use such systems. This way a journal editor can select from a database of potential reviewers that is formed by identifying papers with some overlap to a given submission.
</p>
I have become increasingly concerned, however, that use of algorithmically-based systems might leave a journal vulnerable to fraudulent peer reviewers who have accumulated publications by using paper mills. I became interested in this when submitting work to Wellcome Open Research and F1000, where open peer review is used, but it is the author rather than an editor who selects reviewers. Clearly, with such a system, one needs to be careful to avoid malpractice, and strict criteria are imposed. As explained <a href="https://wellcomeopenresearch.org/for-authors/tips-for-finding-referees">here</a>, reviewers need to be:
<ol style="text-align: left;"><li>Qualified: typically hold a doctorate (PhD/MD/MBBS or equivalent). </li><li>Expert: have published at least three articles as lead author in a relevant topic, with at least one article having been published in the last five years. </li><li>Impartial: No competing interests and no co-authorship or institutional overlap with current authors. </li><li>Global: geographically diverse and from different institutions. </li><li>Diverse: in terms of gender, geographic location and career stage
</li></ol><p>
Unfortunately, now that we have paper mills, which allow authors, for a fee, to generate and publish a large number of fake papers, these criteria are inadequate.
Consider the case of Mohammed Sahab Uddin, who features in <a href="https://retractionwatch.com/2022/08/24/how-a-tweet-sparked-an-investigation-that-led-to-a-phd-student-leaving-his-program/." target="_blank">this account</a> in Retraction Watch. As far as I am aware, he does not have a doctorate*, but I suspect people would be unlikely to query the qualifications of someone who had 137 publications and an H-index of 37. By the criteria above, he would be welcomed as a reviewer from an underrepresented location. And indeed, he was frequently used as a reviewer: Leslie McIntosh, who unmasked Uddin’s deception, noted that before he wiped his Publons profile, he had been listed as a reviewer on 300 papers. </p><p>This is not an isolated case. We are only now beginning to get to grips with the scale of the problem of paper mills. There are undoubtedly many other cases of individuals who are treated as trusted reviewers on the back of fraudulent publications. Once in positions of influence, they can further distort the publication process. As I noted in <a href="http://deevybee.blogspot.com/2022/09/so-do-we-need-editors.html" target="_blank">last week's blogpost</a>, open peer review offers a degree of defence against this kind of malpractice, as readers will at least be able to evaluate the peer review, but it is disturbing to consider how many dubious authors will have already found themselve promoted to positions of influence based on their apparently impressive track record of publishing, reviewing and even editing.
</p><p>
I started to think about this might interact with other moves to embrace artificial intelligence. A recent <a href="https://www.timeshighereducation.com/news/funders-mull-robot-reviewers-research-excellence-framework" target="_blank">piece in Times Higher Education</a> stated: “<i>Research England has commissioned a study of whether artificial intelligence could be used to predict the quality of research outputs based on analysis of journal abstracts, in a move that could potentially remove the need for peer review from the Research Excellence Framework (REF).</i>” This seems to me to be the natural endpoint of the move away from trusting the human brain in the publication process. We could end up with a system where <a href=" https://pdos.csail.mit.edu/archive/scigen/">algorithms write the papers</a>, which are <a href="https://www.nature.com/articles/d41586-021-00733-5" target="_blank">attributed to fake authors</a>, peer reviewed by fake peer reviewers, and ultimately evaluated in the Research Excellence Framework by machines. Such a system is likely to be far more successful than mere mortals, as it will be able to rapidly and flexibly adapt to changing evaluation criteria. At that point, we will have dispensed with the need for human academics altogether and have reached peak academia. </p><p><span style="font-size: x-small;"> *Correction 30/9/22: Leslie McIntosh tells me he does have a doctorate and was working on a postdoc. </span><br /></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com1tag:blogger.com,1999:blog-5841910768079015534.post-74027084553053870422022-09-11T12:03:00.000+01:002022-09-11T12:03:05.678+01:00So do we need editors?<p>It’s been an interesting week in world politics, and I’ve been distracting myself by pondering the role of academic editors. The week kicked off with a rejection of <a href="https://psyarxiv.com/2yf8z/ " target="_blank">a preprint </a>written with co-author Anna Abalkina, who is an expert sleuth who tracks down academic paper mills – organisations that will sell you a fake publication in an academic journal. Our paper describes a paper mill that had placed six papers in the Journal of Community Psychology, a journal which celebrated its <a href=" https://onlinelibrary.wiley.com/doi/full/10.1002/jcop.22721" target="_blank">50th anniversary in 2021</a>. We had expected rejection, as we submitted the paper to the Journal of Community Psychology, as a kind of stress test to see whether the editor, Michael B. Blank, actually reads papers that he accepts for the journal. I had started to wonder, because you can read his decision letters on Publons, and they are identical for every article he accepts. I suspected he may be an instance of <a href="http://deevybee.blogspot.com/2010/09/science-journal-editors-taxonomy.html" target="_blank">Editoris Machina,</a> or automaton, one who just delegates editorial work to an underling, waits until reviewer reports converge on a recommendation, and then accepts or rejects accordingly without actually reading the paper. I was wrong, though. He did read our paper, and rejected it with the comment that it was a superficial analysis of six papers. We immediately posted it as a preprint and plan to publish it elsewhere. </p><p>
Although I was quite amused by all of this, it has a serious side. As we note in our preprint, when paper mills succeed in breaching the defences of a journal, this is not a victimless crime. First, it gives competitive advantage to the authors who paid the paper mill – they do this in order to have a respectable-looking publication that will help their career. I used to think this was a minor benefit, but when you consider that the paper mills can also ensure that the papers they place are heavily cited, you start to realise that authors can edge ahead on conventional indicators of academic prestige, while their more honest peers trail behind. The second set of victims are those who publish in the journal in good faith. Once its reputation is damaged by the evidence that there is no quality control, then all papers appearing in the journal are tainted by association. The third set of victims are busy academics who are trying to read and integrate the literature, who can get tangled up in the weeds as they try to navigate between useful and useless information. And finally, we need to be concerned about the cynicism induces in the general public when they realise that for some authors and editors, the whole business of academic publishing is a game, which is won not by doing good science, but by paying someone to pretend you have done so.</p><p><a href="http://deevybee.blogspot.com/2022/09/we-need-to-talk-about-editors.html" target="_blank">Earlier this week I shared my thoughts</a> on the importance of ensuring that we have some kind of quality control over journal editors. They are, after all, the gatekeepers of science. When I wrote my taxonomy of journal editors back in 2010, I was already concerned at the times I had to deal with editors who were lazy or superficial in their responses to authors. I had not experienced ‘hands off’ editors in the early days of my research career, and I wondered how far this was a systematic change over time, or whether it was related to subject area. In the 1970s and 1980s, I mostly published in journals that dealt with psychology and/or language, and the editors were almost always heavily engaged with the paper, adding their own comments and suggesting how reviewer comments might be addressed. That’s how I understood the job when I myself was an editor. But when I moved to publishing work in journals that were more biological (genetics, neuroscience) things seemed different, and it was not uncommon to find editors who really did nothing more than collate peer reviews. </p><p>
The next change I experienced was when, as a Wellcome-funded researcher, I started to publish in Wellcome Open Research (WOR), which adopts a very different publication model, based on that <a href="https://f1000research.com/about" target="_blank">initiated by F1000</a>. In this model, there is no academic editor. Instead, the journal employs staff who check that the paper complies with rigorous criteria: the proposed peer reviewers much have a track record of publishing and be clear from conflict of interest. Data and other materials must be openly available so that the work can be reproduced. And the peer review is published. The work is listed on PubMed if and when peer reviewers agree that it meets a quality threshold: otherwise the work remains visible but with status shown as not approved by peer review. </p><p>The F1000/WOR model shows that editors are not needed, but I generally prefer to publish in journals that do have academic editors – provided the editor is engaged and does their job properly. My papers have benefitted from input from a wise and experienced editor on many occasions. In a specialist journal, such an editor will also know who are the best reviewers – those who have the expertise to give a detailed and fair appraisal of the work. However, in the absence of an engaged editor, I prefer the F1000/WOR model, where at least everything is transparent. The worst of all possible worlds is when you have an editor who doesn’t do more than collate peer reviews, but where everything is hidden: the outside world cannot know who the editor was, how decisions were made, who did the reviews, and what they said. Sadly, this latter situation seems to be pretty common, especially in the more biological realms of science. To test my intuitions, I ran a little Twitter poll for different disciplines, asking, for instance: <br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigidZsJWXQ6XxareOwnJ1RQKavYAyzy1ZjlAOBNeIoj7iJ7Kp66ijKbWEtwNj2aYjXGlIYJ_XqLDlA-E2MU8JGLxizVoG_AmckePGqcr_c2LIOE9VQDwKFKXAvsYoHdffwOTFXbHLAFmyxXlKrwhueuFPcgu92zPE4RcGtl-e8EPza0OJlvpH7PcvD5A/s609/Screenshot%202022-09-11%20at%2011.59.25.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="244" data-original-width="609" height="160" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigidZsJWXQ6XxareOwnJ1RQKavYAyzy1ZjlAOBNeIoj7iJ7Kp66ijKbWEtwNj2aYjXGlIYJ_XqLDlA-E2MU8JGLxizVoG_AmckePGqcr_c2LIOE9VQDwKFKXAvsYoHdffwOTFXbHLAFmyxXlKrwhueuFPcgu92zPE4RcGtl-e8EPza0OJlvpH7PcvD5A/w400-h160/Screenshot%202022-09-11%20at%2011.59.25.png" width="400" /></a> </div><div class="separator" style="clear: both; text-align: center;"> </div><div class="separator" style="clear: both; text-align: left;"> Results are below</div><p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGA8BtGDhUz_WRlrj8Jak5z2L1eh0Du0pZZILR7nZTGr2-KvtGXmDZjklluZaWDJ0Y9lEFP4oV7gcA-tke0bQp931v0q7ZAXrRa1_-F0RdDw6qoYoFoBsuRpIC0jpM-WCjg66tARN7Rt3NYcBdWFaO4x8G6rPqOSOB6FFWlu-nDa1W5huXvZX0xhoRfw/s938/Screenshot%202022-09-11%20at%2011.57.02.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="322" data-original-width="938" height="138" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGA8BtGDhUz_WRlrj8Jak5z2L1eh0Du0pZZILR7nZTGr2-KvtGXmDZjklluZaWDJ0Y9lEFP4oV7gcA-tke0bQp931v0q7ZAXrRa1_-F0RdDw6qoYoFoBsuRpIC0jpM-WCjg66tARN7Rt3NYcBdWFaO4x8G6rPqOSOB6FFWlu-nDa1W5huXvZX0xhoRfw/w400-h138/Screenshot%202022-09-11%20at%2011.57.02.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">% respondents stating Not Read, Read Superfially, or Read in Depth</td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td></tr></tbody></table><br /> </p><p>Such polls of course have to be taken with a pinch of salt, as the respondents are self-selected, and the poll allows only very brief questions with no nuance. It is clear that within any one discipline, there is wide variability in editorial engagement. Nevertheless, I find it a matter of concern that in all areas, some respondents had experienced a journal editor who did not appear to have read the paper they had accepted, and in areas of biomedicine, neuroscience, and genetics, and also in mega journals, this was as high as around 25-33% </p><p>So my conclusion is that it is not surprising that we are seeing phenomena like paper mills, because the gatekeepers of the publication process are not doing their job. The solution would be either to change the culture for editors, or, where that is not feasible, to accept that we can do without editors. But if we go down that route, we should move to a model such as F1000 with much greater quality control over reviewers and COI, and much more openness and transparency. </p><p><span style="font-size: x-small;"> As usual comments are welcome: if you have trouble getting past comment moderation, please email me.</span><br /></p>
deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com0tag:blogger.com,1999:blog-5841910768079015534.post-5878210692965133032022-09-06T12:02:00.009+01:002022-09-10T09:50:24.707+01:00We need to talk about editors<br /><table cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNIXAUEhhqSWqvSx_0k3UnTe09Wu-2EUPpS2EHKauzedAcbNtog5HqhI7qpRUcLGezPZFay2D2B14HkfO6AnlwbsgQEntZ9hP1x1oCntq0jBoAL1JL5ctrBB2-dVoYMSIGDLG8h0eBB5sgfh7S-caazq7NCq4YxEGAAhdtJJr_rwC5hs26zjVubTu4bQ/s495/exportspiv%20small.png" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" data-original-height="495" data-original-width="413" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNIXAUEhhqSWqvSx_0k3UnTe09Wu-2EUPpS2EHKauzedAcbNtog5HqhI7qpRUcLGezPZFay2D2B14HkfO6AnlwbsgQEntZ9hP1x1oCntq0jBoAL1JL5ctrBB2-dVoYMSIGDLG8h0eBB5sgfh7S-caazq7NCq4YxEGAAhdtJJr_rwC5hs26zjVubTu4bQ/w167-h200/exportspiv%20small.png" title="Editoris Spivia" width="167" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Editoris spivia<br /></td></tr></tbody></table><br />The role of journal editor is powerful: you decide what is accepted or rejected for publication. Given that publications count as an academic currency – indeed <a href="https://www.science.org/content/article/cash-bonuses-peer-reviewed-papers-go-global" target="_blank">in some institutions they are literally fungible</a> – a key requirement for editors is that they are people of the utmost integrity. Unfortunately, there are few mechanisms in place to ensure editors are honest – and indeed there is mounting evidence that many are not. I argue here that we can no longer take editorial honesty for granted, and systems need to change to weed out dodgy editors if academic publishing is to survive as a useful way of advancing science. In particular, the phenomenon of paper mills has shone a spotlight on editorial malpractice.<br /><h4 style="text-align: left;">
Questionable editorial practices</h4><p>
Back in 2010, I described <a href="http://deevybee.blogspot.com/2010/09/science-journal-editors-taxonomy.html" target="_blank">a taxonomy of journal editors</a> based on my own experience as an author over the years. Some were negligent, others were lordly, and others were paragons – the kind of editor we all want, who is motivated solely by a desire for academic excellence, who uses fair criteria to select which papers are published, who aims to help an author improve their work, and provides feedback in a timely and considerate fashion. My categorisation omitted another variety of editor that I have sadly become acquainted with in the intervening years: the spiv. The spiv has limited interest in academic excellence: he or she sees the role of editor as an opportunity for self-advancement. This usually involves promoting the careers of self or friends by facilitating publication of their papers, often with minimal reviewing, and in some cases may go as far as working hand in glove with paper mills to receive financial rewards for placing fraudulent papers. <br /><br />
When I first discovered <a href="http://deevybee.blogspot.com/2015/03/will-elsevier-say-sorry.html " target="_blank">a publication ring that involved journal editors scratching one another’s backs</a>, in the form of rapid publication of each other’s papers, I assumed this was a rare phenomenon. After I blogged about this, one of the central editors was replaced, but others remained in post. </p><p>
I subsequently found <a href="http://deevybee.blogspot.com/2020/08/pepiops-prolific-editors-who-publish-in.html" target="_blank">journals where the editor-in-chief authored an unusually high percentage of the articles published in the journal</a>. I drew these to the attention of integrity advisors of the publishers that were involved, but did not get the impression that they regarded this as particularly problematic or were going to take any action about it. Interestingly, there was one editor, George Marcoulides, who featured twice in a list of editors who authored at least 15 articles in their own journal over a five year period. Further evidence that he equates his editorial role with omnipotence came when his name cropped up in connection with a scandal where a reviewer, Fiona Fidler, complained after she found her positive report on a paper had been modified by the editor to justify rejecting the paper: see this <a href="https://twitter.com/fidlerfm/status/1321650813668794368?s=20&t=ck6lODacRLiQGpZGkbTBBQ" target="_blank">Twitter thread</a> for details. It appears that the publishers regard this as acceptable: Marcoulides is still editor-in-chief at the Sage journal Educational and Psychological Measurement, and at Taylor and Francis’ Structural Equation Modeling, though his rate of publishing in both journals has declined since 2019; maybe someone had a word with him to explain that publishing most of your papers in a journal you edit is not a good look. <br /><br />
<a href="https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3001133">Scanff et al (2021)</a> did a much bigger investigation of what they termed “self-promotion journals” - those that seemed to be treated as the personal fiefdom of editors, who would use the journal as an outlet for their own work. This followed on from a study by <a href="https://ebm.bmj.com/content/27/3/133">Locher et al (2021),</a> which found editors who were ready to accept papers by a favoured group of colleagues with relatively little scrutiny. This had serious consequences when low-quality studies relating to the Covid-19 pandemic appeared in the literature and subsequently influenced clinical decisions. Editorial laxness appears in this case to have done real harm to public health. <br /><br />
So, it's doubtful that all editors are paragons. And this is hardly surprising: doing a good job as editor is hard and often thankless work. On the positive side, an editor may obtain kudos for being granted an influential academic role, but often there is little or no financial reimbursement for the many hours that must be dedicated to reading and evaluating papers, assigning reviewers, and dealing with fallout from authors who react poorly to having their papers rejected. Even if an editor starts off well, they may over time start to think “What’s in this for me?” and decide to exploit the opportunities for self-advancement offered by the position. The problem is that there seems little pressure to keep them on the straight and narrow; it's like when a police chief is corrupt. Nobody is there to hold them to account. </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_QZLTMrOJLdiTso4t3xQug3wL0cS5Q9iV9hoEpYOiH9U6T1lzD_ludCwJrM0X18n6nxMY5F0I2cTOBwLjX1-pip6JPN_SzkzgSyI2JJMFE2SPkeHSDQ_Vaf2zJABqlrjxWpQNzEOUJzIgOoQ_UOsYSYIJ3fjQpZQrc7bkHzER8tnYemdQ28_GpHBSYA/s960/Screenshot%202022-09-04%20at%2011.54.20.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="73" data-original-width="960" height="24" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_QZLTMrOJLdiTso4t3xQug3wL0cS5Q9iV9hoEpYOiH9U6T1lzD_ludCwJrM0X18n6nxMY5F0I2cTOBwLjX1-pip6JPN_SzkzgSyI2JJMFE2SPkeHSDQ_Vaf2zJABqlrjxWpQNzEOUJzIgOoQ_UOsYSYIJ3fjQpZQrc7bkHzER8tnYemdQ28_GpHBSYA/s320/Screenshot%202022-09-04%20at%2011.54.20.png" width="320" /></a></div><h4 style="text-align: left;">
Paper mills</h4><p>
Many people are shocked when they read about the phenomenon of academic paper mills – defined in a <a href="https://publicationethics.org/node/55256" target="_blank">recent report </a>by the Committee on Publication Ethics (COPE) and the Association of Scientific, Tehcnical and Medical Publishers (STM) as “<i>the process by which manufactured manuscripts are submitted to a journal for a fee on behalf of researchers with the purpose of providing an easy publication for them, or to offer authorship for sale.</i>” The report stated that “<i>the submission of suspected fake research papers, also often associated with fake authorship, is growing and threatens to overwhelm the editorial processes of a significant number of journals</i>.” It concluded with a raft of recommendations to tackle the problem from different fronts: changing the incentives adopted by institutions, investment in tools to detect paper mill publications, education of editors and reviewers to make them aware of paper mills, introduction of protocols to impede paper mills succeeding, and speeding up the process of retraction by publishers. <br /><br />
However, no attention was given to the possibility that journal editors may contribute to the problem: there is talk of “educating” them to be more aware of paper mills, but this is not going to be effective if the editor is complicit with the paper mill, or so disengaged from editing as to not care about them. </p><p>It’s important to realise that not all paper mill papers are the same. Many generate outputs that look plausible. As <a href="https://link.springer.com/article/10.1007/s11192-016-2209-6" target="_blank">Byrne and Labbé (2017) </a>noted, in biomedical genetic studies, fake papers are generated from a template that is based on a legitimate paper, and just vary in terms of the specific genetic sequence and/or phenotype that is studied. There are so many genetic sequences and phenotypes, that the number of possible combinations of these is immense. In such cases, a diligent editor may get tricked into accepting a fake paper, because the signs of fakery are not obvious and aren’t detected by reviewers. But at the other extreme, some products of paper mills are clearly fabricated. The most striking examples are those that contain what Guillaume Cabanac and colleagues term <a href="https://doi.org/10.48550/arXiv.2107.06751 " target="_blank">“tortured phrases</a>”. These appear to be generated by taking segments of genuine articles and running them through an AI app that will use a thesaurus to alter words, with the goal of evading plagiarism detection software. In other cases, the starting point appears to be text from an essay mill. The results are often bizarre and so incomprehensible that one only needs read a few sentences to know that something is very wrong. Here’s an example from Elsevier’s International Journal of Biological Macromolecules, which those without access can pay $31.50 for (see analysis on Pubpeer,<a href="https://pubpeer.com/publications/3C7BB2433653097220FD4EC3388674?utm_source=Firefox&utm_medium=BrowserExtension&utm_campaign=Firefox" target="_blank"> here</a>). <br /></p><div style="margin-left: 40px;"><i>
"Wound recuperating camwood a chance to be postponed due to the antibacterial reliance of microorganisms concerning illustration an outcome about the infection, wounds are unable to mend appropriately, furthermore, take off disfiguring scares [150]. Chitin and its derivatives go about as simulated skin matrixes that are skilled should push a fast dermal redesign after constantly utilized for blaze treatments, chitosan may be wanton toward endogenous enzymes this may be a fundamental preference as evacuating those wound dressing camwood foundation trauma of the wounds and harm [151]. Chitin and its derivatives would make a perfect gas dressing. Likewise, they dampen the wound interface, are penetrability will oxygen, furthermore, permit vaporous exchange, go about as a boundary with microorganisms, and are fit about eliminating abundance secretions" </i><br /></div><br />
And here’s the start of an Abstract from a Springer Nature collection called Modern Approaches in Machine Learning and Cognitive Science (see <a href="https://pubpeer.com/publications/FA9EC3A2916659B789B21C61DB273A" target="_blank">here</a> for some of the tortured phrases that led to detection of this article). The article can be yours for £19.95: <br /><br /><div style="margin-left: 40px; text-align: left;">
<i>“Asthma disease are the scatters, gives that influence the lungs, the organs that let us to inhale and it’s the principal visit disease overall particularly in India. During this work, the matter of lung maladies simply like the trouble experienced while arranging the sickness in radiography are frequently illuminated. There are various procedures found in writing for recognition of asthma infection identification. A few agents have contributed their realities for Asthma illness expectation. The need for distinguishing asthma illness at a beginning period is very fundamental and is an exuberant research territory inside the field of clinical picture preparing. For this, we’ve survey numerous relapse models, k-implies bunching, various leveled calculation, characterizations and profound learning methods to search out best classifier for lung illness identification. These papers generally settlement about winning carcinoma discovery methods that are reachable inside the writing.” </i><br /></div><p>These examples are so peculiar that even a layperson could detect the problem. In more technical fields, the fake paper may look superficially normal, but is easy to spot by anyone who knows the area, and who recognises that the term “signal to noise” does not mean “flag to commotion”, or that while there is such a thing as a “Swiss albino mouse” there is no such thing as a “Swiss pale-skinned person mouse”. These errors are not explicable as failures of translation by someone who does not speak good English. They would be detected by any reviewer with expertise in the field. Another characteristic of paper mill outputs, featured in this recent <a href="https://forbetterscience.com/2022/09/05/cyclotron-branch-before-the-fall/" target="_blank">blogpost</a>, are fake papers that combine tables and figures from different publications in nonsensical contexts. </p><p>Sleuths who are interested in unmasking paper mills have developed automated methods for identifying such papers, and the number is both depressing and astounding. As we have seen, though some of these outputs appear in obscure sources, many crop up in journals or edited collections that are handled by the big scientific publishing houses, such as Springer Nature, Elsevier and Wiley. When sleuths find these cases, they report the problems on the website PubPeer, and this typically raises an incredulous response as to how on earth did this material get published. It’s a very good question, and the answer has to be that somehow an editor let this material through. As explained in the COPE&STM report, sometimes a nefarious individual from a paper mill persuades a journal to publish a “special issue” and the unwitting journal is then hijacked and turned into a vehicle for publishing fraudulent work. If the special issue editor poses as a reputable scientist, using a fake email address that looks similar to the real thing, this can be hard to spot.<br /><br />
But in other cases, we see clearcut instances of paper mill outputs that have apparently been approved by a regular journal editor. In a recent <a href="https://psyarxiv.com/2yf8z/" target="_blank">preprint</a>, Anna Abalkina and I describe finding putative paper mill outputs in a well-established Wiley journal, the Journal of Community Psychology. Anna identified six papers in the journal in the course of a much larger investigation of papers that came from faked email addresses. For five of them the peer review and editorial correspondence was available on Publons. The papers, from addresses in Russia or Kazakhstan, were of very low quality and frequently opaque. I had to read and re-read to work out what the paper was about, and still ended up uncertain. The reviewers, however, suggested only minor corrections. They used remarkably similar language to one another, giving the impression that the peer review process had been compromised. Yet the Editor-in-Chief, Michael B. Blank, accepted the papers after minor revisions, with a letter concluding: “Thank you for your fine contribution”. </p><p>There are two hypotheses to consider when a journal publishes incomprehensible or trivial material: either the editor was not doing their job of scrutinising material in the journal, or they were in cahoots with a paper mill. I wondered whether the editor was <a href="http://deevybee.blogspot.com/2010/09/science-journal-editors-taxonomy.html" target="_blank">what I have previously termed an automaton</a> – one who just delegates all the work to a secretary. After all, authors are asked to recommend reviewers, so all that is needed is for someone to send out automated requests to review, and then keep going until there are sufficient recommendations to either accept or reject. If that were the case, then maybe the journal would accept a paper by us. Accordingly, we submitted our manuscript about paper mills to the Journal of Community Psychology. But it was desk rejected by the Editor in Chief with a terse comment: “This a weak paper based on a cursory review of six publications”. So we can reject hypothesis 1 – that the editor is an automaton. But that leaves hypothesis 2 – that the editor does read papers submitted to his journal, and had accepted the previous paper mill outputs in full knowledge of their content. This raises more questions than it answers. In particular, why would he risk his personal reputation and that of his journal by behaving that way? But perhaps rather than dwelling on that question, we should think positively about how journals might protect themselves in future from attacks by paper mills. </p><h4 style="text-align: left;">
A call for action<br /></h4><p>
My proposal is that, in addition to the useful suggestions from the COPE&STM report, we need additional steps to ensure that those with editorial responsibility are legitimate and are doing their job. Here are some preliminary suggestions:<br /></p><ol style="text-align: left;"><li>
Appointment to the post of editor should be made in open competition among academics who meet specified criteria.</li><li>
It should be transparent who is responsible for final sign-off for each article that is published in the journal.</li><li>
Journals where a single editor makes the bulk of editorial decisions should be discouraged. (N.B. I looked at the 20 most recent papers in Journal of Community Psychology that featured on Publons and all had been processed by Michael B. Blank).<br /></li><li>
There should be an editorial board consisting of reputable people from a wide range of institutional backgrounds, who share the editorial load, and meet regularly to consider how the journal is progressing and to discuss journal business.</li><li> Editors should be warned about the dangers of special issues and should not delegate responsibility for signing off on any papers appearing in a special issue.</li><li>
Editors should be required to follow COPE guidelines about publishing in their own journal, and publishers should scrutinise the journal annually to check whether the recommended procedures were followed.</li><li>Any editor who allows gibberish to be published in their journal should be relieved of their editorial position immediately.<br /></li></ol><p>
Many journals run by academic societies already adopt procedures similar to these. Particular problems arise when publishers start up new journals to fill a perceived gap in the market, and there is no oversight by academics with expertise in the area. The COPE&STM report has illustrated how dangerous that can be – both for scientific progress and for the reputation of publishers. <br /><br />
Of course, when one looks at this list of requirements, one may start to wonder why anyone would want to be an editor. Typically there is little financial renumeration, and the work is often done in a person’s “spare time”. So maybe we need to rethink how that works, so that paragons with a genuine ability and interest in editing are rewarded more adequately for the important work they do. <br /><br />
<span style="font-size: x-small;">P.S. Comment moderation is enabled for this blog to prevent it being overwhelmed by spam, but I welcome comments, and will check for these in the weeks following the post, and admit those that are on topic. </span></p><p><span style="font-size: x-small;"> </span></p><p><b><span style="font-size: small;">Comment by Jennifer Byrne, 9th Sept 2022 </span></b></p><p><span style="font-size: x-small;">(this comment by email, as Blogger seems to eat comments by Jennifer for some reason, while letting through weird spammy things!).</span><b><span style="font-size: small;"><br /></span></b></p><p><span style="font-size: x-small;"><span style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; display: inline !important; float: none; font-family: Calibri, sans-serif; font-size: 14.666666984558105px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">This is a fantastic list of suggestions to improve the important contributions of journal editors. I would add that journal editors should be appointed for defined time periods, and their contributions regularly reviewed. If for any reason it becomes apparent that an editor is not in a position to actively contribute to the journal, they should be asked to step aside. In my experience, editorial boards can include numerous inactive editors. These can provide the appearance of a large, active and diverse editorial board, when in practice, the editorial work may be conducted by a much smaller group, or even one person. Journals cannot be run successfully without a strong editorial team, but such teams require time and resources to establish and maintain.</span> <br /></span></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com0tag:blogger.com,1999:blog-5841910768079015534.post-41040851507261428222022-08-09T18:30:00.077+01:002022-08-18T08:52:32.770+01:00Can systematic reviews help clean up science?<p> <br /></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiO9Cl4am0x40UA1-yrEmGK27fOksFmKsv0O6B6IS8ZJlXvWOBYMZIouDMhG-Dzy1cp36Ow8FljoiwoOhwru5Y1uhHw_hQSqdu-6NHlQyuqU72Nz7k6QMl_lRgUU6Mq3EFSmZPlmENE6BfFX9j5J6kx4nKnT0_fWSGZsDXkvyQJyIqJU63YhIF1OnJOow/s1393/cartoon%20systematic%20rev.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1153" data-original-width="1393" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiO9Cl4am0x40UA1-yrEmGK27fOksFmKsv0O6B6IS8ZJlXvWOBYMZIouDMhG-Dzy1cp36Ow8FljoiwoOhwru5Y1uhHw_hQSqdu-6NHlQyuqU72Nz7k6QMl_lRgUU6Mq3EFSmZPlmENE6BfFX9j5J6kx4nKnT0_fWSGZsDXkvyQJyIqJU63YhIF1OnJOow/s320/cartoon%20systematic%20rev.png" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><i>The systematic review was not turning out as Lorna had expected</i><br /></td></tr></tbody></table><p>Why do people take the risk of publishing fraudulent papers, when it is easy to
detect the fraud? One answer is that they don’t expect to be caught. A consequence of the growth in systematic reviews is that this
assumption may no longer be safe. </p><p>In June I participated in a symposium
organised by the LMU Open Science Center in Munich entitled “How paper mills
publish fake science industrial-style – is there really a problem and how does
it work?” The presentations are available <a href="https://osf.io/47utb/?view_only= ">here</a>. I focused on the weird phenomenon of papers containing “tortured phrases”,
briefly reviewed <a href="https://www.nature.com/articles/d41586-021-02134-0">here</a>. For a
fuller account see <a href="https://arxiv.org/abs/2107.06751" target="_blank">here</a>. These are fakes that are
easy to detect, because, in the course of trying to circumvent plagiarism
detection software, they change words, with often unintentionally hilarious
consequences. For instance, “breast cancer” becomes “bosom peril” and “random
value” becomes “irregular esteem”. Most of these papers make no sense at all –
they may include recycled figures from other papers. They are typically highly
technical and so to someone without expertise in the area they may seem valid,
but anyone familiar with the area will realise that someone who writes “flag to
commotion” instead of “signal to noise” is a hoaxer. </p><p>Speakers at the symposium
drew attention to other kinds of paper mill whose output is less conspicuously
weird. Jennifer Byrne documented industrial-scale research fraud in papers on
single gene analyses that were created by templates, and which purported to
provide data on under-studied genes in human cancer models. Even an expert in
the field may be hoodwinked by these. I addressed the question of “does it matter?” For the
nonsense papers generated using tortured phrases, it could be argued that it
doesn’t, because nobody will try to build on that research. But there are still
victims: authors of these fraudulent papers may outcompete other, honest
scientists for jobs and promotion, journals and publishers will suffer
reputational damage, and public trust in science is harmed. But what intrigued
me was that the authors of these papers may also be regarded as victims, because
they will have on public record a paper that is evidently fraudulent. It seems
that either they are unaware of just how crazy the paper appears, or that they
assume nobody will read it anyway. </p><p>The latter assumption may have been true a
couple of decades ago, but with the growth of systematic reviews, researchers
are scrutinizing many papers that previously would have been ignored. I was
chatting with John Loadsman, <a href="https://retractionwatch.com/2021/06/10/meet-a-sleuth-whose-work-has-led-to-the-identification-of-hundreds-of-fraudulent-papers/">who in his role as editor of Anaesthesia and Intensive Care has uncovered numerous cases of fraud</a>.
He observed that many paper mill outputs never get read because, just on the
basis of the title or abstract, they appear trivial or uninteresting. However,
when you do a systematic review, you are supposed to read everything relevant to
the research question, and evaluate it, so these odd papers may come to light. </p><p>I’ve
<a href="http://deevybee.blogspot.com/2018/07/standing-on-shoulders-of-giants-or.html">previously blogged </a>about the importance of systematic reviews for avoiding
cherrypicking of the literature. Of
course, evaluation of papers is often done poorly or not at all, in which case the
fraudulent papers just pollute the literature when added to a meta-analysis. But I’m
intrigued at the idea that systematic reviews might also serve the purpose of putting the
spotlight on dodgy science in general, and fraudsters in particular, by forcing
us to read things thoroughly. I therefore asked Twitter for examples – I asked
specifically about meta-analysis but the responses covered systematic reviews
more broadly, and were wide-ranging both in the types of issue that were
uncovered and the subject areas. </p><p>Twitter did not disappoint: I received numerous
examples – more than I can include here. Much of what was described did not
sound like the work of paper mills, but did include fraudulent data
manipulation, plagiarism, duplication of data in different papers, and analytic
errors. Here are some examples: </p><p><b>Paper mills and template papers</b></p><p>Jennifer Byrne noted how she became aware of paper mills when looking for studies of a
particular gene she was interested in, which was generally under-researched. Two
things raised her suspicions: a sudden spike in studies of the gene, plus<a href="https://osf.io/kqv4f"> series of papers that had the same structure, as if constructed from a template</a>.
Subsequently, with Cyril Labbé, who developed an automated
<a href="http://scigendetection.imag.fr/TPD52/Vb/">Seek & Blastn </a>tool to assess nucleotide sequences, she found numerous errors in the
reagents and specification of genetic sequences of these repetitive papers, and
it became clear that they were fraudulent. </p><p>An example of a <a href="https://doi.org/10.1111/1471-0528.14267">systematic review</a> that discovered a
startling level of inadequate and possibly fraudulent research was focused on the
effect of tranexamic acid on post-partum haemorrhage: out of 26 reports, eight had sections
of identical or very similar text, despite apparently coming from different
trials. This is similar to what has been described for papers from paper mills,
which are constructed from a template. And, as might be expected for a paper
mill output, there were also numerous statistical and methodological errors, and
some cases without ethical approval. (Thanks to @jd_wilko for pointing me to
this example). </p><p><b>Plagiarism </b></p><p>Back in 2006, Iain Chalmers, who is generally ahead of his time, noted that systematic
reviews could root out cases of plagiarism, citing the example of Asim Kurjak,
whose <a href="https://www.bmj.com/content/333/7568/594">paper on epidural analgesia in labour was heavily plagiarised</a>. </p><p><b>Data duplication </b></p><p>Meta-analysis can throw up cases
where the same study is reported in two or more papers, with no indication that
this is the same data. Although this might seem like a minor problem compared
with fraud, it can be serious, because if the duplication is missed in a
meta-analysis, that study will be given more weight than it should have. Ioana Cristea noted that such ‘zombie papers’ have cropped up in a meta-analysis she is currently
analysing. </p><p><b>Tampering with peer review </b></p><p>When a paper considered for a
meta-analysis seems dubious, it raises the question of whether proper peer review
procedures were followed. It helps if the journal adopts open peer
review. Robin N. Kok reported a paper where<a href="https://twitter.com/robinnkok/status/916692674618449920?s=20&t=bSFZ953tX04wDRpelY5tOQ"> the same person was listed as an author and a peer reviewer</a>. This was eventually retracted. </p><p><b>Data seem too good to be true </b></p><p>This <a href="https://www.science.org/content/article/research-linking-violent-entertainment-aggression-retracted-after-scrutiny">piece in Science</a> tells the story of Qian
Zhang, who published a series of studies on impact of cartoon violence in
children which on the one hand had remarkably large samples of children all at
the same age, and on the other hand had similar samples across apparently
different studies. Because of their enormous size, Zhang’s papers distorted any meta-analysis
they were included in. </p><p>Aaron Charlton cited another case, where <a href="https://statmodeling.stat.columbia.edu/2016/02/09/28905/ ">serious anomalies were picked up in a study on marketing in the course of a meta-analysis.</a> The paper
was ultimately retracted 3 years after the concerns were raised, after <a href="https://openmkt.org/blog/2021/retracted-article-why-money-meanings-matter-in-decisions-to-donate-time-and-money/">defensive responses</a> from some of the authors, challenging the meta-analysts. </p><p><a href="https://journals.lww.com/pain/Abstract/9900/Investigating_the_veracity_of_a_sample_of.70.aspx">This case</a> flagged by Neil O’Connell is especially useful, as it documents a
range of methods used to evaluate suspect research. The dodgy work was first flagged up in
a meta-analysis of cognitive behaviour therapy for chronic pain.
Three papers with the same lead author, M. Monticone, obtained results that were discrepant with the rest of the literature, with much bigger effect
sizes. The meta-analysts then looked at other trials by the same team and found that there was a 6-fold difference between the lower confidence
interval of the Monticone studies and the upper confidence interval of all
others combined. The paper also reports email exchanges with Dr Monticone that
may be of interest to readers. </p><p><b>Poor methodology</b> </p><p>Fiona Ramage told me that in
the course of doing a preclinical systematic review and meta-analysis of
nutritional neuroscience, she encountered numerous errors of basic methodology and statistics, e.g. dozens of papers where
error bars were presented without indicating if they show SE or SD; studies
claiming differences between groups without a direct statistical comparison.
This is more likely to be due to ignorance or honest error than to malpractice,
but it needs to be flagged up so that the literature is not polluted by
erroneous data.</p><p><b>What are the consequences?</b></p><p>Of course, the potential of systematic reviews to detect bad science is only
realised if the dodgy papers are indeed weeded out of the literature, and people
who commit scientific fraud are fired. Journals and publishers have started to
respond to paper mills, but, as <a href="https://www.nature.com/articles/d41586-022-02071-6">Ivan Oransky has commented</a>, this is a game of
Whac-a-Mole, and "the process of retracting a paper remains comically clumsy,
slow and opaque”. </p><p>I was surprised
that even when confronted with an obvious case of a paper that had both
numerous tortured phrases and plagiarism, the response from the publisher was
slow – e.g. this <a href="https://pubpeer.com/publications/B087B8BA51622FF9EC97A41D27CF75#">comically worded example</a> is still not retracted, even though
the publisher’s research integrity office acknowledged my email expressing
concern over 2 months ago. But 2 months is nothing. Guillaume
Cabanac <a href="https://twitter.com/gcabanac/status/1556197962648281089?s=20&t=RYVpWAtHklaQ449alsS5eQ">recently tweeted</a><a href="https://twitter.com/gcabanac/status/1556197962648281089?s=20&t=RYVpWAtHklaQ449alsS5eQ"> </a>about a "barn door" case of plagiarism that has just
been retracted 20 years after it was first flagged up. When I discuss
the slow responses to concerns with publishers, they invariably say that they
are being kept very busy with a huge volume of material from paper mills. To
which I answer, you are making immense profits, so perhaps some could be
channeled into employing more people to tackle this problem. As I am fond of
pointing out, I regard a publisher who leaves seriously problematic studies in
the literature as analogous to a restauranteur that serves poisoned food to
customers. </p><p>Publishers may be responsible for correcting the scientific record,
but it is institutional employers who need to deal with those who commit
malpractice. Many institutions don’t seem to take fraud seriously. This point
was made back in 2006 by Iain Chalmers, who described the lenient treatment of
Asim Kurjak, and argued for public naming and shaming of those who are found
guilty of scientific misconduct. Unfortunately, there’s not much evidence that
his advice has been heeded. Consider this <a href="https://www.science.org/content/article/primate-research-center-head-will-keep-job-despite-misconduct-provoking-shock-and">recent example</a> of a director of a primate reseach lab who admitted fraud, but is still in post.
(Here the fraud was highlighted by a whistleblower rather than a systematic
review, but this illustrates the difficulty of tackling fraud when there are
only minor consequences for fraudsters). </p><p>Could a move towards "slow science" help? In the humanities, literary scholars pride themselves on “close
reading” of texts. In science, we are often so focused on speed and concision,
that we tend to lose the ability to focus deeply on a text, especially if it is
boring. The practice of doing a systematic review should in principle develop
better skills in evaluation of individual papers, and in so doing help cleanse
the literature from papers that should never have got published in the first
place. John Loadsman has suggested we should not only read papers carefully,
but should recalibrate ourselves to have a very high “index of suspicion” rather
than embracing the default assumption that everyone is honest. </p><p><b>P.S</b>. </p><p>Many thanks
to everyone who sent in examples. Sorry I could not include everything. Please
feel free to add other examples or reactions in the Comments – these tend to get
overwhelmed with adverts for penis enlargement or (ironically) essay mills, and
so are moderated, but I do check them and relevant comments will eventually appear. </p><p><b>PPS</b>. Florian Naudet sent a couple of relevant links that readers might enjoy: <br /></p><p>Fascinating article by Fanelli et al who looked at how inclusion of retracted papers affected meta-analyses: https://www.tandfonline.com/doi/full/10.1080/08989621.2021.1947810 </p><p>And this piece by Lawrence et al shows the dangers of meta-analyses when there is insufficient scrutiny of the papers that are included: https://www.nature.com/articles/s41591-021-01535-y <br /></p><p>Also, Joseph Lee tweeted about this paper about inclusion of papers from predatory publications in meta-analyses: https://jmla.pitt.edu/ojs/jmla/article/view/491 <br /></p><p><b>PPPS. 11th August 2022</b></p><p>A couple of days after posting this, I received a copy of "Systematic Reviews in Health Research" edited by Egger, Higgins and Davey Smith. Needless to say, the first thing I did was to look up "fraud" in the index. Although there are only a couple of pages on this, the examples are striking. </p><p>First, a study by <a href="https://www.bmj.com/content/348/bmj.g2688">Nowbar et al (2014)</a> on bone marrow stem cells for heart disease found that in a review of 133 reports, over 600 discrepancies were found, and the number of discrepancies increased with the reported effect size. There's a trail of comments on Pubpeer relating to some of the sources, e.g. <a href="https://pubpeer.com/publications/B346354468C121A468D30FDA0E295E">https://pubpeer.com/publications/B346354468C121A468D30FDA0E295E</a>.</p><p>Another example concerns the use of beta-blockers during surgery. A series of studies from one centre (the DECREASE trials) showing good evidence of effectiveness was investigated and found to be inadequate, with missing data and failure to follow research protocols. When these studies were omitted from a meta-analysis, the conclusion was that, far from receiving benefit from beta-blockers, patients in the treatment group were more likely to die <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3932762/#R3">(Bouri et al, 2014</a>). </p><p> PPPPS, 18th August 2022</p><p>This comment by Jennifer Byrne was blocked by Blogger - possibly because it contained weblinks.</p><p>Anyhow, here is what she said:</p><p><span style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; display: inline !important; float: none; font-family: Calibri, sans-serif; font-size: 14.666666984558105px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">I agree, reading both widely and deeply can help to identify problematic papers, and an ideal time for this to happen is when authors are writing either narrative or systematic reviews. Here's another two examples where Prof Carlo Galli and colleagues identified similar papers that may have been based on templates:<span class="Apple-converted-space"> </span></span><a href="https://www.mdpi.com/2304-6775/7/4/67" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; color: #954f72; font-family: Calibri, sans-serif; font-size: 14.666666984558105px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-decoration: underline; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">https://www.mdpi.com/2304-6775/7/4/67</a><span style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; display: inline !important; float: none; font-family: Calibri, sans-serif; font-size: 14.666666984558105px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">,<span class="Apple-converted-space"> </span></span><a href="https://link.springer.com/article/10.1007/s11192-022-04434-2" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; color: #954f72; font-family: Calibri, sans-serif; font-size: 14.666666984558105px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-decoration: underline; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">https://link.springer.com/article/10.1007/s11192-022-04434-2</a> </p><p> <br /></p><p><br /></p><p><br /></p><p><br /></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com3tag:blogger.com,1999:blog-5841910768079015534.post-63319428450128818722022-08-03T06:59:00.005+01:002022-08-03T06:59:56.579+01:00Contagion of the political system<br /><p class="MsoNormal"> </p>
<p class="MsoNormal">Citizens of the UK have in recent weeks watched in amazement
as the current candidates for leadership of the Conservative party debate their
policies. Whoever wins will replace Boris Johnson as Prime Minister, with the
decision made by a few thousand members of the Conservative Party. All options
were bad, and we are now down to the last two: Liz Truss and Rishi Sunak. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">For those of us who are not Conservatives, and for many who
are, there was immense joy at the ousting of Boris Johnson. The man seemed like
a limpet, impossible to dislodge. Every week brought a new scandal that would
have been more than sufficient to lead to resignation 10 years ago, yet he hung
on and on. Many people thought that, after a vote of no confidence in his
leadership, he would step down so that a caretaker PM could run the country
while the debate over his successor took place, but the limpet is still clinging
on. He’s not doing much running of the country, but that’s normal, and perhaps
for the best. He’s much better at running parties than leading the Conservative
party.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I have to say I had not expected much from Truss and Sunak,
but even my low expectations have not been met. The country is facing immense
challenges, from climate change, from coronavirus, and from escalating energy
prices. These are barely mentioned: instead the focus is on reducing taxes,
with the candidates now competing for just how much tax they can cut. As far as
I can see, these policies will do nothing to help the poorest in society, whose
benefits will shrink to pay for tax cuts; the richer you are the more tax you
pay and so this is a rich person’s policy.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">What has surprised me is just how ill-informed the two
candidates are. The strategy seems to be to pick a niche topic of interest to Conservative
voters, make up a new policy overnight and announce it the next day. So we have
Rishi Sunak proposing that the solution to the crisis in the NHS is to charge
people who miss doctor’s appointments. Has he thought this through? Think of
the paperwork. Think of the debt collectors tasked with collecting £10 from a
person with dementia. Think of the cost of all of this.<span style="mso-spacerun: yes;"> </span>And on Education, his idea is to reintroduce selective
(grammar) schools: presumably because he thinks that our regular schools are inadequate
to educate intelligent children. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">On Education, Liz Truss is even worse. Her idea is that all children
who score top marks in their final year school examinations should get an interview
to go to Oxford or Cambridge University. This is such a crazy idea that others
have written at length to point out its flaws (e.g.<a href=" https://www.newstatesman.com/quickfire/2022/08/liz-truss-oxbridge-plan-is-truly-baffling"> this cogent analysis by Sam Freedman</a>).
Suffice it to say that it has a similar vibe to the Sunak grammar schools plan:
it implies that only two British universities have any value. Conservatives do
seem obsessed with creating divisions between haves and have-nots, but only if
they can ensure their children are among the haves. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Another confused statement from Truss is that, as far as Scotland
goes, she plans to ignore Nicola Sturgeon, the First Minister of Scotland and
leader of the Scottish National Party. At a time when relationships between
Scotland and England are particularly fraught, this insensitive statement is
reminiscent of the gaffes of Boris Johnson. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Oh, and yesterday she also announced – and then quickly
U-turned – an idea that would limit the pay of public sector workers in the
North of England, because it was cheaper to live there. </p>
<p class="MsoNormal"><span style="mso-spacerun: yes;"> </span></p>
<p class="MsoNormal">What I find so odd about both Sunak and Truss is that they
keep scoring own goals. Nobody requires them to keep coming up with new
policies in niche areas. <span style="mso-spacerun: yes;"> </span>Why don’t they
just hold on to their original positions, and if asked about anything else,
just agree to ‘look at’ it when in power? Johnson was always very good at
promising to ‘look at’ things: when he’s not being a limpet, he’s a basilisk. The
more you probe Sunak and Truss, the more their shallowness and lack of expertise
show through. They’d do well to keep schtum. Or, better still, show some indication that they could, for instance, get a grip on the crisis in the NHS.<br /></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">What all this demonstrates is how an incompetent and
self-promoting leader causes damage far beyond their own term. Johnson’s
cabinet was selected purely on one criterion: loyalty to him. The first
requirement was to “believe in Brexit” – reminiscent of the historical wars
between Protestants and Catholics, where the first thing you ask of a candidate
is what their religion is. Among Conservative politicians, it seems that an
accusation of not really being a Brexiteer is the worst thing you can say about
a candidate. Indeed, that is exactly the charge that her opponents level against
Truss, who made cogent arguments for remaining in the EU before the referendum.
Like a Protestant required to recant their beliefs or face the flames, she is
now reduced to defending Brexit in the strongest possible terms, saying that “predictions
of doom have not come true”, as farmers, fishermen, and exporters go out of
business, academics leave in droves, and holidaymakers sit in queues at Dover. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">It's known that Johnson does not want to give up the top
job. I’m starting to wonder if behind all of this is a cunning plan. The people
he’s appointed to cabinet are so incompetent that maybe he hopes that, when
confronted with a choice between them, the Conservative Party will decide that
he looks better than either of them. </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p><style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 1 0;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073697537 9 0 511 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style></p>deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com1tag:blogger.com,1999:blog-5841910768079015534.post-88942627247813278812022-06-29T10:28:00.002+01:002022-07-05T10:19:52.068+01:00A proposal for data-sharing that discourages p-hacking<p>Open data is a great way of helping give confidence in the reproducibility of research findings. Although we are still a long way from having adequate implementation of data-sharing in psychology journals (see, for example, <a href="https://www.sciencedirect.com/science/article/pii/S0749596X22000389" target="_blank">this commentary</a> by Kathy Rastle, editor of Journal of Memory and Language), things are moving in the right direction, with an increasing number of journals and funders requiring sharing of data and code. But there is a downside, and I've been thinking about it this week, as we've just published a <a href="https://doi.org/10.1016/j.cortex.2022.05.013" target="_blank">big paper on language lateralisation</a>, where all the <a href="https://osf.io/g9tqh/wiki/home/" target="_blank">code and data</a> are available on Open Science Framework. </p><p>One problem is p-hacking. If you put a large and complex dataset in the public domain, anyone can download it and then run multiple unconstrained analyses until they find something, which is then retrospectively fitted to a plausible-sounding hypothesis. The potential to generate non-replicable false positives by such a process is extremely high - far higher than many scientists recognise. I illustrated this with a fictitious example <a href="https://royalsocietypublishing.org/doi/full/10.1098/rsos.160109" target="_blank">here</a>. </p><p>Another problem is <a href="https://deevybee.blogspot.com/2017/06/prospecting-for-kryptonite-value-of.html" target="_blank">self-imposed publication bias</a>: the researcher runs a whole set of analyses to test promising theories, but forgets about them as soon as they turn up a null result. With both of these processes in operation, data sharing becomes a poisoned chalice: instead of increasing scientific progress by encouraging novel analyses of existing data, it just means more unreliable dross is deposited in the literature.
So how can we prevent this? </p><p>In <a href="https://royalsocietypublishing.org/doi/full/10.1098/rsos.160109" target="_blank">this Commentary paper</a>, I noted several solutions. One is to require anyone accessing the data to submit a protocol which specifies the hypotheses and the analyses that will be used to test them. In effect this amounts to preregistration of secondary data analysis. This is the method used for some big epidemiological and medical databases. But it is cumbersome and also costly - you need the funding to support additional infrastructure for gatekeeping and registration. For many psychology projects, this is not going to be feasible. </p><p>A simpler solution would be to split the data into two halves - those doing secondary data analysis only have access to part A, which allows them to do exploratory analyses, after which they can then see if any findings hold up in part B. Statistical power will be reduced by this approach, but with large datasets it may be high enough to detect effects of interest. I wonder if it would be relatively easy to incorporate this option into Open Science Framework: i.e. someone who commits a preregistration of a secondary data
analysis on the basis of exploratory analysis of half a dataset then receives a code that unlocks the second half of the data (the hold-out sample). A rough outline of how this might work is shown in Figure 1. <br /></p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-Isk1yoajLhEPLLdUKjjJCFwjP5PiRRV1hYO957VU7x_7FFprMRopxmCuBJnxHBuu0p48jfZMZxCDlbeiecSai5vwPZhY2Qxc3R7Be28pbfEY9PcfERPH7dzfvQLYJvsFr1qa7J5E3RLcEitKoiepSD1EUCrvCMRL_Kvl2sFOiPcnOwSZlQAEbD_mqg/s1112/vertical%20datashareplot.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1112" data-original-width="720" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-Isk1yoajLhEPLLdUKjjJCFwjP5PiRRV1hYO957VU7x_7FFprMRopxmCuBJnxHBuu0p48jfZMZxCDlbeiecSai5vwPZhY2Qxc3R7Be28pbfEY9PcfERPH7dzfvQLYJvsFr1qa7J5E3RLcEitKoiepSD1EUCrvCMRL_Kvl2sFOiPcnOwSZlQAEbD_mqg/w414-h640/vertical%20datashareplot.jpg" width="414" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><i>Figure 1: A possible flowchart for secondary data analysis on a platform such as OSF<br /></i></td></tr></tbody></table><br /><p></p><p>An alternative that has been discussed by <a href=" https://www.nature.com/articles/526187a" target="_blank">MacCoun and Perlmutter </a>is blind analysis - "temporarily
and judiciously removing data labels and altering data values to fight
bias and error". The idea is that you can explore a dataset and run a
planned analysis on it, but it won't be possible for the results to
affect your analysis, because the data have been changed, so you won't
know what is correct. A variant of this approach would be multiple
datasets with shuffled data in all but one of them. The shuffling would
be similar to what is done in permutation analysis - so there might be
ten versions of the dataset deposited, with only one having the original
unshuffled data. Those downloading the data would not know whether or
not they had the correct version - only after they had decided on an
analysis plan, would they be told which dataset it should be run on. </p><p>I don't know if these methods would work, but I think they have
potential for keeping people honest in secondary data analysis, while
minimising bureaucracy and cost. On a platform such as Open Science
Framework it is already possible to create a time-stamped
preregistration of an analysis plan. I assume that within OSF there is
already a log that indicates who has downloaded a dataset. So someone
who wanted to do things right and just download one dataset (either a
random half, or one of a set of shuffled datasets) would just need to
have a mechanism that allowed them to gain access to the full, correct
data after they had preregistered an analysis, similar to that outlined above. </p><p>These methods are not foolproof. Two researchers could collude - or one researcher could adopt multiple personas - so that they get to see the correct data as person X and then start a new process as person B, when they can preregister an analysis where results are already known. But my sense is that there are many honest researchers who would welcome this approach - precisely because it would keep them honest. Many of us enjoy exploring datasets, but it is all too easy to fool yourself into thinking that you've turned up something exciting when it is really just a fluke arising in the course of excessive data-mining. </p><p>Like a lot of my blogposts, this is just a brain dump of an idea that is not fully thought through. I hope by sharing it, I will encourage people to come up with criticisms that I haven't thought of, or alternatives that might work better. Comments on the blog are moderated to prevent spam, but please do not be deterred - I will post any that are on topic. </p><p><b>P.S. 5th July 2022 </b></p><p>Florian Naudet drew my attention to this v relevant paper: </p><div class="csl-bib-body" style="line-height: 2; margin-left: 2em; text-indent: -2em;">
<div class="csl-entry">Baldwin, J. R., Pingault, J.-B., Schoeler, T., Sallis, H. M., & Munafò, M. R. (2022). Protecting against researcher bias in secondary data analysis: Challenges and potential solutions. <i>European Journal of Epidemiology</i>, <i>37</i>(1), 1–10. <a href="https://doi.org/10.1007/s10654-021-00839-0">https://doi.org/10.1007/s10654-021-00839-0</a></div>
<span class="Z3988" title="url_ver=Z39.88-2004&ctx_ver=Z39.88-2004&rfr_id=info%3Asid%2Fzotero.org%3A2&rft_id=info%3Adoi%2F10.1007%2Fs10654-021-00839-0&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Protecting%20against%20researcher%20bias%20in%20secondary%20data%20analysis%3A%20challenges%20and%20potential%20solutions&rft.jtitle=European%20Journal%20of%20Epidemiology&rft.stitle=Eur%20J%20Epidemiol&rft.volume=37&rft.issue=1&rft.aufirst=Jessie%20R.&rft.aulast=Baldwin&rft.au=Jessie%20R.%20Baldwin&rft.au=Jean-Baptiste%20Pingault&rft.au=Tabea%20Schoeler&rft.au=Hannah%20M.%20Sallis&rft.au=Marcus%20R.%20Munaf%C3%B2&rft.date=2022-01-01&rft.pages=1-10&rft.spage=1&rft.epage=10&rft.issn=1573-7284&rft.language=en"></span>
</div><br />deevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.com6