tag:blogger.com,1999:blog-5841910768079015534.post5462378397585333881..comments2024-03-29T08:40:11.883+00:00Comments on BishopBlog: Improving reproducibility: the future is with the youngdeevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-5841910768079015534.post-83869869599157852822018-02-22T08:48:07.125+00:002018-02-22T08:48:07.125+00:00So the answer to my first question is "yes&qu...So the answer to my first question is "yes".<br /><br />I have looked through your slides and you have clearly put a lot of thought into this subject. However, you don't have much discussion on the conduct of the experiment - ie the process of producing data in the first place. In my experience (agriculture, pesticides) this is where things can go terribly wrong. And if the data does not contain measurable signal, or the signal is confounded with one of the treatments, then no amount of analysis and reanalysis will produce anything useful. <br /><br />A simple example: an field experiment on the effect of neonicotinoids on bee activity. Beehives were placed in cages and the ecologists had worked out a way of measuring bee activity. Unfortunately bees are like us in that they take time to wake up and the ecologists decided to measure all of the neonic cages first and the untreated cages afterwards. Result: measured effect is confounded with time of day and it appeared that the pesticide was having a massive effect on the bees. pe51terhttps://www.blogger.com/profile/01967023613826064039noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-91604987705206432402018-02-13T17:17:41.954+00:002018-02-13T17:17:41.954+00:00Thanks for the questions.
1. There's no one c...Thanks for the questions.<br /><br />1. There's no one cause. Lots of factors - some from scientists, others from journals, institutions and funders. Brief summary here: https://www.slideshare.net/deevybishop/what-is-the-reproducibility-crisis-in-science-and-what-can-we-do-about-it<br /><br />2. There is more than one consensus: different disciplines have used the terms differently. In fields where much is done with analysis of large existing datasets (e.g. politics, economics, sociology) there is interest in reproducibility in the strictest sense - i.e. if you get the same data set and try to do the same analysis, do you come to the same results. In psychology and biomedicine there's more focus on replicability and generalisability - if you try to repeat an experiment that someone else did, do you get broadly the same result. This is sometimes encompassed under a broader meaning of reproducibility. There are some nice slides by Kirstie Whitaker explaining these distinctions here: zhttps://figshare.com/articles/Showing_your_working_A_guide_to_reproducible_neuroimaging_analyses/4244996<br /><br />Bottom line is you can't be replicable if you aren't reproducible.deevybeehttps://www.blogger.com/profile/15118040887173718391noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-2326976345787927262018-02-12T19:30:03.881+00:002018-02-12T19:30:03.881+00:00A couple of questions:
If you are running a cours...A couple of questions:<br /><br />If you are running a course on this subject does this mean that you know the cause of poor reproducibility?<br /><br />In the blog you refer to both "reproducibility" and "replicability". Is there a consensus on what these words mean? Do they refer to different or identical concepts?<br /><br />pe51terhttps://www.blogger.com/profile/01967023613826064039noreply@blogger.com