tag:blogger.com,1999:blog-5841910768079015534.post4849437943019454973..comments2024-03-25T17:14:36.888+00:00Comments on BishopBlog: Data sharing: Exciting but scarydeevybeehttp://www.blogger.com/profile/15118040887173718391noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-5841910768079015534.post-80559612477782936422014-07-05T18:34:34.039+01:002014-07-05T18:34:34.039+01:00I was experimenting with LabArchives but am waitin...I was experimenting with LabArchives but am waiting until paper is accepted before turning it live. Since then I have been also experimenting with using OpenScienceFrameworkdeevybeehttps://www.blogger.com/profile/15118040887173718391noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-51850162261853851652014-07-02T13:10:15.502+01:002014-07-02T13:10:15.502+01:00Out of curiosity, where did you post the data set ...Out of curiosity, where did you post the data set and R code? GitHub? Joehttps://www.blogger.com/profile/10216250557919588707noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-65288534504229140692014-06-14T17:22:06.101+01:002014-06-14T17:22:06.101+01:00An example of data sharing at its best: https://ww...An example of data sharing at its best: https://www.youtube.com/watch?v=N2zK3sAtr-4jrkrideauhttps://www.blogger.com/profile/04869979887929067657noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-55570279894873376112014-06-01T15:28:53.050+01:002014-06-01T15:28:53.050+01:00Re : Piketty
A rather good (devastating?) respons...Re : Piketty<br /><br />A rather good (devastating?) response by Thomas Piketty to Chris Giles criticisims in the Financial Times.<br /><br />ineteconomics.org/sites/inet.civicactions.net/files/Piketty2014TechnicalAppendixResponsetoFT.pdf<br /><br />One of the things that is very impressive is that he appears to have made every scrap of data available on line.<br />jrkrideauhttps://www.blogger.com/profile/04869979887929067657noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-7116055023300861932014-05-28T19:49:57.537+01:002014-05-28T19:49:57.537+01:00Thanks for directing me to Nate's website. He...Thanks for directing me to Nate's website. He makes some good points and I have not yet had a change to read Chris Giles' article as my local university library does not have an electronic version of the FT. I am going to have to track it down in hard copy. It's got to be in the university library somewhere--I'm not likely to be able to buy a copy here in a small city in Canada.<br /><br />Re Excel. I gave up on it some time ago simply because I don't like the new interface (and I was a beta tester for the Mac version back in the 80's). When I really need a spreadsheet I use Apache Open Office.<br /><br /> I find it's usually a lot faster and easier to just go directly to R for any analyses. Matter of taste I guess although I really don't trust the Excel stats routines. I know of one instance, some years ago, where someone ran a linear regression and ended up with a negative Rsq.<br /><br />jrkrideauhttps://www.blogger.com/profile/04869979887929067657noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-33712850911160850412014-05-28T09:14:25.940+01:002014-05-28T09:14:25.940+01:00thanks for your comment.Re Excel : I think it has ...thanks for your comment.Re Excel : I think it has its uses- I will often do a preliminary quick and dirty look at a dataset in Excel, not least because it is so easy to see data and plots alongside one another. I then do the serious analysis in R or SPSS, but having the Excel version provides a good double check, and I have trapped errors when different approaches give discrepant results.<br />I am fascinated by the current debate on Piketty - I had only been very vaguely aware of this until someone on Twitter asked if my post was inspired by the Piketty case. I can now see why - very interesting parallels in terms of error detection. I liked this account of the story, which I think is v balanced:<br />http://fivethirtyeight.com/features/be-skeptical-of-both-piketty-and-his-skeptics/deevybeehttps://www.blogger.com/profile/15118040887173718391noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-83769996597228310792014-05-27T19:06:00.238+01:002014-05-27T19:06:00.238+01:00Speaking of making errors I just wiped out all my ...Speaking of making errors I just wiped out all my comments. If the preceeding seems a bit out of context that's because this was supposed to preceed it:<br /><br />Well done Dr. Bishop.<br /><br />Yes, it is all too easy to make a mistake. I remember as a graduate students, some (cough) years ago, running a correlation with the result of r = 1. I was quite excited until common sense took hold. One should not correlate line numbers with sequential ID numbers.<br /><br />Recently in the economics field there seem to have been some rather dismaying data entry and analysis errors--none of which look deliberate but very distressing particularly the Reinhart-Rogoff paper which has had a very significant influence on government policy in many countries. It was several years later (4-5?) before they released the data to a grad student who proceeded to point out a multitude of errors. <br /><br />Having the data released immediately, probably would have allowed some immediate corrections and damage control rather than having it help set monetary policy for a country like the USA.<br /><br />I doubt that economists are naturally more prone to these mistake than any other researchers but they have been having a bit of a rough time at the moment (see below). I will point out that Piketty for his book, Capitalism in the 21 Century did publish his data at the same time as the book.<br /><br />As a pet peeve of mine, it looks like all three examples used Excel as their main analysis tool. I personally feel that a spreadsheet has no place in serious, or even frivolous, data analysis.jrkrideauhttps://www.blogger.com/profile/04869979887929067657noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-36906222903563307282014-05-27T19:03:59.387+01:002014-05-27T19:03:59.387+01:00The Reinhart-Rogoff error – or how not to Excel at...The Reinhart-Rogoff error – or how not to Excel at economics <br />http://theconversation.com/the-reinhart-rogoff-error-or-how-not-to-excel-at-economics-13646<br /><br />Richard Tol<br />Errors in estimates of the aggregate economic impacts of climate change <br />http://www.lse.ac.uk/GranthamInstitute/Media/Commentary/2014/April/Errors-in-estimates-of-the-aggregate-economic-impacts-of-climate-change-%E2%80%93-Part-II.aspx<br /><br />Financial Times Finds “Many” Errors in Piketty Analysis, Argues They Undermine His Thesis<br />http://www.nakedcapitalism.com/2014/05/financial-times-finds-many-errors-piketty-analysis-argues-undermine-thesis.html<br />jrkrideauhttps://www.blogger.com/profile/04869979887929067657noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-30357178641547008142014-05-27T09:32:40.797+01:002014-05-27T09:32:40.797+01:00Thanks, Mark. My experience exactly.
Also think th...Thanks, Mark. My experience exactly.<br />Also think this post by Betsy Levy Paluck is worth a read as a riposte to those who say they don't have time to prepare data for sharing. She agrees that it slows you down, but points out that this is no bad thing:<br />http://www.betsylevypaluck.com/blog/2014/5/25/what-i-stand-for-in-this-discussion-about-scientific-rigor<br />deevybeehttps://www.blogger.com/profile/15118040887173718391noreply@blogger.comtag:blogger.com,1999:blog-5841910768079015534.post-35543317264649508742014-05-27T08:28:50.581+01:002014-05-27T08:28:50.581+01:00Hi Dorothy,
Great post! I completely agree that p...Hi Dorothy,<br /><br />Great post! I completely agree that posting data online also improves one's own approach to the analysis. The simple act of preparing the data for someone else to understand is a useful debugging tool, gives you a different perspective and forces you to be a bit more organised/systematic than you might otherwise be. Russ Poldrack made a similar point recently, in a nice post dissecting a coding error he only detected when he shared his analysis scripts: http://www.russpoldrack.org/2013/02/anatomy-of-coding-error.html. Basically, error is inevitable, especially for bespoke script-based analyses, so we really do need another pair of eyes. I am always struck by how much scrutiny we put to the text of a manuscript, send it around to colleagues and co-authors for endless proof-reads, edits, corrections, but rarely show anyone else the original working out of analyses. This must be the wrong way around. Moreover, I worry that error is not random. We are far more likely to double check anomalies that contradict our hypotheses than nice publishable results (biased debugging: http://the-brain-box.blogspot.co.uk/2013/02/biased-debugging.html). Preparing data (and analysis scripts) for public scrutiny is a great way to improve the reliability of research findings. <br /><br />Mark<br /><br />StokesBloghttps://www.blogger.com/profile/00890404304081225894noreply@blogger.com