Monday, May 01, 2006

For Science's Gatekeepers, a Credibility Gap

NYT story about the refereeing process and it's shortcomings... For Science's Gatekeepers, a Credibility Gap. It should be noted that this article focuses on medical studies which have very different concerns and conflicts of interest than more basic science research (e.g., on planet formation).

8 Comments:

Blogger Vincent said...

Indeed, the situations described in the article sound pretty foreign to me. Maybe I'm oblivious, but I get the impression that astronomy doesn't suffer from this same sort of problem (at least not to the same degree).

5/01/2006 11:57:00 PM  
Blogger Eric said...

Agreed. I recently was talking with Sandra Faber (astronomer at UCSC who enjoys participating in some meta-level discussions), and she pointed out that she is not aware of a single case where an astronomer deliberately falsified data (and got cought). I wonder whether that's because the pressures to get results are not as extreme in astronomy, or whether biomedical/applied chemical research is more likely to attract people who might stray into the dishonest, or whether astronomers are just less likely to get caught (perhaps because there isn't as much redundancy as in the sciences that are more applied and better funded). As an astronomer, I'd like to beleive that it's one of the first two.

5/03/2006 03:46:00 AM  
Blogger Vincent said...

Plus, there's always the profit motive in biomedical research, which may override the scientific motive to uncover the truth. As far as I know, nobody gets rich off astronomical research.

5/03/2006 11:57:00 PM  
Blogger finou said...

yea but wasn't there some issue a while back about astronomy results that were stolen or something like that? I thought I remember reading something about some guy who published stuff that he got from some other group.... (of course, I could be totally wrong)
and of course there was the physics guy who totally falsified data over several years that came to light a year or so ago... I don't think this is limitted to medical research...

5/04/2006 09:39:00 AM  
Blogger Justin said...

I would suspect that astronomy is a bit of a special case because individual researchers don't have their own lab setups, they're all sharing the same telescopes. When you have your own lab, it's a heck of a lot harder for someone else to replicate your setup and verify your data, increasing the apparent safety of falsifying data. When the next guy in line can point the same telescope at the same point in the sky, it's a lot more risky to claim you saw an object there that you didn't.

5/04/2006 11:24:00 AM  
Blogger finou said...

I think disagree with that. For the most part (except for special home built setups that are few and far between), the individual labs of people that work in the same are have pretty much the same equipment and they have access to the same reagents and cell lines. You can pretty much replicate most anything you read. Granted sometimes it takes a while and you have to talk to people to get all the steps but usually new projects start with a "replicate this data" part.

Astronomy on the otherhand you need to get time on joint facilities and if you have your time scheduled to do something for yourself, it seems to me you wouldn't want to waste it repeating someone elses work?

Anyway, the physics thing I was thinking of was the 2002 thing that Mwal had mentioned on this blog a while back

5/04/2006 05:07:00 PM  
Blogger Eric said...

Uhm, that post refers to cloning (biology), making transistors (material science), and high Z elements (chemsitry). Granted each of those projects depend on a fair bit of physics (what doesn't), but I wouldn't have classified them as physics. That said, I can see how some people would could it physics (at least for the latter two), and some of the scientists are likely to have been trained as physicists.

I think most of the data on scientific misconduct are consistant with a profit motive being the primary problem.

Yes, many observatories are open to many people, but most are not avaliable to everybody. Somebody has to pay for them, and the people who provide the money usually get most of the observing time. (People/organizations that provide land, instruments, expertise, etc. may also get their own time.) And some astronomy projects are so big, that it's unlikely that they will be made again any time soon. That said, most of the facilities have a policity that after so much time the data becomes public. So others can get access to the raw data and analyze it for themselves. That means that if you make a sufficiently exciting/controversial/important discovery, then there's a good chance that someone else will reanalyze the data (often not to test if you're honest, but as the first step in doing a deeper analysis).

5/05/2006 01:52:00 AM  
Blogger finou said...

The Jan guy was a physicist, he got a bunch of physics awards, and the story was a big deal for the APS (not the MRS btw) so I would classify that as physics. In the press, it was labeled as a scientific contraversy in Physics so for better or worse I think the general American public sees it as Physics.
I think most scientific misconduct in academia (perhaps not industry) happens due to people wanting to make a big name for themselves not so much for the monetary profit. They start fudging data a little bit and then it goes in a downward spiral...

5/05/2006 02:21:00 PM  

Post a Comment

<< Home