Meta-Blog

SEARCH QandO

Email:
Jon Henke
Bruce "McQ" McQuain
Dale Franks
Bryan Pick
Billy Hollis
Lance Paddock
MichaelW

BLOGROLL QandO

 
 
Recent Posts
The Ayers Resurrection Tour
Special Friends Get Special Breaks
One Hour
The Hope and Change Express - stalled in the slow lane
Michael Steele New RNC Chairman
Things that make you go "hmmmm"...
Oh yeah, that "rule of law" thing ...
Putting Dollar Signs in Front Of The AGW Hoax
Moving toward a 60 vote majority?
Do As I Say ....
 
 
QandO Newsroom

Newsroom Home Page

US News

US National News
Politics
Business
Science
Technology
Health
Entertainment
Sports
Opinion/Editorial

International News

Top World New
Iraq News
Mideast Conflict

Blogging

Blogpulse Daily Highlights
Daypop Top 40 Links

Regional

Regional News

Publications

News Publications

 
Scientific Studies: More wrong than right?
Posted by: McQ on Saturday, September 15, 2007

Frankly, given the "science" we've seen displayed through various studies on various issues over the last few years, this comes as no real surprise.
Dr. Ioannidis is an epidemiologist who studies research methods at the University of Ioannina School of Medicine in Greece and Tufts University in Medford, Mass. In a series of influential analytical reports, he has documented how, in thousands of peer-reviewed research papers published every year, there may be so much less than meets the eye.

These flawed findings, for the most part, stem not from fraud or formal misconduct, but from more mundane misbehavior: miscalculation, poor study design or self-serving data analysis. "There is an increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims," Dr. Ioannidis said. "A new claim about a research finding is more likely to be false than true."

The hotter the field of research the more likely its published findings should be viewed skeptically, he determined.
The last sentence is easy to explain - there's money in them thar studies, especially if they return the desired results.

Of course, you should also be skeptical of the Ioannidis findings as well, since they were the result of an, ahem, scientific study. Heh ...
 
TrackBacks
Return to Main Blog Page
 
 

Previous Comments to this Post 

Comments
Many people see science as either
a) a source of oracular pronouncements... "Truth" being handed down from on high, or
b) agenda-driven quackery (to be dismissed unless it supports one’s own agendas).

Science is a process. It is a means of gathering new information. It is inherently capable of errors along the way. Indeed, it wouldn’t work without them. There is nothing wrong or surprising about the statement, "A new claim about a research finding is more likely to be false than true."

The only reason this presents any problem is because so many people are, deep down, authoritarian minded. Science - with its inherent missteps along the road to greater knowledge - is used to back up government policies. That’s the danger.

If you had an incorrect understanding about the gas planets orbiting distant stars or the connection between cigarette smoke and healing bones, it wouldn’t actually affect me. At the risk of sounding like a damned crazy libertarian or anarchist, the problem is not that the conclusions of scientists are often false. The problem is that the government has higher priorities than the protection of my rights.
 
Written By: Wulf
URL: http://www.atlasblogged.com
As Isaac Asimov put it in "The Relativity of Wrong"...
When people thought the earth was flat, they were wrong. When people thought the earth was spherical, they were wrong. But if you think that thinking the earth is spherical is just as wrong as thinking the earth is flat, then your view is wronger than both of them put together.
Science is about gradually getting things less and less wrong. Research is often wrong, but it is usually a step closer to correct.
 
Written By: Jon Henke
URL: http://QandO.net
I work in academic medical research, so there are just a couple things I wanted to point out. First and foremost is that these studies he talks about are not published to intentionally mislead.

The false conclusions reached and published (there is a distinction with "published" that I will get to in a second) stem from 2 distinct problems.

False conclusions are reached when a researcher is so sure of his/her theory that they acknowledge evidence to the contrary as an anomaly. Certainly with the complexity of biological research these days, this is almost a necessity. For an example: I worked on projects dealing with cell targeting. Sometimes we would have 30% success rate, other times we would have 50% success rate, and still others would have 5% success rate (control targeting = 0% success). From the data, we could show that we were targeting, but not perfectly. All 3 experiments were completely equal, but held divergent results. Figuring out why we had divergent results would take years and millions of dollars in funding. Common practice in these situations is to cherry-pick your good results and go on. Doing this consistently can skew subsequent conclusions based on compounded data.

The second (and worse IMHO), is in the peer review bias. Experiments are performed, data is collected, and the results are written into a paper. The paper is sent off to journals for peer review. During the review process, reviewers look at the experiments and conclusions and decide whether the paper is good as-is, if conclusions need to be reconsidered, or if additional experiments are required. The reviewers for the journal also decide whether the paper is ’worthy’ enough to be published in their journal. The peer review process serves as a gatekeeper to publication. For each niche of research, there are so-called ’experts’ that review the papers. If you are working in a hot or controversial area of research, politics can grossly affect whether the [ultimately] correct conclusions are published. There may be 3 competing theories for how something works. The expert reviewers have their own pet theories, and discount contrary theories. If your theory clashes with theirs, they are going to be especially hard on the review process and your chances of publication drop. It’s not so much a retribution idea, but there is a subconscious bias that your study must be flawed b/c they are sure of their studies. In other cases, incorrect conclusions may be published because the reviewers exercise a subconscious confirmation bias, unintentionally ignoring holes in data and logical leaps. One of the worst travesties in research is when correct conclusions are unintentionally suppressed for years because they do not fit the conventional wisdom of reviewers.

I forget who said it, but to paraphrase: "the problem lies not so much in what don’t know, but what we know just isn’t right"
 
Written By: researcher
URL: http://
Aren’t you guys forgetting the importance of consensus? If there’s a ’scientific consensus’ then all the articles that support it must be right, right?
 
Written By: JorgXMcKie
URL: http://
...and this is the anti-science rhetoric I’ve been talking about seeing here. Of course most studies are wrong: if they weren’t, we’d be done by now. Both Wulf and Jon’s responses are spot-on.
 
Written By: Mithras
URL: http://harnly.net
I took a Sociology of Science class in college (so since I’ve read two books on the subject, I’m officialy an expert, LOL). I have to say that Wulf’s comment is spot-on. The deification of science by society is the problem. I don’t know if the cause of this deification is, as Wulf claims, an innate desire for authoritarian control; personally I think it’s more that people need a god and something to believe in that gives purpose to their lives, and if they’ve already ruled out the traditional all-powerful being and working toward a heaven, they have to have something, so they believe in science and scientific progress. And then we get travesties like "scientific consensus", because if it’s a god there’s just no way it can have feet of clay, right?
 
Written By: Wacky Hermit
URL: http://organicbabyfarm.blogspot.com
Mithras, what is anti-science about pointing out a study which demonstrates that published research can be flawed more than most realize? If anything McQ has been more anti-science BS than plain anti-science.

I too agree that Wulf, Jon, and Researcher make excellent points.

 
Written By: Grimshaw
URL: http://
Of course most studies are wrong: if they weren’t, we’d be done by now.
Bull. We would still have plenty to explore that the studies didn’t and often couldn’t address.

 
Written By: Don
URL: http://

 
Add Your Comment
  NOTICE: While we don't wish to censor your thoughts, we do blacklist certain terms of profanity or obscenity. This is not to muzzle you, but to ensure that the blog remains work-safe for our readers. If you wish to use profanity, simply insert asterisks (*) where the vowels usually go. Your meaning will still be clear, but our readers will be able to view the blog without worrying that content monitoring will get them in trouble when reading it.
Comments for this entry are closed.
Name:
Email:
URL:
HTML Tools:
Bold Italic Blockquote Hyperlink
Comment:
   
 
Vicious Capitalism

Divider

Buy Dale's Book!
Slackernomics by Dale Franks

Divider

Divider