Lessons from a year of ‘Science-ish’

By Julia Belluz - Thursday, May 31, 2012 at 6:28 PM - 0 Comments

Left: Erik K Veland/Flickr Right: ShironekoEuro/Flickr

It’s been a year since Science-ish launched with the aim of scrutinizing the news coverage of health topics, and holding politicians, opinion leaders and journalists to account for misusing or misrepresenting science. The modest goal of the blog was, and remains, to help readers wade through the evidence on a given subject and get a sense of what the science actually shows.

So what has Science-ish learned in 12 months of fact-checking the reporting on everything from the health effects of asbestos, to whether the benefits of urban cycling outweigh its harms, how and if cancer screening will save your life, the “cures” for autism, dubious mental health statistics, and just about every health story in between? Here are five key lessons for telling science from science-ish:

Continue…

I bet Mr. Trower has a delightful accent

By Colby Cosh - Tuesday, August 24, 2010 at 7:27 PM - 0 Comments

“Cold War weapons expert warns Wi-Fi could cause birth defects” cries the National Post, heralding Barrie Trower’s arrival unto the good microwave-fearing people of Simcoe County. Who is Barrie Trower, you ask?

Barrie Trower, who specialized in microwave “stealth” warfare during the Cold War, was to lecture at the University of Toronto on Tuesday night…

“When I realized these same frequencies and powers [as weapons during the Cold War] were being used as Wi-Fi in schools, I decided to come out of retirement and travel around the world free of charge and explain exactly what the problem is going to be in the future,” Mr. Trower told Postmedia News in an interview on Tuesday.

…“What you are doing in schools is transmitting at low levels,” said Mr. Trower, who teaches at Britain’s Dartmoor College and holds a degree in physics.

You will notice what’s very specifically not been said here, which is that Mr. Trower teaches physics at a university. Lest anyone should carelessly arrive at this impression, it ought to be said that what the Post calls “Dartmoor College” is South Dartmoor Community College, a state comprehensive school for children aged 11-18. They are doubtless lucky to have a “weapons expert” like Mr. Trower on staff (assuming he is on staff), although it is damned hard to be a military expert in anything for any length of time without inadvertently getting your name on any patents or peer-reviewed papers to speak of. Trower has said he worked for what he called the “Government Microwave Warfare Establishment”; it’s possible the Post judged this a strong claim after Googling “Government Microwave Warfare Establishment“, or just “Microwave Warfare Establishment“, and finding links to loads of pages related to Barrie Trower and not much else. Excellent work.

[UPDATE, 1:15 a.m. Eastern: the Post's original story has vanished from the Web, so you'll have to visit the Vancouver Sun's site to read it.]

Science-reporting smell test of the week

By Colby Cosh - Tuesday, May 4, 2010 at 2:26 PM - 80 Comments

Here’s the lede of a science story from Saturday’s Winnipeg Free Press:

WINNIPEG — Depression and substance abuse plague about half of American women who reported having an abortion, according to a new University of Manitoba study.

The study, published in the current issue of the Canadian Journal of Psychology, suggests there’s an association between mental disorders and abortion…

Eager to investigate this shocking headline claim—the Edmonton Journal, picking up the story, literally gave it the headline “Depression or drug abuse found in half of women who aborted”—I set out to find the study. This presented something of a problem, since there has not been a “Canadian Journal of Psychology” since 1993. I spent a little while rifling through Canadian Psychology and the Canadian Journal of Experimental Psychology until a helpful reader on Twitter clued me in. Yes, you guessed it: it can be found in the Canadian Journal of Psychiatry. First place I should have looked, really.

That’s an understandable mistake. It’s a bit more of a problem that the first sentence of the article—an article that includes a warning from the lead author to the effect that it is “important the study is not misinterpreted”—is totally false. Because of, y’know, misinterpretation.

The paper, entitled “Associations Between Abortion, Mental Disorders, and Suicidal Behaviour in a Nationally Representative Sample”, does what it says on the tin: the data are taken from interviews with a demographically representative subset of the U.S.’s National Comorbidity Survey Replication project. It is hard to know what numbers the reporter added or multiplied or pulled out of a hat to reach the conclusion that “Depression and substance abuse plague about half of American women who reported having an abortion.” (I spoke to the lead author of the study, and she can’t figure it out either.) But a good guess would be that she looked at this section from the article’s main chart—

—and simply added together the estimated lifetime incidence of depression among women who had had an abortion (29.3%) and the lifetime incidence of substance-use disorders (24.6%). It will probably have occurred to you that there might be some overlap there between depression and substance abuse, which go together like poached eggs and hollandaise. You don’t need a Ph.D. to know that the depression group is likely to contain almost all of the women in the substance-abuse group.

And this naïve math (which is hardly attributable to a failure to grasp hyper-advanced statistics) is compounded by the wording of the offending sentence, which doesn’t say that “some percentage of abortion recipients have, at some point before or after getting an abortion, experienced depression or substance abuse or both.” It uses present tense, unjustifiably implying that all the women in question are plagued by both problems now.

This mess is already being picked up, “carelessly” garbled even further, and circulated around the globe by pro-lifers, despite the personal entreaties of the scientist who helped the newspaper with its reporting and the many, many methodological and interpretive caveats in the original study. This kind of thing is exactly why a lot of scientists hate talking to reporters. Nor does it make sincere research into therapeutic abortion any easier. The UM study can’t be used to attribute psychiatric morbidity to abortion, but it could be used by fair-minded pro-lifers (let’s assume for the sake of argument that there were some) to raise questions about abortion’s place in our society and argue for a research program.

Oh, I know: we’re a hundred years away from that kind of discussion being possible. But the inadvertent propagation of urban legends only pushes that day further into the future.

Bittersweet justice for Singh

By Colby Cosh - Thursday, April 15, 2010 at 9:37 AM - 48 Comments

Simon Singh MBE, the celebrated science writer and documentarian, has officially won his libel tilt with the British Chiropractic Association. In April 2008 Singh wrote a column for the Guardian about the persistence of pre-scientific ideas in the British chiropractic trade. What most people now think of as merely an expert form of massage began with the claim that spinal maladjustments were the source of virtually all disease in humans, and some chiropractors still believe they can cure a lot more than back and joint pain. Singh wrote:

You might think that modern chiropractors restrict themselves to treating back problems, but in fact they still possess some quite wacky ideas. The fundamentalists argue that they can cure anything. And even the more moderate chiropractors have ideas above their station. The British Chiropractic Association claims that their members can help treat children with colic, sleeping and feeding problems, frequent ear infections, asthma and prolonged crying, even though there is not a jot of evidence. This organisation is the respectable face of the chiropractic profession and yet it happily promotes bogus treatments.

I can confidently label these treatments as bogus because I have co-authored a book about alternative medicine with the world’s first professor of complementary medicine, Edzard Ernst. He learned chiropractic techniques himself and used them as a doctor. This is when he began to see the need for some critical evaluation. Among other projects, he examined the evidence from 70 trials exploring the benefits of chiropractic therapy in conditions unrelated to the back. He found no evidence to suggest that chiropractors could treat any such conditions.

(Singh’s column went on to discuss the controversy surrounding the 1998 death of Saskatchewan chiropractic patient Laurie Jean Mathiason.)

English libel law is so tough on defendants that the world’s rich and offended will torture jurisdictional logic to the point of incoherence if it means their complaint can be heard in an English courtroom. But the BCA had it easy; Singh was trapped right there on the island with them. They took him to court. And only him; they chose not to name the Guardian in their claim at all.

English libel requires the judge to issue pre-trial rulings on the meanings of offending passages. Singh, whose piece had appeared in the Comment section of the Guardian, argued before Sir David Eady that his use of the word “bogus” meant only that there is no good evidence for the effectiveness of the impugned treatments. But the judge not only closed off the fair comment defence; he ruled, without giving much indication that he was paying close attention to the arguments or the relevant text, that the term denoted conscious and deliberate dishonesty. This shifted a frightful burden of proof onto Singh, requiring him to show not only that British chiropractors were offering useless and unverifiable treatments, but that they did so with the certain and specific knowledge that they were useless and unverifiable.

It became clear almost immediately that the BCA had overplayed its hand. Eady’s ruling rightly raised a worldwide clamour against the depraved state of the law and the health of free inquiry in the land of Newton and Darwin. (This has helped put libel reform on the agendas of all major parties in the current UK election.) It is, after all, almost not enough to say that science “depends” on the freedom to make tough evidentiary criticisms; considered socially, science is practically equivalent to the possibility of making them. Meanwhile, the beam of a million-watt searchlight had been attracted to the claims and conduct of the British chiropractic business. In a canonical demonstration of the Streisand Effect, the country’s statutory regulator of chiropractic, which holds the activity and advertising of practitioners to an explicitly scientific standard, was obliged to launch literally hundreds of investigations into strip-mall spine-crackers.

The harm that British chiropractic has done to itself is incalculable; meanwhile, it has had to give up hope of impoverishing Singh, who had Eady’s ruling overturned by the England and Wales Court of Appeal on April 1. In asking a public controversy concerning a question of evidence to be a matter for a libel suit, wrote the Lord Chief Justice, the BCA was inviting the court to serve as “an Orwellian ministry of truth”. The court, he added, must decline to do so. (He did not neglect to throw in a pinch of old John Milton and his Areopagitica.)

One ought not to admonish the BCA for abandoning its libel action; it was self-evidently the right thing to do. But what does it say about British libel law that the Association did so almost immediately once the fair comment defence was made available to Singh—a commentator by profession, one whose standing to assess and challenge scientific evidence could hardly be higher? Just one month ago, Singh announced that he would be ceasing his newspaper column for good. One hopes he will reconsider, but it is still uncertain that he will recover his own defence costs, and the time and effort he has expended will never be recouped.