Beyond The Commons

Aaron Wherry covers all the goings-on in and around Parliament Hill. Follow Aaron on Twitter: @aaronwherry

Categories: Beyond The Commons, Capital Read

Let us now argue passionately about data collection

by Aaron Wherry on Friday, July 2, 2010 12:51pm - 61 Comments

The government has announced, on the grounds of “personal privacy,” a rather important change to the census. Stephen Gordon, in response, is scathing.

It’s not often that sample selection bias becomes an issue of national importance, but then again, it’s not often that census sampling design is outsourced to drunken monkeys. The people who thought that increasing the number of long forms to be sent out will somehow compensate for making the long-form responses voluntary simply do not understand the gravity of the error they are making.

Tags: census, Statistics Canada, StatsCan, Stephen Gordon, Tony Clement

More by Aaron Wherry

Emily

This 'voluntary' idea is purely an import from the US where census takers were actually shot at!
http://intensedebate.com/people/LynnTO LynnTO

I know Gordon sounds melodramatic, but he's right.
http://intensedebate.com/people/tobyornotoby tobyornotoby

I get off the census bandwagon at the point where StatsCan starts selling the information they get from me, as opposed to providing it free to researchers, etc. At that point they are behaving the same as the so-called market research companies that I routinely hang up on.

For instance, a few years ago I somehow got on the list for a Stats Can labour market questionairre administered by phone once a quarter to ask detailed questions about my labour history, current employment, hours of work and over-time, all of which they claimed was compulsory and which took a half an hour on the phone each time. (Same as last time wasn't an acceptable answer).

I did it for a year, and then when they called me again the next year, I said I didn't want to have to do this forever, that I had done my bit and they should find another victim. They said it was compulsory, and that I would be breaking the law if I didn't answer, at which point I suggested they send me a letter explaining that so my lawyer could look it over and advise me on my rights. The letter never arrived.

http://bigcitylib.blogspot.com bigcitylib

Trying to make money off statscan/other Canadian stats publications was Mulroney's idea.

tobyornotoby

Yes, and it was a mistake and that, plus the highhanded attitude above, is undermining the entire census. If someone like me who actually uses and values some Stats Can information resents their collection methods, I'm sure someone who doesn't is going to be even less thrilled.

Anon 001

Tony Clement, one freaking disaster after another.
http://intensedebate.com/people/MostlyCivil MostlyCivil

Actually, a lot of statscan's best customers are provincial and municipal governments. As an example, the "how many rooms in your house" question referred to in the article has a purpose.

When you're trying to figure out where to put the new hospital, how many schools to build (and where to put them) or judge capacity for new highways/public transport, it's nice to know how many people COULD live in an area. Just because our three-bedroom house has two people in it right now doesn't mean we'll always be there. There could be 5 or 6, in a year or two. Multiply this by the 10,000 homes in my school district, or 50,000 in the hospital service area….

http://intensedebate.com/people/bill_y bill_y

From the 2006 long form guide:

Information on the number of rooms and bedrooms in homes and on housing costs is
combined with data on the number of persons in households to assess the economic
situation of families in different regions. Provincial and municipal governments use
this information to measure levels of crowding within households and to develop
http://www12.statcan.gc.ca/census-recensement/200…

appropriate housing programs.
http://bigcitylib.blogspot.com bigcitylib

Actually, governments are hardly the only users of statscan census data. If you, for example, are interested in setting up an outlet for an Italian restaurant franchise in Canadian city X, this material is the best way to find out where all the Italians are. The private sector uses this material in all sorts of ways.

http://intensedebate.com/people/ChrisInKW ChrisInKW

Bottom line, this is going to adversely affect the quality of policy made in this country at all levels of government. Direct correlation between quality of data and quality of governance. Quality of data will go down and will take the quality of decisions down with it.

This is ultimately going to result in inefficiency (read wasted money). However, I'm not surprised Harper dreamed this up. He hates facts and figures, they get in the way of his ideological tendencies and emotion-based policy.

Guy

It takes StatsCan 2 years to process the information. It is useless by then.

http://intensedebate.com/people/Ottawa_Centrist Ottawa_Centrist

Stephen is right.
http://intensedebate.com/people/Gaunilon Gaunilon

They may not actually be drunken. It's difficult to tell.

http://intensedebate.com/people/PhilCP Phil

You accept that monkeys were/are involved, but you're just not so sure that they were drunk? Gotcha! ;-)
madeyoulook

Good luck approaching them apes with the Breathalyzer.

http://intensedebate.com/people/Stewart_Smith Stewart_Smith

Once monkeys start to party, you can expect flinging to beginimmediately. Once they achieve drunkendom, their aim is way off.
Lord Kitchener's Own

They're not apes, they're monkeys! Can't you tell the difference?

Are YOU drunk?

lol

hosertohoosier

I welcome corporate and government intrusion into my privacy. The more the government knows about me, the better it can serve me. When resources are allocated to different municipalities, census counts of how many people, with whatever set of characteristics play a big role in who gets what. Supermarket scan cards are an even more valuable "intrusion". By knowing what products everybody likes, supermarkets can stock products that only a few customers like. Without that kind of technology you wouldn't have microbrews or fancy teas in supermarkets (NB: I live in the US).
http://intensedebate.com/people/Jenn_ Jenn_

This would be a better point of Harper's if I didn't get a call, this past Wednesday, from Ekos Research. They were calling about a taxpayer (remember, I'm in the public accounting realm) and wanted to ask some questions of his accountant.

Now, who this taxpayer's accountant is, is not public knowledge. It's not like we're listed in the phone book under "Joe Smith's accountant" or anything like that. So right away as the privacy officer, I've got a problem.with Canada Revenue Agency providing Ekos with the name of the client, the name of the accountant, and the accountant's phone number. But beyond that, I have a real problem as a taxpayer with CRA or any government body issuing a contract to Ekos for this sort of thing. Because, of course, we don't have authorization from the client to talk to Ekos! And I'm willing to bet 99.99% of public accountants don't have authorization, either. So we're paying Ekos to phone a bunch of accountants only to be told they can't answer their questions?

But they care about privacy as it relates to StatsCan?

Gaunilon

At this point us righties throw up our hands and say "See? This is why you don't want the government running anything more than it has to….they'll screw it up and you won't even have the satisfaction of knowing that you can take your business elsewhere."

Yes MYL (aka CtG), "we righties".

madeyoulook

CtG?

(UPDATE: Never mind. Took me a while, but I got it.)
Lord Kitchener's Own

However, wasn't at least part of this post about the government contracting out to a private firm something they should be doing themselves?

http://intensedebate.com/people/LynnTO LynnTO

Jenn, I know I'm LTTP here, but a couple of questions:

1. In your profession, is there any sort of codified privilege or expectation of privacy that exists between you and your clients?
2. Was EKOS calling on behalf of the government?

I'm curious.

http://intensedebate.com/people/YSP YSP

There is no legal right of priviledge in Jenn's (& my former) profession in the same way that lawyers have. We can be subpoenaed & forced to testify against clients, which is why corporate lawyers sometimes do the same work we do for much higher pay.

However, short of being subpoenaed, we accountants can lose their certification if they blab about clients.

On the other hand, I've had some crazy right-wing clients who kind of believe there are other people in the world somewhere, and that they're called the government, and refer all inquiries to their accountant, because they don't talk to the government, and everyone they don't know is the government. (And yes, these people are capable of running a small business. Which is kind of why "boss" is a four-letter word.)
Jenn_

Further to YSP, who is correct IMO, we also have engagement letters for both personal and corporate clients that refer to our keeping personal information confidential. We have husband's and wives (and children) fill out and sign a form that allows us to give all tax returns to the one who comes in to pick them up–otherwise they each need to come in, even when we know the people involved and know that one spouse does that sort of thing while the other one, uh, doesn't at all.
For our discussing things with CRA, we have a form that the client signs specifically allowing CRA to discuss their tax returns with either a specific person, or the firm. Then CRA mails the client to ask if they in fact signed this form, if so, fine, if not, call the number on the letter. It is always top-of-mind (look at the scope and variety of stuff we know! Age, address, SIN, religion, medical procedures, income, assets, relationships) and is taken seriously. Okay, yes, I'm the privacy officer.

Yes, the gentleman did disclose that he was conducting the survey on behalf of the government. I've thought about this because I cannot believe even this government is that stupid, but I see no benefit for an identity thief–plus, how did he know this person was our client? Was it a test of the privacy provisions? It makes some sense, but . . . really? We're not particularly 'scared' by CRA, after all, the way some of our clients might be.

http://intensedebate.com/people/madeyoulook madeyoulook

I get, and I accept, the "personal privacy" rationale. But then this non-random "optional" participation makes the data pretty meaningless. So they may as well have scrapped the long form altogether.

And if they recoil in horror at the scrapping suggestion, because it is so crucial to have reliable data from the long form, then they shouldn't make it optional.

So, basically, they have, likely with good intentions, opted to have the worst of both worlds.

Makes you wonder if anyone in Cabinet ever took (or remembers) an introductory statistics course.

http://intensedebate.com/people/Stewart_Smith Stewart_Smith

You are thinking to much like Stephen G …. not enough like Ian Brodie.

1. Will this outrage the general population and lead to a Liberal majority?
Answer, No.

2. Will this hurt them with their core voters?
No, so this puts the crippled census reform on the table.

3. Will a note sent to their membership asking for financial help to keep the coalition, power-mongering whores from invading your personal life be successful in raising funds for the Conservative party?

So this is not weak math skills on the part of the Conservatives, they are simply optimizing a different parameter.

Glum

But Tory-friendly Big Business isn't going to be happy about this, either. They use that information to market to their customers. The marketing firms that Big Business employs also uses that information.

Guy

The problem here is that by the time StatsCan has the data ready for use it is 2 years old. So it's pretty meaningless to begin with.

http://intensedebate.com/people/PolJunkie PolJunkie

I saw this story a couple of days ago and have been hoping ever since that the MSM would latch on to it.

This policy decision very much symbolizes all that is wrong with these amateurs running our government. It has no basis in facts, data, analysis or science. It is purely based on rightwing ideology.

One has to wonder how many more of such decisions have been made in this fashion and have not made it into the papers…
http://intensedebate.com/people/Stewart_Smith Stewart_Smith

I find it hard to see evidence of Conservative incompetence in this new measure. On law&order, tax reform, foreign policy etc, this government has been frustrated and opposed by nasties who stoop to evidence-based arguments and even facts to argue against proposed government policies. When someone routinely is using facts to beat you over the head, the only logical response is to take away those facts.

http://intensedebate.com/people/NorthernPoV NorthernPoV

Facts?
we don't need no stinkin' facts!

wilson

''I don't really care if some lady in Saskatchewan sues over privacy''

Perhaps she is about to win the case, so the govenment is getting infront of the issue.
Lord Kitchener's Own

If the government needs information about us, why don't they just ask Google?

I'm pretty sure Google knows more about my life than I do.
http://intensedebate.com/people/Stewart_Smith Stewart_Smith

I am (somewhat) sympathetic to privacy issues, which certainly dates me. One compromise would be to have a mandatory short form, that provides some of the information needed for the most important correlations, followed by the remainder of the long form as voluntary. The return rate of the voluntary portion could then be corrected, wrt at least the variables in the mandatory part.

http://intensedebate.com/people/madeyoulook madeyoulook

Isn't that pretty much what they are doing? I presume the 20% who get the "optional" long form still must complete at least the short-form content. No?

http://intensedebate.com/people/Stewart_Smith Stewart_Smith

who knows? … not I

inge

I don't think this is about privacy at all. This government is much happier without information and analysis interferring with their spin. Clement admitted that they did not consult with anyone (not even the department apparently!) The privacy issue is merely a convenient rationale for doing what suits them for their own reasons. It is however mind boggling that they would even suggest that voluntary responses would have statistical validity.
Holly Stick

There was a long campaign by genealogists, historians, etc., to have Census records released after 92 years instead of never; they got legislation passed in 2005. Possibly this is a way for the government to avoid the requirements of the 2005 legislation.
http://globalgenealogy.com/globalgazette/gazgw/ga…

This will not make genealogists happy. Many of them are retirees, possibly somewhat conservative in their political views. They may now change their votes.

http://intensedebate.com/people/YSP YSP

Nah, cause logically, genealogists are concerned about society, & study statistics & worry about facts nobody else has cared about for a hundred years. That probably makes them Dippers or Greens.

Lou

Wrong! Most of my friends who research their families vote CPC. The census short-form was already ruined in 2006 by making release optional without Stats Can explaining that people when they get older (and some jistorically-minded young people) like to understand where they came from.

http://intensedebate.com/people/Geiseric Geiseric

WTF?

I can't say 'Fascist' in here?
Steve M

Personally, I think thsi is much ado about nothing, and Stephen Gordon's "6-ft tall" ananolgy is wrong. A better analogy would be if only people who were over 6-ft tall replied to the question, except the question had to do with something unrelated to height, like the colour of your hair.

Let's wait and see what percentage of the voluntary forms get filled out before we freak out.
http://intensedebate.com/people/StephenGordon StephenGordon

Nope. Sample selection bias occurs if response rates are correlated with the variable you're trying to measure. If response rates vary with income and education levels, then you no longer have a random sample for income and education levels.
http://intensedebate.com/people/madeyoulook madeyoulook

And since you won't even know how response rate varies with the variable you're trying to estimate, then the voluntary reporting business is a waste of time. Unless you want to know the income or education levels of people who volunteer to answer StatsCan's long form. Which, I don't.

SteveM is incorrect. StephenG is correct. As soon as it is non-random, it is not helpful for extrapolation.
Steve M

And we know that the non-responders will all be similar rather than as diverse as the general population because…?

I'm not saying it's not possible, but it seems like everyone here is taking it as a given.
Guy

You are assuming, quite incorrectly, that all of those filling the long form are providing 100% accurate and truthful information. A random sample of lousy information makes a rotten data set. Garbage in, garbage out. There were several well organized campaigns in 2006 which encouraged Canadians to provide incorrect/incomplete data on their census form.

Let us add to the case the fact that the long form was sent out as a random sample. Any statistician will tell you that there is no absolute guarantee that the forms would be distributed truly randomly amongst all education and income levels.
Steve M

"As soon as it is non-random, it is not helpful for extrapolation."

So all those polls we political junkies fret over are actually meaningless? 'Cuz last time I checked, no one was forcing me to not just hang up when they called my house.
Mike T.

It's pretty much a given in statistical analysis circles, yeah.
http://intensedebate.com/people/NorthernPoV NorthernPoV

based on very limited knowledge (1st year U stats course)
I'd say you are way out of your league Steve M, give it up before you embarrass yourself some more
- or become a Harper MP as you are obviously qualified.
madeyoulook

Not that non-responders will all be similar. Just that non-responders as a group will be different enough from the group of responders, in both known and unknown characteristics, as to make it impossible to confidently apply the results to the entire population.
Amateur Hour

A lot of public policy and spending is influenced by census information. We receive tens of thousands of dollars per year per person in benefits from said policies and spending. More importantly, long-term planning requires the best information available vs. the information we get around to providing, if we care to.

I don't really care if some lady in Saskatchewan sues over privacy. I DO care that the Industry Minister's spokesman has twisted a few people complaining into an omnibus statement for all Canadians vis a vis the census: “Our feeling was that the change was to make a reasonable limit on what most Canadians felt was an intrusion into their personal privacy in terms of answering the longer form".

Most Canadians feel the census long form is an intrusion into personal privacy?

Prove it. (Oh, you'd have to intrude on people's privacy to do that.)
http://intensedebate.com/people/madeyoulook madeyoulook

The greater the refusal rate, the more meaningless the survey results.
Stewart_Smith

There is a significant difference, because the big question that political polls seek to answer is what would happen in an election. That means that in principle it should be possible for the stats-geeks to use election results to develop models that describe how the polls should be interpreted. The other political questions that are polled are providing entertaining "information" for political junkies. (Our version of Entertainment Tonight)

An analogy would be to have a mandatory form in every other census. The results of that mandatory census could be used to calibrate the preceding voluntary census information.
Lord Kitchener's Own

So all those polls we political junkies fret over are actually meaningless?

Not entirely meaningless, no, because most polling firms correct for their sample biases. They do so by comparing their biased sample to the more accurate representation of the population found in the CENSUS.
http://intensedebate.com/people/Jenn_ Jenn_

I've never been given the opportunity to fill out a long-form census, at least that I recall. Do you have to provide your name and social insurance number?
Guy

In 2006, several well-organized groups were formed to encourage Canadians to provide inaccurate information on the census. It was mainly to protest awarding census software contracts to a US company. Quite a large number of Canadians supported this cause. So, yes, many Canadians feel that the long form is an intrusion to their privacy. Respecting privacy trumps census information, anywhere, anytime.
Steve M

Then I repeat, let's wait and see what percentage of the voluntary forms get filled out before we freak out.
http://intensedebate.com/people/madeyoulook madeyoulook

Fear not, Steve. I am not freaking out. But the quality of the data collected will suffer. The quality will be somewhere between (A) acceptably inferior to what a random obligatory sampling would have given (say, 5 to 10% decline to answer) and (Z) a web-based survey with four geek respondents, three of whom are the same guy. People can place bets about where between (A) and (Z) it will end up. My guess: about (P) or (Q), maybe.
http://intensedebate.com/people/Party_of_One Party_of_One

Your name and address goes on the cover page, same as the short form. That cover page is used only to confirm that you have responded. It is separated from the rest of the information before that information is collected from the rest of the form (hence the seration, if you've noticed it).
Guy

You're assuming that all of the long forms have been answered 100% accurately. No way that happened.