Inkless Wells

Inkless Wells

Paul Wells on all the latest out of Ottawa—along with the occasional post about jazz. Follow Paul on Twitter: @InklessPW

Workfare for scientists: cheaper AND more productive!

by Paul Wells on Wednesday, April 15, 2009 9:27am - 31 Comments

I’m going to get in trouble for the title on this thing, which is meant to be a bit satirical. A helpful reader sent me a link to this abstract in the provocatively-named journal Accountability in Research, which argues that the costs of (a) preparing NSERC grant applications (a cost borne by applicants) and (b) sifting through grant applications in peer review to decide which are worthy (a cost borne by NSERC) is so onerous that it would be cheaper just to give every qualified scientist a $30,000 baseline grant. Here’s their abstract in full; to me, just about every part of it is intriguing:

Using Natural Science and Engineering Research Council Canada (NSERC) statistics, we show that the $40,000 (Canadian) cost of preparation for a grant application and rejection by peer review in 2007 exceeded that of giving every qualified investigator a direct baseline discovery grant of $30,000 (average grant). This means the Canadian Federal Government could institute direct grants for 100% of qualified applicants for the same money. We anticipate that the net result would be more and better research since more research would be conducted at the critical idea or discovery stage. Control of quality is assured through university hiring, promotion and tenure proceedings, journal reviews of submitted work, and the patent process, whose collective scrutiny far exceeds that of grant peer review. The greater efficiency in use of grant funds and increased innovation with baseline funding would provide a means of achieving the goals of the recent Canadian Value for Money and Accountability Review. We suggest that developing countries could leapfrog ahead by adopting from the start science grant systems that encourage innovation.

This paper has provoked an unholy ruckus (sorry, James Lunney) on assorted science blogs; a handy summary of the arguments and counter-arguments is here. From the author of that blog post:

This does not mean, in their proposal, that all of the Canadian money earmarked for science would be given this way – this is still just a small part of it. If you have a big lab or do expensive research and need to apply for much bigger grants, that would be done by the traditional peer review. But in order to get to the point where you have a good proposal, you need to have some neat stuff done (the “preliminary data”). With the proposed system, that preliminary data can be really exciting or revolutionary, something that, as an initial proposal, would never fly by peers.

Would people send out proposals for crap? Some would, I’m sure, but that doesn’t matter. Most would not. Scientists are curious about nature and would like to test their hunches. Some will flop, some will be amazing – it is the latter that this new system is worth doing for, as they may never be done otherwise. Anyway, how many $5,000,000 grants produced amazing stuff? All? I.Don’t. Think.So.

I’m just getting to all this now, after spending much of yesterday writing for our print edition. I hope to read Gordon and Poulin’s paper today, and I’ll be contributing more intelligently to the conversation later today. But the reason why I’m calling attention to this rather erudite debate on a general-interest blog is (a) to show how cumbersome and costly our country’s research apparatus has become and (b) to emphasize an occasional Inkless talking point: because nobody can ever predict where a discovery will come from, it is not necessarily better to add layers of “accountability” mechanisms between a scientist and her lab. Anyway, poke around in all of this, if you’re so inclined, and we’ll continue this conversation this afternoon.

Bookmark and Share
  • tobyornotoby

    It isn’t just research grants, try applying for a grant from HRSDC or Environment Canada, which have a more is safer approach to process. After the HRDC scandal circa 2001, the Canadian government began to use a gruelling contribution agreement process, that forces organizations to eseentially begin their projects, with no source of funding, in order to predict the costs and results. I’d have no problem with this, except the government A) almost always requires matching funding, and b) never allows the organizatiosn to recoup the cost of developing the proposals. Only the work performed after the agreement is signed can be funded.

    Compare that for example with a for-profit supplier to the government, which would be able to recoup those development costs as profit, but health, environmental, education and development groups that “supply” innovative projects and services, have hidden costs forced onto them by the process?

  • Andrew (not Potter or Coyne)

    I’m sold, Paul. I’ve heard lots of horror stories about the NSERC application process from the researchers I know. And getting rid of expensive bureaucracy sounds good to me.

  • Bill Simpson

    From a contrarian point of view, making the initial proposal very costly to prepare will weed out trivial submissions and to force all submissions into the approved mold, saving the bureaucracy.

    Of course, it is obviously easier on scientists to leave bags of money on the steps for anyone in particular to pick up and enjoy, but it is wise not to force discussions on the cost/benefit of this kind of activity too far.

    • Ian

      Contrarian in the sense that we then wouldn’t have any time or resources to conduct any research?

      • Bill Simpson

        Contrary to the point of the referenced article. But truly, if a serious research effort cannot attract $40,000 worth of backing, then it is probably not going to get a grant anyway.

  • http://www.maple-leaf-forever.com Lord Bob

    My essential objection can be summed up as “if you make something free, more people will come take it.”

    The fact that an applicant has to fill out a grant appication while facing the possibility of rejection means that some scientists are deterred from attempting it. If our system changes to “send us a photocopy of your diploma, get money”, well, I bet even the laziest scientist could manage to walk to the fax machine.

    And I don’t buy that “scientists are naturally curious and would use their powers for good, not evil” stuff. A great many scientists are very good people, but they’re people regardless. We’ve all seen decent, hard-working people talk themselves into accepting something unethical for their own benefit. Sample rationalisation: Scientists in this country don’t get paid enough, you know, the government never supports them, really, getting a spurious grant to test the impact of tropical sunshine on a Ph.D’s suntanning body is just levelling things out a bit…

    We have to accept the fact that the base cost would be much higher than $30,000 * [number of grant applications].

    That said, I’m still not entirely sure if you’re seriously supporting this or just raising an interesting point about how dysfunctional the existing system is, Mr. Wells, so possibly disregard all of this.

    • Ian

      Bob, researchers are employed by universities and junior researchers (which these grants would disproportionately benefit) are under enormous pressure to produce results for the department. Publish or perish.

      • http://www.maple-leaf-forever.com Lord Bob

        There’s no reason to believe that free money would disproportionately benefit any one group that is eligible for it. If I’m the senior professor of so-and-sology at the University of I’ve Got Tenure and the government offers me $30,000 for whatever “research” I want you bet I’m getting some of that.

        • Ian

          Sure, but even in that case the amount is probably some fraction of the researcher’s total funding, whereas for a junior researcher it’s everything and it’s the chance to actually get started. And despite what you might think, there is some accountability for how the money is spent – you don’t get to throw a wicked kegger for the department every Friday. The authors of this paper (which I know most people probably can’t access) even acknowledge this when they suggest this as an alternative:

          “2. Baseline funding, in which each researcher gets the same amount per year (Gordon, 1993; Poulin and Gordon, 2001). This protects each scientist from peer review. It is open to the objection that some scientists will do nothing useful with the money, and so it needs some modest checks and balances, such as requiring a public accounting of what research was accomplished. It also underfunds expensive research.”

          The “protects from peer review” point is important too. These discovery grants (http://www.nserc-crsng.gc.ca/Professors-Professeurs/Grants-Subs/DGIGP-PSIGP_eng.asp) are designed to support long-term research programs and not short term “we have to discover something we can sell” research. But there is still a lot of competition and so the award grants are weighted heavily towards projects that are halfway along and thus are probably going to yield positive results, or projects that are highly similar to previous work and thus are probably going to yield positive results.

          So maybe your senior prof has a great off-the-wall idea that would make a innovative research project for a new grad student, but he can’t squeeze it into his current research funding. Some baseline funding would pay for a student and some supplies and maybe they find something really novel.

          • http://www.maple-leaf-forever.com Lord Bob

            What sort of accountability do they have in mind?

            From the excerpt (thank you for that, by the way), the public accounting comes after the research has been done, i.e. after the money has already been spent. Obviously, therefore, they’re not going to recoup that. When Professor Lazybones’s empirical study on the impact of Black Russians on his blood alcohol level is soundly rejected in the public accounting, what do they propose to do about it?

            Get him to pay it back? He hasn’t got it anymore. Fine him $30,000 or some other significant amount? Even assuming the grant foundation has that sort of authority, I don’t know many academics with thirty grand in ready cash. What they’d probably end up doing is freezing or reducing access to future grants, which might just make the problem worse for young academics: if your grant is rejected in peer review than so be it, but if you’re cut off from future grants then your career is over.

          • Ian

            In your example the university administration would have to be complicit by accepting bar receipts as charges against the grant account. A grant isn’t a bag of money sitting beside the lab bench. And even if you slip stuff by everyone and party hard in Year 1 then your progress report is going to look pretty thin and Year 2-5 will probably be yanked.

            I’m not debating that the system could be abused (and probably would be by some) but it’s worth considering as an alternative to the current system. I would argue that the benefit in increased funding, and increased research freedom, is outweighed by the problem of some researchers doing nothing useful with the money. Remember that you’re not just giving 30 grand in “free money” to someone on the street. This is a highly-selected population that in general are motivated to use the grant for what it’s intended. The vast majority of scientists place a higher value on publications, good students and “research success” then they do on some highballs and a new laptop.

          • http://www.maple-leaf-forever.com Lord Bob

            Thanks for the well-informed answers, Ian. As has become obvious, I don’t really know that much about grant procedure, and for whatever reason I was under the impression that the money, once granted, would be administered by the researcher and not by the school.

            Obviously, having a faculty asking you why you felt the need to charge seventy rums-and-coke on Thursday is going to cut down on the worst excesses, except in cases where the faculty is corrupt or otherwise in the researcher’s corner. You agree that some abuse would happen, but at least it wouldn’t be as epidemic as I feared.

            With that said, I’m still worried about a) a potential chilling effect if a young researcher’s grant is condemned after the fact when normally it would have been snuffed in peer review, b) the accounting on this. Does the article account for the fact that, if you remove the barrier to entry on getting a grant, the number of grants could go way up?

          • Lord Kitchener’s Own

            Plus, i don’t think anyone’s suggesting that we eliminate applications ALL TOGETHER. It’s not REALLY as though you’d just show your diploma and get a briefcase full of cash. There’d at least be SOME sort of application, it just wouldn’t require you to hire additional staff to complete it (any idea how many departments have grant writing advisers, or how many professors hire private consultants to help them get through the grant process? A LOT).

            Presumably, there’d be an application with a line where you’d have to write “what I plan to research”, and, I’m guessing “How drunk I can get” wouldn’t get money. There’s surely some wiggle room between “Please spend tens of thousands of dollars preparing the grant application telling us why we should give you $40,000″ and “Gee, you have a pulse, here’s a sack of crisp new hundred dollar bills”.

            I’d be willing to bet that the amount of money wasted by the occasional nare-do-well would be infinitesimal in comparison to the amount of money currently wasted by researchers and the granting agencies as they slog through the process of creating, and then approving, the grant applications (which is the point of the article, I think). The idea is, the money hypothetically wasted by a couple dozen people having parties is nowhere near the amount of money currently being wasted by hundreds of researchers slogging through paperwork and red tape.

          • Ian

            Well, the primary barrier to entry is that you have to be faculty at an academic institution. The authors are suggesting that everyone get a baseline grant instead of only 70% of applicants. In 2007, for example, there were 3592 applicants and 2522 of those were successful. So they can’t really go way up. That said, probably “I didn’t get a grant” looks less bad on your CV than “I got a grant but didn’t accomplish anything with it”. So having the low-quality applications weeded out in peer-review may help there.

            The point the authors are trying to make is that with the current system it costs very roughly $40,000 for each of the unsuccessful applications (for a $30,000 grant). Thus a 100% success rate would save money. They have a lot of suggestions for alternative ways to structure the funding, and it’s oversimplifying to say their conclusion is to just hand money to everyone who asks. They (and I) are not advocating for dropping peer review. But, for small core-funding type grants like these, it seems to me that the waste of scientists pissing away the money doing useless, poorly-considered experiments is probably less than the overhead of the peer-review administration.

            However, note the comments from beentheredonethat and Bill Shipley below. They both clearly have a lot more experience with this than I do, are both are less than impressed with the paper’s suggestions.

  • http://www.obesitypanacea.blogspot.com Travis

    Keep in mind the paper does not say that we should dispense with the peer-review process on the whole, merely for “baseline” grants, which are quite small in the grand scheme of research funding. It’s also important to remember that in order to receive larger grants, pilot data can be incredibly helpful, but without any funding pilot data is very hard to collect. Thus giving every *qualified* applicant a baseline grant might also help researchers collect pilot data and decide whether or not it is worth applying for a larger grant in this same area of investigation, or whether the area should be abandoned.

    Some people will obviously milk such a system, but I would rather look at ways to limit people taking advantage of such a system rather than discounting the system outright.

    • beentheredonethat

      After having spent the better part of a decade as a PhD working as a granting council program oficer and middle manager, I am reminded of several things. It would take much too long to disucss them all and in detail but here’s a few head lines and sub tittles (based on actual comments received from univeristy researchers).

      Just give me a grant for 30 years (i.e., my whole career): the peers know nothing anyway.

      $30K will be enough to hire a student and take a trip to a conference: no research going on around here (unless one is in the humanities, which gets the best bang for the buck).

      Every professor hired by a university is a genius and needs no oversight (doesn’t explain all those crappy profs we all had, does it?)

      I CAN go on. However, I trust that my point is made. Let me ensure it is with a reductio argument: the newocon mantra is this: trust the market’s to regulate themselves. Well, we have seen what happens when this mantra is turned into policy: listerosis outbreak, markets collapse, banking systems in taters, etc. etc. etc. So, tell me now, why not just give researchers the $30K? Each and every one of them can regulate their own selves; It’s not like their peers know anything anyway.

      Sorry, in the world I prefer to live in, I want to be sure that the research funded (even at a mere $30K) is top of the line as judged by peers not by the applicants themselves. It’s not like the councils have all the money in the world to work with; it’s not like researchers are God; and, it’s not like all projects are deserving of funding.

      Moreover, I am also of the mind of “public purse, public purpose.” I am not sure I am suggesting following the lead of the Australian Research Council (all funding – period – is targeted, which, by what I have heard, was found acceptable by their research communities after an initial period of complaint) – I do believe that basic research in all disciplines is necessary.

      But I also believe that all researchers should at some point in their careers (i.e., at least once) give back to society and engage in research which is of public interest as opposed to the interest of 12 other researchers in the WHOLE world; we all know a few of these.

      I am not denigrating the important social function of teaching (even if many reseachers hardly engage in that activity much any more, leaving it to teaching assistants and contract professors). In fact, I would encourage more of it (but to get there certain policy changes need to be made like how CFI formulae are devised, how University Admininstrator are obsessede with seeking the grant buck, the hiring of MBAs to run universities, etc.)

      I am not advocating bureaucracy but rather that good research is funded and that good research is determiend to be good by peers – not by the applicants, not by the politicians, not by university administrators and not by public opinion. If this means that some money needs to be spent in order to ensure that the best is funded, so be it.

      One also needs to account for the cost savings of the peer-review process. Comittee members join together in a central location allowing for more than the peer-reivew work for which they are there. There are opportunties for network building, exchanges of idea, new teams can develop, new partnerships can emerge. If there were no in person peer-review opportunties, these unanticipated outcomes will be limited and the costs associated with learning about new activities and opportunties would soar.

      In the end, I guess that I am saying this is hardly a black and white issue. Sure I’d love to have $30K year to do my own thing, which is above and beyond my salary. But then so would all other members of our society. Do we realy want to start down this slope?

  • archangel

    It makes so much sense (caveats noted) that it can never happen. Now if some corporation could benefit without risk — well that’s different.

  • Sisyphus

    Could be a media make-work project. Consider the potential for faux outrage stories on the grants for study of the reproductive function of garden slugs.

    • Lord Kitchener’s Own

      Until, of course, some PhD student realizes that the enzymes used in the reproductive process of garden slugs might cure cancer!

      • Sisyphus

        I don’t think Kevin Gaudet cares.

  • Ian

    In not really related news, Gary Goodyear responds Nature (the journal, not the ecosystem):

    http://www.nature.com/nature/journal/v458/n7240/full/458830c.html

    • http://www.jackmitchell.ca Jack Mitchell

      Goodyear II: When Nature Calls

  • http://pages.usherbrooke.ca/jshipley/recherche/ Bill Shipley

    A couple of points from a university scientist who has been funded by NSERC for many years and who has also served on a peer grant review committee (committee 18 -ecology and evolution). First, it is not true that “sifting through grant applications in peer review to decide which are worthy” is a cost borne only by NSERC. These grant applications are read and evaluated by review committees made up of university researchers (8 when I served for my committee). This involves several hundred hours of work per person per year and the researchers on these committees are not paid a cent for this work. We do it because we know that to do otherwise would destroy the system as currently constructed.
    Having said that, I am not in favour of the proposal in the article. No one can predict, before the research is complete, if the result will produce a Nobel prize or not. However, we can make reasonable predictions of competence and creativity based on a researcher’s past research performance as judged from peer-reviewed publications and subsequent citation impact within the research area. A much simpler – and less expensive – system would be to provide grant amounts per researcher on a short (say, 3 year) time frame based on past performance without reviewing the research details that form most of the grant application. This would consist of (i) numbers of graduate students successfully produced, (ii) numbers of peer-reviewed publications and (iii) citation rates of the researcher’s papers. People who are good at converting grant money into productive and influential scientific publications will be well funded. People who cannot do this would not be funded. It would take about 5 minutes to judge this based on a standard formula.

  • http://www.umanitoba.ca/faculties/medicine/radiology/stafflist/rgordon.html Dick Gordon

    If you would like to read:

    Gordon, R. & B.J. Poulin (2009). Cost of the NSERC science grant peer review system exceeds the cost of giving every qualified researcher a baseline grant. Accountability in Research: Policies and Quality Assurance 16(1), 1-28.

    Poulin, B.J. & R. Gordon (2001). How to organize science funding: the new Canadian Institutes for Health Research (CIHR), an opportunity to vastly increase innovation. Canadian Public Policy 27(1), 95-112.

    Gordon, R. (1993). Grant agencies versus the search for truth. Accountability in Research: Policies and Quality Assurance 2(4), 297-301.

    let me know. -Dick Gordon, gordonr@cc.umanitoba.ca

  • Terry

    I hate bureaucracy, but I hate funding anything where people feel they have an automatic right to government money.

    So either way the issue works out is fine by me.

  • http://200blueprint.com/r/today200/freereport Stjepan

    Dear Sirs, dear friends!

    Allow to greet, and ask you, how are you? Do you still breathe O2 or GO2?

    Human species has no borders, in order to earn, to destroy nature, air, animals, plants …? Apprentice ..???
    Please, do not want to laugh at you in the face, but you are naive
    Monvi, MONY, MONY … that’s overture modern society, that all tread, in order to achieve profit. G-minor food is evil, that we destroy. In molecule G-minor, trainees can put all: Otrov to destroy the people, the state .. cancer, ADIS
    castration of men …
    Not enough water to drink, does not have enough oxygen and healthy
    nature, poisons near us, recession .. cause new divisions over the world and also the wars …
    Scientists and intelligent people do not say? Do not have the funds for a serious
    work and those who live in old glory, and without scruple, enjoy up to his death, and which will be after them, that they are interested … Thank you and regards

  • http://www.umanitoba.ca/faculties/medicine/radiology/stafflist/rgordon.html Dick Gordon

    It is interesting how much distrust of scientists is expressed in some of the above comments, by people who may have sat on the hiring, promotion and tenure committees of the people they are complaining about, and let them stay. If the deadwood hasn’t been screened out by these three processes, which are of much greater breadth, depth and duration than any grant committee has time for, then the problem lies deeper: in a way, universities have abrogated their responsibilities in selecting their own to the granting committees, which are not equipped to do that job.

    To me the primary issue is that of freedom and responsibility to explore versus control by peers and administrators. This control stifles innovation. $30,000/year is perhaps 30% of a university professor’s salary. In just about any other job in this world, the institution one works for is expected to provide the means to do the job. As professors we are told that teaching, service and research are what we’re judged on. But we must first beg for the funds to do our work. As we are our peers, this leads to a system of terribly uneven distribution of the available funds via simple positive feedback mechanisms. Baseline funding, utilizing perhaps 50% of the funding agencies’ money, would greatly enhance innovation. The other 50% can stand in the current system of peer review, allowing more ambitious projects to compete with one another, each perhaps well designed through the careful use of baseline funding.

    If you haven’t read:

    Gordon, R. & B.J. Poulin (2009). Cost of the NSERC science grant peer review system exceeds the cost of giving every qualified researcher a baseline grant. Accountability in Research: Policies and Quality Assurance 16(1), 1-28.

    ask for a reprint. -Dick Gordon, gordonr@cc.umanitoba.ca

  • http://www.umanitoba.ca/faculties/medicine/radiology/stafflist/rgordon.html Dick Gordon

    There two hypotheses, alternative to lax committees, that could be tested, about where “deadwood” comes from amongst science professors.

    First, it is possible that a string of rejections of grant applications kills the will to apply to grant agencies altogether. As salaries in Canadian universities are generally not dependent on whether or not one does research, some may lose the incentive, especially those that have achieved tenure. The danger in making this argument is, of course, that some will use it as an excuse to undermine the tenure system itself, which is the major support of academic freedom and independence of thought. In fact, baseline funding would be essentially an underpinning of the privilege of tenure with a little cash to be productive. Tenure is presently an underfunded institution.

    Second, there are others, especially in fields that don’t require expensive equipment and lab resources, who would rather do research than write grants. These professors are “deadwood” from the point of view of university administrators who value people in proportion to the amount of money they bring in. Any internal university public relations publication, with its dollar headlines, announces to all of us this predominant criterion by which we are judged.

    Those who value people according to their “success” in getting grants will not be swayed by these possibilities, because that is their one dimensional criterion for “excellence”.

  • http://www.umanitoba.ca/faculties/medicine/radiology/stafflist/rgordon.html Dick Gordon

    Typo:
    Thee are two hypotheses…

  • http://www.umanitoba.ca/faculties/medicine/radiology/stafflist/rgordon.html Dick Gordon

    I see it’s late and I’m tired:
    There are two hypotheses…

    • Paul Wells

      Or “Thou art two hypotheses…”

From Macleans