Hacker reveals climatic research fraud

Politics and religion
41

From Revkin’s blog at the NYT:

http://dotearth.blogs.nytimes.com/2009/11/23/your-dot-concerns-about-climate-files/

  1. What appears to be a conspiracy to organize the deletion of e-mails in contravention of Freedom of Information laws. Do you reject this type of action? “Mike, Can you delete any e-mails you may have had with Keith re AR4? Keith will do likewise. He’s not in at the moment – minor family crisis. Can you also email Gene and get him to do the same? I don’t have his new e-mail address. We will be getting Caspar to do likewise.”

  2. What appears to be collusion to “oust” journal editors who decide to publish articles your associates disagree with. Is this a course of action that you believe is reasonable? “Proving bad behavior here is very difficult. If you think that Saiers is in the greenhouse skeptics camp, then, if we can find documentary evidence of this, we could go through official AGU channels to get him ousted.”

  3. What appears to be cooperation to modify data sets and papers to achieve defined goals rather than clear presentation. Is there a defense of these actions? “I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (i.e. from 1981 onwards) and from 1961 for Keith’s to hide the decline.”

  4. What appears to be attempts to keep peer-reviewed, scientific papers from being considered in the I.P.C.C. process. Do you think that lead authors in the I.P.C.C. process should be the ones to judge their own work? “I can’t see either of these papers being in the next I.P.C.C. report. Kevin and I will keep them out somehow - even if we have to redefine what the peer-review literature is!”

From Monbiot’s blog at the Guardian:

It’s no use pretending this isn’t a major blow. The emails extracted by a hacker from the climatic research unit at the University of East Anglia could scarcely be more damaging. I am now convinced that they are genuine, and I’m dismayed and deeply shaken by them.

Yes, the messages were obtained illegally. Yes, all of us say things in emails that would be excruciating if made public. Yes, some of the comments have been taken out of context. But there are some messages that require no spin to make them look bad. There appears to be evidence here of attempts to prevent scientific data from being released, and even to destroy material that was subject to a freedom of information request.

Worse still, some of the emails suggest efforts to prevent the publication of work by climate sceptics, or to keep it out of a report by the Intergovernmental Panel on Climate Change. I believe that the head of the unit, Phil Jones, should now resign. Some of the data discussed in the emails should be re-analysed.

Neither of these is to suggest global warming is just a big scam, there is a great deal to be concerned about, and most responsible “skeptics” will readily agree. However there is plenty here to suggest what many have long suspected, that many of the most prominent names in the climatology field are engaging in advocacy, at the expense if not the exclusion of actual science. This is a very bad thing.

42

There’s a good overview on Salon that explains some of what happened here for those interested in context:

Bluntly put, the climate scientists who have devoted their careers to proving global warming is happening do not believe that Steve McIntyre is a legitimate scientist whose real goal is the advancement of climate change science. They believe his primary goal is to undermine their work by any means necessary, and that any data they give him will be misused, abused and ultimately become political fodder for the conservative forces who are fighting any efforts to do anything about climate change.

You can’t put it more clearly than does Thomas Karl, the director of the National Oceanic and Atmospheric Administration’s National Climatic Data Center, in one of the hacked e-mails.

[quote]We should be able to conduct our scientific research without constant fear of an “audit” by Steven McIntyre; without having to weigh every word we write in every email we send to our scientific colleagues. In my opinion, Steven McIntyre is the self-appointed Joe McCarthy of climate science. I am unwilling to submit to this McCarthy-style investigation of my scientific research. As you know, I have refused to send McIntyre the “derived” model data he requests, since all of the primary model data necessary to replicate our results are freely available to him. I will continue to refuse such data requests in the future. Nor will I provide McIntyre with computer programs, email correspondence, etc. I feel very strongly about these issues. We should not be coerced by the scientific equivalent of a playground bully. I will be consulting LLNL’s Legal Affairs Office in order to determine how the DOE and LLNL should respond to any FOI requests that we receive from McIntyre.

From the perspective of the climate scientists involved, it seems clear that they did not trust McIntyre, did not feel that his FOI requests were legitimate scientific inquiry, and were determined to do whatever possible to resist him.

Does that exculpate them? Absolutely not. Does it explain why Phil Jones thought that private e-mails from climate researchers discussing the Fourth Assessment Report of the IPCC should be deleted? Nope, not at all. Does it demonstrate that scientific progress, despite supposedly being based on the accumulation of data and the testing of theories, can be a messy, messy business, full of personal intrigue and antipathies? Absolutely yes.[/quote]

And, the previous day, this information:

Here’s an e-mail that has gotten particular attention, with the supposedly damning language bolded:

[quote]Dear Ray, Mike and Malcolm,

Once Tim's got a diagram here we'll send that either later today or first thing tomorrow.

I've just completed Mike's Nature [b]trick[/b] of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) amd from 1961 for Keith's to [b]hide the decline[/b]. Mike's series got the annual land and marine values while the other two got April-Sept for NH land N of 20N. The latter two are real for 1999, while the estimate for 1999 for NH combined is +0.44C wrt 61-90. The Global estimate for 1999 with data through Oct is +0.35C cf. 0.57 for 1998.

Thanks for the comments, Ray.

Cheers, Phil

Here’s RealClimate’s explanation:

The paper in question is the Mann, Bradley and Hughes (1998) Nature paper on the original multiproxy temperature reconstruction, and the “trick” is just to plot the instrumental records along with reconstruction so that the context of the recent warming is clear. Scientists often use the term “trick” to refer to a “a good way to deal with a problem”, rather than something that is “secret”, and so there is nothing problematic in this at all. As for the “decline,” it is well known that Keith Briffa’s maximum latewood tree ring density proxy diverges from the temperature records after 1960 (this is more commonly known as the “divergence problem” – see e.g. the recent discussion in this paper) and has been discussed in the literature since Briffa et al in Nature in 1998 (Nature, 391, 678-682). Those authors have always recommend not using the post 1960 part of their reconstruction, and so while “hiding” is probably a poor choice of words (since it is “hidden” in plain sight), not using the data in the plot is completely appropriate, as is further research to understand why this happens.

So what’s going on here? Put aside the question of whether the words “trick” or “hide” have nefarious or innocuous meanings. The scientific problem is that in attempting to reconstruct temperatures in the past, climate scientists are often faced with the problem that there were no humans standing around holding thermometers and writing down temperatures. So scientists use “proxies” – tree rings, or ice cores, or fossilized clams, or lake pollen trapped in sediment. The “divergence problem” referred to above references a case where in one particular instance, tree ring variations in density did not match actual recorded temperatures after 1960.[/quote]

43

Remember that this is an English university, and their FOIA stuff works differently.

None of the other emails seem at all sinister to me.

44

If those emails ended up at a @nasa.gov address, they are subject to FOIA. If the NASA scientist involved in this is actually a university employee then they would not be subject to FOIA. The only university employees that have both an @nasa.gov address and are not subject to FOIA are JPL employees, and this doesn’t appear to involve JPL.

45

It’s not an American university!

46

And? The UK has its own FOIA. One that the director of the Climate Research Unit of the university in question made clear he would do everything he could to avoid complying with. Try to keep an open mind, and read this:

http://blogs.news.com.au/heraldsun/andrewbolt/index.php/heraldsun/comments/the_warmist_conspiracy_tthe_emails_that_really_damn_professor_jones#63704

There is tons of damning stuff there. Tons.

47

I dunno about that. I see a guy with a clear agenda harping on about “alarmist” global warming data and praising McIntyre to high heaven, someone who seems to be far more about ideological wrangling than scientific inquiry if the head of the NOAA’s Climatic Data Center is to be believed.

Seems like it’s more dancing around the totem pole and waving the magic wand of wishful thinking on the right’s part.

48

Then ignore him but note the emails quoted and the Monbiot article I quoted earlier. George_Monbiot has been among the most stalwart defenders of the global warming orthodoxy for years, regularly mocking “denialists” in the Guardian for years. He’s calling for Dr. Jones’s head and says the emails could “could scarcely be more damaging”.

Some of Monbiot’s responses to comments to his post:

Monbiot
23 Nov 2009, 9:18PM
Contributor Sabraguy:

But now I suggest you review your file of correspondence and articles, and figure out who you need to apologize to.

I apologise. I was too trusting of some of those who provided the evidence I championed. I would have been a better journalist if I had investigated their claims more closely.

Monbiot
23 Nov 2009, 9:21PM
Contributor contrarian2:

But if the science is that “settled,” why refuse to disclose the data? If global warming so obvious and incontrovertible, why be in such a panic about FOI, why talk openly about re-defining “peer review”, why threaten to (or actually) delete data?

I agree. It is exactly for those reasons that Phil Jones should resign. There’s a word for his lack of openness and control of the data: unscientific.

It’s staggering, just staggering to read comments like that from Monbiot. To say that there is nothing here is just ignorant. It’s unethical, it’s horribly damaging to science in general, and it plays right into the hands of the wingnut faction of the Right. None of this is good at all, even to those of us skeptical of “the science is settled” blanket statements.

49

And if you’re read my previous postings you’d know the answers to Monbiot’s questions.

50

The coverup is always worse than the crime.

51

Our resident wingnut already asked me if I changed my stance on Climate Change. I told him it would take a few out-of-context emails to change the opinion I made while reading dozens of scientific studies.

52

Monbiot didn’t ask any questions. Did you read my previous postings?

53

Ease up on the frothing there.
George believes the science. I trust scientists when they tell me how electricity works, how protein works, how vitamins work, and how I should exercise and eat a balanced diet to stay fit.
I don’t spend my life calling them 'defenders of the vitamin orthodoxy", because to get so agitated about scientific theory and evidence is just silly.

I don’t see why people expect scientists to work to a different standard to every other group on people on earth. They can be sloppy, bitchy, argumentative, confrontational and bitter, just like anyone else. A few emails from a very few climate scientists does not refute the overwhelming evidence pointing to man made climate change.

But hey, I’ll keep buying shares in renewable energy, and you keep buying shares in Exxon Mobil. lets see who wins.

54

Goldman Sachs?

55

zing!

56

These seem like questions to me.

57

That was from a reader’s comment to his article, Monbiot’s response was as noted above:

58

I think if Jones violated any laws he should resign too at the very least. On that point I agree. However it does help to understand the siege mentality that led to his bad behavior.

59

Hardware guys get their knickers in a twist when software guys try to do hardware. Software guys pull their hair out when hardware guys try and do software. And both weep bitterly into their beer when scientists try to do either.

The emails are probably less interesting than the source code

As the leaked messages, and especially the HARRY_READ_ME.txt file, found their way around technical circles, two things happened: first, programmers unaffiliated with East Anglia started taking a close look at the quality of the CRU’s code, and second, they began to feel sympathetic for anyone who had to spend three years (including working weekends) trying to make sense of code that appeared to be undocumented and buggy, while representing the core of CRU’s climate model.

One programmer highlighted the error of relying on computer code that, if it generates an error message, continues as if nothing untoward ever occurred. Another debugged the code by pointing out why the output of a calculation that should always generate a positive number was incorrectly generating a negative one. A third concluded: “I feel for this guy. He’s obviously spent years trying to get data from undocumented and completely messy sources.”

Programmer-written comments inserted into CRU’s Fortran code have drawn fire as well. The file briffa_sep98_d.pro says: “Apply a VERY ARTIFICAL correction for decline!!” and “APPLY ARTIFICIAL CORRECTION.” Another, quantify_tsdcal.pro, says: “Low pass filtering at century and longer time scales never gets rid of the trend - so eventually I start to scale down the 120-yr low pass time series to mimic the effect of removing/adding longer time scales!”

If the fate of human civilization is at stake, you’d think we’d have some unit testing in our numerical frameworks. Somebody should put these guys on the case (heck, their software is getting retired, finally). What’s $50 million a year in software engineering costs when you’re trying to model outcomes for trillion dollar global economic policy decisions?

[quote]

60

Could not agree more. Spend more money on the models before making huge economic decisions. PS: I don’t know if it is the actual people, but rather the process and the willingness to pay the extra money to support the process. I think it is a worthwhile effort to have the G20 pool a nice pile of cash and set up a large combined research effort. And when I say a large pile of cash I mean many billions not some chump change a few million here and there.