Wednesday, December 23, 2009

Phew! Got the limited coverage error sorted out

I had been worried about reproducing the error range shown in Brohan et al. section 6.1 because I was getting much larger error bars than in the paper.

But I wrote some tests and dug into the code and discovered a problem where my regridder/subsampler was ignoring some data because of a difference in the way longitude was reported between my gridding program and the NetCDF file from the reanalysis data.

Bottom line: I now get values much, much closer to the published information and all my worries about the temperature for 2009 appearing inside the error bars for the 1860s are gone.

Phew.

Here's the new chart showing the smoothed trend line with 95% confidence bars.


And here for comparison is the chart from the paper (mine is northern hemisphere and this is global but you can clearly see the similarity now).


Now, I can retire from amateur climatology... I've worked through Brohan et al. using the Met Office data, found a few problems, but mostly reproduced their results.

3 comments:

JZ said...

Why does the std error/dev compress going forward?

Is it a function of the number of observation stations, i.e. a form of the law of large numbers? - i.e. that std dev will compress the more samples you have?

This is never clearly explained but is visually represented over and over. (An amateur writes...)

Paul said...

Thank you for the nice piece of code! I was planning to do something similar. You saved me a lot of work.

I downloaded and ran the code. There is a small error on line 535. $n should be $m. At least then it worked. :)

John Graham-Cumming said...

@Paul

Sorry about that. Managed to commit a version half-way through refactoring something.

You are correct about that line being wrong (although you fix isn't the right one).

I've now committed the correct code.