I suppose I need to begin this by declaring that I'm not a climate change nutjob... I am a mathematician by training who's had a great deal of fun using your recently released land surface dataset to reproduce some of your gridded data. If you are curious about what I did with your data, this YouTube video I made will take 9 minutes of your time and tell you about it:
A couple of questions have arisen, and I'm hopeful that you can answer them for me.
1. I've noticed that there seems to be a big difference between the 'Normals' given in many of the datafiles for Australisia and the actual normal values calculated from the 1961-1990 data. See for example, this blog of entry of mine about one of them:
Do you know of any issues with these values?
2. In following through the calculations in section 6.1 of the Brohan paper I'm surprised that the sampling error isn't much greater for the mid-1800s. In the mid-1800s there were less than 100 stations reporting temperature and yet if you look at Figure 11 (top) in the paper the green band is the 95% error value based on sampling error.
It appears to be of similar size no matter what year is being examined. Yet my own recalculation of the values (by sub-sampling the NCEP/NCAR dataset) show that the standard deviation in the anomalies is about 1 whereas for 2009 it's around 0.05. Intuitively this makes sense given that the number of stations increased greatly over the years. Do you know why this isn't reflected in the paper?
3. The sub-sampling of NCEP/NCAR also gives a mean value for the sampling error. The paper doesn't seem to talk about using that mean value to adjust the calculated values. Is that correct? It would seem to me that you'd want to use the mean to adjust for sampling error. If you do it reduces the temperatures up until the mid-1970s by around 0.5C and then has a small effect on the warming trend after that. Is it reasonable to use the mean from the sub-sampled data in that way?
Thank you for taking the time to answer me questions and Merry Christmas.
And the full reply:
Thank you for your interest in the data.
1. First off, thank you for bringing this to our attention. We have undertaken further investigation upon the full dataset and confirmed this. The error affects <1% of the network and is primarily in Oceania. It arises because normals were calculated outside of the update cycle and the normals for these stations were not updated when extra data were added in the normals period as CRUTEM3 was being finalised for publication.
We intend to add this information to our online Q and A for the data and we would like to credit you with pointing out the error. Would you be happy to be mentioned in this way?
2. Figure 11 shows annual hemispheric-averages combined land and sea-surface temperature data, which have been smoothed using a 21-point binomial filter. A large proportion of the coverage error (the green area on the diagram) arises because there are few observations over the Arctic and Antarctic. Temperatures in these areas exhibit high variability and therefore contribute significantly to the uncertainty on estimates of hemispheric and global average temperature. Because this is a large component of the uncertainty on global average temperatures and because coverage at high latitudes is still relatively poor, the uncertainty range is not much narrower now than it was in the 19th century.
3. We do not use the mean NCEP error to 'correct' the data because it is important that CRUTEM3 and HadCRUT3 provide an observational estimate of temperature change that is independent of all models. Furthermore, the mean error would only be representative of changes during the period covered by the reanalysis and not applicable outside this range.
Best regards and Merry Christmas,
I'm still not sure I 100% understand part 2 there, and I've sent a follow-up question to the Met Office. I'm guessing from their reply that the sub-sampling mentioned in the Brohan et al. paper isn't done in a simple fashion (as I had imagined) since they are specifically talking there about taking into account the variability at the high latitudes.
Will publish their response when I get it.