Articles | Volume 3, issue 2
Geosci. Instrum. Method. Data Syst., 3, 187–200, 2014
Geosci. Instrum. Method. Data Syst., 3, 187–200, 2014

Research article 25 Sep 2014

Research article | 25 Sep 2014

A framework for benchmarking of homogenisation algorithm performance on the global scale

K. Willett1, C. Williams2, I. T. Jolliffe3, R. Lund4, L. V. Alexander5, S. Brönnimann6, L. A. Vincent7, S. Easterbrook8, V. K. C. Venema9, D. Berry10, R. E. Warren11, G. Lopardo12, R. Auchmann6, E. Aguilar13, M. J. Menne2, C. Gallagher4, Z. Hausfather14, T. Thorarinsdottir15, and P. W. Thorne16 K. Willett et al.
  • 1Met Office Hadley Centre, FitzRoy Road, Exeter, UK
  • 2National Climatic Data Center, Ashville, NC, USA
  • 3Exeter Climate Systems, University of Exeter, Exeter, UK
  • 4Department of Mathematical Sciences, Clemson University, Clemson, SC, USA
  • 5ARC Centre of Excellence for Climate System Science and Climate Change Research Centre, University of New South Wales, Sydney, Australia
  • 6Oeschger Center for Climate Change Research & Institute of Geography, University of Bern, Bern, Switzerland
  • 7Climate Research Division, Science and Technology Branch, Environment Canada, Toronto, Canada
  • 8Department of Computer Science, University of Toronto, Toronto, Canada
  • 9Meteorologisches Institut, University of Bonn, Bonn, Germany
  • 10National Oceanography Centre, Southampton, UK
  • 11College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, UK
  • 12Istituto Nazionale di Ricerca Metrologica (INRiM), Torino, Italy
  • 13Centre for Climate Change, Universitat Rovira i Virgili, Tarragona, Spain
  • 14Berkeley Earth, Berkeley, CA, USA
  • 15Norwegian Computing Center, Oslo, Norway
  • 16Nansen Environmental and Remote Sensing Center, Bergen, Norway

Abstract. The International Surface Temperature Initiative (ISTI) is striving towards substantively improving our ability to robustly understand historical land surface air temperature change at all scales. A key recently completed first step has been collating all available records into a comprehensive open access, traceable and version-controlled databank. The crucial next step is to maximise the value of the collated data through a robust international framework of benchmarking and assessment for product intercomparison and uncertainty estimation. We focus on uncertainties arising from the presence of inhomogeneities in monthly mean land surface temperature data and the varied methodological choices made by various groups in building homogeneous temperature products. The central facet of the benchmarking process is the creation of global-scale synthetic analogues to the real-world database where both the "true" series and inhomogeneities are known (a luxury the real-world data do not afford us). Hence, algorithmic strengths and weaknesses can be meaningfully quantified and conditional inferences made about the real-world climate system. Here we discuss the necessary framework for developing an international homogenisation benchmarking system on the global scale for monthly mean temperatures. The value of this framework is critically dependent upon the number of groups taking part and so we strongly advocate involvement in the benchmarking exercise from as many data analyst groups as possible to make the best use of this substantial effort.