Conclusions and recommendations

Data rescue is an area of UK strength, and offers an opportunity to make a big impact. It is so diverse in both sources and customers that it’s hard to choose priority sources - it does not really make sense to try and choose between ship and station data, for example. Even so, we can perhaps agree on some general principles and some short-term plans.

General principles

• Data rescue is a necessary part of Met Office activity, both as part of our responsibility as the public memory of the weather, and to support scientific and other projects.
• Long observational records are an area where the UK could be world-leading. We should aim not only to be seen to have the best historical observational records in the world, but also to encourage research on UK climate (as opposed to other regions).
• The long-term aim should be to digitise everything. Current customers are mostly dataset-builders, and there are already enough of those to generate a use-case for almost every observation. The customer requirements for digital data are likely to grow as data science and machine learning expand further.

• As there are already requirements for many different sets of observations, short term focus should be pragmatic:

  • What’s easy
  • What’s externally resourced
  • What’s at risk
  • What do we already have a customer and a database for.

• Data rescue is not a activity unique to the UK or the Met Office. We should engage more actively with European and international projects in archive work, data rescue methods, and distribution of digitised observations.

Short-term specific priorities

• We should complete the digitisation of the UK Daily Weather Reports. This data has a current major customer (20CR and the forthcoming Copernicus Reanalysis). It’s a large enough project to make a big scientific impact, and a small enough one to be completeable in a year or two. We’ve done a test period and shown the effect of the new observations, and we have an active partner in the University of Reading. This would not require heroic efforts to do, and could be a big success.
• We should collaborate actively with the new international in-situ observations dataset being developed by Copernicus C3S. If this works, it will provide a way of getting our observations widely used; if we put our data in it, any products derived from it will have the best possible representation over the UK. This would involve clarifying the relationship between MIDAS and the international datasets.
• We should provide images from our archives to the research community working on computer vision tools for data rescue. Let’s get the people working on this problem working on the documents we want digitised.
• We should continue to collaborate with the teams in the US and New Zealand working on effective ways to rescue ship data.