What this is, and why

This is a detailed review of a variety of methods that have been used for historical weather data rescue, an overview of the lessons learned from each method, and some suggestions for future projects.

Such data rescue is now a well established activity, with a good selection of reports and papers offering justifications and advice. Rather than just adding to this collection, here we are providing a much more detailed description of a wide-ranging set of sample data rescue projects: In each case aiming to describe exactly what was done - in enough detail to let others reproduce the work, or to use these projects as the basis for a new one. In addition to the detailed project descriptions, we add a brief discussion of the lessons learned from each project, and an overview of the costs, strengths, and weaknesses of each approach.

This is not a paper, or a report. Instead it borrows a standard approach from the modern software industry. This is a collection of GitHub repositories. Each repository is a comprehensive collection of the data, software, and results of one example data-rescue project; some of them also contain an associated web site (made with GitHub pages). The repositories included are:

• Weather observations from marine expeditions: Repository, Web site.
• New UK-region weather observations digitised from the Daily Weather Reports: Repository, Web site.
• Royal Navy observations for the WW2 period: Repository.
• Weather observations from English East India Company ships: Repository.
• oldWeather.org phases 1&2: Repository.
• oldWeather.org phase 3: Repository.
• Auto-transcription benchmark 1: Fort William pressures: Repository, Web site.

As well as these project repositories, there is one more containing the overview, the web site of which is what you are currently reading.

A virtue of this approach is that it contains vastly more information than any possible paper document; this should make it much more useful to anyone setting up a new data rescue project. It is far from comprehensive - there are many other projects out there not included - but it does cover a wide range of approaches.

This is version 1.0.0, dated at 2018-10-02. If you have found a problem with it, or want to make a comment, please raise an issue.