Global observations count¶

The total number of observations, in ISPD3.2.9, over the period jan 1st 1851 to Dec 31st 2000, is 1,133,185,756.

Code to count them¶

Download all the observations for the period:

# Retrieve ISPD3.2.9 observations from NERSC

import datetime
import IRData.twcr as twcr

for year in range(1851,2001):
    dte=datetime.datetime(year,1,1)
    twcr.fetch_observations(dte,version='2c')

The data files have 1 observation per line, so it’s fastest to count the lines

#!/usr/bin/env python

# Count the total number of observations, 1851-2000
#  Fastest way is to count the lines in the text files

import os
import sys

# How many lines in a file
def file_len(fname):
    with open(fname) as f:
        for i, l in enumerate(f):
            pass
    return i + 1

total=0
for year in range(1851,2001):
    ydir=("%s/20CR/version_2c/observations/%04d" %
                            (os.getenv('SCRATCH'),year))
    files=os.listdir(ydir)
    for file in files:
        total=total+file_len("%s/%s" % (ydir,file))
    print "%04d %11d" % (year,total)
    sys.stdout.flush()
print total

    

Note, the unix utility ‘wc’ would do this, except that ‘wc -l $SCRATCH/20CR/version_2c/observations//.txt’ wll fail, as there are too many files.

Counts up to the end of each year¶

Output of the script above.

Table of Contents

Get the source code

Found a bug, or have a suggestion?

Global observations count¶

Code to count them¶

Counts up to the end of each year¶