Archives for posts with tag: ds570.0

Since the last post I mostly worked on processing the ds570.0 dataset. I implement a parser in Java to feed the ClimateSOS ( via the transactional profile – meaning I user the SOS operations RegisterSensor and InsertObservation via http POST to insert data. I base the code on the transactional feeder from the OX-Framework, and will also make the programme available for download here (see below).

The data insertion into the SOS takes me longer than planned, but for good reasons. First, I took some time to think about how to model the data that I have in O&M. I have to be able to parse the observation markup in R later and cannot prepare for all eventualities that are possible in O&M. Therefore I outlined an idea for an O&M profile for simple measurements (I will write about that soon). Second, I also spent some time to think about how to model the data in R once I requested and parsed it – luckily I am not the only one needing such a feature – see the sptX project.

So, a lot of things happening “on the side”, that do contribute though not directly.

But back to the parser: Read the rest of this entry »

I mentioned the decision process for the dataset ds570.0 in a previous post. Today I would like to introduce the dataset a little further. I also made some first steps with the package lattice for plotting, and will show the results of that. From the dataset website:

World Monthly Surface Station Climatology, 1738-cont

This world monthly surface station climatology has data for over 4700 different stations (2600 in more recent years). Data for some stations goes as far back as the mid-1700’s. See decedal coverage for more detail. Most of the data was obtained directly from the National Climatic Data Center (NCDC), Asheville, North Carolina. However, much of the data prior to 1951 came from John Wolbach of Harvard College Observatory, who contracted to have this data key entered at NCDC. The first six months of 1961 were key entered at NCAR. Sharon Nicholson, Florida State University, provided African precipitation data to extend the records of over 250 stations. Dennis Shea, NCAR/CGD, has been a valuable source for data obtained directly from various countries.

Check out the detailed information if you want to know more. I took the following graphic from that page, showing the land stations’ positions (also available as a KML file):
Read the rest of this entry »

The ClimateSOS will use offerings to give the data some structure, and a regional subdivision seems like the natural choice. But as the subset format of ds570.0 does not contain this kind of information, I had to look elsewhere. Luckily, the dataset uses WMO Identifiers (World Meteorological Organization) for the included stations. And the WMO’s Volume A dataset contains exactly the fields that I needed, namely

But it wasn’t that simple. Read the rest of this entry »