METEO(1)

NAME
meteo - introduction to meteo station tools

DESCRIPTION
The meteo package contains programs that allow to retrieve data from a Davis Instruments Weather Monitor II or a Vantage Pro station, store the data in a MySQL database and extract suitable averages to create graphs to be embedded in Web pages. Other Davis Instruments stations may work as well, but have not been tested due to the lack of access to such hardware. The documentation to the meteo package consists of several distinct programs which have their own manual pages. This page serves to explain the overall architecture and common prerequisites for the individual programs. All programs use a common configuration file which defaults to ${prefix}/etc/meteo.xml, or can be specified with the -f command line switch, the syntax of the configuration file is explained in meteo.xml(5).

ARCHITECTURE
The minimum installation for the meteo tools uses the two programs meteopoll(1), meteoavg(1), and meteodraw(1). meteopoll(1) connects to the weather station either using a serial interface or a tcp connection to a terminal server or a serial device server like the Lantronix UDS-10. It speaks the protocol of the weather station to collect samples from the station at a reasonable rate, usually in one second increments. These samples are then averaged (or summed, e.g. for the rain measurements), and SQL update queries prepared to be sent to a MySQL database. The database schema uses two tables, stationdata and averages. The stationdata contains the averaged data points for every minute, which for all postprocessing purposes are considered to be raw data. It also contains some fields about the 5min, 30min, 2h or 1day interval the data point belongs to, to simplify the computation of averages for the averages table or to produce graphs. Of course, this is a partial denormalization. The averages table contains averages (or sums, or integrals) over intervals of 5 minutes, 30 minutes, 2 hours and a full day. The idea is to speed up selection of such averages to produce graphs that span a day (400 intervals of length 5 min), a week (400 intervals of length 30 min), a month (400 intervals of length 2h) or a year (400 intervals of length 1 day). The averages table is filled by the meteoavg(1) program. The data can be visualized by the meteodraw(1) program. Given an interval and a configuration file, it computes all graphs for that interval that end at the current point in time. It should be run from a cron job to keep up to date graphs available. A sample cron job script is meteojob(1), as included in the distribution. The meteopoll(1) program can also maintain a map file, which contains the current data in XML format. Since its contents may change at any time, an atomic read shoulc be used to read and display this data.

UNIFORM RESOURCE LOCATORS
The meteo packages uses new kinds of URLs to specify meteo stations and log destinations. Please see the meteo.xml(5) man page for meteo station URLs. Log URLS have one of two forms. If the log should be written to a file logfile, use a URL of the form file://logfile. Note that in most cases, logfile must be absolute. One special case is the URL file:///-, which represents standard error. If messages should be sent to a syslog(3) facility f, use an url of the form syslog:f.

TIME LABELS
The browser uses so called time labels to specify the resolution and the point in time for which a graph should be produced. Time labels try to be human readable. They always start with a single character indicating the resolution. The possible formats are:
label | interval | strftime format string
hYYYYmmddHH | 60 | h%Y%m%d%H
dYYYYmmdd | 300 | d%Y%m%d
wYYYYww | 1800 | w%Y%V
mYYYYmm | 7200 | m%Y%m
yYYYY | 86400 | y%Y
Note that the ISO week number is used, for more information about the strftime format strings used see strftime(3).

PERFORMANCE ISSUES
There are two areas where performance can become a problem. In both cases, the database is the bottleneck. Especially producing graphs spanning a whole year (which means computing averages over more than half a million 1min intervals) can be very time consuming. As most averages would be computed over and over again without any change, storing them in a special table, the averages table, is beneficial even for very small data sets. On the development system (a 266MHz Pentium MMX Notebook running Linux 2.4.10), producing the 1 day graphs from the averages table takes about 1 second, while it takes more than two seconds when recomputing the averages. The difference becomes even more notable for graphs spanning larger time spans: producing these graphs from the averages table does not take any more time, while computing the averages for a 1 year graph can take minutes. To maintain the averages table, the meteoavg(1) program should be run in daemon mode. In Version 0.8.0, the possibility to draw graphs directly from the stationdata table has been removed except for the 60 second resolution graphs. During the computation of averages, the stationdata table may be locked, preventing the meteopoll(1) program from updating the database. As it will block in the database call, some data from the weather station may be lost. To help decouple the weather station from the database, the meteodequeue(1) program has been added. It reads queries sent to a message queue, and forwards them to the database. The meteopoll(1) program accepts the -q option which directs it to send its SQL queries not to the database but the a message queue (the details of the message queue are configured in the configuration file).

SEE ALSO
meteo.xml(5), meteopoll(1), meteoavg(1), meteodraw(1), meteodequeue(1), meteojob(1)

AUTHOR
Dr. Andreas Mueller <afm@othello.ch>