1. Overview

1.1. Purpose and organization of the User’s Guide

The goal of this User’s Guide is to provide basic information for users of the METdatadb database and the METviewer and METexpress display systems to enable users to create plots from their MET output statistics.

The METdatadb User’s Guide is organized as follows. Currently, the METdatadb User’s Guide only contains information about METdbload, a utility of METdatadb. Eventually, there will be utilities to add and delete databases, delete duplicate records, etc.

1.2. The Developmental Testbed Center (DTC)

METdatadb has been developed, and will be maintained and enhanced, by the Developmental Testbed Center (DTC; http://www.dtcenter.org/ ). The main goal of the DTC is to serve as a bridge between operations and research, to facilitate the activities of these two important components of the numerical weather prediction (NWP) community. The DTC provides an environment that is functionally equivalent to the operational environment in which the research community can test model enhancements; the operational community benefits from DTC testing and evaluation of models before new models are implemented operationally. METdatadb serves both the research and operational communities in this way - offering capabilities for researchers to test their own enhancements to models and providing a capability for the DTC to evaluate the strengths and weaknesses of advances in NWP prior to operational implementation.

METdatadb will also be available to DTC visitors and to the modeling community for testing and evaluation of new model capabilities, applications in new environments, and so on.

1.3. METdatadb goals and design philosophy

METdatadb is a Python rewrite of the capabilities in METviewer. METdbload, a METdatadb utility, reads MET verification statistics ASCII files, and loads them into a database for plotting with METviewer and METexpress. The specification for which files to load is written in XML.

The METdatadb code and documentation is maintained by the DTC in Boulder, Colorado. The MET package is freely available to the modeling, verification, and operational communities, including universities, governments, the private sector, and operational modeling and prediction centers.

1.4. METdatadb Requirements

METdatadb requires installation of Python 3.6+ and MySQL, MariaDB or AuroraDB.

1.5. METdatadb release notes

When applicable, release notes are followed by the GitHub issue number which describes the bugfix, enhancement, or new feature: METdatadb GitHub issues.

1.5.1. Version 1.0.0 release notes (2021-05-12)

1.5.1.1. Version 1.0.0 release notes (20210512)

New Functionality:

  • Read MET (.stat) files (#1)

  • Transform MET data for any database (#3)

  • Write MET (.stat) files to a SQL database (#6)

  • Read VSDB (.vsdb) files (#2)

  • Transform VSDB data for any database (#4)

  • Apply and Drop SQL Indexes (#8)

  • Implement date_list tag (#15)

  • Load MODE files (#22)

  • Load MTD files (#23)

  • Enhance METdatadb to load the TCST output line types from TC-Pairs. (#28)

  • Add new optional command line argument to change default location of temporary files (#30)

Enhancements:

  • Load new DMAP line type created by Grid-Stat in MET version 9.0. (#19)

  • Load new MTD columns for MET version 9.0. (#20)

  • Update METdb to handle changes in met-9.1. (#25)

  • Enhance METdbLoad to read new RPS line type from Ensemble-Stat and Point-Stat. (#26)

  • Add support for ANOM_CORR_RAW which is new in the CNT line type for met-9.1. (#29)

  • Add version number to usage (#41)

  • Create initial METdbload documentation (#46)

  • Enhance METdatadb to load the additional CRPS columns in the ECNT line type updated for met-10.0.0. (#48)

  • Enhance METdatadb to load the additional climatology column in the ORANK line type updated for met-10.0.0. (#49)

Bugfixes:

  • Fix bug - handle non-existent directory with templates (#12)

  • Error loading ECLV(ECON) line type from VSDB files (#32)

Internal:

  • Put flags from XML load_spec into a dictionary (#5)

  • Handle transformations related to PSTD, RHIST, ECNT, and ORANK (#7)

  • Handle variable length line types in stat records (#9)

  • Speed up reading files (#13)

  • Check to make sure XML file exists (#18)

1.6. Future development plans

METdatadb is an evolving verification database package. New capabilities are planned in controlled, successive version releases. Bug fixes and user-identified problems will be addressed as they are found. Plans are also in place to incorporate many new capabilities and options in future releases of METdatadb. Additionally, updates to accommodate new features of the MET software are often required.

1.7. Code support

Support for METdatadb is provided through the METplus GitHub Discussions Forum. We will endeavor to respond to requests for help in a timely fashion.

We welcome comments and suggestions for improvements to METdatadb, especially information regarding errors. In addition, comments on this document would be greatly appreciated. While we cannot promise to incorporate all suggested changes, we will certainly take all suggestions into consideration.

METdatadb is a “living” application. Our goal is to continually enhance it and add to its capabilities.