Differences
This shows you the differences between two versions of the page.
wdb:manuals:wdb_call_interface [2012-04-04 12:56:38] michaeloa [WDB Call Interface User Manual] |
wdb:manuals:wdb_call_interface [2022-05-31 09:29:32] |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== WDB Call Interface User Manual ====== | ||
- | |||
- | **Abstract** | ||
- | |||
- | WDB is a data storage solution for weather and water data based on the PostgreSQL object-relational database system. The system utilizes PostGIS for GIS (Geographic Information Systems) support and handles regular grids (e.g., forecast fields) and point (e.g., observation) data. | ||
- | |||
- | This is the User's Manual for WDB Call Interface. | ||
- | ===== Chapter�1.�Introduction ===== | ||
- | |||
- | **Table of Contents** | ||
- | |||
- | [[# | ||
- | |||
- | The WDB Call Interface (WCI) is the official API used for retrieving data from the WDB system. | ||
- | |||
- | ===== About this Manual ===== | ||
- | |||
- | This manual is intended for system designers, application developers and programmers. It describes in technical terms how weather data can be retrieved from the WDB system through the WCI. It is assumed that the reader has some familiarity with databases and weather data. | ||
- | |||
- | For a more elaborate description of the vision and system architecture of the WDB system, see the WDB Developer' | ||
- | |||
- | ===== Intended Usage ===== | ||
- | |||
- | WCI is intended to be used for retrieving weather data (e.g., temperature, | ||
- | |||
- | ===== Outline ===== | ||
- | |||
- | This document is structured as follows: | ||
- | |||
- | * Key Concepts: introduction to the concepts and terms used in the documentation | ||
- | * Using the WCI: an introduction to how the WCI is used | ||
- | * Function Reference: reference documentation for each of the WCI functions | ||
- | * Data Types Reference: reference documentation for the WCI data types | ||
- | * Examples: Example programs for WCI | ||
- | * FAQ: Frequently asked questions | ||
- | * Known bugs and limitations | ||
- | |||
- | ===== Chapter�2.�Key Concepts ===== | ||
- | |||
- | **Table of Contents** | ||
- | |||
- | [[# | ||
- | |||
- | This section provides an overview of the WCI, including some important definitions and explanations. | ||
- | |||
- | ===== Overview ===== | ||
- | |||
- | WCI is an API designed to retrieve weather data from the WDB system. The WCI API resides on the database server. | ||
- | |||
- | All function calls on WCI are executed as SQL statements. The SQL statements can be executed through a dedicated SQL client application, | ||
- | |||
- | A data item in WDB could be an observation, | ||
- | |||
- | A dimension could be a description of the data item (e.g., temperature), | ||
- | |||
- | In order to access the data items in WDB, the user should have an understanding of the dimensions as they are the entry point into the database. When data is retrieved from the database, a set of search criteria is given for each of the WCI dimensions. If the search criteria for a dimension is not specified (i.e., all NULLs), it is assumed the user wants to retrieve all data items, regardless of their dimension description. Please do not attempt this unless the database you are querying is extremely small. | ||
- | |||
- | In the context of WCI search criteria, NULL is always interpreted to mean " | ||
- | |||
- | The last parameter of a WCI function call that is intended to return data is always reserved for specifying the data type to be returned (this is because most functions are overloaded to return many different forms of data. By convention, this parameter is usually given as a NULL cast to the relevant data type; for example: | ||
- | |||
- | NULL:: | ||
- | |||
- | The dimensions are described in the following sections. | ||
- | |||
- | ===== Data Provider ===== | ||
- | |||
- | The Data Provider identifies the source of the data; literally, //the entity that provides the data//. Where multiple sources could be identified as the source of the data, the entity that can be identified closest to the creation of the data item at the time of loading is usually used. | ||
- | |||
- | A data provider can be a software process (e.g., Hirlam), a meteorological or climate station, an aircraft, or a person. A data provider is identified by a DataProviderName. DataProviderNames are used to search for the data. For convenience, | ||
- | |||
- | When searching for data, the user may specify a single DataProviderName, | ||
- | |||
- | DataProviderNames exist within a data provider namespace. A namespace can be defined by the WDB administrator, | ||
- | |||
- | To retrieve all of the DataProviderNames (excluding data provider groups) that are currently stored in the database for the currently specified namespace, the following wci.browse function call could be used: | ||
- | |||
- | SELECT * FROM wci.browse( NULL:: | ||
- | |||
- | To retrieve all of the DataProviderNames (including data provider groups) that the database currently has the capacity to store and display in the currently specified namespace, the following wci.info function call could be used: | ||
- | |||
- | SELECT * FROM wci.getDataProvider( NULL ); | ||
- | |||
- | ===== Place (Geographic Location) ===== | ||
- | |||
- | The place (geographic location) of a data item is the position of the item on the earth in a 2D space. In WDB, the geographic location is by default specified using longitude and latitude in a WGS84 coordinate system (though this can be changed when the database is set up; consult your system administrator). The geographic dimension is specified using a geometry object and can be either a point or a polygon. | ||
- | |||
- | In addition to using geometry objects to retrieve data, the user can also use a PlaceName - a pre-specified name that defines a geometry object in the database - to specify location. | ||
- | |||
- | As for DataProviderName, | ||
- | |||
- | To retrieve the full list of the PlaceNames for the data values currently being stored in the database, use the following wci.browse function call: | ||
- | |||
- | SELECT * FROM wci.browse( NULL:: | ||
- | |||
- | At the same time as the user specifies a location to retrieve data from, it is also possible to specify an interpolation option for location calculations. These permit the user a great deal of flexibility in searching. Currently, interpolations such as " | ||
- | |||
- | The form of place specification then becomes one of the two following: | ||
- | |||
- | ' | ||
- | |||
- | ' | ||
- | |||
- | ' | ||
- | |||
- | ' | ||
- | |||
- | Specifying NULL for the place definition of a query is logically equivalent to asking for all of the data items in the database, irrespective of their location. | ||
- | |||
- | To retrieve all of the place names that are currently registered in the database, the following WCI function call can be used: | ||
- | |||
- | SELECT * FROM wci.getPlaceDefinition( NULL ); | ||
- | |||
- | ===== Reference Time ===== | ||
- | |||
- | The reference time of a data item is the moment when the data item is referenced from. For forecast data, this would typically be the reference time of the data values the forecast is based upon; for observation data, it would typically be the time when the observation data was recorded. | ||
- | |||
- | When searching for data, the user specifies the reference time (with time zone), or NULL. NULL indicates that the user wants all data items, regardless of the reference time of the data. Time is specified in WCI as a text string with datetime' | ||
- | |||
- | To retrieve a list of the reference times currently being stored in the database, use the wci.browse functionality: | ||
- | |||
- | SELECT * FROM wci.browse( NULL:: | ||
- | |||
- | In addition to time points, it is also possible for the user to specify time periods using a variety of keywords. The keywords are: exact (default), before, after, inside, and contains. And interval can be specified using two timestamps and separating them with the keyword TO, or a timestamp and a duration separated by the keyword FOR. As examples, consider: | ||
- | |||
- | exact 2009-03-31T11: | ||
- | |||
- | Denotes the specific time point of 11:20 UTC, on the 31st of March, 2009. The keyword " | ||
- | |||
- | before 2009-03-31T11: | ||
- | |||
- | Denotes that the user wants to return all data for the time period prior to the time point of 11:20 UTC, on the 31st of March, 2009. | ||
- | |||
- | after 2009-03-31T11: | ||
- | |||
- | Denotes that the user wants to return all data for the time period after the time point of 11:20 UTC, on the 31st of March, 2009. | ||
- | |||
- | inside 2009-03-31T11: | ||
- | |||
- | Denotes that the user wants to return all data inside the time period between the time point of 11:20 UTC, on the 31st of March, 2009 and the same time two months later. | ||
- | |||
- | inside 2009-03-31T11: | ||
- | |||
- | Denotes that the user wants to return all data inside the time period between the time point of 11:20 UTC, on the 31st of March, 2009 and the same time two months later. | ||
- | |||
- | ===== Valid Time ===== | ||
- | |||
- | The valid time of a data item is the time period for which the data item is valid. The valid time is always stored in the database as a time interval. | ||
- | |||
- | When searching for data, the user specifies the valid time (with time zone), or NULL. NULL indicates that the user wants all data items, regardless of the valid time of the data. Time is specified in WCI as a text string with datetime' | ||
- | |||
- | As usual, the list of valid times stored in the database can be retrieved using the wci.browse functionality. | ||
- | |||
- | SELECT * FROM wci.browse( NULL:: | ||
- | |||
- | Time is specified in the same way for time periods as for time points. Examples of how the qualifiers should be interpreted are given below: | ||
- | |||
- | exact 2009-03-31T11: | ||
- | |||
- | The exact keyword returns only time periods that match precisely with the time period specified. | ||
- | |||
- | before 2009-03-31T11: | ||
- | |||
- | The before keyword returns only time periods that end before the specified time point. In other words, the time period from 2009-03-29 to 2009-03-30 would be returned by the query but the time period from 2009-03-29 to 2009-04-01 would not. | ||
- | |||
- | after 2009-03-31T11: | ||
- | |||
- | The after keyword returns only time periods that start after the specified time point. | ||
- | |||
- | inside 2009-03-31T11: | ||
- | |||
- | The inside keyword for time periods is true if the time period both starts and ends within the designated time (the time point specified inclusive). It does not return time periods that merely overlap (i.e., where only the start time or the end time are within the given time period). | ||
- | |||
- | contains 2009-03-31T11: | ||
- | |||
- | The contains keyword returns all the data with valid times beginning prior to the specified time and ending after the specified time. Semantics are inclusive, so a valid time that matches exactly would be returned as a row by this query. | ||
- | |||
- | ===== Value Parameter ===== | ||
- | |||
- | Each data value can be described using a "value parameter" | ||
- | |||
- | The user searches the database for a specific parameter in the database using the ValueParameterName. ValueParameterNames are specified within a parameter namespace. As with the other namespaces, the ParameterNameSpaceId 0 is reserved for the default WDB namespace, and is based on international english. Additional namespaces can be defined by the administrator of the database. | ||
- | |||
- | To retrieve a list of the value parameters currently being stored in the database, use the wci.browse functionality: | ||
- | |||
- | SELECT * FROM wci.browse( NULL:: | ||
- | |||
- | To list all of the value parameter names that can be stored in the database using the currently selected parameter namespace, the following wci.info function call could be used: | ||
- | |||
- | SELECT * FROM wci.getValueParameter( NULL ); | ||
- | |||
- | ===== Level ===== | ||
- | |||
- | The //Level// dimension is normally used to designate the altitude or depth of the data value. Level is designated using a level interval (height from and to) and a level parameter (e.g., height above sea level, pressure). The level definition can be prefixed with a level interpolation option that permits the user a large degree of flexibility in expressing searches. Levels are specified using a level specification in string form that is parsed by the WCI. | ||
- | |||
- | The level designation in the WCI queries is a string. The form of the level designation when querying the database is as follows: | ||
- | |||
- | interpolation levelFrom TO levelTo leveparametername | ||
- | |||
- | The interpolations available are //exact//, //below//, //above//, //inside//, and //any//. The default interpolation is exact (specifying the interpolation is optional). One may also omit the //TO levelTo// construction if the level being specified is a point. | ||
- | |||
- | Specifying //any// for the level interpolation is logically equivalent to specifying a NULL for the level; i.e., it retrieves all data items regardless of their level (height). | ||
- | |||
- | The level parameter is specified using a LevelParameterName. LevelParameterNames are specified within the parameter namespace. Consequently, | ||
- | |||
- | Examples of level designations are given below: | ||
- | |||
- | exact 2 height | ||
- | |||
- | above 0 height above mean sea level distance | ||
- | |||
- | inside 300 TO 500 isobaric surface pressure | ||
- | |||
- | To retrieve a full list of the levels and level parameters currently being stored in the database, use the wci.browse functionality: | ||
- | |||
- | SELECT * FROM wci.browse( NULL:: | ||
- | |||
- | To list all of the level parameter names that can be stored in the database using the currently selected parameter namespace, the following wci.info function call could be used: | ||
- | |||
- | SELECT * FROM wci.getLevelParameter( NULL ); | ||
- | |||
- | ===== Data Version ===== | ||
- | |||
- | There can be several different versions of the same data value that is valid for the same time, position, etc. This can happen with probability forecast calculations or when a data value is edited (in which case the new value may often be inserted with a higher data version). A data version is always a positive whole number. | ||
- | |||
- | The user may specify one or several data versions as search criteria in WCI, as well as NULL. NULL indicates that the user wants all versions of the data items specified. In addition, the user may specify the data version as a negative number; e.g., //-1//. Specifying //-1// retrieves the maximum data version in the database (i.e., the most recent version of the data item), //-2// retrieves the second most recent, and so on. | ||
- | |||
- | There is no wci.browse functionality to list all of the data versions of the data in the database. | ||
- | |||
- | ===== Chapter�3.�Using the WCI ===== | ||
- | |||
- | **Table of Contents** | ||
- | |||
- | [[# | ||
- | |||
- | This section gives an overview on how to use the WCI to retrieve data values from WDB. | ||
- | |||
- | ===== Outline ===== | ||
- | |||
- | The following list provides a brief overview of the actions that the user takes in order to retrieve data from WCI. | ||
- | |||
- | * Create a connection to the database | ||
- | * Initialize WCI | ||
- | * Read data from WDB using WCI functions (one or more times) | ||
- | * Release WCI | ||
- | * Close database connection | ||
- | |||
- | ===== Open a connection ===== | ||
- | |||
- | To enable communication with WCI, the user must open a connection to the database. How this is done depends very much on the application or API that is being used (consult the documentation of the program or API for information). | ||
- | |||
- | Using libpq, | ||
- | |||
- | connection_ = PQconnectdb(" | ||
- | |||
- | The connection must be open and valid as long as commands are being sent and data retrieved from the database. It is worth noting that connections are a resource; the database will have limits on how many connections can be open at the same time. This is particularly important for massively multi-user applications (e.g., web applications). In such cases, the database connections must be managed with some care. There is also an amount overhead involved with setting up a connection, so it will always be more efficient to connect once and perform multiple database operations, than to connect repeatedly for each operation. | ||
- | |||
- | ===== Initialize WCI ===== | ||
- | |||
- | Before WCI may be used by the user, it must be initialized. | ||
- | |||
- | SELECT wci.begin(' | ||
- | |||
- | The username is used to identify the user with WCI (this is used in setting up internal WCI variables). The three following numbers are the namespace codes for data provider namespace, place name space, and parameter name space. The wci.begin can also be called without setting these three variables, in which case they are set to the default (usually 0, 0, 0) by the system. | ||
- | |||
- | The username of the wci user can be a different identity than the one the user connected with (always assuming, of course, that your user has appropriate rights). This allows, for instance, for a single system to act as a multiplexing front-end to retrieve data available from several distinct users. | ||
- | |||
- | Name spaces are used to identify what set of names should be used for searching the database. While the user is in a namespace, only the data defined in that namespace is visible to the user. If the user should wish, for some reason, to combine data from different namespaces within an application, | ||
- | |||
- | ===== Read from the WCI ===== | ||
- | |||
- | As soon as the user has initialized WCI using wci.begin, the database is ready for retrieval of data. Data is retrieved using the wci.read function, and data is searched for by setting the appropriate dimensions in that call. | ||
- | |||
- | ==== Reading Point Data from WCI ==== | ||
- | |||
- | The wci.read function is overloaded to permit the user to read either point data or binary data (entire grids) from the database. The last parameter in the wci.read function allows the user to specify the return type of the query. To return point data values, the user specifies wci.returnFloat. | ||
- | |||
- | SELECT * | ||
- | FROM wci.read ( ARRAY[' | ||
- | ' | ||
- | ' | ||
- | NULL, -- Valid Time | ||
- | ARRAY[' | ||
- | '2 height', | ||
- | ARRAY[-1], | ||
- | NULL:: | ||
- | ) | ||
- | |||
- | The above query returns Hirlam data for a specific region (given by the place geometry), whose referencetime is on or between 12:00 to 18:00 on January 1, 1980; the query will return data for any valid times, for any air temperature. It will return data at 2 metre above ground, but only the latest version of such data. Values will be returned as point data. | ||
- | |||
- | The data provider, place definition, value parameter and level parameter must all be defined within the namespaces given in the wci.begin call that preceded the wci.read call. In this particular case, the place definition is a geometry definition, but it would also have been possible to use a placename. Consider, for example, the following query: | ||
- | |||
- | SELECT * | ||
- | FROM wci.read ( ARRAY[' | ||
- | ' | ||
- | ' | ||
- | ' | ||
- | ARRAY[' | ||
- | NULL, | ||
- | NULL, -- Data version | ||
- | NULL:: | ||
- | ) | ||
- | |||
- | The above call returns all precipitation parameters valid for the 10 days following the given referencetime measured at the nearest point to oslo. Oslo, in this case, is a place name usually defined as a geographical point within the database. Prefixing the place name is the interpolation that we wish to use when retrieving points; we could also have used any of the alternatives defined in the reference manual below. Note the use of SQL wildcard characters in the value parameter; although these slow down queries (since wildcard searching is not as efficient as a straight comparison), | ||
- | |||
- | === Return values === | ||
- | |||
- | The return value from this call is one row for each value. This means that if you have chosen an interpolation type that will return several values for each grid, you will get many rows for each grid. | ||
- | |||
- | The actual data value is given in the column " | ||
- | |||
- | ==== Reading Grid Data from WCI ==== | ||
- | |||
- | Reading grid data from the database is a two-step process. First the user retrieves the Grid ID with wci.read function (with wci.returnGId), | ||
- | |||
- | SELECT * | ||
- | FROM wci.read ( ARRAY[' | ||
- | ' | ||
- | ' | ||
- | NULL, | ||
- | ARRAY[' | ||
- | '2 height', | ||
- | ARRAY[-1], | ||
- | NULL:: | ||
- | ) | ||
- | |||
- | The above query returns the list of grids available for Hirlam data, defined on the hirlam 10km grid (i.e., if there were data on 20km or 4km grids, these would not be returned), whose referencetime is 12:00 on January 1, 1980. The query will return all grid references to air temperature forecasts for the above configuration. | ||
- | |||
- | Grid IDs are essentially 8-byte integers, that are then passed to the wci.fetch command, as given below: | ||
- | |||
- | SELECT * | ||
- | FROM wci.fetch ( 7411403, | ||
- | NULL:: | ||
- | ) | ||
- | |||
- | The above call returns a single row, containing a binary string containing the actual grid data. It also includes additional information detailing the configuration of the grid. For details about the wci.fetch call, see the function reference. | ||
- | |||
- | === Return values === | ||
- | |||
- | The return value from this call is one row for each grid. The number given in the " | ||
- | |||
- | ===== Release WCI ===== | ||
- | |||
- | Once the user has finished retrieving data using WCI, it is a good idea to close down WCI. | ||
- | |||
- | SELECT wci.end(); | ||
- | |||
- | This is not strictly necessary, but allows the system to graciously deallocate resources and reset any settings that may have been changed by WCI. | ||
- | |||
- | ===== Close Connection ===== | ||
- | |||
- | Closes down the connection to the database, freeing up the resources on the database to be used by other applications. | ||
- | |||
- | Using libpq, | ||
- | |||
- | PQfinish(connection_); | ||
- | |||
- | ===== Chapter�4.�Function Reference ===== | ||
- | |||
- | **Table of Contents** | ||
- | |||
- | [[# |