mlair.helpers.join¶
Functions to access join database.
Module Contents¶
Functions¶
|
Read data from JOIN/TOAR. |
|
Transform to the standard data format. |
|
Download join data using requests framework. |
|
|
|
List all series ids that are available for given station id and network name. |
|
Select distinct series ids for all variables. Also check if a parameter is from REA or not. |
|
Save given data in data frame. |
|
Map given statistic name to new namespace defined by mapping dict. |
|
Lower all elements of given list. |
|
Create a request url with given base url, service type and arbitrarily many additional keyword arguments. |
Attributes¶
-
mlair.helpers.join.__date__= 2019-10-16¶
-
mlair.helpers.join.str_or_none¶
-
exception
mlair.helpers.join.EmptyQueryResult¶ Bases:
ExceptionException that get raised if a query to JOIN returns empty results.
-
mlair.helpers.join.download_join(station_name: Union[str, List[str]], stat_var: dict, station_type: str = None, network_name: str = None, sampling: str = 'daily', data_origin: Dict = None) → [pandas.DataFrame, pandas.DataFrame]¶ Read data from JOIN/TOAR.
- Parameters
station_name – Station name e.g. DEBY122
stat_var – key as variable like ‘O3’, values as statistics on keys like ‘mean’
station_type – set the station type like “traffic” or “background”, can be none
network_name – set the measurement network like “UBA” or “AIRBASE”, can be none
sampling – sampling rate of the downloaded data, either set to daily or hourly (default daily)
data_origin – additional dictionary to specify data origin as key (for variable) value (origin) pair. Valid origins are “REA” for reanalysis data and “” (empty string) for observational data.
- Returns
data frame with all variables and statistics and meta data frame with all meta information
-
mlair.helpers.join.correct_data_format(data)¶ Transform to the standard data format.
For some cases (e.g. hourly data), the data is returned as list instead of a dictionary with keys datetime, values and metadata. This functions addresses this issue and transforms the data into the dictionary version.
- Parameters
data – data in hourly format
- Returns
the same data but formatted to fit with aggregated format
-
mlair.helpers.join.get_data(opts: Dict, headers: Dict) → Union[Dict, List]¶ Download join data using requests framework.
Data is returned as json like structure. Depending on the response structure, this can lead to a list or dictionary.
- Parameters
opts – options to create the request url
headers – additional headers information like authorization, can be empty
- Returns
requested data (either as list or dictionary)
-
mlair.helpers.join.retries_session(max_retries=3)¶
-
mlair.helpers.join.load_series_information(station_name: List[str], station_type: str_or_none, network_name: str_or_none, join_url_base: str, headers: Dict, data_origin: Dict = None) → [Dict, Dict]¶ List all series ids that are available for given station id and network name.
- Parameters
station_name – Station name e.g. DEBW107
station_type – station type like “traffic” or “background”
network_name – measurement network of the station like “UBA” or “AIRBASE”
join_url_base – base url name to download data from
headers – additional headers information like authorization, can be empty
data_origin – additional information to select a distinct series e.g. from reanalysis (REA) or from observation (“”, empty string). This dictionary should contain a key for each variable and the information as key
- Returns
all available series for requested station stored in an dictionary with parameter name (variable) as key and the series id as value.
-
mlair.helpers.join._select_distinct_series(vars: List[Dict], data_origin: Dict = None) → [Dict, Dict]¶ Select distinct series ids for all variables. Also check if a parameter is from REA or not.
-
mlair.helpers.join._save_to_pandas(df: Union[pandas.DataFrame, None], data: dict, stat: str, var: str) → pandas.DataFrame¶ Save given data in data frame.
If given data frame is not empty, the data is appened as new column.
- Parameters
df – data frame to append the new data, can be none
data – new data to append or format as data frame containing the keys ‘datetime’ and ‘<stat>’
stat – extracted statistic to get values from data (e.g. ‘mean’, ‘dma8eu’)
var – variable the data is from (e.g. ‘o3’)
- Returns
new created or concatenated data frame
-
mlair.helpers.join._correct_stat_name(stat: str) → str¶ Map given statistic name to new namespace defined by mapping dict.
Return given name stat if not element of mapping namespace.
- Parameters
stat – namespace from JOIN server
- Returns
stat mapped to local namespace
-
mlair.helpers.join._lower_list(args: List[str]) → Iterator[str]¶ Lower all elements of given list.
- Parameters
args – list with string entries to lower
- Returns
iterator that lowers all list entries
-
mlair.helpers.join.create_url(base: str, service: str, **kwargs: Union[str, int, float, None]) → str¶ Create a request url with given base url, service type and arbitrarily many additional keyword arguments.
- Parameters
base – basic url of the rest service
service – service type, e.g. series, stats
kwargs – keyword pairs for optional request specifications, e.g. ‘statistics=maximum’
- Returns
combined url as string
-
mlair.helpers.join.var_all_dic¶