`mlair.data_handler.data_handler_with_filter`¶

Data Handler using kz-filtered data.

Module Contents¶

Classes¶

`DataHandlerFilterSingleStation`	General data handler for a single station to be used by a superior data handler.
`DataHandlerFilter`	Data handler using FIR filtered data.
`DataHandlerFirFilterSingleStation`	Data handler for a single station to be used by a superior data handler. Inputs are FIR filtered.
`DataHandlerFirFilter`	Data handler using FIR filtered data.
`DataHandlerClimateFirFilterSingleStation`	Data handler for a single station to be used by a superior data handler. Inputs are FIR filtered. In contrast to
`DataHandlerClimateFirFilter`	Data handler using climatic adjusted FIR filtered data.

Attributes¶

`__author__`
`__date__`
`str_or_list`

mlair.data_handler.data_handler_with_filter.__author__ = Lukas Leufen¶

mlair.data_handler.data_handler_with_filter.__date__ = 2020-08-26¶

mlair.data_handler.data_handler_with_filter.str_or_list¶

class mlair.data_handler.data_handler_with_filter.DataHandlerFilterSingleStation(*args, filter_dim=DEFAULT_FILTER_DIM, **kwargs)¶

Bases: mlair.data_handler.data_handler_single_station.DataHandlerSingleStation

General data handler for a single station to be used by a superior data handler.

_hash¶

DEFAULT_FILTER_DIM = filter¶

setup_transformation(self, transformation: Union[None, dict, Tuple]) → Tuple[Optional[dict], Optional[dict]]¶: Adjust setup of transformation because filtered data will have negative values which is not compatible with the log transformation. Therefore, replace all log transformation methods by a default standardization. This is only applied on input side.

_check_sampling(self, **kwargs)¶

make_input_target(self)¶

abstract apply_filter(self)¶

abstract create_filter_index(self) → pandas.Index ¶: Create name for filter dimension.

get_transposed_history(self) → xarray.DataArray¶

Return history.

Returns: history with dimensions datetime, window, Stations, variables, filter.

abstract _create_lazy_data(self)¶

_extract_lazy(self, lazy_data)¶

class mlair.data_handler.data_handler_with_filter.DataHandlerFilter(*args, use_filter_branches=False, **kwargs)¶

Bases: mlair.data_handler.DefaultDataHandler

Data handler using FIR filtered data.

data_handler¶

data_handler_transformation¶

_requirements¶

get_X_original(self)¶

class mlair.data_handler.data_handler_with_filter.DataHandlerFirFilterSingleStation(*args, filter_cutoff_period, filter_order, filter_window_type=DEFAULT_WINDOW_TYPE, plot_path=None, filter_plot_dates=None, **kwargs)¶

Bases: DataHandlerFilterSingleStation

Data handler for a single station to be used by a superior data handler. Inputs are FIR filtered.

_hash¶

DEFAULT_WINDOW_TYPE = ['kaiser', 5]¶

static _prepare_filter_order(filter_order, removed_index, fs)¶

static _prepare_filter_cutoff_period(filter_cutoff_period, fs)¶: Frequency must be smaller than the sampling frequency fs. Otherwise remove given cutoff period pair.

static _get_kzf_cutoff_period(kzf_settings, fs)¶

static _period_to_freq(cutoff_p)¶

static _get_fs(**kwargs)¶: Return frequency in 1/day (not Hz)

apply_filter(self)¶: Apply FIR filter only on inputs.

create_filter_index(self, add_unfiltered_index=True) → pandas.Index ¶

Round cut off periods in days and append ‘res’ for residuum index.

Round small numbers (<10) to single decimal, and higher numbers to int. Transform as list of str and append ‘res’ for residuum index. Add index unfiltered if the raw / unfiltered data is appended to data in addition.

_create_lazy_data(self)¶

_extract_lazy(self, lazy_data)¶

transform(self, data_in, dim: Union[str, int] = 0, inverse: bool = False, opts=None, transformation_dim=None)¶

Transform data according to given transformation settings.

This function transforms a xarray.dataarray (along dim) or pandas.DataFrame (along axis) either with mean=0 and std=1 (method=standardise) or centers the data with mean=0 and no change in data scale (method=centre). Furthermore, this sets an internal instance attribute for later inverse transformation. This method will raise an AssertionError if an internal transform method was already set (‘inverse=False’) or if the internal transform method, internal mean and internal standard deviation weren’t set (‘inverse=True’).

Parameters

dim (string/int) – This param is not used for inverse transformation. | for xarray.DataArray as string: name of dimension which should be standardised | for pandas.DataFrame as int: axis of dimension which should be standardised
inverse – Switch between transformation and inverse transformation.

Returns

xarray.DataArrays or pandas.DataFrames: #. mean: Mean of data #. std: Standard deviation of data #. data: Standardised data

class mlair.data_handler.data_handler_with_filter.DataHandlerFirFilter(*args, use_filter_branches=False, **kwargs)¶

Bases: DataHandlerFilter

Data handler using FIR filtered data.

data_handler¶

data_handler_transformation¶

_requirements¶

class mlair.data_handler.data_handler_with_filter.DataHandlerClimateFirFilterSingleStation(*args, apriori=None, apriori_type=None, apriori_diurnal=False, apriori_sel_opts=None, extend_length_opts=DEFAULT_EXTEND_LENGTH_OPTS, **kwargs)¶

Bases: DataHandlerFirFilterSingleStation

Data handler for a single station to be used by a superior data handler. Inputs are FIR filtered. In contrast to the simple DataHandlerFirFilterSingleStation, this data handler is centered around t0 to have no time delay. For values in the future (t > t0), this data handler assumes a climatological value for the low pass data and values of 0 for all residuum components.

Parameters

apriori – Data to use as apriori information. This should be either a xarray dataarray containing monthly or any other heuristic to support the clim filter, or a list of such arrays containing heuristics for all residua in addition. The 2nd can be used together with apriori_type residuum_stats which estimates the error of the residuum when the clim filter should be applied with exogenous parameters. If apriori_type is None/zeros data can be provided, but this is not required in this case.
apriori_type – set type of information that is provided to the clim filter. For the first low pass always a calculated or given statistic is used. For residuum prediction a constant value of zero is assumed if apriori_type is None or zeros, and a climatology of the residuum is used for residuum_stats.
apriori_diurnal – use diurnal anomalies of each hour as addition to the apriori information type chosen by parameter apriori_type. This is only applicable for hourly resolution data.
apriori_sel_opts – specify some parameters to select a subset of data before calculating the apriori information. Use this parameter for example, if apriori shall only calculated on a shorter time period than available in given data.
extend_length_opts – use this parameter to use future data in the filter calculation. This parameter does not affect the size of the history samples as this is handled by the window_history_size parameter. Example: set extend_length_opts=7*24 to use the observation of the next 7 days to calculate the filtered components. Which data are finally used for the input samples is not affected by these 7 days. In case the range of history sample exceeds the horizon of extend_length_opts, the history sample will also include data from climatological estimates.

DEFAULT_EXTEND_LENGTH_OPTS = 0¶

_hash¶

_store_attributes¶

apply_filter(self)¶: Apply FIR filter only on inputs.

create_filter_index(self, add_unfiltered_index=True) → pandas.Index ¶

Round cut off periods in days and append ‘res’ for residuum index.

Round small numbers (<10) to single decimal, and higher numbers to int. Transform as list of str and append ‘res’ for residuum index. Add index unfiltered if the raw / unfiltered data is appended to data in addition.

_create_lazy_data(self)¶

_extract_lazy(self, lazy_data)¶

static _prepare_filter_cutoff_period(filter_cutoff_period, fs)¶: Frequency must be smaller than the sampling frequency fs. Otherwise remove given cutoff period pair.

static _period_to_freq(cutoff_p)¶

make_history_window(self, dim_name_of_inputs: str, window: int, dim_name_of_shift: str) → None ¶

Create a xr.DataArray containing history data. As ‘input_data’ already consists of a dimension ‘window’, this method only shifts the data along ‘window’ dimension x times where x is given by ‘window_history_offset’. Results are stored in history attribute.

Parameters

dim_name_of_inputs – Name of dimension which contains the input variables
window – this parameter is not used in the inherited method
dim_name_of_shift – Dimension along shift will be applied

call_transform(self, inverse=False)¶

class mlair.data_handler.data_handler_with_filter.DataHandlerClimateFirFilter(*args, use_filter_branches=False, **kwargs)¶

Bases: DataHandlerFilter

Data handler using climatic adjusted FIR filtered data.

data_handler¶

data_handler_transformation¶

_requirements¶

_store_attributes¶

mlair.data_handler.data_handler_with_filter¶

Module Contents¶

Classes¶

Attributes¶

`mlair.data_handler.data_handler_with_filter`¶