:py:mod:`mlair.data_handler.data_handler_with_filter` ===================================================== .. py:module:: mlair.data_handler.data_handler_with_filter .. autoapi-nested-parse:: Data Handler using kz-filtered data. Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: mlair.data_handler.data_handler_with_filter.DataHandlerFilterSingleStation mlair.data_handler.data_handler_with_filter.DataHandlerFilter mlair.data_handler.data_handler_with_filter.DataHandlerFirFilterSingleStation mlair.data_handler.data_handler_with_filter.DataHandlerFirFilter mlair.data_handler.data_handler_with_filter.DataHandlerClimateFirFilterSingleStation mlair.data_handler.data_handler_with_filter.DataHandlerClimateFirFilter Attributes ~~~~~~~~~~ .. autoapisummary:: mlair.data_handler.data_handler_with_filter.__author__ mlair.data_handler.data_handler_with_filter.__date__ mlair.data_handler.data_handler_with_filter.str_or_list .. py:data:: __author__ :annotation: = Lukas Leufen .. py:data:: __date__ :annotation: = 2020-08-26 .. py:data:: str_or_list .. py:class:: DataHandlerFilterSingleStation(*args, filter_dim=DEFAULT_FILTER_DIM, **kwargs) Bases: :py:obj:`mlair.data_handler.data_handler_single_station.DataHandlerSingleStation` General data handler for a single station to be used by a superior data handler. .. py:attribute:: _hash .. py:attribute:: DEFAULT_FILTER_DIM :annotation: = filter .. py:method:: setup_transformation(self, transformation: Union[None, dict, Tuple]) -> Tuple[Optional[dict], Optional[dict]] Adjust setup of transformation because filtered data will have negative values which is not compatible with the log transformation. Therefore, replace all log transformation methods by a default standardization. This is only applied on input side. .. py:method:: _check_sampling(self, **kwargs) .. py:method:: make_input_target(self) .. py:method:: apply_filter(self) :abstractmethod: .. py:method:: create_filter_index(self) -> pandas.Index :abstractmethod: Create name for filter dimension. .. py:method:: get_transposed_history(self) -> xarray.DataArray Return history. :return: history with dimensions datetime, window, Stations, variables, filter. .. py:method:: _create_lazy_data(self) :abstractmethod: .. py:method:: _extract_lazy(self, lazy_data) .. py:class:: DataHandlerFilter(*args, use_filter_branches=False, **kwargs) Bases: :py:obj:`mlair.data_handler.DefaultDataHandler` Data handler using FIR filtered data. .. py:attribute:: data_handler .. py:attribute:: data_handler_transformation .. py:attribute:: _requirements .. py:method:: get_X_original(self) .. py:class:: DataHandlerFirFilterSingleStation(*args, filter_cutoff_period, filter_order, filter_window_type=DEFAULT_WINDOW_TYPE, plot_path=None, filter_plot_dates=None, **kwargs) Bases: :py:obj:`DataHandlerFilterSingleStation` Data handler for a single station to be used by a superior data handler. Inputs are FIR filtered. .. py:attribute:: _hash .. py:attribute:: DEFAULT_WINDOW_TYPE :annotation: = ['kaiser', 5] .. py:method:: _prepare_filter_order(filter_order, removed_index, fs) :staticmethod: .. py:method:: _prepare_filter_cutoff_period(filter_cutoff_period, fs) :staticmethod: Frequency must be smaller than the sampling frequency fs. Otherwise remove given cutoff period pair. .. py:method:: _get_kzf_cutoff_period(kzf_settings, fs) :staticmethod: .. py:method:: _period_to_freq(cutoff_p) :staticmethod: .. py:method:: _get_fs(**kwargs) :staticmethod: Return frequency in 1/day (not Hz) .. py:method:: apply_filter(self) Apply FIR filter only on inputs. .. py:method:: create_filter_index(self, add_unfiltered_index=True) -> pandas.Index Round cut off periods in days and append 'res' for residuum index. Round small numbers (<10) to single decimal, and higher numbers to int. Transform as list of str and append 'res' for residuum index. Add index unfiltered if the raw / unfiltered data is appended to data in addition. .. py:method:: _create_lazy_data(self) .. py:method:: _extract_lazy(self, lazy_data) .. py:method:: transform(self, data_in, dim: Union[str, int] = 0, inverse: bool = False, opts=None, transformation_dim=None) Transform data according to given transformation settings. This function transforms a xarray.dataarray (along dim) or pandas.DataFrame (along axis) either with mean=0 and std=1 (`method=standardise`) or centers the data with mean=0 and no change in data scale (`method=centre`). Furthermore, this sets an internal instance attribute for later inverse transformation. This method will raise an AssertionError if an internal transform method was already set ('inverse=False') or if the internal transform method, internal mean and internal standard deviation weren't set ('inverse=True'). :param string/int dim: This param is not used for inverse transformation. | for xarray.DataArray as string: name of dimension which should be standardised | for pandas.DataFrame as int: axis of dimension which should be standardised :param inverse: Switch between transformation and inverse transformation. :return: xarray.DataArrays or pandas.DataFrames: #. mean: Mean of data #. std: Standard deviation of data #. data: Standardised data .. py:class:: DataHandlerFirFilter(*args, use_filter_branches=False, **kwargs) Bases: :py:obj:`DataHandlerFilter` Data handler using FIR filtered data. .. py:attribute:: data_handler .. py:attribute:: data_handler_transformation .. py:attribute:: _requirements .. py:class:: DataHandlerClimateFirFilterSingleStation(*args, apriori=None, apriori_type=None, apriori_diurnal=False, apriori_sel_opts=None, extend_length_opts=DEFAULT_EXTEND_LENGTH_OPTS, **kwargs) Bases: :py:obj:`DataHandlerFirFilterSingleStation` Data handler for a single station to be used by a superior data handler. Inputs are FIR filtered. In contrast to the simple DataHandlerFirFilterSingleStation, this data handler is centered around t0 to have no time delay. For values in the future (t > t0), this data handler assumes a climatological value for the low pass data and values of 0 for all residuum components. :param apriori: Data to use as apriori information. This should be either a xarray dataarray containing monthly or any other heuristic to support the clim filter, or a list of such arrays containing heuristics for all residua in addition. The 2nd can be used together with apriori_type `residuum_stats` which estimates the error of the residuum when the clim filter should be applied with exogenous parameters. If apriori_type is None/`zeros` data can be provided, but this is not required in this case. :param apriori_type: set type of information that is provided to the clim filter. For the first low pass always a calculated or given statistic is used. For residuum prediction a constant value of zero is assumed if apriori_type is None or `zeros`, and a climatology of the residuum is used for `residuum_stats`. :param apriori_diurnal: use diurnal anomalies of each hour as addition to the apriori information type chosen by parameter apriori_type. This is only applicable for hourly resolution data. :param apriori_sel_opts: specify some parameters to select a subset of data before calculating the apriori information. Use this parameter for example, if apriori shall only calculated on a shorter time period than available in given data. :param extend_length_opts: use this parameter to use future data in the filter calculation. This parameter does not affect the size of the history samples as this is handled by the window_history_size parameter. Example: set extend_length_opts=7*24 to use the observation of the next 7 days to calculate the filtered components. Which data are finally used for the input samples is not affected by these 7 days. In case the range of history sample exceeds the horizon of extend_length_opts, the history sample will also include data from climatological estimates. .. py:attribute:: DEFAULT_EXTEND_LENGTH_OPTS :annotation: = 0 .. py:attribute:: _hash .. py:attribute:: _store_attributes .. py:method:: apply_filter(self) Apply FIR filter only on inputs. .. py:method:: create_filter_index(self, add_unfiltered_index=True) -> pandas.Index Round cut off periods in days and append 'res' for residuum index. Round small numbers (<10) to single decimal, and higher numbers to int. Transform as list of str and append 'res' for residuum index. Add index unfiltered if the raw / unfiltered data is appended to data in addition. .. py:method:: _create_lazy_data(self) .. py:method:: _extract_lazy(self, lazy_data) .. py:method:: _prepare_filter_cutoff_period(filter_cutoff_period, fs) :staticmethod: Frequency must be smaller than the sampling frequency fs. Otherwise remove given cutoff period pair. .. py:method:: _period_to_freq(cutoff_p) :staticmethod: .. py:method:: make_history_window(self, dim_name_of_inputs: str, window: int, dim_name_of_shift: str) -> None Create a xr.DataArray containing history data. As 'input_data' already consists of a dimension 'window', this method only shifts the data along 'window' dimension x times where x is given by 'window_history_offset'. Results are stored in history attribute. :param dim_name_of_inputs: Name of dimension which contains the input variables :param window: this parameter is not used in the inherited method :param dim_name_of_shift: Dimension along shift will be applied .. py:method:: call_transform(self, inverse=False) .. py:class:: DataHandlerClimateFirFilter(*args, use_filter_branches=False, **kwargs) Bases: :py:obj:`DataHandlerFilter` Data handler using climatic adjusted FIR filtered data. .. py:attribute:: data_handler .. py:attribute:: data_handler_transformation .. py:attribute:: _requirements .. py:attribute:: _store_attributes