:py:mod:`mlair.helpers` ======================= .. py:module:: mlair.helpers .. autoapi-nested-parse:: Collection of different supporting functions and classes. Subpackages ----------- .. toctree:: :titlesonly: :maxdepth: 3 data_sources/index.rst Submodules ---------- .. toctree:: :titlesonly: :maxdepth: 1 datastore/index.rst filter/index.rst geofunctions/index.rst helpers/index.rst logger/index.rst meteo/index.rst statistics/index.rst tables/index.rst testing/index.rst time_tracking/index.rst Package Contents ---------------- Classes ~~~~~~~ .. autoapisummary:: mlair.helpers.PyTestRegex mlair.helpers.TimeTracking mlair.helpers.TimeTrackingWrapper mlair.helpers.Logger Functions ~~~~~~~~~ .. autoapisummary:: mlair.helpers.PyTestAllEqual mlair.helpers.check_nested_equality mlair.helpers.remove_items mlair.helpers.float_round mlair.helpers.dict_to_xarray mlair.helpers.to_list mlair.helpers.extract_value mlair.helpers.select_from_dict mlair.helpers.make_keras_pickable mlair.helpers.sort_like mlair.helpers.filter_dict_by_value .. py:class:: PyTestRegex(pattern: Union[str, Pattern], flags: int = 0) Assert that a given string meets some expectations. Use like >>> PyTestRegex(r"TestString\d+") == "TestString" False >>> PyTestRegex(r"TestString\d+") == "TestString2" True :param pattern: pattern or string to use for regular expresssion :param flags: python re flags .. py:method:: __eq__(self, actual: str) -> bool Return whether regex matches given string actual or not. .. py:method:: __repr__(self) -> str Show regex pattern. .. py:function:: PyTestAllEqual(check_list: List) .. py:function:: check_nested_equality(obj1, obj2, precision=None, skip_args=None) Check for equality in nested structures. Use precision to indicate number of decimals to check for consistency .. py:class:: TimeTracking(start=True, name='undefined job', logging_level=logging.INFO, log_on_enter=False) Bases: :py:obj:`object` Track time to measure execution time. Time tracking automatically starts on initialisation and ends by calling stop method. Duration can always be shown by printing the time tracking object or calling get_current_duration. It is possible to start and stop time tracking by hand like .. code-block:: python time = TimeTracking(start=True) # start=True is default and not required to set do_something() time.stop(get_duration=True) A more comfortable way is to use TimeTracking in a with statement like: .. code-block:: python with TimeTracking(): do_something() The only disadvantage of the latter implementation is, that the duration is logged but not returned. .. py:method:: _start(self) -> None Start time tracking. .. py:method:: _end(self) -> None Stop time tracking. .. py:method:: _duration(self) -> float Get duration in seconds. .. py:method:: __repr__(self) -> str Display current passed time. .. py:method:: run(self) -> None Start time tracking. .. py:method:: stop(self, get_duration=False) -> Optional[float] Stop time tracking. Will raise an error if time tracking was already stopped. :param get_duration: return passed time if enabled. :return: duration if enabled or None .. py:method:: duration(self) -> float Return duration in seconds. .. py:method:: __enter__(self) Context manager. .. py:method:: __exit__(self, exc_type, exc_val, exc_tb) -> None Stop time tracking on exit and log info about passed time. .. py:class:: TimeTrackingWrapper(func) Wrapper implementation of TimeTracking class. Use this implementation easily as decorator for functions, classes and class methods. Implement a custom function and decorate it for automatic time measure. .. code-block:: python @TimeTrackingWrapper def sleeper(): print("start") time.sleep(1) print("end") >>> sleeper() start end INFO: foo finished after 00:00:01 (hh:mm:ss) .. py:method:: __call__(self, *args, **kwargs) Start time tracking. .. py:method:: __get__(self, instance, cls) Create bound method object and supply self argument to the decorated method. .. py:class:: Logger(log_path=None, level_file=logging.DEBUG, level_stream=logging.INFO) Basic logger class to unify all logging outputs. Logs are saved in local file and returned to std output. In default settings, logging level of file logger is DEBUG, logging level of stream logger is INFO. Class must be imported and initialised in starting script, all subscripts should log with logging.info(), debug, ... .. py:method:: setup_logging_path(path: str = None) :staticmethod: Check if given path exists and creates if not. If path is None, use path from main. The logging file is named like `logging_.log` where runtime=`%Y-%m-%d_%H-%M-%S` of current run. :param path: path to logfile :return: path of logfile .. py:method:: logger_console(self, level: int) Define a stream handler which writes messages of given level or higher to std out. :param level: logging level as integer, e.g. logging.DEBUG or 10 :return: defines stream handler .. py:function:: remove_items(obj: Union[List, Dict, Tuple], items: Any) Remove item(s) from either list, tuple or dictionary. :param obj: object to remove items from (either dictionary or list) :param items: elements to remove from obj. Can either be a list or single entry / key :return: object without items .. py:function:: float_round(number: float, decimals: int = 0, round_type: Callable = math.ceil) -> float Perform given rounding operation on number with the precision of decimals. :param number: the number to round :param decimals: numbers of decimals of the rounding operations (default 0 -> round to next integer value) :param round_type: the actual rounding operation. Can be any callable function like math.ceil, math.floor or python built-in round operation. :return: rounded number with desired precision .. py:function:: dict_to_xarray(d: Dict, coordinate_name: str) -> xarray.DataArray Convert a dictionary of 2D-xarrays to single 3D-xarray. The name of new coordinate axis follows . :param d: dictionary with 2D-xarrays :param coordinate_name: name of the new created axis (2D -> 3D) :return: combined xarray .. py:function:: to_list(obj: Any) -> List Transform given object to list if obj is not already a list. Sets are also transformed to a list. :param obj: object to transform to list :return: list containing obj, or obj itself (if obj was already a list) .. py:function:: extract_value(encapsulated_value) .. py:function:: select_from_dict(dict_obj: dict, sel_list: Any, remove_none: bool = False, filter_cond: bool = True) -> dict Extract all key values pairs whose key is contained in the sel_list. Does not perform a check if all elements of sel_list are keys of dict_obj. Therefore, the number of pairs in the returned dict is always smaller or equal to the number of elements in the sel_list. If `filter_cond` is given, this method either return the parts of the input dictionary that are included or not in `sel_list`. .. py:function:: make_keras_pickable() .. py:function:: sort_like(list_obj: list, sorted_obj: list) Sort elements of list_obj as ordered in sorted_obj. Length of sorted_obj as allowed to be higher than length of list_obj, but must contain at least all objects of list_obj. Will raise AssertionError, if not all elements of list_obj are also in sorted_obj. Also it is required for list_obj and sorted_obj to have only unique elements. :param list_obj: list to sort :param sorted_obj: list to use ordering from :return: sorted list .. py:function:: filter_dict_by_value(dictionary: dict, filter_val: Any, filter_cond: bool) -> dict Filter dictionary by its values. :param dictionary: dict to filter :param filter_val: search only for key value pair with a value equal to filter_val :param filter_cond: indicate to use either all dict entries that fulfil the filter_val criteria (if `True`) or that do not match the criteria (if `False`) :returns: a filtered dict with either matching or non-matching elements depending on the `filter_cond`