:py:mod:`mlair.helpers.datastore` ================================= .. py:module:: mlair.helpers.datastore .. autoapi-nested-parse:: Implementation of experiment's data store. Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: mlair.helpers.datastore.AbstractDataStore mlair.helpers.datastore.DataStoreByVariable mlair.helpers.datastore.DataStoreByScope .. py:exception:: NameNotFoundInDataStore Bases: :py:obj:`Exception` Exception that get raised if given name is not found in the entire data store. .. py:exception:: NameNotFoundInScope Bases: :py:obj:`Exception` Exception that get raised if given name is not found in the provided scope, but can be found in other scopes. .. py:exception:: EmptyScope Bases: :py:obj:`Exception` Exception that get raised if given scope is not part of the data store. .. py:class:: AbstractDataStore Bases: :py:obj:`abc.ABC` Abstract data store for all settings for the experiment workflow. Save experiment parameters for the proceeding run_modules and predefine parameters loaded during the experiment setup phase. The data store is hierarchically structured, so that global settings can be overwritten by local adjustments. .. py:attribute:: tracker .. py:method:: set(self, name: str, obj: Any, scope: str, log: bool = False) -> None Abstract method to add an object to the data store. :param name: Name of object to store :param obj: The object itself to be stored :param scope: the scope / context of the object, under that the object is valid :param log: log which objects are stored if enabled (default false) .. py:method:: get(self, name: str, scope: str) -> None Abstract method to get an object from the data store. :param name: Name to look for :param scope: scope to search the name for :return: the stored object .. py:method:: get_default(self, name: str, scope: str, default: Any) -> Any Retrieve an object with `name` from `scope` and return given default if object wasn't found. Same functionality like the standard get method. But this method adds a default argument that is returned if no data was stored in the data store. Use this function with care, because it will not report any errors and just return the given default value. Currently, there is no statement that reports, if the returned value comes from the data store or the default value. :param name: Name to look for :param scope: scope to search the name for :param default: default value that is return, if no data was found for given name and scope :return: the stored object or the default value .. py:method:: search_name(self, name: str) -> None Abstract method to search for all occurrences of given `name` in the entire data store. :param name: Name to look for :return: search result .. py:method:: search_scope(self, scope: str) -> None Abstract method to search for all object names that are stored for given scope. :param scope: scope to look for :return: search result .. py:method:: list_all_scopes(self) -> None Abstract method to list all scopes in data store. :return: all found scopes .. py:method:: list_all_names(self) -> None Abstract method to list all names available in the data store. :return: all names .. py:method:: clear_data_store(self) -> None Reset entire data store. Warning: This will remove all entries of the data store without any exception. .. py:method:: create_args_dict(self, arg_list: List[str], scope: str) -> Dict Create dictionary from given argument list (as keys) and the stored data inside data store (as values). Try to load all stored elements for `arg_list` and create an entry in return dictionary for each valid key value pair. Not existing keys from arg_list are skipped. This method works on a single scope only and cannot create a dictionary with values from different scopes. Depending on the implementation of the __get__ method, all superior scopes are included in the parameter search, if no element is found for the given subscope. :param arg_list: list with all elements to look for :param scope: the scope to search in :return: dictionary with all valid elements from given arg_list as key and the corresponding stored object as value. .. py:method:: set_from_dict(self, arg_dict: Dict, scope: str, log: bool = False) -> None Store multiple objects from dictionary under same `scope`. Each object needs to be parsed as key value pair inside the given dictionary. All new entries are stored under the same scope. :param arg_dict: updates for the data store, provided as key value pairs :param scope: scope to store updates :param log: log which objects are stored if enabled (default false) .. py:class:: DataStoreByVariable Bases: :py:obj:`AbstractDataStore` Data store for all settings for the experiment workflow. Save experiment parameters for the proceeding run_modules and predefine parameters loaded during the experiment setup phase. The data store is hierarchically structured, so that global settings can be overwritten by local adjustments. This implementation stores data as .. code-block:: : value : value : value : value .. py:method:: set(self, name: str, obj: Any, scope: str, log: bool = False) -> None Store an object `obj` with given `name` under `scope`. In the current implementation, existing entries are overwritten. :param name: Name of object to store :param obj: The object itself to be stored :param scope: the scope / context of the object, under that the object is valid :param log: log which objects are stored if enabled (default false) .. py:method:: get(self, name: str, scope: str) -> Any Retrieve an object with `name` from `scope`. If no object can be found in the exact scope, take an iterative look on the levels above. Raise a NameNotFoundInDataStore error, if no object with given name can be found in the entire data store. Raise a NameNotFoundInScope error, if the object is in the data store but not in the given scope and its levels above (could be either included in another scope or a more detailed sub-scope). :param name: Name to look for :param scope: scope to search the name for :return: the stored object .. py:method:: _stride_through_scopes(self, name, scope, depth=0) .. py:method:: search_name(self, name: str) -> List[str] Search for all occurrences of given `name` in the entire data store. :param name: Name to look for :return: list with all scopes and sub-scopes containing an object stored as `name` .. py:method:: search_scope(self, scope: str, current_scope_only=True, return_all=False) -> List[str or Tuple] Search for given `scope` and list all object names stored under this scope. For an expanded search in all superior scopes, set `current_scope_only=False`. To return the scope and the object's value too, set `return_all=True`. :param scope: scope to look for :param current_scope_only: look only for all names for given scope if true, else search for names from superior scopes too. :param return_all: return name, definition scope and value if True, else just the name :return: list with all object names (if `return_all=False`) or list with tuple of object name, object scope and object value ordered by name (if `return_all=True`) .. py:method:: list_all_scopes(self) -> List[str] List all available scopes in data store. :return: names of all stored objects .. py:method:: list_all_names(self) -> List[str] List all names available in the data store. :return: all names .. py:class:: DataStoreByScope Bases: :py:obj:`AbstractDataStore` Data store for all settings for the experiment workflow. Save experiment parameters for the proceeding run_modules and predefine parameters loaded during the experiment setup phase. The data store is hierarchically structured, so that global settings can be overwritten by local adjustments. This implementation stores data as .. code-block:: : value : value : value : value .. py:method:: set(self, name: str, obj: Any, scope: str, log: bool = False) -> None Store an object `obj` with given `name` under `scope`. In the current implementation, existing entries are overwritten. :param name: Name of object to store :param obj: The object itself to be stored :param scope: the scope / context of the object, under that the object is valid :param log: log which objects are stored if enabled (default false) .. py:method:: get(self, name: str, scope: str) -> Any Retrieve an object with `name` from `scope`. If no object can be found in the exact scope, take an iterative look on the levels above. Raise a NameNotFoundInDataStore error, if no object with given name can be found in the entire data store. Raise a NameNotFoundInScope error, if the object is in the data store but not in the given scope and its levels above (could be either included in another scope or a more detailed sub-scope). :param name: Name to look for :param scope: scope to search the name for :return: the stored object .. py:method:: _stride_through_scopes(self, name, scope, depth=0) .. py:method:: search_name(self, name: str) -> List[str] Search for all occurrences of given `name` in the entire data store. :param name: Name to look for :return: list with all scopes and sub-scopes containing an object stored as `name` .. py:method:: search_scope(self, scope: str, current_scope_only: bool = True, return_all: bool = False) -> List[str or Tuple] Search for given `scope` and list all object names stored under this scope. For an expanded search in all superior scopes, set `current_scope_only=False`. To return the scope and the object's value too, set `return_all=True`. :param scope: scope to look for :param current_scope_only: look only for all names for given scope if true, else search for names from superior scopes too. :param return_all: return name, definition scope and value if True, else just the name :return: list with all object names (if `return_all=False`) or list with tuple of object name, object scope and object value ordered by name (if `return_all=True`) .. py:method:: list_all_scopes(self) -> List[str] List all available scopes in data store. :return: names of all stored objects .. py:method:: list_all_names(self) -> List[str] List all names available in the data store. :return: all names