`mlair.helpers.datastore`¶

Implementation of experiment’s data store.

Module Contents¶

Classes¶

`AbstractDataStore`	Abstract data store for all settings for the experiment workflow.
`DataStoreByVariable`	Data store for all settings for the experiment workflow.
`DataStoreByScope`	Data store for all settings for the experiment workflow.

exception mlair.helpers.datastore.NameNotFoundInDataStore¶

Bases: Exception

Exception that get raised if given name is not found in the entire data store.

exception mlair.helpers.datastore.NameNotFoundInScope¶

Bases: Exception

Exception that get raised if given name is not found in the provided scope, but can be found in other scopes.

exception mlair.helpers.datastore.EmptyScope¶

Bases: Exception

Exception that get raised if given scope is not part of the data store.

class mlair.helpers.datastore.AbstractDataStore¶

Bases: abc.ABC

Abstract data store for all settings for the experiment workflow.

Save experiment parameters for the proceeding run_modules and predefine parameters loaded during the experiment setup phase. The data store is hierarchically structured, so that global settings can be overwritten by local adjustments.

tracker¶

set(self, name: str, obj: Any, scope: str, log: bool = False) → None ¶

Abstract method to add an object to the data store.

Parameters

name – Name of object to store
obj – The object itself to be stored
scope – the scope / context of the object, under that the object is valid
log – log which objects are stored if enabled (default false)

get(self, name: str, scope: str) → None ¶

Abstract method to get an object from the data store.

Parameters

name – Name to look for
scope – scope to search the name for

Returns

the stored object

get_default(self, name: str, scope: str, default: Any) → Any¶

Retrieve an object with name from scope and return given default if object wasn’t found.

Same functionality like the standard get method. But this method adds a default argument that is returned if no data was stored in the data store. Use this function with care, because it will not report any errors and just return the given default value. Currently, there is no statement that reports, if the returned value comes from the data store or the default value.

Parameters

name – Name to look for
scope – scope to search the name for
default – default value that is return, if no data was found for given name and scope

Returns

the stored object or the default value

search_name(self, name: str) → None ¶

Abstract method to search for all occurrences of given name in the entire data store.

Parameters: name – Name to look for
Returns: search result

search_scope(self, scope: str) → None ¶

Abstract method to search for all object names that are stored for given scope.

Parameters: scope – scope to look for
Returns: search result

list_all_scopes(self) → None ¶

Abstract method to list all scopes in data store.

Returns: all found scopes

list_all_names(self) → None ¶

Abstract method to list all names available in the data store.

Returns: all names

clear_data_store(self) → None ¶

Reset entire data store.

Warning: This will remove all entries of the data store without any exception.

create_args_dict(self, arg_list: List[str], scope: str) → Dict¶

Create dictionary from given argument list (as keys) and the stored data inside data store (as values).

Try to load all stored elements for arg_list and create an entry in return dictionary for each valid key value pair. Not existing keys from arg_list are skipped. This method works on a single scope only and cannot create a dictionary with values from different scopes. Depending on the implementation of the __get__ method, all superior scopes are included in the parameter search, if no element is found for the given subscope.

Parameters

arg_list – list with all elements to look for
scope – the scope to search in

Returns

dictionary with all valid elements from given arg_list as key and the corresponding stored object as value.

set_from_dict(self, arg_dict: Dict, scope: str, log: bool = False) → None ¶

Store multiple objects from dictionary under same scope.

Each object needs to be parsed as key value pair inside the given dictionary. All new entries are stored under the same scope.

Parameters

arg_dict – updates for the data store, provided as key value pairs
scope – scope to store updates
log – log which objects are stored if enabled (default false)

class mlair.helpers.datastore.DataStoreByVariable¶

Bases: AbstractDataStore

Data store for all settings for the experiment workflow.

Save experiment parameters for the proceeding run_modules and predefine parameters loaded during the experiment setup phase. The data store is hierarchically structured, so that global settings can be overwritten by local adjustments.

This implementation stores data as

<variable1>
    <scope1>: value
    <scope2>: value
<variable2>
    <scope1>: value
    <scope3>: value

set(self, name: str, obj: Any, scope: str, log: bool = False) → None ¶

Store an object obj with given name under scope.

In the current implementation, existing entries are overwritten.

Parameters

name – Name of object to store
obj – The object itself to be stored
scope – the scope / context of the object, under that the object is valid
log – log which objects are stored if enabled (default false)

get(self, name: str, scope: str) → Any¶

Retrieve an object with name from scope.

If no object can be found in the exact scope, take an iterative look on the levels above. Raise a NameNotFoundInDataStore error, if no object with given name can be found in the entire data store. Raise a NameNotFoundInScope error, if the object is in the data store but not in the given scope and its levels above (could be either included in another scope or a more detailed sub-scope).

Parameters

name – Name to look for
scope – scope to search the name for

Returns

the stored object

_stride_through_scopes(self, name, scope, depth=0)¶

search_name(self, name: str) → List[str]¶

Search for all occurrences of given name in the entire data store.

Parameters: name – Name to look for
Returns: list with all scopes and sub-scopes containing an object stored as name

search_scope(self, scope: str, current_scope_only=True, return_all=False) → List[str or Tuple]¶

Search for given scope and list all object names stored under this scope.

For an expanded search in all superior scopes, set current_scope_only=False. To return the scope and the object’s value too, set return_all=True.

Parameters

scope – scope to look for
current_scope_only – look only for all names for given scope if true, else search for names from superior scopes too.
return_all – return name, definition scope and value if True, else just the name

Returns

list with all object names (if return_all=False) or list with tuple of object name, object scope and object value ordered by name (if return_all=True)

list_all_scopes(self) → List[str]¶

List all available scopes in data store.

Returns: names of all stored objects

list_all_names(self) → List[str]¶

List all names available in the data store.

Returns: all names

class mlair.helpers.datastore.DataStoreByScope¶

Bases: AbstractDataStore

Data store for all settings for the experiment workflow.

Save experiment parameters for the proceeding run_modules and predefine parameters loaded during the experiment setup phase. The data store is hierarchically structured, so that global settings can be overwritten by local adjustments.

This implementation stores data as

<scope1>
    <variable1>: value
    <variable2>: value
<scope2>
    <variable1>: value
    <variable3>: value

set(self, name: str, obj: Any, scope: str, log: bool = False) → None ¶

Store an object obj with given name under scope.

In the current implementation, existing entries are overwritten.

Parameters

name – Name of object to store
obj – The object itself to be stored
scope – the scope / context of the object, under that the object is valid
log – log which objects are stored if enabled (default false)

get(self, name: str, scope: str) → Any¶

Retrieve an object with name from scope.

If no object can be found in the exact scope, take an iterative look on the levels above. Raise a NameNotFoundInDataStore error, if no object with given name can be found in the entire data store. Raise a NameNotFoundInScope error, if the object is in the data store but not in the given scope and its levels above (could be either included in another scope or a more detailed sub-scope).

Parameters

name – Name to look for
scope – scope to search the name for

Returns

the stored object

_stride_through_scopes(self, name, scope, depth=0)¶

search_name(self, name: str) → List[str]¶

Search for all occurrences of given name in the entire data store.

Parameters: name – Name to look for
Returns: list with all scopes and sub-scopes containing an object stored as name

search_scope(self, scope: str, current_scope_only: bool = True, return_all: bool = False) → List[str or Tuple]¶

Search for given scope and list all object names stored under this scope.

For an expanded search in all superior scopes, set current_scope_only=False. To return the scope and the object’s value too, set return_all=True.

Parameters

scope – scope to look for
current_scope_only – look only for all names for given scope if true, else search for names from superior scopes too.
return_all – return name, definition scope and value if True, else just the name

Returns

list with all object names (if return_all=False) or list with tuple of object name, object scope and object value ordered by name (if return_all=True)

list_all_scopes(self) → List[str]¶

List all available scopes in data store.

Returns: names of all stored objects

list_all_names(self) → List[str]¶

List all names available in the data store.

Returns: all names

mlair.helpers.datastore¶

Module Contents¶

Classes¶

`mlair.helpers.datastore`¶