mlair.workflows

Submodules

Package Contents

Classes

Workflow

Abstract workflow class to handle sequence of stages (run modules). An inheriting class has to first initialise

DefaultWorkflow

A default workflow executing ExperimentSetup, PreProcessing, ModelSetup, Training and PostProcessing in exact

DefaultWorkflowHPC

A default workflow for Jülich HPC systems executing ExperimentSetup, PreProcessing, PartitionCheck, ModelSetup,
class mlair.workflows.Workflow(name=None, log_level_stream=None)

Abstract workflow class to handle sequence of stages (run modules). An inheriting class has to first initialise this mother class and can afterwards add an arbitrary number of stages by using the add method. The execution order is equal to the ordering of the stages have been added. To run the workflow, finally, a single call of the run method is sufficient. It must be taken care for inter-stage dependencies, this workflow class only handles the execution but not the dependencies (workflow would probably fail in this case).

add(self, stage, **kwargs)

Add a new stage with optional kwargs.

run(self)

Run workflow embedded in a run environment and according to the stage’s ordering.

class mlair.workflows.DefaultWorkflow(stations=None, train_model=None, create_new_model=None, window_history_size=None, experiment_date='testrun', variables=None, statistics_per_var=None, start=None, end=None, target_var=None, target_dim=None, window_lead_time=None, dimensions=None, interpolation_method=None, time_dim=None, limit_nan_fill=None, train_start=None, train_end=None, val_start=None, val_end=None, test_start=None, test_end=None, use_all_stations_on_all_data_sets=None, fraction_of_train=None, experiment_path=None, plot_path=None, forecast_path=None, bootstrap_path=None, overwrite_local_data=None, sampling=None, permute_data_on_training=None, extreme_values=None, extremes_on_right_tail_only=None, transformation=None, train_min_length=None, val_min_length=None, test_min_length=None, plot_list=None, model=None, batch_size=None, epochs=None, data_handler=None, log_level_stream=None, **kwargs)

Bases: mlair.workflows.abstract_workflow.Workflow

A default workflow executing ExperimentSetup, PreProcessing, ModelSetup, Training and PostProcessing in exact the mentioned ordering.

_setup(self, **kwargs)

Set up default workflow.

class mlair.workflows.DefaultWorkflowHPC(stations=None, train_model=None, create_new_model=None, window_history_size=None, experiment_date='testrun', variables=None, statistics_per_var=None, start=None, end=None, target_var=None, target_dim=None, window_lead_time=None, dimensions=None, interpolation_method=None, time_dim=None, limit_nan_fill=None, train_start=None, train_end=None, val_start=None, val_end=None, test_start=None, test_end=None, use_all_stations_on_all_data_sets=None, fraction_of_train=None, experiment_path=None, plot_path=None, forecast_path=None, bootstrap_path=None, overwrite_local_data=None, sampling=None, permute_data_on_training=None, extreme_values=None, extremes_on_right_tail_only=None, transformation=None, train_min_length=None, val_min_length=None, test_min_length=None, plot_list=None, model=None, batch_size=None, epochs=None, data_handler=None, log_level_stream=None, **kwargs)

Bases: DefaultWorkflow

A default workflow for Jülich HPC systems executing ExperimentSetup, PreProcessing, PartitionCheck, ModelSetup, Training and PostProcessing in exact the mentioned ordering.

_setup(self, **kwargs)

Set up default workflow.