Install MLAir¶
MLAir is based on several python frameworks. To work properly, you have to install all packages from the
requirements.txt
file. Additionally to support the geographical plotting part it is required to install geo
packages built for your operating system. Unfortunately, the names of these package may differ for different systems.
In this instruction, we try to address users of different operating systems namely openSUSE Leap, Ubuntu and macOS.
If the installation is still not working, we recommend skipping the geographical plot. We have put together a small
workaround here. For special instructions to install MLAir on the Juelich
HPC systems, see section Installation on Jülich HPC systems.
Pre-requirements¶
Make sure to have the python3.6 version installed (We are already using python3.8, but will refer to python3.6 here as this was used for long time and is therefore tested well.)
(geo) A c++ compiler is required for the installation of the program cartopy
(geo) Install proj and GEOS on your machine using the console.
Install the python3.6 develop libraries.
Installation of MLAir¶
Install all requirements from requirements.txt preferably in a virtual environment
Either clone MLAir from the gitlab repository
or download the distribution file (current version) and install it via
pip install <dist_file>.whl
. In this case, you can simply import MLAir in any python script inside your virtual environment usingimport mlair
.
Special Instructions for Installation¶
openSUSE Leap 15.1¶
c++ compiler
sudo zypper install gcc-c++
geo packages
sudo zypper install proj geos-devel
depending on the pre-installed packages it could be required to install further packages
sudo zypper install libproj-devel binutils gdal-devel graphviz graphviz-gnome
python develop libraries
sudo zypper install python3-devel
Ubuntu 20.04.1¶
c++ compiler
sudo apt install build-essential
geo packages
sudo apt install proj-bin libgeos-dev libproj-dev
depending on the pre-installed packages it could be required to install further packages
sudo apt install graphviz libgeos++-dev
python develop libraries
sudo apt install python3.6-dev
macOS & windows¶
The installation on macOS is not tested yet. The following commands are possibly needed:
brew install geos
sudo port install graphviz
The installation on Windows is not tested yet.
Installation on Jülich HPC systems¶
Please note, that the HPC setup is customised for JUWELS and HDFML. When using another HPC system, you can use the HPC setup files as a skeleton and customise it to your needs.
The following instruction guide you through the installation on JUWELS and HDFML.
Clone the repo to HPC system (we recommend to place it in
/p/projects/<project name>
).Setup venv by executing
source setupHPC.sh
. This script loads all pre-installed modules and creates a venv for all other packages. Furthermore, it creates slurm/batch scripts to execute code on compute nodes. You have to enter the HPC project’s budget name (–account flag).The default external data path on JUWELS and HDFML is set to
/p/project/deepacf/intelliaq/<user>/DATA/toar_<sampling>
.To choose a different location open
run.py
and add the following keyword argument toExperimentSetup
:data_path=<your>/<custom>/<path>
.Execute
python run.py
on a login node to download example data. The program will throw an OSerror after downloading.Execute either
sbatch run_juwels_develgpus.bash
orsbatch run_hdfml_batch.bash
to verify that the setup went well.Currently cartopy is not working on our HPC system, therefore PlotStations does not create any output.
Note: The method PartitionCheck
currently only checks if the hostname starts with ju
or hdfmll
.
Therefore, it might be necessary to adopt the if
statement in PartitionCheck._run
.
Workaround to skip geographical plot¶
If it is not possible to install all required geo libraries on your system, a good compromise is to skip the creation
of the geographical plot. Therefore, it is required to remove the plot from the plot_list
manually. We recommend
to use this code snippet as a starting point.
from mlair.helpers import remove_items
from mlair.configuration.defaults import DEFAULT_PLOT_LIST
mlair.run(plot_list=remove_items(DEFAULT_PLOT_LIST, "PlotStationMap"))