User Tools

Site Tools


cluster-lbt:extra-tools:conda

Conda on Baal

Conda, more specifically Miniconda, has been installed on Baal cluster. Conda is an open source package management system and environment management which allows you to install packages in separate environments. It is mainly for Python but can be used also for R. Here the website: https://docs.conda.io/projects/conda/en/latest/

On Baal, there is two conda modules :

  1. miniconda-py2/latest for Python 2
  2. miniconda-py3/latest for Python 3

Unless absolute requirement, you should always use the version for Python 3.

On the cluster, all commands to prepare an environment (create, install packages, etc) have to be done on the master (or login) node. Conda dont do compilation but just downloading and extracting archives

Create Environments

Conda works with environments which are isolated from the rest of the OS. You can have as much as environments you like. To create one :

[baal master node] $ conda create -n "test" python=3.7

will create an environment called test with python 3.7

By default, conda create environments in your workdir (/workdir/my_team/username/.conda/envs) so they can be used easily on the compute nodes.

Using Environments

Now we created an environment called test, let's use it :

[baal master node] $ conda activate test
(test) [baal master node] $ 

When activated, you should see the name of the environment between parenthesis, like above.

To stop using the environment, juste deactivate it:

(test) [baal master node] $ conda deactivate

Install packages

Once your environment is activate, you can install packages inside.

(test) [baal master node] $ conda install numpy

Using environment on the compute nodes

Once you have created your environment and installed packages inside it, you can use it on the computer nodes within your job script. Just add the line to activate it inside your script :

#!/bin/bash
#PBS -S /bin/bash
#PBS -N <job-name>
#PBS -o <job-name>.out
#PBS -e <job-name>.err
 
[...]
#Modules
module load miniconda-py3/latest
 
conda activate test
 
python myscript.py
 
[...]
cluster-lbt/extra-tools/conda.txt ยท Last modified: 2020/06/18 15:37 by santuz