SCIAMA

High Performance Compute Cluster

Using CONDA environments on SCIAMA

Creating and Activating Environments

To create a new conda environment on SCIAMA you will need to load the anaconda3 module first. On creating a new environment it will be created in /mnt/lustre/shared_conda/envs/userid/
e.g. when I create a new conda environment called map2map it is created in /mnt/lustre/shared_conda/envs/tmaule/map2map
Do NOT create environments in your home directory as this will exceed your quota of 20GB!
Please email sciama-support@port.ac.uk if you would like to share your environment.

module load anaconda3

To see which conda environments are already available, use:

conda env list

There are a few base environments created that can be used and then cloned:

af2 - For AlphFold
gpu_lensing - GPU Lensing
gpubase - Base environment for GPUs
map2map - Map2Map

You can clone an environment or create a new one. If you load someone else's environment you will not be able to add or remove packages so you should clone it, remember to call it something different though!

conda create --name cloned_env --clone original_env
conda create -n name

To activate the environment, if you haven't done conda init:

source activate name

If you have done conda init and your .bashrc file activates the base environment on login, use:

conda activate name

Installing Packages

All packages are located in a shared directory on Lustre. If the package you require is not already there, it will install automatically when you use conda install packagename. This allows us to save disk space by only needing one copy of the package which can be shared among many environments. When you install the package to your environment, hardlinks to the package are created.

conda install packagename

You can add the conda-forge channel with

conda config --add channels conda-forge

If you find a package is not available in conda, or you want to install a downloaded package you can use pip but only as a last resort, please contact sciama-support for assistance.

pip install packagename or pip install -e .

Submitting a Job to SLURM

Below is a simple job submission script when using a CONDA environment; basically you need to source the environment.

#!/bin/bash
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=2
#SBATCH --time=0-2:00
#SBATCH --job-name=conda-test
#SBATCH -p sciama4.q
#SBATCh --error=conda.err.%j
#SBATCH --output=conda.out.%j

module purge
module add anaconda3/2022.05

source activate envname
srun ....

Top of Page