Metacentrum Python installation

Metacentrum (https://metavo.metacentrum.cz/) is virtual organization providing grid computing infrastructure for academic purposes of Czech Universities and Research Groups.

Servers under this organization contain so called modules which are software packages that can be loaded via module commmand. These modules contain also python software packages like TensorFlow, Caffe, Keras, and others. But these modules contain old versions of these python software packages, and most of time, it is difficult to use them, due to wrong installation with different versions of python librabries. Most of time you get exception with missing library name.

Best way to use python software packages is to install them locally, without loading any available modules.

Prepare environment

First step is to prepare Python environment, install pip and install required python software packages.

Setup Python ENV

export PYTHONUSERBASE="/storage/praha1/home/${USER}/.local"
export PYTHONPATH=$PYTHONUSERBASE/lib/python2.7/site-packages:$PYTHONPATH/
export PATH=$PYTHONUSERBASE/bin:$PATH

Install PIP (if not already installed)

Do not use python modu

wget 'https://bootstrap.pypa.io/get-pip.py'
python get-pip.py

Instal modules

pip instal -UI ...

Basic job template

#!/bin/bash
export PYTHONUSERBASE="/storage/praha1/home/${USER}/.local"
export PYTHONPATH=$PYTHONUSERBASE/lib/python2.7/site-packages:$PYTHONPATH/
export PATH=$PYTHONUSERBASE/bin:$PATH

python script.py