The Dockerfile provides a project container that contains Jupyter and provides a conda environment specified by environment.yml
.
The container uses mamba
to install the environment.
Within the container, you can install new packages with mamba
.
-
create a custom
environment.yml
-
build the container
docker build -t PROJECT_NAME:VERSION .
-
run
-
Jupyter:
# Jupyter will be available on port `27101` docker run --name CONTAINER_NAME -v "${PWD}":/workspace -p 27101:8888 -it PROJECT_NAME:VERSION start.sh jupyter
-
zsh
docker run --name CONTAINER_NAME -v "${PWD}":/workspace -it PROJECT_NAME:VERSION
-
- run with
--gpus all
to support GPUs; if you are running in rootless mode (which you should!), make sureno-cgroups = true
in/etc/nvidia-container-runtime/config.toml
(source). Also, for an environment.yml you need to specify the right pytorch version, e.g., via pytorch=*=cuda11.7", otherwise the CPU version is installed by mamba (see bug report). --shm-size=128gb
increases the shared memory size which might be necessary when fitting and training larger models
-
I am acutally not sure whether mamba/micromamba is a good idea at this point (but
conda
is just too low :(). I already encountered a couple of issues now. -
replace
mamba
withmicromamba
as soon as the following bug is fixed (we are not usingconda
because it is tremendeously, unbelievably, horrendously slow when installing pytorch!!!): mamba-org/mamba#2167 (comment) -
maybe support user mode like the Jupyter Docker Stack containers (or replace this with Jupyter Stack containers?)
-
secure Jupyterlab!
-
container musings:
podman
might be able to actually map local users as we wanted to -.- ... maybe it can also do this with root though- what about singularity? how do
podman
andsingularity
compare? - easy way to make an image user specific linked by Singularity Jupyter Image
- Docker
-
I also tried running with a command like this in rootless mode (with the correct user and group ids, i.e., the same as the calling user) but it fucked up my file permission to some weird number:
docker run -it --rm \ -p 8889:8888 \ --user root \ -e NB_USER="mgbckr" -e NB_UID=1001 -e NB_GID=1001 \ -e CHOWN_HOME=yes -e CHOWN_HOME_OPTS='-R' \ -w "/home/mgbckr" \ -v "${PWD}:/home/mgbckr/work" jupyter/base-notebook
-
running Docker with
---user
also didn't get me anywhere ... I was still seeing things as root and the scripts didn't work anymore
-
@mgbckr what is your
environment-jupyter.yml