Running notebook in a Docker container inside a VM via ssh tunnel

Inside VM, determine the IP address of the docker container:

docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id

Container ID would be a hash string like f9bb667e8d59.

Next, shell into a running Docker container on your VM:

docker exec -it f9bb667e8d59 /bin/bash

Then, launch an instance of jupyter in the container:

jupyter notebook --allow-root --no-browser --port=8889 --ip=0.0.0.0

NOTE: make sure the port 8889 is exposed on the container (or use whichever you decide to expose), to check that do:

docker port f9bb667e8d59
8889/tcp -> 0.0.0.0:8889

To expose a port in docker container during launch, use -p flag.

Finally, on the local machine (e.g. laptop) do:

ssh -N -f -L 8889:<IP address of container>:8889 yourusername@<IP address of the VM>

Then notebook is now accessible at http://localhost:8889 from local browser (on laptop).

Running the notebook via ssh-tunnel on the headnode or a VM

Run web browser on your local machine, while running jupyter notebook on the cluster, and access it via ssh tunnel

On the remote machine (mcmillan cluster headnode in this case), launch the jupyter notebook:

jupyter notebook --no-browser --port=8889 --ip=127.0.0.1

On the local machine type:

ssh -N -f -L localhost:8889:localhost:8889 <yourusername>@della.princeton.edu

The first option -N tells SSH that no remote commands will be executed, and is useful for port forwarding. The second option -f has the effect that SSH will go to background, so the local tunnel-enabling terminal remains usable. The last option -L lists the port forwarding configuration (remote port 8889 to local port 8889).

Note: tunnel will be running in the background. The notebook can now be accessed from your browser at http://localhost:8889

Running the notebook via ssh-tunnel on a worker node via salloc

If you use salloc to allocate a worker node, you cannot do the local port forwarding the way as in the previous example, because worker nodes have no internet access.

The workaraound is simple.

First, allocate a node on a cluster:

salloc -N 1 --ntasks-per-node=4 --ntasks-per-socket=2 --gres=gpu:4 -t 0-6:00

Say, you allocated tiger-i23g4.

On that node, start Jupyter:

export XDG_RUNTIME_DIR=""
jupyter notebook --no-browser --port=8889 --ip=0.0.0.0

In that case you would need to unset the XDG_RUNTIME_DIR to avoid permission issue.

Do the ssh-tunnel as follows:

ssh -N -f -L 8889:tiger-i23g4:8889 <yourusername>@tigergpu.princeton.edu

The notebook can now be accessed from your browser at http://localhost:8889