Skip to content

Instantly share code, notes, and snippets.

@askmi
Last active April 17, 2022 13:05
Show Gist options
  • Save askmi/daa5021f2a1377b8ddb57f90e9053d97 to your computer and use it in GitHub Desktop.
Save askmi/daa5021f2a1377b8ddb57f90e9053d97 to your computer and use it in GitHub Desktop.

cgroup man

http://manpages.ubuntu.com/manpages/focal/en/man7/cgroups.7.html

nsenter man

http://manpages.ubuntu.com/manpages/xenial/man1/nsenter.1.html/
https://man7.org/linux/man-pages/man1/nsenter.1.html

Enter to namespace of a particular process?


Finding docker container PID

docker inspect -f '{{.State.Pid}}' <container id>

Entering a Running Docker Container

lsns -p $PID
nsenter --target $PID --mount --uts --ipc --net --pid
echo "sudo /usr/bin/nsenter --target \$1 --mount --uts --ipc --net --pid env -i - \$(sudo cat /proc/\$1/environ | xargs -0) bash" > enterDocker.sh; chmod +x enterDocker.sh;
  • To disable user namespaces for a specific container, add the --userns=host
  • In v1.5 and later of Docker, you can make the host's process ID ns visible from inside a container by specifying the --pid=host option to docker run.
systemd-cgtop
systemd-cgls
ls /sys/fs/cgroup/memory/docker
systemctl set-property docker-${cid}.scope CPUShares=512

docker ps --quiet --all | xargs docker inspect --format '{{ .Id }}: PidMode={{ .HostConfig.PidMode }}'
DID=$(docker inspect -f '{{.State.Pid}}' <Container ID>);ps --ppid $DID -o pid,ppid,cmd
cat "/proc/${pid}/cgroup"
sudo ps -e -o pid,comm,cgroup | grep "/docker/${cid}"

How to detect when a Docker container reaches its configured memory limit

docker inspect --format '{{.State.OOMKilled}}' ${container_id}

docker events --filter type=container --filter event=oom
docker stats --no-stream | tr -d "%" | awk '{if($7>10)print "Warning: " $1 "/" $2" is consuming more than 80% of its availabile memory ("$7"%)"}'
docker stats --no-stream | tr -d "%" | awk '{if($7>10)print "Warning: " $1 "/" $2" is consuming more than 80% of its availabile memory ("$7"%)"}'

    #! /bin/bash
    for i in $(docker ps --no-trunc | awk '{print $1}' | grep -v CONTAINER); do \
    LIMIT=`cat /sys/fs/cgroup/memory/docker/${i}/memory.limit_in_bytes`; \
    USAGE=`systemd-cgtop -b --iterations 1 | grep ${i} | awk '{print $4}'`; \
    echo Container ${i} is using $USAGE and its limited at $LIMIT; done;

One way to monitor this is with docker stats command. You could monitor usage of container even before limit is enforced. https://docs.docker.com/config/containers/runmetrics/#docker-stats docker stats container1 container2 You can do some bash scripts too: how to find MAX memory from docker stats? docker stats --format 'CPU: {{.CPUPerc}}\tMEM: {{.MemPerc}}' see https://stackoverflow.com/questions/42809500/how-to-find-max-memory-from-docker-stats Another option is to look at docker events: https://www.systutorials.com/docs/linux/man/1-docker-events/ Docker containers will report the following events: attach, commit, copy, create, destroy, detach, die, exec_create, exec_detach, exec_start, export, kill, oom, pause, rename, resize, restart, start, stop, top, unpause, update


DOCKER DAEMON

sudo journalctl -fu docker.service
service docker status | restart
sudo systemctl restart docker
sudo /etc/init.d/docker restart

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment