Skip to content

Instantly share code, notes, and snippets.

@ivan4th
Created September 10, 2024 20:09
Show Gist options
  • Save ivan4th/865431218bc7e1e3bb9cc94c6317b9cf to your computer and use it in GitHub Desktop.
Save ivan4th/865431218bc7e1e3bb9cc94c6317b9cf to your computer and use it in GitHub Desktop.
Multiple instances of a program with a big DB on small-ish drive

The idea here is to use LVM copy-on-write snapshot feature to reduce disk usage needed for the multiple instances.

  1. Create a 300G sparse file. This file will not be using much disk space initially, despite its apparent size.
truncate -s 300G image.img
  1. Attach a loop device to the file and grab its name
loop=$(losetup --show -f image.img)
  1. Create an LVM Physical Volume on the loop device, and make a Volume Group containing that volume
pvcreate "${loop}"
vgcreate vgloop "${loop}"
  1. Create an LVM Thin Pool on the Volume Group. The need for it can be avoided by using a larger sparse file, but IMO this way fewer calculations are needed.
lvcreate -l100%FREE -T vgloop/tpool
  1. Create a 250G volume for the original database.
lvcreate -V250G -T vgloop/tpool -n orig
  1. Set up an ext4 filesystem on the volume and mount it. -o discard is supposed to give up freed blocks back to the host filesystem from the image file, but I'm not sure how well it works. For me, fstrim -v mountpoint/ did the trick.
mkfs.ext4 /dev/vgloop/orig
mount -o discard /dev/vgloop/orig orig/
  1. Copy the database (or other files) to the "original" volume. If you have it all in a single file, this dd command will do it faster and without flushing kernel caches with large read/write operations. If you copy the file by other means, you may want to do a sync, too.
dd if=state.db of=orig/state.db bs=1G iflag=direct oflag=direct status=progress
  1. Create as many thin snapshots of the original volume as you need, one for each copy of the program
lvcreate -s --name snap1 vgloop/orig
lvcreate -s --name snap2 vgloop/orig
...
  1. Make the volumes activable, and before that disable "skip activation" on them, so they can be activated and later re-attached later with just losetup
lvchange -kn vgloop/snap1
lvchange -kn vgloop/snap2
...
lvchange -ay vgloop/snap1
lvchange -ay vgloop/snap2
...
  1. Mount the copies
mount -o discard /dev/vgloop/snap1 snap1/
mount -o discard /dev/vgloop/snap2 snap2/
...

Now you can start multiple instances using snap1/, snap2/ and so on as data directories.

To remove snapshots:

umount snap1
umount snap2
...
lvremove -y vgloop/snap1
lvremove -y vgloop/snap2
...

To detach the volumes and deactivate VG, and also the loop device:

umount orig
umount snap1
umount snap2
...
vgchange -an vgloop
losetup -d "${loop}"

After that you can remove the image.img file, or re-activate the volumes:

losetup --show -f image.img
vgchange -ay vgloop
mount -o discard /dev/vgloop/orig orig/
mount -o discard /dev/vgloop/snap1 snap1/
mount -o discard /dev/vgloop/snap2 snap2/
...

You can check the actual disk usage of the image file like this:

du -h image.img

Note that it will differ from the apparent size you see in ls -lh output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment