Notes

This guide uses Kamal 1.8.1

Motivation

Kamal was designed with 1 service = 1 droplet/VPS in mind.
But I'm cheap and I want to be able to deploy multiple demo/poc apps apps on my $20/month dedicated server.
What the hell, I'll even host my private container registry on it.

Setup your dedicated server

• https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-22-04
• https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-22-04
• https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-22-04
• https://www.digitalocean.com/community/tutorials/how-to-install-and-use-postgresql-on-ubuntu-22-04
• https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-redis-on-ubuntu-22-04
• https://www.digitalocean.com/community/tutorials/how-to-set-up-minio-object-storage-server-in-standalone-mode-on-ubuntu-20-04

Setup your domain names

• Add A/AAAA record in your DNS records for server.mydomain.com, registry.mydomain.com, myapp1.mydomain.com and myapp2.mydomain.com to the IP of your dedicated server

Setup a private container registry

• https://www.digitalocean.com/community/tutorials/how-to-set-up-a-private-docker-registry-on-ubuntu-22-04

Note 1: This tutorial says you need a host server and a client server. In our case, we will use only one server to be both client and server.

Note 2: The port 5000 is already used by datadog-agent by default, so I prefer using 7000:5000 instead.

Install SSL certificates for the registry

• Run sudo certbot certonly --nginx -d registry.mydomain.com
• Add to /etc/nginx/sites-enabled/registry:

  server {
    ...
  
    listen 443 ssl;
    listen [::]:443 ssl;
  
    ssl_certificate /etc/letsencrypt/live/registry.domain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/registry.domain.com/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!MD5;
    
    location / {
      ...
      
      proxy_pass                          http://localhost:7000;
      ...
    }
  }

Open postgresql access to the docker containers

• Add host all all 172.16.0.0/12 scram-sha-256 to /etc/postgresql/16/main/pg_hba.conf
• Add listen_addresses = '*' to /etc/postgresql/16/main/postgresql.conf

Open redis access to the docker containers

• Edit /etc/redis/redis.conf to set bind 0.0.0.0 and protected-mode no
• Allow access for docker containers sudo ufw allow proto tcp from 172.16.0.0/12 to any port 6379

Create a database

• sudo -u postgres createuser myapp1 --createdb --pwprompt --encrypted
• Update database.yml, add DB_HOST, DB_USERNAME, DB_PASSWORD, DB_PORT to your .env.erb file.

  production:
    <<: *default
    host: <%= ENV["DB_HOST"] %>
    database: myapp1_production
    username: <%= ENV["DB_USERNAME"] %>
    password: <%= ENV["DB_PASSWORD"] %>
    port: <%= ENV.fetch("DB_PORT") { 5432 } %>

Setup the traefik manager app with Kamal and Traefik

With the standard use of Kamal, a Traefik instance would be deployed for each app, which would end up with conflict between the Traefik instances. Instead, we are going to deploy an empty web app with a single Traefik instance, while Traefik will be disabled on the other apps.

• Create a new rails app (could also be sinatra or whatever) rails new traefik_manager --minimal --skip-active-record --skip-asset-pipeline --skip-test
• Run Docker locally
• Install Kamal. Follow: [https://kamal-deploy.org/docs/installation]
• Add your registry credentials in .env
• Run bin/kamal env push
• Create the file .kamal/hook/pre-deploy and make it executable chmod +x .kamal/hook/pre-deploy

#!/usr/bin/env bash

REMOTE_HOST="ubuntu@server.mydomain.com"
NETWORK_NAME="traefik-network"
# SSH into the remote host and execute Docker commands
ssh $REMOTE_HOST << EOF
 # Check if the Docker network already exists
 if ! docker network inspect "$NETWORK_NAME" &>/dev/null; then
     # If it doesn't exist, create it
     docker network create "$NETWORK_NAME"
     echo "Created Docker network: $NETWORK_NAME"
 else
     echo "Docker network $NETWORK_NAME already exists, skipping creation."
 fi
EOF

• Choose a unique port for your apps, use sudo lsof -i -P -n | grep LISTEN to check which ports are already in use. For this guide I'll choose 7001 and 7002.
• Update config/deploy.yml

# Name of your application. Used to uniquely configure containers.
service: traefik-manager

# Name of the container image.
image: n-studio/traefik-manager

# Deploy to these servers.
servers:
  web:
    hosts:
      - server.mydomain.com
    options:
      network: "traefik-network"

# Credentials for your image host.
registry:
  # Specify the registry server, if you're not using Docker Hub
  server: registry.mydomain.com
  username:
    - KAMAL_REGISTRY_USERNAME
  # Always use an access token rather than real password when possible.
  password:
    - KAMAL_REGISTRY_PASSWORD

# Inject ENV variables into containers (secrets come from .env).
# Remember to run `kamal env push` after making changes!
env:
  secret:
    - RAILS_MASTER_KEY

# Use a different ssh user than root
ssh:
  user: ubuntu

# Configure custom arguments for Traefik
traefik:
  publish: false
  options:
    network: "traefik-network"
    publish:
      - "7001:7001"
      - "7002:7002"
  args:
    entryPoints.web-7001.address: ':7001'
    entryPoints.web-7002.address: ':7002'

# Configure a custom healthcheck (default is /up on port 3000)
healthcheck:
  cmd: echo "OK"

• Run bin/kamal setup
• Run bin/kamal traefik reboot after any change in the traefik configuration

Install Kamal in your web apps

• Run Docker locally
• Follow: [https://kamal-deploy.org/docs/installation]
• Add database/redis credentials
• Run bin/kamal env push
• Disable traefik with traefik: false in deploy.yml
• Add labels to link the app with port 7001 or 7002 in deploy.yml

The deploy.yml file should look like this:

# Name of your application. Used to uniquely configure containers.
service: my-app-1

# Name of the container image.
image: n-studio/my-app-1

# Deploy to these servers.
servers:
  web:
    hosts:
      - server.mydomain.com
    options:
      network: "traefik-network"
    traefik: false

# Credentials for your image host.
registry:
  # Specify the registry server, if you're not using Docker Hub
  server: registry.mydomain.com

  # Always use an access token rather than real password when possible.
  username:
    - KAMAL_REGISTRY_USERNAME
  password:
    - KAMAL_REGISTRY_PASSWORD

# Inject ENV variables into containers (secrets come from .env).
# Remember to run `kamal env push` after making changes!
env:
  clear:
    DB_HOST: 172.17.0.1
    DB_PORT: 5432
    REDIS_URL: redis://172.17.0.1:6379/1 # give a unique db number for each app
  secret:
    - RAILS_MASTER_KEY
    - DB_USERNAME
    - DB_PASSWORD

# Use a different ssh user than root
ssh:
  user: ubuntu

labels:
  traefik.http.routers.my-app-1-web.rule: PathPrefix(`/`)
  traefik.http.routers.my-app-1-web.entrypoints: web-7001
  traefik.http.routers.my-app-1-web.priority: 10
  traefik.http.services.my-app-1-web.loadbalancer.server.port: 3000

• Note: if you use destination with Kamal (eg. staging), you might want to write: traefik.http.routers.my-app-1-web-staging.rule: PathPrefix(`/`)
• Run bin/kamal setup
• Repeat the same for your other app with a different port (7002)

Create SSL certificates

• Run sudo certbot certonly --nginx -d myapp1.mydomain.com
• Edit production.rb in your app to set config.assume_ssl = true

Install reverse proxy with nginx

server {
    server_name myapp1.mydomain.com;
    listen 80;
    listen [::]:80;

    ## redirect http to https ##
    rewrite ^ https://$server_name$request_uri? permanent;
}

server {
    server_name myapp1.mydomain.com;
    listen 443 ssl;
    listen [::]:443 ssl;

    ssl_certificate /etc/letsencrypt/live/myapp1.mydomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/myapp1.mydomain.com/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!MD5;

    location / {
        proxy_pass                          http://localhost:7001;
        proxy_set_header  Host              $http_host;   # required for docker client's sake
        proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
        proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header  X-Forwarded-Proto $scheme;
        proxy_set_header  X-Forwarded-Ssl   on;
        proxy_set_header  X-Forwarded-Port  $server_port;
        proxy_set_header  X-Forwarded-Host  $host;
        proxy_read_timeout                  900;
    }
    
    location /cable {
        proxy_pass                           http://localhost:7001/cable;
        proxy_http_version                   1.1;
        proxy_set_header   Upgrade           $http_upgrade;
        proxy_set_header   Connection        "Upgrade";
        proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header   Host              $http_host;
        proxy_set_header   X-Real-IP         $remote_addr;
        proxy_set_header   X-Forwarded-Proto https;
        proxy_redirect                       off;
    }
    
    location ~ ^/assets/ {
        proxy_pass               http://localhost:7001;
    
        expires                  1y;
        add_header Cache-Control public;
        add_header ETag          "";
    }
}

• Run sudo service nginx restart

Visit website

• Open https://app1.mydomain.com in your browser

Run console (for Rails)

• kamal app exec -i 'bin/rails console'