Skip to content

Instantly share code, notes, and snippets.

@acogoluegnes
Last active May 4, 2021 08:28
Show Gist options
  • Save acogoluegnes/7aeddc72b4de0d31d7122916e30c391b to your computer and use it in GitHub Desktop.
Save acogoluegnes/7aeddc72b4de0d31d7122916e30c391b to your computer and use it in GitHub Desktop.
RabbitMQ and load balancing

Set up a working directory:

cd /tmp
rm -rf stream-load-balancer
mkdir stream-load-balancer
cd stream-load-balancer

docker-compose configuration to create 3-node RabbitMQ cluster with haproxy in front of it:

echo 'version: "3.9"
services:
  rabbitmq-1:
    image: pivotalrabbitmq/rabbitmq-stream
    hostname: rabbitmq-1
    ports:
      - 15672:15672
    volumes:
      - "./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf"
    environment:
      - RABBITMQ_ERLANG_COOKIE=dummy

  rabbitmq-2:
    image: pivotalrabbitmq/rabbitmq-stream
    hostname: rabbitmq-2
    volumes:
      - "./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf"
    ports:
      - 15673:15672
    environment:
      - RABBITMQ_ERLANG_COOKIE=dummy

  rabbitmq-3:
    image: pivotalrabbitmq/rabbitmq-stream
    hostname: rabbitmq-3
    volumes:
      - "./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf"
    ports:
      - 15674:15672
    environment:
      - RABBITMQ_ERLANG_COOKIE=dummy

  load-balancer:
    image: haproxy:2.3
    volumes:
      - "./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro"
    ports:
      - 5555:5555
      - 8100:8100' > docker-compose.yml

haproxy configuration to load balance connections to the stream plugin:

echo 'global
        log 127.0.0.1   local0 info
        maxconn 4096
        stats socket /tmp/haproxy.socket uid haproxy mode 770 level admin

defaults
        log     global
        mode    tcp
        option  tcplog
        option  dontlognull
        retries 3
        option redispatch
        maxconn 2000
        timeout connect 5s
        timeout client 120s
        timeout server 120s

listen stream_front
        bind :5555
        mode tcp
        balance roundrobin
        server rabbitmq-1 rabbitmq-1:5551
        server rabbitmq-2 rabbitmq-2:5551
        server rabbitmq-3 rabbitmq-3:5551

listen  stats
        bind :8100
        mode http
        option httplog
        stats enable
        stats uri       /stats
        stats refresh 5s' > haproxy.cfg

RabbitMQ configuration:

echo 'loopback_users = none
default_user = guest
default_pass = guest
cluster_formation.peer_discovery_backend = classic_config
cluster_formation.classic_config.nodes.1 = rabbit@rabbitmq-1
cluster_formation.classic_config.nodes.2 = rabbit@rabbitmq-2
cluster_formation.classic_config.nodes.3 = rabbit@rabbitmq-3' > rabbitmq.conf

Make sure to delete the existing image of RabbitMQ stream you can have locally:

docker rmi pivotalrabbitmq/rabbitmq-stream

Start the cluster:

docker-compose up

Go to localhost:15672, connect with guest/guest and check the cluster is up (all this can take about a minute or 2)

In another terminal tab, set up the stream performance tool:

cd /tmp/stream-load-balancer
wget https://github.com/rabbitmq/rabbitmq-java-tools-binaries-dev/releases/download/v-stream-perf-test-latest/stream-perf-test-latest.jar
java -jar stream-perf-test-latest.jar --rate 10000 --uris rabbitmq-stream://localhost:5555 --load-balancer

The tool should start properly. Check at http://localhost:15672/#/stream/connections that stream connections are spread across 2 nodes at least.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment