The goal of this article is to explain how to install one or many Raspberry Pi 3 B+ devices connected with one to four Raspberry Pi Zero 1.3 devices each via USB OTG networking.
Each device - Pi and Zero - will be connected to the network 192.168.0.0/24
. Each Zero will be connected via USB OTG to the Pi. That will establish a network between the two devices. On each Pi a network bridge will be configured to forward the traffic between the Pi ethernet port and the connected USB ports.
Their individual hostname/IP pattern will be:
10.lab | 192.168.0.10 (1st Pi)
20.lab | 192.168.0.20 (2nd Pi)
11.lab | 192.168.0.11 (1st Zero on 1st Pi)
12.lab | 192.168.0.12 (2nd Zero on 1st Pi)
22.lab | 192.168.0.22 (2nd Zero on 2nd Pi)
And so on. Here is a small diagram that shows the setup for one Pi:
┌────────┐ 192.168.0.11
│ Zero 1 │ 00:22:82:ff:ff:01 usb0 ──────────────┐
└────────┘ 11.lab │
│ ┌──────────── br0 ────────────┐
┌────────┐ 192.168.0.12 │
│ Zero 2 │ 00:22:82:ff:ff:02 usb0 ──────┐ │
└────────┘ 12.lab │ ethusb1 00:22:82:ff:ff:11 ┌────────┐ 192.168.0.10
└── ethusb2 00:22:82:ff:ff:12 │ Pi 1 │ eth0 ────────────────────
┌────────┐ 192.168.0.13 ┌── ethusb3 00:22:82:ff:ff:13 └────────┘ 10.lab
│ Zero 3 │ 00:22:82:ff:ff:03 usb0 ──────┘ ethusb4 00:22:82:ff:ff:14
└────────┘ 13.lab │
│
┌────────┐ 192.168.0.14 │
│ Zero 4 │ 00:22:82:ff:ff:04 usb0 ──────────────┘
└────────┘ 14.lab
On all SD cards - for the Pi and the Zero - perform the following steps before booting the devices:
Connect an empty SD card (at least 2GB). Figure out the device e.g. via watching the dmesg -w
on your Linux host output while inserting it.
Assuming that your SD card device is /dev/sdb
you can now perform the following step do download the Raspbian Lite image and burn it to your SD card:
wget -O - https://downloads.raspberrypi.org/raspbian_lite_latest | dd of=/dev/sdb bs=4096
That command may take several minutes. After that your SD card has two partitions: One for /boot
and one for /
.
Mount the /boot
partition with mkdir -p /media/boot && mount /dev/sdb1 /media/boot
Then enable the SSH daemon by createing an empty ssh
file on the /boot
partition by typing touch /boot/ssh
.
The following instructions should only be executed on the SD cards for the Zero before booting.
The dwc2 and g_ether kernel modules are required to be loaded for this setup. Therefore change the file /boot/cmdline.txt
from:
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
to:
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait modules-load=dwc2,g_ether g_ether.host_addr=00:22:82:ff:ff:01 g_ether.dev_addr=00:22:82:ff:ff:11 quiet
Each OTG networking connection requires both MAC address setups. One address for the Zero (host) and a second one for the Pi (dev).
The pattern is:
host_addr=00:22:82:ff:ff:01 dev_addr=00:22:82:ff:ff:11 (1st Zero)
host_addr=00:22:82:ff:ff:02 dev_addr=00:22:82:ff:ff:12 (2nd Zero)
host_addr=00:22:82:ff:ff:03 dev_addr=00:22:82:ff:ff:13 (3rd Zero)
host_addr=00:22:82:ff:ff:04 dev_addr=00:22:82:ff:ff:14 (4th Zero)
Now the modules need to be activated. To enable the OTG overlay and provide a bit more memory edit the /boot/config.txt
file and add at the end of the file:
dtoverlay=dwc2
gpu_mem=16
To enable ssh add an empty file to the /boot
folder:
touch /boot/ssh
Configure the IP address statically by editing the file /etc/dhcpcd.conf
:
interface usb0
static ip_address=192.168.0.11/24
static routers=192.168.0.10
static domain_name_servers=9.9.9.9 1.1.1.1
The following instructions are required only to be done on the Raspberrey Pi, not the Zero.
We require network packet forwarding between the ethernet port and the USB OTG ports. As it is disabled by default we need to enable it by typing sysctl -w net.ipv4.ip_forward=1
. You might want to add this property to the /etc/sysctl.conf
file as well to keep it persistent (run sudo sysctl -p /etc/sysctl.conf
afterwards).
Check also iptables for blocking forwarding rules with iptables -S
and iptables -t filter -L
.
Adjust iptables to accept forwarding. By default it is set to DROP:
iptables -P FORWARD ACCEPT
iptables-save
Now we need to give static device names to USB OTG network devices. Sometimes they would show up as ethX, sometimes as usbX. We don't want them to toggle so we fix their names by creating a /etc/udev/rules.d/90-pi-network.rules
file with the following content:
SUBSYSTEM=="net", ATTR{address}=="00:22:82:ff:ff:01", NAME="ethusb1"
SUBSYSTEM=="net", ATTR{address}=="00:22:82:ff:ff:02", NAME="ethusb2"
SUBSYSTEM=="net", ATTR{address}=="00:22:82:ff:ff:03", NAME="ethusb3"
SUBSYSTEM=="net", ATTR{address}=="00:22:82:ff:ff:04", NAME="ethusb4"
The USB network devices will be named ethusbX. The MAC addresses will be explicitly defined by setting up the Zeros in the next chapter.
Each USB OTG network requires two MAC addresses. One for each side. The USB OTG host (Zero) and the connected device (Pi). The link between the 1st Zero and the Pi will 00:22:82:ff:ff:11
on the Zero side and 00:22:82:ff:ff:01
on the Pi side. The second one 00:22:82:ff:ff:12
/00:22:82:ff:ff:02
etc.
Now we need to setup the bridge on the Raspberry Pi. All the connected USB network devices together with eth0 need to be added.
The first step is to make sure that the /etc/network/interfaces
file contains the following line:
source /etc/network/interfaces.d/*
Some older versions contain a different syntax which won't work any more.
Then we add our bridge configuration to the file /etc/network/interfaces.d/bridge.conf
:
auto eth0
auto ethusb1
auto ethusb2
auto ethusb3
auto ethusb4
iface eth0 inet manual
iface ethusb1 inet manual
iface ethusb2 inet manual
iface ethusb3 inet manual
iface ethusb4 inet manual
auto br0
iface br0 inet dhcp
bridge_ports eth0 ethusb1 ethusb2 ethusb3 ethusb4
bridge_stp off
bridge_fd 0
bridge_maxage 1
In addition we need to modify the dhcpcd configuration to ignore the devices and to setup a static IP configuration for the bridge. This is done in the /etc/dhcpcd.conf
file:
denyinterfaces eth0 ethusb1 ethusb2 ethusb3 ethusb4
interface br0
static ip_address=192.168.0.10/24
static routers=192.168.0.1
static domain_name_servers=9.9.9.9 1.1.1.1
Now the bridge is configured. We just need to change the host name in /etc/hostname
to 10
(or 20, 30 according to the pattern described above).
The hostname should also be added to the /etc/hosts
file like:
127.0.1.1 10.lab 10
Now you can boot your devices. On all of them - Pi and Zero - you should perform the following steps:
To expand the filesystem to the full size of the SD card execute the raspi-config
command. A wizard shows up. Go to advanced settings and there you will find an option to expand the filesystem to the full SD card size.
Execute passwd
to set an individual password.
Since each Raspbian image has pre-configured SSH host keys they need to be re-created individually. Therefore execute the following:
rm /etc/ssh/ssh_host_*
dpkg-reconfigure openssh-server
In addition you might want to add your SSH public key to /home/pi/.ssh/authorized_keys
to enable logins via public key.
In case you don't want to use IPv6 you can turn it off with sysctl -w net.ipv6.conf.all.disable_ipv6=1
. You might want to add this property to the /etc/sysctl.conf
file as well to keep it persistent (run sudo sysctl -p /etc/sysctl.conf
afterwards).
To watch the OTG network devices becoming ready type the following command on the Pi:
dmesg -ew
To get more information about the configured bridge install the bridge-utils
and have a look at its current state:
apt install bridge-utils
brctl show br0
brctl showstn br0
brctl showmac br0
ip a
ip r
This document is based on the Raspberry Pi OTG network article. That describes a similar setup but with a NAT network configuration instead of a bridged one.
Also an interesting read: Build your PiZero Swarm with OTG networking
You might want to install your own DNS server like dnsmasq to resolve hosts by the pattern described above e.g. 10.lab or 21.lab.
A very easy solution is to install a pi-hole
on one one Pi. That comes with a dnsmasq
tool which you can configure to resolve your hosts. More information: https://pi-hole.net/