- Distributed Peer Table (DPT) / Node Discovery
- RLPx Transport Protocol
- Ethereum Wire Protocol (ETH) / Application Layer
- v4 is currently the most widely used - (besides LES which uses v5)
- v5 adoption is coming soon (check felix lange's devcon video below)
- This summary based on geth implementation (~ 70% of ethereum nodes)
- Similar to Kademlia - except in ethereum p2p network is only used to discover new peers (vs. traditionally for sharing content) (though this might change in eth once sharding comes about)
- Content (value) associated with a key
- Stored only at peers whose nodeId is 'close' to associated peer
- Closeness given by
nodeId1
XORnodeId2
- Each node has
b
distinct buckets - Each bucket stores entwork information about
k
peers at a distancei
lookup(t)
- t=target- Node finds nodeIds in her bucket closest to
t
- Send request for
t
or nodeIds in requested bucket that are closer - Repeat until
t
is found
- Node finds nodeIds in her bucket closest to
- Public key (64byte cryptographic ECDSA public key)
- !NOT! tied to a unique network address (i.e. easily generate many nodeIds from one machine)
-
UDP
- Only used to exchange information about the peer-to-peer network
- No limit on connections except max 16 concurrent connections
- 4 types of messages
ping
solicitspong
in returnfindnode
solicitsneighbor
neighbor
returns 16 nodes closer tot
that have been seen by the responder- Node will ONLY respond to
findnode
if querying node is already indb
- All UDP messages are timestamped and authentiacted w/ senders nodeId (i.e.public key)
- To prevent replay attacks - UDP messages older than 20 seconds (relative to local time) are dropped
pong
s also contain hash ofping
it's responding to to prevent ip spoofing
-
TCP
- Used to exchange all blockchain information
- Encrypted and authenticated
- Total number of TCP connections at any time is
maxpeers
(default = 25) - TO ECLIPSE A NODE - ATTACKER CONTINUOUSLY OCCUPIES ALL OF THESE CONNECTIONS
outgoing
if initiated by client //incoming
otherwise- Client can initiate MAX (1/2 *
maxpeers
) (default = 13)
db
- Used for long-term storage of network information (i.e. nodes the client has seen)
- Seen = client received a valid
pong
from node after sending validping
- Seen = client received a valid
- Stored on disk // persists across client reboots
db
entry- nodeid
- IP address
- TCP port
- UDP port
- Time of last ping sent to node
- Time of last pong received from node
- Number of times node has failed to respond to
findnode
- Node's age = time elapsed since receiving
pong
from that node - Every hour client evicts all nodes from
db
older than 1 day
- Used for long-term storage of network information (i.e. nodes the client has seen)
table
- Used to select peers (i.e. outgoing tcp connections)
- Short-term database
- ALWAYS EMPTY WHEN THE CLIENT REBOOTS
- Contains 256 buckets
- Each bucket can hold up to 16 entries
- Entry Info
- nodeId
- IP address
- TCP port
- UDP port
- Entries sorted in order in which they were added to bucket
- If new entry maps to full bucket
- Oldest node in bucket is
ping
ed - If node fails to respond with
pong
- Node is kicked out of bucket
- If node responds with
pong
- New entry is not added
- Oldest node in bucket is
- Nodes are removed from
table
if they fail to respond tofindnode
more than 4 times - Map (bucket => nodeId) via
logdist()
- This mapping is PUBLIC!!
- Easy for adversary to predict which nodeId will map to which bucket in victims table
- This mapping is PUBLIC!!
logdist()
(ethereum's modification of XOR)- Measures the distance between two nodeIds
logdist(nodeIdA, nodeIdB) = r
dA = SHA3(nodeIdA)
dB = SHA3(nodeIdB)
r
= # of most significant bits b/wdA
anddB
that are the same- i.e.
r + 1
bit ofdA
anddB
are different
- i.e.
- nodeId with
logdist() = r
from client is mapped to bucket256 - r
in the client'stable
logdist()
results in skewed mapping of nodeIds to buckets- ~ 1/2 of all nodes map to bucket 256
- ~ 1/4 of all nodes map to bucket 255
- ~ 1/8 of all nodes map to bucket 254
- ....
table
has capacity of 256 * 16 = 4096 nodes- BUT a client with ~ all ethereum nodes will only store ~ 168 nodes in it's table
- Bootstrap nodes
- When a client is booted up for the first time - it's
db
is empty & only knows about 6 hardcoded bootstrap nodes
- When a client is booted up for the first time - it's
- Bonding
- Used to populate
db
andtable
- When client bods with a node - checks
- Node exists in his
db
db
records 0 failed responses tofindnode
requestsdb
records that node has responded topong
in last 24 hours
- Node exists in his
- If ^ checks pass - client immediately tries to add node to
table
(successfully if there's space intable
) - Client sends
ping
to node - Successful bonding == node responds with
pong
- If successful - add node's entry to
db
and try to add totable
- If successful - add node's entry to
- Used to populate
- Unsolicited pings
- Client receives unsolicited
ping
responds withpong
and bonds to node
- Client receives unsolicited
lookup(t)
- Closeness to a target
t
(256-bit string)dA = SHA3(nodeIdA) XOR t
dB = SHA3(nodeIdB) XOR t
- nodeIdA is closer to t if
dA < dB
and vice versa
- 1st. 16 nodes closest to
t
in client'stable
are selected - 2nd. query each of those 16 nodes with
findnode
msg that containst
- 3rd. upon receiving
findnode
msg - other node returns 16 nodes closest tot
from it's own table in aneighbor
msg - 4th. orig. client combines original nodes and all new nodes returned via
neighbor
msg - 5th. from up to (16 * 16) nodes - selects 16 nodes closest to
t
- Repeat from step 2
- Continue repeating 2-5 until set of 16 closest node stabilizes (doesnt change) b/w iterations
- Add the finalized 16 nodes to
lookup_buffer
(FIFO queue) - If a node fails to respond tofindnode
request 5 times in a row - it's evicted fromtable
.
- Closeness to a target
- Seeding
- Triggered when
- Node reboots
- Every hour
lookup()
called on emptytable
- 1st. checks if
table
is non-empty- if yes - nothing happens
- 2nd. client bonds to each of the 6 bootstrap nodes
- 3rd. bonds to
min(30, l)
seed nodes randomly selected fromdb
and younger than 5 daysl
= number of nodes indb
younger than 5 days
- 4th. client runs
lookup(self)
- Intended to populate other nodes' buckets with the client's newly-online nodeId
self
=SHA3(nodeId)
with client's own nodeId
- Triggered when
- ~ 1/2 selected from
lookup_buffer
(result oflookup()
) - ~ 1/2 selected from
table
- When client starts - task runner is started and run continuously
- Task runner
- Populates client's
db
andtable
- Creates up to
1/2(1 + maxpeers)
(default=13) outgoing TCP connections - Has MAX 16 concurrent tasks and a task queue
- If < 16 tasks && task queue is empty => new tasks are created via task-creation algorithm (details below)
- Two types of tasks
discover_task
- Call of
lookup(t)
w/t
= 256-bit target string
- Call of
dial_task
- Attempt to make new TCP connection (aka dial) another node
- Pre-dial checks - target node is . . .
- Not currently being dailed
- Not already a connected peer
- Not itself
- Not blacklisted
- Not recently dialed
- Populates client's
- Resolving an unknown IP
- Usually client knows ip address associated with nodeId
- Except 2 cases . . .
- nodeId statically configured by user w/o ip address
- Ip address field is empty
- If unknown IP - call
lookup(t)
- Task-Creation Algorithm
- Initialize
x = [1/2(1 + maxpeers)]
(default=13) - Decrement
x
for each outgoing peer connection currently connected || currently being dialed - If client has no peers && 20+ seconds since client restarted => select bootstrap node
- If bootstrap node passes 5
dial_task
pre-checks => decrementx
and createdial_task
to boostrap node
- If bootstrap node passes 5
- Fill
random_buffer
with1/2[1/2(1 + maxpeers)]]
(default=6) random nodes fromtable
- If
random_buffer
is not empty - overwrite what's there - Select first
1/2(x)
nodes in buffer - decrement x - and createdial_task
for all nodes that pass 5 pre-checks
- If
- While
x > 0
- Remove first node in
lookup_buffer
- and createdial_task
if it passes checks - If
len(lookup_buffer) < x
andlookup()
is not running => creatediscover_task
- Remove first node in
- Initialize
- p2p wiki
- Kademlia paper - Maymounkov
- Intro to Kademlia
- Kademlia DHT python implementation
- Queues don't fix overload
- How eth protocols map to OSI
- RLP encoding
- EthereumJS RLP Encoding Implementation
- Node Discovery Protocol
- Devp2p Wire Protocol
- Devp2p White-Paper
- Light Client Protocol
- described above