From 4f85b4cb85284193eb2be1c6021167ec02b43b00 Mon Sep 17 00:00:00 2001 From: Matthias Radestock Date: Wed, 10 Dec 2014 18:41:35 +0000 Subject: [PATCH 1/9] explain crypto and encapsulation --- site/how-it-works.md | 240 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 240 insertions(+) diff --git a/site/how-it-works.md b/site/how-it-works.md index dceb89d4..29f585cb 100644 --- a/site/how-it-works.md +++ b/site/how-it-works.md @@ -33,5 +33,245 @@ also establish UDP "connections", possibly encrypted, for the aforementioned packet forwarding. These "connections" are duplex and can traverse firewalls. +### Encapsulation + +When the weave router forwards packets, the encapsulation looks +something like this: + + +-----------------------------------+ + | IP headers | + +-----------------------------------+ + | UDP headers | + +-----------------------------------+ + | Name of sending peer | + +-----------------------------------+ + | Frame 1: Name of capturing peer | + +-----------------------------------+ + | Frame 1: Name of destination peer | + +-----------------------------------+ + | Frame 1: Captured payload length | + +-----------------------------------+ + | Frame 1: Captured payload | + +-----------------------------------+ + | Frame 2: Name of capturing peer | + +-----------------------------------+ + | Frame 2: Name of destination peer | + +-----------------------------------+ + | Frame 2: Captured payload length | + +-----------------------------------+ + | Frame 2: Captured payload | + +-----------------------------------+ + | ... | + +-----------------------------------+ + | Frame N: Name of capturing peer | + +-----------------------------------+ + | Frame N: Name of destination peer | + +-----------------------------------+ + | Frame N: Captured payload length | + +-----------------------------------+ + | Frame N: Captured payload | + +-----------------------------------+ + +The standard IP and UDP headers are followed the name of the sending +peer, which enables the receiving peer to identify who sent this UDP +packet. Next come the meta data and payload for one or more captured +frames. The router performs batching: if it captures several frames +very quickly that all need forwarding to the same peer, it will fit as +many of them as possible into a single UDP packet. + +The meta data for each frame contains the names of the capturing and +destination peers. Since the name of the capturing peer name is +associated with the source MAC of the captured payload, it allows +receivings peers to build up their mappings of which client MAC +addresses are local to which peers. The destination peer name enables +the receiving peer to identify whether this frame is destined for +itself or whether it should be forwarded on to some other peer, +accomodating multi-hop routing. This will work even when the receiving +intermediate peer has no knowledge of the destination MAC: only the +original capturing peer needs to determine the destination peer from +the MAC. This way weave peers never need to exchange the MAC addresses +of clients and need not take any special action for ARP traffic and +MACs discovery. + +### Crypto + +Weave encrypts traffic over both TCP and UDP between peers. Weave +uses the NaCl crypto libraries, using Curve25519, XSalsa20 and Poly1305 +to encrypt and authenticate messages. Weave protects against injection +and replay attacks for traffic forwarded between peers. + +NaCl was selected because of its good reputation both in terms of +selection and implementation of ciphers, but equally importantly, its +clear APIs and good documentation. It is quite difficult to use NaCl +incorrectly. Contrast this with libraries such as OpenSSL where the +size of the library and its APIs are vast, poorly documented, and +easily used wrongly. + +#### Establishing the Ephemeral Session Key + +When a weave peer has been started with the `-password` option, the +initial handshake between the peers over TCP includes the exchange of +the peers' public keys. Each peer generates a fresh public and private +key pair for each connection, and these are created from the NaCl +crypto library's `GenerateKey` function. Peers that were started with +`-password` do not continue with connection establishment unless they +receive a public key from the remote peer. Thus either all peers in a +weave network must be started with `-password`, or none of them. + +When a peer has received a public key from the remote peer, it uses +this to form the ephemeral session key for this connection. The public +key from the remote peer is combined with the private key for the +local peer in the usual Diffie-Hellman way, resulting in both peers +arriving at the same shared key. To this is appended the password from +the command line `-password` option, and the result is hashed through +SHA256, to form the final ephemeral session key. Thus the supplied +password is never exchanged directly, and is thoroughly mixed into the +shared secret. The shared key formed by Diffie-Hellman is 256 bits +long, appending the password to this obviously makes it longer by an +unknown amount, and the use of SHA256 reduces this back to 256 bits, +to form the final ephemeral session key. This late combination with +the password eliminates "Man In The Middle" attacks: sniffing the +public key exchange between the two peers and faking their responses +will not grant an attacker knowledge of the password, and so an +attacker would not be able to form valid ephemeral session keys. + +The same ephemeral session key is used for both TCP and UDP traffic +between two peers. + +#### TCP + +TCP connection are only used to exchange topology information between +peers, via a message-based protocol. The router generates a fresh +192-bit random nonce for every message to be sent, and prepends the +nonce to the encrypted message so that the receiver knows the +nonce. Encryption of each message is carried out using NaCl's +`secretbox.Seal` function using the ephemeral session key. Each TCP +connection has a monotonically incrementing message counter, the +current value of which is included in the encrypted part of the +message. Given the assumption that TCP is reliable and ordered, a +message received via TCP is only acted upon if the message counter in +the received message is the expected message counter. This prevents +replay attacks on the TCP connection. + +As TCP connections do not carry captured traffic, minimising message +size or latency is not a major concern, so the potentially substantial +increase in length of messages by prepending the full nonce, or the +cost of generating a fresh random nonce for each message is not +considered likely to cause problems. The random nonces are created by +crypto library's rand module, which implements a cryptographically +secure pseudorandom number generator. + +#### UDP + +UDP connection carry captured traffic between peers. For a UDP packet +sent between peers that are using crypto, the encapsulation looks as +follows: + + +-----------------------------------+ + | IP headers | + +-----------------------------------+ + | UDP headers | + +-----------------------------------+ + | Name of sending peer | + +-----------------------------------+ + | Nonce offset and flags | + +-----------------------------------+ + | NaCl SecretBox overheads | + +-----------------------------------+ -+ + | Frame 1: Name of capturing peer | | + +-----------------------------------+ | This section is encrypted + | Frame 1: Name of destination peer | | using the ephemeral session + +-----------------------------------+ | key between the weave peers + | Frame 1: Captured payload length | | sending and receiving this + +-----------------------------------+ | packet. + | Frame 1: Captured payload | | + +-----------------------------------+ | + | Frame 2: Name of capturing peer | | + +-----------------------------------+ | + | Frame 2: Name of destination peer | | + +-----------------------------------+ | + | Frame 2: Captured payload length | | + +-----------------------------------+ | + | Frame 2: Captured payload | | + +-----------------------------------+ | + | ... | | + +-----------------------------------+ | + | Frame N: Name of capturing peer | | + +-----------------------------------+ | + | Frame N: Name of destination peer | | + +-----------------------------------+ | + | Frame N: Captured payload length | | + +-----------------------------------+ | + | Frame N: Captured payload | | + +-----------------------------------+ -+ + +This is very similar to the [non-crypto encapsulation](#encapsulation). + +All of the frames are encrypted with the same ephemeral session key +and all must be decrypted by the receiving peer. Frames which are to +be forwarded on to some further peer will be re-encrypted with the +relevant ephemeral session keys for the onwards connections. Thus all +traffic is fully decrypted on every peer it passes through. +Encryption is again done with the NaCl `secretbox.Seal` function. + +The name of the sending peer enables the receiver to identify the peer +who sent this UDP packet, and in turn to determine which ephemeral +session key was used and which nonce, and perform decryption. To avoid +sending a fresh 192-bit nonce with every UDP packet, which would pose +an unacceptable overhead, each UDP packet only carries the lowest 15 +bits of the nonce, which is treated as an offset from the "established +nonce". The lifecycle of a nonce for UDP is: + +1. A fresh nonce is generated and the most significant 177 bits are +sent to the receiving peer over the TCP connection. These upper most +177 bits are used for the next 2^15 (32768) UDP messages. This is the +"established nonce". + +2. Each UDP message carries the lowest 15 bits as a unique counter. Thus +the lowest 15 bits are combined with (appended to) the uppermost 177 +bits to form the unique nonce for that message. + +3. Once the sending side has sent the 16384'th message on the current +nonce (50% of the way through the available range), it generates a new +nonce (upper 177 bits only) and sends that over the TCP connection, +thus hopefully ensuring it arrives and is ready before it is +needed. The sending side will switch to using the new nonce once the +full 32768 messages of the current nonce have been used. At this +point, the sending side resets the offset to 0. + +4. The receiving side must deal with the fact that UDP is unreliable +and unordered. The receiving side keeps track of the highest offset +seen for each established nonce. The condition to switching to the new +nonce it has received via the TCP connection is: the highest offset +seen for the current nonce must be above 24576 (75% of the available +range) _and_ the current message just received must have an offset +less than 8192 (25% of the available range) _and_ the new offset must +be less than 8192 messages *ahead* of the highest seen offset so far +(assuming modulo 32768). If these conditions are met, the highest +offset is set to the offset of the current message and the new nonce +is used. If those conditions are not met, the receiving side continues +with the current nonce (according to the rules below), updating the +highest offset seen as appropriate. Thus in order for the nonce to not +be updated incorrectly would require a loss of at least 8192 messages. + +5. When the receiving side switches to the new nonce, it does not +discard the old nonce. If the highest offset seen is below 8192 (25% of +the range) _and_ the current message offset is above 24576 (75% of the +range), _and_ the current message offset is less than 8192 behind the +highest seen nonce (assuming modulo 32768), then the current message is +decoded with the old nonce. + +6. In the remaining cases, the current message is only decoded if its +offset is within 8192 either side of the highest seen offset. + +7. To avoid replay attacks, the receiving side keeps a set of which +offsets have been used with the current and previous nonce. If the +offset doesn't exist in the set, and the message can be correctly +decoded, the offset is added to the relevant set (thus we avoid +poisoning attacks). If the offset already exists in the set, or the +message cannot be correctly decoded, the message is not processed +further. + +### Further reading More details on the inner workings of weave can be found in the [architecture documentation](https://github.com/zettio/weave/blob/master/docs/architecture.txt). From 2a1316553c4ed2b874f0677576f42db75cda918f Mon Sep 17 00:00:00 2001 From: Matthias Radestock Date: Fri, 12 Dec 2014 17:44:23 +0000 Subject: [PATCH 2/9] simplify encapsulation diagrams by dropping the standard packet headers --- site/how-it-works.md | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/site/how-it-works.md b/site/how-it-works.md index 29f585cb..535f7a75 100644 --- a/site/how-it-works.md +++ b/site/how-it-works.md @@ -38,10 +38,6 @@ can traverse firewalls. When the weave router forwards packets, the encapsulation looks something like this: - +-----------------------------------+ - | IP headers | - +-----------------------------------+ - | UDP headers | +-----------------------------------+ | Name of sending peer | +-----------------------------------+ @@ -72,12 +68,12 @@ something like this: | Frame N: Captured payload | +-----------------------------------+ -The standard IP and UDP headers are followed the name of the sending -peer, which enables the receiving peer to identify who sent this UDP -packet. Next come the meta data and payload for one or more captured -frames. The router performs batching: if it captures several frames -very quickly that all need forwarding to the same peer, it will fit as -many of them as possible into a single UDP packet. +The name of the sending peer enables the receiving peer to identify +who sent this UDP packet. This is followed by the meta data and +payload for one or more captured frames. The router performs batching: +if it captures several frames very quickly that all need forwarding to +the same peer, it will fit as many of them as possible into a single +UDP packet. The meta data for each frame contains the names of the capturing and destination peers. Since the name of the capturing peer name is @@ -167,10 +163,6 @@ UDP connection carry captured traffic between peers. For a UDP packet sent between peers that are using crypto, the encapsulation looks as follows: - +-----------------------------------+ - | IP headers | - +-----------------------------------+ - | UDP headers | +-----------------------------------+ | Name of sending peer | +-----------------------------------+ From f99fd9b8fb3382d49c1562c4072dc1ed6f9f456d Mon Sep 17 00:00:00 2001 From: Matthias Radestock Date: Fri, 12 Dec 2014 18:29:15 +0000 Subject: [PATCH 3/9] some rewording and some links --- site/how-it-works.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/site/how-it-works.md b/site/how-it-works.md index c67fad48..7abb314b 100644 --- a/site/how-it-works.md +++ b/site/how-it-works.md @@ -97,17 +97,20 @@ MACs discovery. ### Crypto -Weave encrypts traffic over both TCP and UDP between peers. Weave -uses the NaCl crypto libraries, using Curve25519, XSalsa20 and Poly1305 -to encrypt and authenticate messages. Weave protects against injection -and replay attacks for traffic forwarded between peers. +Weave can be configures to encrypt both the data passing over the TCP +connections and the payloads of UDP packets sent between peers. This +is accomplished using the [NaCl](http://nacl.cr.yp.to/) crypto +libraries, employing Curve25519, XSalsa20 and Poly1305 to encrypt and +authenticate messages. Weave protects against injection and replay +attacks for traffic forwarded between peers. NaCl was selected because of its good reputation both in terms of selection and implementation of ciphers, but equally importantly, its -clear APIs and good documentation. It is quite difficult to use NaCl -incorrectly. Contrast this with libraries such as OpenSSL where the -size of the library and its APIs are vast, poorly documented, and -easily used wrongly. +clear APIs, good documentation and high-quality +[go implementation](https://godoc.org/golang.org/x/crypto/nacl). It is +quite difficult to use NaCl incorrectly. Contrast this with libraries +such as OpenSSL where the library and its APIs are vast in size, +poorly documented, and easily used wrongly. #### Establishing the Ephemeral Session Key From b7d129b353c6955cc4a7629eb91cc638285f7fd6 Mon Sep 17 00:00:00 2001 From: Matthias Radestock Date: Fri, 12 Dec 2014 18:41:40 +0000 Subject: [PATCH 4/9] fix typo --- site/how-it-works.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site/how-it-works.md b/site/how-it-works.md index 7abb314b..61afc2a6 100644 --- a/site/how-it-works.md +++ b/site/how-it-works.md @@ -97,7 +97,7 @@ MACs discovery. ### Crypto -Weave can be configures to encrypt both the data passing over the TCP +Weave can be configured to encrypt both the data passing over the TCP connections and the payloads of UDP packets sent between peers. This is accomplished using the [NaCl](http://nacl.cr.yp.to/) crypto libraries, employing Curve25519, XSalsa20 and Poly1305 to encrypt and From ee19e3971289c602ac4b429154564663b4d8c92f Mon Sep 17 00:00:00 2001 From: Matthias Radestock Date: Fri, 12 Dec 2014 18:45:36 +0000 Subject: [PATCH 5/9] make it clear that key pairs are created per connection --- site/how-it-works.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/site/how-it-works.md b/site/how-it-works.md index 61afc2a6..d8cc8356 100644 --- a/site/how-it-works.md +++ b/site/how-it-works.md @@ -114,14 +114,13 @@ poorly documented, and easily used wrongly. #### Establishing the Ephemeral Session Key -When a weave peer has been started with the `-password` option, the -initial handshake between the peers over TCP includes the exchange of -the peers' public keys. Each peer generates a fresh public and private -key pair for each connection, and these are created from the NaCl -crypto library's `GenerateKey` function. Peers that were started with -`-password` do not continue with connection establishment unless they -receive a public key from the remote peer. Thus either all peers in a -weave network must be started with `-password`, or none of them. +For every connection between peers, a fresh public/private key pair is +created at both ends, using NaCl's `GenerateKey` function, and the +public key portion is sent to the other end as part of the initial +handshake performed over TCP. Peers that were started with `-password` +do not continue with connection establishment unless they receive a +public key from the remote peer. Thus either all peers in a weave +network must be started with `-password`, or none of them. When a peer has received a public key from the remote peer, it uses this to form the ephemeral session key for this connection. The public From 5ab6d7bd1fc04d5cd70f16eb8e511669fa8fc1a9 Mon Sep 17 00:00:00 2001 From: Matthias Radestock Date: Fri, 12 Dec 2014 18:46:21 +0000 Subject: [PATCH 6/9] cosmetic --- site/how-it-works.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/site/how-it-works.md b/site/how-it-works.md index d8cc8356..5319b562 100644 --- a/site/how-it-works.md +++ b/site/how-it-works.md @@ -115,12 +115,12 @@ poorly documented, and easily used wrongly. #### Establishing the Ephemeral Session Key For every connection between peers, a fresh public/private key pair is -created at both ends, using NaCl's `GenerateKey` function, and the -public key portion is sent to the other end as part of the initial -handshake performed over TCP. Peers that were started with `-password` -do not continue with connection establishment unless they receive a -public key from the remote peer. Thus either all peers in a weave -network must be started with `-password`, or none of them. +created at both ends, using NaCl's `GenerateKey` function. The public +key portion is sent to the other end as part of the initial handshake +performed over TCP. Peers that were started with `-password` do not +continue with connection establishment unless they receive a public +key from the remote peer. Thus either all peers in a weave network +must be started with `-password`, or none of them. When a peer has received a public key from the remote peer, it uses this to form the ephemeral session key for this connection. The public From 6b741bef27e4e0080d33c29466a39f9a6f1123c7 Mon Sep 17 00:00:00 2001 From: Matthias Radestock Date: Fri, 12 Dec 2014 18:50:09 +0000 Subject: [PATCH 7/9] state that nonce prepending is normal in NaCl --- site/how-it-works.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/site/how-it-works.md b/site/how-it-works.md index 5319b562..b5afa7e1 100644 --- a/site/how-it-works.md +++ b/site/how-it-works.md @@ -147,14 +147,14 @@ between two peers. TCP connection are only used to exchange topology information between peers, via a message-based protocol. The router generates a fresh 192-bit random nonce for every message to be sent, and prepends the -nonce to the encrypted message so that the receiver knows the -nonce. Encryption of each message is carried out using NaCl's -`secretbox.Seal` function using the ephemeral session key. Each TCP -connection has a monotonically incrementing message counter, the -current value of which is included in the encrypted part of the -message. Given the assumption that TCP is reliable and ordered, a -message received via TCP is only acted upon if the message counter in -the received message is the expected message counter. This prevents +nonce to the encrypted message, as is normal in NaCl, so that the +receiver knows the nonce. Encryption of each message is carried out +using NaCl's `secretbox.Seal` function using the ephemeral session +key. Each TCP connection has a monotonically incrementing message +counter, the current value of which is included in the encrypted part +of the message. Given the assumption that TCP is reliable and ordered, +a message received via TCP is only acted upon if the message counter +in the received message is the expected message counter. This prevents replay attacks on the TCP connection. As TCP connections do not carry captured traffic, minimising message From b1bde561bd4389ae4340d384404400434ffecb65 Mon Sep 17 00:00:00 2001 From: Matthias Radestock Date: Sat, 13 Dec 2014 10:06:40 +0000 Subject: [PATCH 8/9] fix typo --- site/how-it-works.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site/how-it-works.md b/site/how-it-works.md index b5afa7e1..1073d409 100644 --- a/site/how-it-works.md +++ b/site/how-it-works.md @@ -252,7 +252,7 @@ offset is set to the offset of the current message and the new nonce is used. If those conditions are not met, the receiving side continues with the current nonce (according to the rules below), updating the highest offset seen as appropriate. Thus in order for the nonce to not -be updated incorrectly would require a loss of at least 8192 messages. +be updated correctly would require a loss of at least 8192 messages. 5. When the receiving side switches to the new nonce, it does not discard the old nonce. If the highest offset seen is below 8192 (25% of From 9f25726bc28c685c1e7587f569242e2fbebdb03b Mon Sep 17 00:00:00 2001 From: David Wragg Date: Fri, 12 Dec 2014 16:59:38 +0000 Subject: [PATCH 9/9] Typos --- site/how-it-works.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/site/how-it-works.md b/site/how-it-works.md index 1073d409..27723f52 100644 --- a/site/how-it-works.md +++ b/site/how-it-works.md @@ -84,7 +84,7 @@ UDP packet. The meta data for each frame contains the names of the capturing and destination peers. Since the name of the capturing peer name is associated with the source MAC of the captured payload, it allows -receivings peers to build up their mappings of which client MAC +receiving peers to build up their mappings of which client MAC addresses are local to which peers. The destination peer name enables the receiving peer to identify whether this frame is destined for itself or whether it should be forwarded on to some other peer, @@ -93,7 +93,7 @@ intermediate peer has no knowledge of the destination MAC: only the original capturing peer needs to determine the destination peer from the MAC. This way weave peers never need to exchange the MAC addresses of clients and need not take any special action for ARP traffic and -MACs discovery. +MAC discovery. ### Crypto @@ -162,12 +162,12 @@ size or latency is not a major concern, so the potentially substantial increase in length of messages by prepending the full nonce, or the cost of generating a fresh random nonce for each message is not considered likely to cause problems. The random nonces are created by -crypto library's rand module, which implements a cryptographically +the go `crypto/rand` package, which implements a cryptographically secure pseudorandom number generator. #### UDP -UDP connection carry captured traffic between peers. For a UDP packet +UDP connections carry captured traffic between peers. For a UDP packet sent between peers that are using crypto, the encapsulation looks as follows: