From 5960ecb3d1064673a6618fa9a314d4c52d96648b Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Wed, 17 Jun 2015 11:12:10 +0100 Subject: [PATCH 1/2] Expand description of -initpeercount --- site/ipam.md | 46 +++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 43 insertions(+), 3 deletions(-) diff --git a/site/ipam.md b/site/ipam.md index 7e3ea17a..05a0457b 100644 --- a/site/ipam.md +++ b/site/ipam.md @@ -43,9 +43,7 @@ order to avoid formation of isolated groups, which could lead to inconsistency, i.e. the same IP address being allocated to two different containers. Therefore, you must either supply the list of all peers in the network to `weave launch` or add the `-initpeercount` -flag to specify how many peers there will be. It isn't a problem to -over-estimate by a bit, but if you supply a number that is too small -then multiple independent groups may form. +flag to specify how many peers there will be. To illustrate, suppose you have three hosts, accessible to each other as `$HOST1`, `$HOST2` and `$HOST3`. You can start weave on those three @@ -66,6 +64,48 @@ time, you can give the number of peers like this: host3$ weave launch -initpeercount 3 $HOST2 +### More on `-initpeercount` + +TL;DR: it isn't a problem to over-estimate by a bit, but if you supply +a number that is too small then multiple independent groups may form. + +Weave uses the estimate of the number of peers at initialization to +compute a majority or quorum number - specifically floor(n/2) + 1. So, +if the actual number of peers is less than half the number stated then +they will keep waiting for someone else to join to reach a quorum. On +the other hand, if the actual number is more than twice the quorum +number then you could have two sets of peers each reach a quorum and +initialize independent data structures. You'd have to be quite unlucky +for that to happen in practice, as they have to go through the whole +agreement process without learning about each other, but it's +definitely possible. + +The quorum number is only used once at start-up (specifically, the +first time someone tries to allocate or claim an IP address), so once +a set of peers is initialized you can add more and they will join on +to the data structure used by the existing set. The one thing you +have to watch is if the early ones get restarted, you must restart +them with the current number of peers - if they use the smaller number +that was correct when they first started then they could form an +independent set again. + +To illustrate this last point, the following sequence of operations +would be safe wrt weave's startup quorum: + + host1$ weave launch + +[time passes] + + host2$ weave launch $HOST1 + +[time passes] + + host3$ weave launch $HOST1 $HOST2 + +[time passes; host1 is rebooted] + + host1$ weave launch $HOST2 $HOST3 + ## Choosing an allocation range By default, weave will allocate IP addresses in the 10.128.0.0/10 From 40efc347b61d0cdc6457e32ee0b547181a29cfec Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Wed, 17 Jun 2015 11:41:46 +0100 Subject: [PATCH 2/2] Cosmetic --- site/ipam.md | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/site/ipam.md b/site/ipam.md index 05a0457b..f4eeee7c 100644 --- a/site/ipam.md +++ b/site/ipam.md @@ -93,17 +93,12 @@ To illustrate this last point, the following sequence of operations would be safe wrt weave's startup quorum: host1$ weave launch - -[time passes] - + ...time passes... host2$ weave launch $HOST1 - -[time passes] - + ...time passes... host3$ weave launch $HOST1 $HOST2 - -[time passes; host1 is rebooted] - + ...time passes... + ...host1 is rebooted... host1$ weave launch $HOST2 $HOST3 ## Choosing an allocation range