Add release v1.5.1

Document `weave status ipam`. Fixes #1876.

CHANGELOG.md

@@ -1,3 +1,22 @@
+## Release 1.5.1
+
+Bug fixes and minor improvements
+
+* Persisted data that was rendered invalid by changing peer name or
+  allocation range is detected and removed automatically, preventing
+  crashes and hangs #2246/#2209/#2249
+* `weave rmpeer` persists the range takeover in case the peer on which
+  it was executed dies subsequently #2238
+* Launching a container with an explicit `WEAVE_CIDR` in the
+  allocation range now waits instead of erroring if the allocator
+  hasn't finished initialising #2232/#2265
+* Weave DNS now responds to AAAA queries with an empty answer section,
+  instead of NXDOMAIN which could be cached and block subsequent
+  resolution of A records #2244/#2252
+* Many improvements to the documentation.
+
+More details in the [change log](https://github.com/weaveworks/weave/issues?q=milestone%3A1.5.1).
+
 ## Release 1.5.0
 
 ## Highlights

README.md

@@ -4,178 +4,18 @@
 
 # About Weaveworks
 
-[Weaveworks](http://weave.works) is the company that delivers the most productive way for developers to connect, observe and control
-Docker containers. The first product developed by Weaveworks, with nearly 5 million downloads to date, is Weave Net, enabling users to get started with Docker clusters and portable apps in a fraction of the time compared to other solutions. In addition, Weaveworks offers Weave Scope, a powerful container monitoring tool that automatically maps Docker containers and their interactions, as well as Weave Flux, a microservice router that automates the access of containers as services.
- 
+[Weaveworks](https://www.weave.works) is the company that delivers the most productive way for developers to connect, observe and control
+Docker containers.
 
-# Introduction
-This document is intended to cover some of the more technical aspects of Weave Net. To learn about our products, including getting
-started tutorials, visit our [website](http://weave.works) and
-[documentation](http://docs.weave.works).
+This repository contains [Weave Net](https://www.weave.works/products/weave-net/), the first product developed by Weaveworks, with nearly 5 million downloads to date, which enables users to get started with Docker clusters and portable apps in a fraction of the time compared with other solutions.
+
+Other products from Weaveworks include [Weave Scope](https://www.weave.works/products/weave-scope/), a powerful container monitoring tool that automatically maps Docker containers and their interactions, and [Weave Flux](https://www.weave.works/products/weave-flux/), a microservice router that automates the access of containers as services.
 
 # Weave Net
 
-Weave Net creates a virtual network that connects Docker containers
-deployed across multiple hosts and enables their automatic discovery.
+Weave Net creates a virtual network that connects Docker containers across multiple hosts and enables their automatic discovery. With Weave Net, portable microservices-based applications consisting of multiple containers can run anywhere: on one host, multiple hosts or even across cloud providers and data centers. Applications use the network just as if the containers were all plugged into the same network switch, without having to configure port mappings, ambassadors or links.
 
-![Weave Virtual Network](/docs/virtual-network.png?raw=true "Weave Virtual Network")
+Services provided by application containers on the Weave network can be exposed to the outside world, regardless of where they are running. Similarly, existing internal systems can be opened to accept connections from application containers irrespective of their location.
 
-Applications use the network just as if the containers were all
-plugged into the same network switch, with no need to configure port
-mappings, links, etc. Services provided by application containers on
-the weave network can be made accessible to the outside world,
-regardless of where those containers are running. Similarly, existing
-internal systems can be exposed to application containers irrespective
-of their location.
-
-![Weave Net Deployment](/docs/deployment.png?raw=true "Weave Net Deployment")
-
-Weave Net can traverse firewalls and operate in partially connected
-networks. Traffic can be encrypted, allowing hosts to be connected
-across an untrusted network.
-
-With Weave Net you can easily construct applications consisting of
-multiple containers, running anywhere.
-
-Weave Net works alongside Docker's existing (single host) networking
-capabilities, so these can continue to be used by containers.
-
-## Installation
-
-To get started, you should be running Linux (kernel 3.8 or later) and have Docker
-(version 1.6.0 or later) installed. Then install Weave Net with
-
-    sudo curl -L git.io/weave -o /usr/local/bin/weave
-    sudo chmod a+x /usr/local/bin/weave
-
-For usage on OSX (with Docker Machine) you first need to
-[make sure that a VM is running and configured](https://docs.docker.com/installation/mac/#from-your-shell).
-Then you can launch Weave Net directly from the OSX host.
-
-For installing Weave Net on other platforms, follow the [integration guides](http://weave.works/product/integrations/).
-
-Weave Net respects the environment variable `DOCKER_HOST`, so you can run
-it locally to control a weave network on a remote host.
-
-Weave Net
-[periodically contacts Weaveworks servers for available versions](https://github.com/weaveworks/go-checkpoint).
-New versions are announced in the log and in
-[the status summary](http://docs.weave.works/weave/latest_release/troubleshooting.html#weave-status).
-To disable this check, run:
-
-```
-export CHECKPOINT_DISABLE=1
-```
-
-before launching Weave Net.
-
-## Quick Start Screencast
-
-<a href="https://youtu.be/kihQCCT1ykE" alt="Click to watch the screencast" target="_blank">
-  <img src="/docs/hello-screencast.png" />
-</a>
-
-## Example
-
-Say you have docker running on two hosts, accessible to each other as
-`$HOST1` and `$HOST2`, and want to deploy an application consisting of
-two containers, one on each host.
-
-On $HOST1 you would run:
-
-    host1$ weave launch
-    host1$ eval $(weave env)
-    host1$ docker run --name a1 -ti ubuntu
-
-> NB: If the first command results in an error like
-> `http:///var/run/docker.sock/v1.19/containers/create: dial unix
-> /var/run/docker.sock: permission denied. Are you trying to connect
-> to a TLS-enabled daemon without TLS?` then you likely need to be
-> 'root' in order to connect to the Docker daemon. If so, run the
-> above and all subsequent commands in a *single* root shell (e.g. one
-> created with `sudo -s`). Do *not* prefix individual commands with
-> `sudo`, since some commands modify environment entries and hence
-> they all need to be executed from the same shell.
-
-The first line runs Weave Net. The second line configures your environment
-so that containers launched via the docker command line are
-automatically attached to the weave network. Finally, you run your
-application container.
-
-That's it! If your application consists of more than one container on
-this host, you can simply launch them with `docker run` as appropriate.
-
-Next you'll repeat similar steps on `$HOST2`...
-
-    host2$ weave launch $HOST1
-    host2$ eval $(weave env)
-    host2$ docker run --name a2 -ti ubuntu
-
-The only difference, apart from the name of the application container,
-is that weave is being told that it should peer with the weave on
-`$HOST1` (specified as the IP address or hostname, and optional
-`:port`, by which `$HOST2` can reach it). NB: if there is a firewall
-between `$HOST1` and `$HOST2`, you must permit traffic to the weave
-control port (TCP 6783) and data ports (UDP 6783/6784).
-
-Note that you could instead have told the Weave Net instance on `$HOST1` to connect to
-`$HOST2`, or told both about each other. Order does not matter here;
-Weave Net automatically (re)connects to peers when they become
-available. You can instruct Weave Net to connect to multiple peers by
-supplying multiple addresses, separated by spaces; you can also
-[add peers dynamically](http://docs.weave.works/weave/latest_release/features.html#dynamic-topologies).
-
-Weave Net must be started once per host. The relevant container images are
-pulled down on demand, but if you wish you can preload them by running
-`weave setup` - this is particularly useful for automated deployments,
-and ensures that there are no delays during later operations.
-
-Now that you've got everything set up, let's confirm that your containers
-can talk to each other...
-
-In the container started on `$HOST1`...
-
-    root@a1:/# ping -c 1 -q a2
-    PING a2.weave.local (10.40.0.2) 56(84) bytes of data.
-    --- a2.weave.local ping statistics ---
-    1 packets transmitted, 1 received, 0% packet loss, time 0ms
-    rtt min/avg/max/mdev = 0.341/0.341/0.341/0.000 ms
-
-Similarly, in the container started on `$HOST2`...
-
-    root@a2:/# ping -c 1 -q a1
-    PING a1.weave.local (10.32.0.2) 56(84) bytes of data.
-    --- a1.weave.local ping statistics ---
-    1 packets transmitted, 1 received, 0% packet loss, time 0ms
-    rtt min/avg/max/mdev = 0.366/0.366/0.366/0.000 ms
-
-So there you have it, two containers on separate hosts happily talking
-to each other.
-
-## Find out more
-
- * [Documentation homepage](http://docs.weave.works/weave/latest_release/)
- * [Getting Started Guides](http://weave.works/guides/)
- * [Features](http://docs.weave.works/weave/latest_release/features.html)
- * [Troubleshooting](http://docs.weave.works/weave/latest_release/troubleshooting.html)
- * [Building](http://docs.weave.works/weave/latest_release/building.html)
- * [How it works](http://docs.weave.works/weave/latest_release/how-it-works.html)
-
-## Contact Us
-
-Found a bug, want to suggest a feature, or have a question?  Please
-[file an issue](https://github.com/weaveworks/weave/issues), or post
-to the
-[Weave Users Google Group](https://groups.google.com/a/weave.works/forum/#!forum/weave-users),
-which you can email at weave-users@weave.works. When reporting a bug, please
-include which version of Weave Net you are running, as shown by `weave
-version`.
-
-Follow us on Twitter:
-[@weaveworks](https://twitter.com/weaveworks).
-
-Check out our blog for the latest product and ecosystem news:
-[Weaveworks Blog](https://www.weave.works/blog/).
-
-IRC:
-[#weavenetwork](https://botbot.me/freenode/weavenetwork/)
+* [Install Weave Net](https://www.weave.works/docs/net/latest/installing-weave/)
+* [Find out more about Weave Net](https://www.weave.works/products/weave-net/)

bin/publish-site

@@ -0,0 +1,46 @@
+#!/bin/bash
+
+set -e
+set -o pipefail
+
+PRODUCT=net
+
+fatal() {
+    echo "$@" >&2
+    exit 1
+}
+
+if [ ! -d .git ] ; then
+    fatal "Current directory is not a git clone"
+fi
+
+if ! BRANCH=$(git symbolic-ref --short HEAD) || [ -z "$BRANCH" ] ; then
+    fatal "Could not determine branch"
+fi
+
+case "$BRANCH" in
+    issues/*)
+        VERSION="${BRANCH#issues/}"
+        TAGS="$VERSION"
+        ;;
+    *)
+        if echo "$BRANCH" | grep -qE '^[0-9]+\.[0-9]+' ; then
+            DESCRIBE=$(git describe --match 'v*')
+            if ! VERSION=$(echo "$DESCRIBE" | grep -oP '(?<=^v)[0-9]+\.[0-9]+\.[0-9]+') ; then
+                fatal "Could not infer latest $BRANCH version from $DESCRIBE"
+            fi
+            TAGS="$VERSION latest"
+        else
+            VERSION="$BRANCH"
+            TAGS="$VERSION"
+        fi
+        ;;
+esac
+
+for TAG in $TAGS ; do
+    echo ">>> Publishing $PRODUCT $VERSION to $1/docs/$PRODUCT/$TAG"
+    wordepress \
+        --url "$1" --user "$2" --password "$3" \
+        --product "$PRODUCT" --version "$VERSION" --tag "$TAG" \
+        publish site
+done

circle.yml

@@ -55,6 +55,12 @@ teardown:
         parallel: true
 
 deployment:
+  release:
+    branch: /[0-9]+\.[0-9]+/
+    owner: weaveworks
+    commands:
+      - go get github.com/weaveworks/wordepress && cd /home/ubuntu/src/github.com/weaveworks/wordepress && git checkout v1.0.0 && cd cmd/wordepress && go get
+      - cd $SRCDIR; bin/publish-site "$WP_LIVE_URL" "$WP_LIVE_USER" "$WP_LIVE_PASSWORD"
   hub:
     branch: master
     owner: weaveworks
@@ -62,3 +68,11 @@ deployment:
       - docker login -e $DOCKER_EMAIL -u $DOCKER_USER -p $DOCKER_PASS
       - cd $SRCDIR; make clean-bin
       - cd $SRCDIR; make publish
+      - go get github.com/weaveworks/wordepress && cd /home/ubuntu/src/github.com/weaveworks/wordepress && git checkout v1.0.0 && cd cmd/wordepress && go get
+      - cd $SRCDIR; bin/publish-site "$WP_DEV_URL" "$WP_DEV_USER" "$WP_DEV_PASSWORD"
+  issues:
+    branch: /.*/
+    owner: weaveworks
+    commands:
+      - go get github.com/weaveworks/wordepress && cd /home/ubuntu/src/github.com/weaveworks/wordepress && git checkout v1.0.0 && cd cmd/wordepress && go get
+      - cd $SRCDIR; bin/publish-site "$WP_DEV_URL" "$WP_DEV_USER" "$WP_DEV_PASSWORD"

common/utils.go

@@ -45,6 +45,7 @@ func WithNetNS(ns netns.NsHandle, work func() error) error {
 }
 
 type NetDev struct {
+	Name  string
 	MAC   net.HardwareAddr
 	CIDRs []*net.IPNet
 }
@@ -97,7 +98,7 @@ func linkToNetDev(link netlink.Link) (*NetDev, error) {
 		return nil, err
 	}
 
-	netDev := &NetDev{MAC: link.Attrs().HardwareAddr}
+	netDev := &NetDev{Name: link.Attrs().Name, MAC: link.Attrs().HardwareAddr}
 	for _, addr := range addrs {
 		netDev.CIDRs = append(netDev.CIDRs, addr.IPNet)
 	}

docs/release-process.md

@@ -136,6 +136,9 @@ Finally, for **Mainline** releases only:
 * Merge the branch you are on into `latest_release_doc_updates` and push to GitHub.
 * Close the [milestone](https://github.com/weaveworks/weave/milestones) in GitHub and create the next milestone
 * Update the `#weavenetwork` topic heading on freenode (requires 'chanops' permission)
+* For a mainline release vX.Y.0, create a release branch X.Y from the
+  tag and push to GitHub - this will result in X.Y.0 site docs being
+  published to https://www.weave.works
 
 ## Troubleshooting
 

ipam/allocator.go

@@ -113,7 +113,20 @@ func NewAllocator(config Config) *Allocator {
 
 // Start runs the allocator goroutine
 func (alloc *Allocator) Start() {
-	alloc.loadPersistedData()
+	loadedPersistedData := alloc.loadPersistedData()
+	switch {
+	case loadedPersistedData && len(alloc.seed) != 0:
+		alloc.infof("Found persisted IPAM data, ignoring supplied IPAM seed")
+	case loadedPersistedData:
+		alloc.infof("Initialising with persisted data")
+	case len(alloc.seed) != 0:
+		alloc.infof("Initialising with supplied IPAM seed")
+		alloc.createRing(alloc.seed)
+	case alloc.paxos.IsElector():
+		alloc.infof("Initialising via deferred consensus")
+	default:
+		alloc.infof("Initialising as observer - awaiting IPAM data from another peer")
+	}
 	actionChan := make(chan func(), mesh.ChannelSize)
 	stopChan := make(chan struct{})
 	alloc.actionChan = actionChan
@@ -263,7 +276,7 @@ func (alloc *Allocator) Lookup(ident string, r address.Range) ([]address.CIDR, e
 }
 
 // Claim an address that we think we should own (Sync)
-func (alloc *Allocator) Claim(ident string, cidr address.CIDR, isContainer, noErrorOnUnknown bool) error {
+func (alloc *Allocator) Claim(ident string, cidr address.CIDR, isContainer, noErrorOnUnknown bool, hasBeenCancelled func() bool) error {
 	resultChan := make(chan error)
 	op := &claim{
 		resultChan:       resultChan,
@@ -271,6 +284,7 @@ func (alloc *Allocator) Claim(ident string, cidr address.CIDR, isContainer, noEr
 		cidr:             cidr,
 		isContainer:      isContainer,
 		noErrorOnUnknown: noErrorOnUnknown,
+		hasBeenCancelled: hasBeenCancelled,
 	}
 	alloc.doOperation(op, &alloc.pendingClaims)
 	return <-resultChan
@@ -402,6 +416,7 @@ func (alloc *Allocator) Shutdown() {
 		alloc.cancelOps(&alloc.pendingPrimes)
 		if heir := alloc.pickPeerForTransfer(); heir != mesh.UnknownPeerName {
 			alloc.ring.Transfer(alloc.ourName, heir)
+			alloc.persistRing()
 			alloc.space.Clear()
 			alloc.gossip.GossipBroadcast(alloc.Gossip())
 			time.Sleep(100 * time.Millisecond)
@@ -439,7 +454,7 @@ func (alloc *Allocator) AdminTakeoverRanges(peerNameOrNickname string) address.C
 		}
 
 		before := alloc.space.NumFreeAddresses()
-		alloc.space.AddRanges(newRanges)
+		alloc.ringUpdated()
 		after := alloc.space.NumFreeAddresses()
 
 		alloc.gossip.GossipBroadcast(alloc.Gossip())
@@ -614,11 +629,13 @@ func (alloc *Allocator) actorLoop(actionChan <-chan func(), stopChan <-chan stru
 		case <-stopChan:
 			return
 		case <-alloc.ticker.C:
+			// Retry things in case messages got lost between here and recipients
 			if alloc.awaitingConsensus {
 				alloc.propose()
+			} else {
+				alloc.tryPendingOps()
 			}
 			alloc.removeDeadContainers()
-			alloc.tryPendingOps()
 		}
 
 		alloc.assertInvariants()
@@ -647,8 +664,8 @@ func (alloc *Allocator) establishRing() {
 func (alloc *Allocator) createRing(peers []mesh.PeerName) {
 	alloc.debugln("Paxos consensus:", peers)
 	alloc.ring.ClaimForPeers(normalizeConsensus(peers))
-	alloc.gossip.GossipBroadcast(alloc.Gossip())
 	alloc.ringUpdated()
+	alloc.gossip.GossipBroadcast(alloc.Gossip())
 }
 
 func (alloc *Allocator) ringUpdated() {
@@ -870,52 +887,57 @@ func (alloc *Allocator) persistRing() {
 	}
 }
 
-func (alloc *Allocator) loadPersistedData() {
+// Returns true if persisted data is to be used, otherwise false
+func (alloc *Allocator) loadPersistedData() bool {
 	var checkPeerName mesh.PeerName
 	nameFound, err := alloc.db.Load(nameIdent, &checkPeerName)
 	if err != nil {
 		alloc.fatalf("Error loading persisted peer name: %s", err)
 	}
-	ringFound, err := alloc.db.Load(ringIdent, &alloc.ring)
+	var persistedRing *ring.Ring
+	ringFound, err := alloc.db.Load(ringIdent, &persistedRing)
 	if err != nil {
 		alloc.fatalf("Error loading persisted IPAM data: %s", err)
 	}
-	ownedFound, err := alloc.db.Load(ownedIdent, &alloc.owned)
+	var persistedOwned map[string]ownedData
+	ownedFound, err := alloc.db.Load(ownedIdent, &persistedOwned)
 	if err != nil {
 		alloc.fatalf("Error loading persisted address data: %s", err)
 	}
 
-	if nameFound {
-		if checkPeerName == alloc.ourName {
-			if ringFound {
-				if len(alloc.seed) != 0 {
-					alloc.infof("Found persisted IPAM data, ignoring supplied IPAM seed")
-				}
-				alloc.space.UpdateRanges(alloc.ring.OwnedRanges())
-			}
-			if ownedFound {
-				for _, d := range alloc.owned {
-					for _, cidr := range d.Cidrs {
-						alloc.space.Claim(cidr.Addr)
-					}
-				}
-			}
-			return
-		}
-		alloc.infof("Deleting persisted data for peername %s", checkPeerName)
+	overwritePersisted := func(fmt string, args ...interface{}) {
+		alloc.infof(fmt, args...)
 		alloc.persistRing()
 		alloc.persistOwned()
 	}
 
-	if len(alloc.seed) != 0 {
-		alloc.infof("Initialising with supplied IPAM seed")
-		alloc.createRing(alloc.seed)
-	} else if alloc.paxos.IsElector() {
-		alloc.infof("Initialising via deferred consensus")
-	} else {
-		alloc.infof("Initialising as observer - awaiting IPAM data from another peer")
+	if !nameFound || !ringFound {
+		overwritePersisted("No valid persisted data")
+		return false
 	}
 
+	if checkPeerName != alloc.ourName {
+		overwritePersisted("Deleting persisted data for peername %s", checkPeerName)
+		return false
+	}
+
+	if persistedRing.Range() != alloc.universe {
+		overwritePersisted("Deleting persisted data for IPAM range %s; our range is %s", persistedRing.Range(), alloc.universe)
+		return false
+	}
+
+	alloc.ring = persistedRing
+	alloc.space.UpdateRanges(alloc.ring.OwnedRanges())
+
+	if ownedFound {
+		alloc.owned = persistedOwned
+		for _, d := range alloc.owned {
+			for _, cidr := range d.Cidrs {
+				alloc.space.Claim(cidr.Addr)
+			}
+		}
+	}
+	return true
 }
 
 func (alloc *Allocator) persistOwned() {

ipam/allocator_test.go

@@ -22,6 +22,14 @@ const (
 
 func returnFalse() bool { return false }
 
+func (alloc *Allocator) SimplyAllocate(ident string, cidr address.CIDR) (address.Address, error) {
+	return alloc.Allocate(ident, cidr, true, returnFalse)
+}
+
+func (alloc *Allocator) SimplyClaim(ident string, cidr address.CIDR) error {
+	return alloc.Claim(ident, cidr, true, true, returnFalse)
+}
+
 func TestAllocFree(t *testing.T) {
 	const (
 		container1 = "abcdef"
@@ -41,11 +49,11 @@ func TestAllocFree(t *testing.T) {
 	cidr2, _ := address.ParseCIDR(subnet2)
 
 	alloc.claimRingForTesting()
-	addr1, err := alloc.Allocate(container1, cidr1, true, returnFalse)
+	addr1, err := alloc.SimplyAllocate(container1, cidr1)
 	require.NoError(t, err)
 	require.Equal(t, testAddr1, addr1.String(), "address")
 
-	addr2, err := alloc.Allocate(container1, cidr2, true, returnFalse)
+	addr2, err := alloc.SimplyAllocate(container1, cidr2)
 	require.NoError(t, err)
 	require.Equal(t, testAddr2, addr2.String(), "address")
 
@@ -54,28 +62,28 @@ func TestAllocFree(t *testing.T) {
 	require.Equal(t, []address.CIDR{address.MakeCIDR(cidr1, addr1), address.MakeCIDR(cidr2, addr2)}, addrs)
 
 	// Ask for another address for a different container and check it's different
-	addr1b, _ := alloc.Allocate(container2, cidr1, true, returnFalse)
+	addr1b, _ := alloc.SimplyAllocate(container2, cidr1)
 	if addr1b.String() == testAddr1 {
 		t.Fatalf("Expected different address but got %s", addr1b.String())
 	}
 
 	// Ask for the first container again and we should get the same addresses again
-	addr1a, _ := alloc.Allocate(container1, cidr1, true, returnFalse)
+	addr1a, _ := alloc.SimplyAllocate(container1, cidr1)
 	require.Equal(t, testAddr1, addr1a.String(), "address")
-	addr2a, _ := alloc.Allocate(container1, cidr2, true, returnFalse)
+	addr2a, _ := alloc.SimplyAllocate(container1, cidr2)
 	require.Equal(t, testAddr2, addr2a.String(), "address")
 
 	// Now delete the first container, and we should get its addresses back
 	require.NoError(t, alloc.Delete(container1))
-	addr3, _ := alloc.Allocate(container3, cidr1, true, returnFalse)
+	addr3, _ := alloc.SimplyAllocate(container3, cidr1)
 	require.Equal(t, testAddr1, addr3.String(), "address")
-	addr4, _ := alloc.Allocate(container3, cidr2, true, returnFalse)
+	addr4, _ := alloc.SimplyAllocate(container3, cidr2)
 	require.Equal(t, testAddr2, addr4.String(), "address")
 
 	alloc.ContainerDied(container2)
 
 	// Resurrect
-	addr1c, err := alloc.Allocate(container2, cidr1, true, returnFalse)
+	addr1c, err := alloc.SimplyAllocate(container2, cidr1)
 	require.NoError(t, err)
 	require.Equal(t, addr1b, addr1c, "address")
 
@@ -170,31 +178,32 @@ func TestAllocatorClaim(t *testing.T) {
 	addr1, _ := address.ParseCIDR(testAddr1)
 
 	// First claim should trigger "dunno, I'm going to wait"
-	err := alloc.Claim(container3, addr1, true, true)
+	err := alloc.SimplyClaim(container3, addr1)
 	require.NoError(t, err)
 
 	alloc.Prime()
 	// Do an allocate on the other peer, which we will try to claim later
 	addrx, err := allocs[0].Allocate(container1, subnet, true, returnFalse)
+	router.Flush()
 
 	// Now try the claim again
-	err = alloc.Claim(container3, addr1, true, true)
+	err = alloc.SimplyClaim(container3, addr1)
 	require.NoError(t, err)
 	// Check we get this address back if we try an allocate
-	addr3, _ := alloc.Allocate(container3, subnet, true, returnFalse)
+	addr3, _ := alloc.SimplyAllocate(container3, subnet)
 	require.Equal(t, testAddr1, address.MakeCIDR(subnet, addr3).String(), "address")
 	// one more claim should still work
-	err = alloc.Claim(container3, addr1, true, true)
+	err = alloc.SimplyClaim(container3, addr1)
 	require.NoError(t, err)
 	// claim for a different container should fail
-	err = alloc.Claim(container1, addr1, true, true)
+	err = alloc.SimplyClaim(container1, addr1)
 	require.Error(t, err)
 	// claiming the address allocated on the other peer should fail
-	err = alloc.Claim(container1, address.MakeCIDR(subnet, addrx), true, true)
+	err = alloc.SimplyClaim(container1, address.MakeCIDR(subnet, addrx))
 	require.Error(t, err, "claiming address allocated on other peer should fail")
 	// Check an address outside of our universe
 	addr2, _ := address.ParseCIDR(testAddr2)
-	err = alloc.Claim(container1, addr2, true, true)
+	err = alloc.SimplyClaim(container1, addr2)
 	require.NoError(t, err)
 }
 
@@ -314,11 +323,11 @@ func TestGossipShutdown(t *testing.T) {
 	defer alloc.Stop()
 
 	alloc.claimRingForTesting()
-	alloc.Allocate(container1, subnet, true, returnFalse)
+	alloc.SimplyAllocate(container1, subnet)
 
 	alloc.Shutdown()
 
-	_, err := alloc.Allocate(container2, subnet, true, returnFalse) // trying to allocate after shutdown should fail
+	_, err := alloc.SimplyAllocate(container2, subnet) // trying to allocate after shutdown should fail
 	require.False(t, err == nil, "no address")
 
 	CheckAllExpectedMessagesSent(alloc)
@@ -453,7 +462,7 @@ func TestAllocatorFuzz(t *testing.T) {
 		allocIndex := rand.Int31n(nodes)
 		alloc := allocs[allocIndex]
 		//common.Log.Infof("Allocate: asking allocator %d", allocIndex)
-		addr, err := alloc.Allocate(name, subnet, true, returnFalse)
+		addr, err := alloc.SimplyAllocate(name, subnet)
 
 		if err != nil {
 			panic(fmt.Sprintf("Could not allocate addr"))
@@ -510,7 +519,7 @@ func TestAllocatorFuzz(t *testing.T) {
 
 		//common.Log.Infof("Asking for %s (%s) on allocator %d again", res.name, addr, res.alloc)
 
-		newAddr, _ := alloc.Allocate(res.name, subnet, true, returnFalse)
+		newAddr, _ := alloc.SimplyAllocate(res.name, subnet)
 		oldAddr, _ := address.ParseIP(addr)
 		if newAddr != oldAddr {
 			panic(fmt.Sprintf("Got different address for repeat request for %s: %s != %s", res.name, newAddr, oldAddr))
@@ -531,7 +540,7 @@ func TestAllocatorFuzz(t *testing.T) {
 		addressIndex := rand.Int31n(int32(subnet.Size()))
 		alloc := allocs[allocIndex]
 		addr := address.Add(subnet.Addr, address.Offset(addressIndex))
-		err := alloc.Claim(name, address.MakeCIDR(subnet, addr), true, true)
+		err := alloc.SimplyClaim(name, address.MakeCIDR(subnet, addr))
 		if err == nil {
 			noteAllocation(allocIndex, name, addr)
 		}

ipam/claim.go

@@ -15,6 +15,7 @@ type claim struct {
 	cidr             address.CIDR
 	isContainer      bool
 	noErrorOnUnknown bool
+	hasBeenCancelled func() bool
 }
 
 // Send an error (or nil for success) back to caller listening on resultChan
@@ -33,6 +34,11 @@ func (c *claim) sendResult(result error) {
 
 // Try returns true for success (or failure), false if we need to try again later
 func (c *claim) Try(alloc *Allocator) bool {
+	if c.hasBeenCancelled() {
+		c.Cancel()
+		return true
+	}
+
 	if !alloc.ring.Contains(c.cidr.Addr) {
 		// Address not within our universe; assume user knows what they are doing
 		alloc.infof("Address %s claimed by %s - not in our range", c.cidr, c.ident)
@@ -51,11 +57,10 @@ func (c *claim) Try(alloc *Allocator) bool {
 		// success
 	case mesh.UnknownPeerName:
 		// If our ring doesn't know, it must be empty.
+		alloc.infof("Claim %s for %s: is in the range %s, but the allocator is not initialized yet; will try later.",
+			c.cidr, c.ident, alloc.universe.AsCIDRString())
 		if c.noErrorOnUnknown {
-			alloc.infof("Claim %s for %s: address allocator still initializing; will try later.", c.cidr, c.ident)
 			c.sendResult(nil)
-		} else {
-			c.sendResult(fmt.Errorf("%s is in the range %s, but the allocator is not initialized yet", c.cidr, alloc.universe.AsCIDRString()))
 		}
 		return false
 	default:
@@ -72,14 +77,14 @@ func (c *claim) Try(alloc *Allocator) bool {
 		return false
 	}
 
+	if c.ident == "_" { // Special "I don't have a unique ID" identifier
+		c.ident = c.cidr.Addr.String()
+	}
 	// We are the owner, check we haven't given it to another container
 	switch existingIdent := alloc.findOwner(c.cidr.Addr); existingIdent {
 	case "":
 		if err := alloc.space.Claim(c.cidr.Addr); err == nil {
 			alloc.debugln("Claimed", c.cidr, "for", c.ident)
-			if c.ident == "_" { // Special "I don't have a unique ID" identifier
-				c.ident = c.cidr.String()
-			}
 			alloc.addOwned(c.ident, c.cidr, c.isContainer)
 			c.sendResult(nil)
 		} else {
@@ -89,7 +94,7 @@ func (c *claim) Try(alloc *Allocator) bool {
 		// same identifier is claiming same address; that's OK
 		alloc.debugln("Re-Claimed", c.cidr, "for", c.ident)
 		c.sendResult(nil)
-	case c.cidr.String():
+	case c.cidr.Addr.String():
 		// Address already allocated via "_" name
 		c.sendResult(fmt.Errorf("address %s already in use", c.cidr))
 	default:

ipam/http.go

@@ -38,32 +38,52 @@ func writeAddresses(w http.ResponseWriter, cidrs []address.CIDR) {
 	}
 }
 
+func hasBeenCanceled(dockerCli *docker.Client, closedChan <-chan bool, ident string, checkAlive bool) func() bool {
+	return func() bool {
+		select {
+		case <-closedChan:
+			return true
+		default:
+			res := checkAlive && dockerCli != nil && dockerCli.IsContainerNotRunning(ident)
+			checkAlive = false // we check only once; if the container dies later we learn about that through events
+			return res
+		}
+	}
+}
+
+func cancellationErr(w http.ResponseWriter, err error) bool {
+	if _, ok := err.(*errorCancelled); ok {
+		common.Log.Infoln("[allocator]:", err.Error())
+		fmt.Fprint(w, "cancelled")
+		return true
+	}
+	return false
+}
+
 func (alloc *Allocator) handleHTTPAllocate(dockerCli *docker.Client, w http.ResponseWriter, ident string, checkAlive bool, subnet address.CIDR) {
-	closedChan := w.(http.CloseNotifier).CloseNotify()
 	addr, err := alloc.Allocate(ident, subnet, checkAlive,
-		func() bool {
-			select {
-			case <-closedChan:
-				return true
-			default:
-				res := checkAlive && dockerCli != nil && dockerCli.IsContainerNotRunning(ident)
-				checkAlive = false // we check only once; if the container dies later we learn about that through events
-				return res
-			}
-		})
+		hasBeenCanceled(dockerCli, w.(http.CloseNotifier).CloseNotify(), ident, checkAlive))
 	if err != nil {
-		if _, ok := err.(*errorCancelled); ok { // cancellation is not really an error
-			common.Log.Infoln("[allocator]:", err.Error())
-			fmt.Fprint(w, "cancelled")
-			return
+		if !cancellationErr(w, err) {
+			badRequest(w, err)
 		}
-		badRequest(w, err)
 		return
 	}
-
 	fmt.Fprintf(w, "%s/%d", addr, subnet.PrefixLen)
 }
 
+func (alloc *Allocator) handleHTTPClaim(dockerCli *docker.Client, w http.ResponseWriter, ident string, cidr address.CIDR, checkAlive, noErrorOnUnknown bool) {
+	err := alloc.Claim(ident, cidr, checkAlive, noErrorOnUnknown,
+		hasBeenCanceled(dockerCli, w.(http.CloseNotifier).CloseNotify(), ident, checkAlive))
+	if err != nil {
+		if !cancellationErr(w, err) {
+			badRequest(w, fmt.Errorf("Unable to claim: %s", err))
+		}
+		return
+	}
+	w.WriteHeader(204)
+}
+
 // HandleHTTP wires up ipams HTTP endpoints to the provided mux.
 func (alloc *Allocator) HandleHTTP(router *mux.Router, defaultSubnet address.CIDR, dockerCli *docker.Client) {
 	router.Methods("GET").Path("/ipinfo/defaultsubnet").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -73,13 +93,10 @@ func (alloc *Allocator) HandleHTTP(router *mux.Router, defaultSubnet address.CID
 	router.Methods("PUT").Path("/ip/{id}/{ip}/{prefixlen}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		vars := mux.Vars(r)
 		if cidr, ok := parseCIDR(w, vars["ip"]+"/"+vars["prefixlen"], false); ok {
-			noErrorOnUnknown := r.FormValue("noErrorOnUnknown") == "true"
+			ident := vars["id"]
 			checkAlive := r.FormValue("check-alive") == "true"
-			if err := alloc.Claim(vars["id"], cidr, checkAlive, noErrorOnUnknown); err != nil {
-				badRequest(w, fmt.Errorf("Unable to claim: %s", err))
-				return
-			}
-			w.WriteHeader(204)
+			noErrorOnUnknown := r.FormValue("noErrorOnUnknown") == "true"
+			alloc.handleHTTPClaim(dockerCli, w, ident, cidr, checkAlive, noErrorOnUnknown)
 		}
 	})
 

nameserver/dns.go

@@ -134,7 +134,7 @@ func (d *DNSServer) createMux(client *dns.Client, defaultMaxResponseSize int) *d
 
 func (h *handler) handleLocal(w dns.ResponseWriter, req *dns.Msg) {
 	h.ns.debugf("local request: %+v", *req)
-	if len(req.Question) != 1 || req.Question[0].Qtype != dns.TypeA {
+	if len(req.Question) != 1 {
 		h.nameError(w, req)
 		return
 	}
@@ -149,6 +149,12 @@ func (h *handler) handleLocal(w dns.ResponseWriter, req *dns.Msg) {
 		h.nameError(w, req)
 		return
 	}
+	// Per RFC4074, if we have an A but another type was requested,
+	// return 'no error' with empty answer section
+	if req.Question[0].Qtype != dns.TypeA {
+		h.respond(w, h.makeResponse(req, nil))
+		return
+	}
 
 	header := dns.RR_Header{
 		Name:   req.Question[0].Name,
@@ -204,7 +210,7 @@ func (h *handler) handleRecursive(w dns.ResponseWriter, req *dns.Msg) {
 	h.ns.debugf("recursive request: %+v", *req)
 
 	// Resolve unqualified names locally
-	if len(req.Question) == 1 && req.Question[0].Qtype == dns.TypeA {
+	if len(req.Question) == 1 {
 		hostname := dns.Fqdn(req.Question[0].Name)
 		if strings.Count(hostname, ".") == 1 {
 			h.handleLocal(w, req)

plugin/README.md

@@ -9,6 +9,6 @@ to integrate Weave Net with [Kubernetes](http://kubernetes.io/).
 The Docker plugin runs automatically when you `weave launch`, provided your
 Docker daemon is version 1.9 or newer.
 
-More detail on the Docker plugin [here](http://docs.weave.works/weave/latest_release/plugin.html).
+More detail on the Docker plugin [here](https://www.weave.works/docs/net/latest/plugin/).
 
-More detail on the CNI plugin [here](https://github.com/weaveworks/weave/blob/master/site/plugin/cni-plugin-how-to.md).
+More detail on the CNI plugin [here](https://www.weave.works/docs/net/latest/cni-plugin/).

prog/weaveutil/addrs.go

@@ -27,7 +27,7 @@ func containerAddrs(args []string) error {
 		}
 
 		for _, netDev := range netDevs {
-			fmt.Printf("%12s %s", containerID, netDev.MAC.String())
+			fmt.Printf("%12s %s %s", containerID, netDev.Name, netDev.MAC.String())
 			for _, cidr := range netDev.CIDRs {
 				prefixLength, _ := cidr.Mask.Size()
 				fmt.Printf(" %v/%v", cidr.IP, prefixLength)

proxy/proxy.go

@@ -103,7 +103,7 @@ func (proxy *Proxy) attachWithRetry(id string) {
 
 	j := &attachJob{id: id, tryInterval: initialInterval}
 	j.timer = time.AfterFunc(time.Duration(0), func() {
-		if err := proxy.attach(id, false); err != nil {
+		if err := proxy.attach(id, false, false); err != nil {
 			// The delay at the nth retry is a random value in the range
 			// [i-i/2,i+i/2], where i = initialInterval * 1.5^(n-1).
 			j.timer.Reset(j.tryInterval)
@@ -383,7 +383,10 @@ func (proxy *Proxy) listen(protoAndAddr string) (net.Listener, string, error) {
 
 // weavedocker.ContainerObserver interface
 func (proxy *Proxy) ContainerStarted(ident string) {
-	proxy.notifyWaiters(ident, proxy.attach(ident, true))
+	// In case of attach failure, if we have a request waiting on the start, kill the container,
+	// otherwise assume it is a Docker-initated restart and kill the process inside.
+	err := proxy.attach(ident, true, proxy.waitChan(ident) == nil)
+	proxy.notifyWaiters(ident, err)
 }
 
 func containerShouldAttach(container *docker.Container) bool {
@@ -438,18 +441,20 @@ func (proxy *Proxy) waitForStart(r *http.Request) error {
 	return nil
 }
 
-// If some other operation is waiting for a container to start, join in the wait
-func (proxy *Proxy) waitForStartByIdent(ident string) error {
-	var ch chan error
+func (proxy *Proxy) waitChan(ident string) chan error {
 	proxy.Lock()
+	defer proxy.Unlock()
 	for _, wait := range proxy.waiters {
 		if ident == wait.ident && !wait.done {
-			ch = wait.ch
-			break
+			return wait.ch
 		}
 	}
-	proxy.Unlock()
-	if ch != nil {
+	return nil
+}
+
+// If some other operation is waiting for a container to start, join in the wait
+func (proxy *Proxy) waitForStartByIdent(ident string) error {
+	if ch := proxy.waitChan(ident); ch != nil {
 		Log.Debugf("Wait for start of container %s", ident)
 		return <-ch
 	}
@@ -461,7 +466,7 @@ func (proxy *Proxy) ContainerDestroyed(ident string) {}
 
 // Check if this container needs to be attached, if so then attach it,
 // and return nil on success or not needed.
-func (proxy *Proxy) attach(containerID string, orDie bool) error {
+func (proxy *Proxy) attach(containerID string, orDie, killProcess bool) error {
 	container, err := proxy.client.InspectContainer(containerID)
 	if err != nil {
 		if _, ok := err.(*docker.NoSuchContainer); !ok {
@@ -498,7 +503,11 @@ func (proxy *Proxy) attach(containerID string, orDie bool) error {
 		args = append(args, "--no-multicast-route")
 	}
 	if orDie {
-		args = append(args, "--or-die")
+		if killProcess {
+			args = append(args, "--or-kill")
+		} else {
+			args = append(args, "--or-die")
+		}
 	}
 	args = append(args, container.ID)
 	return callWeaveAttach(container, args)

site/cni-plugin.md

@@ -1,6 +1,5 @@
 ---
 title: Integrating Kubernetes via the CNI Plugin
-layout: default
 menu_order: 65
 ---
 

site/faq.md

@@ -5,19 +5,14 @@ menu_order: 100
 
 
 
-**Q:How do I obtain the IP of a specific container when I'm using Weave?**
+**Q: How do I obtain the IP of a specific container when I'm using Weave?**
 
 You can use `weave ps <container>` to see the allocated address of a container on a Weave network.  
 
 See [Troubleshooting Weave - List attached containers](/site/troubleshooting.md#list-attached-containers).
 
 
-**Q: How do I expose one of my containers to the outside world?**
-
-Exposing a container to the outside world is described in [Exposing Services to the Outside](/site/using-weave/service-export.md).
-
-
-**Q: My dockerized app needs to check the request of a an application that uses a static IP. Is it possible to manually change the IP of a container?**
+**Q: My dockerized app needs to check the request of an application that uses a static IP. Is it possible to manually change the IP of a container?**
 
 
 You can manually change the IP of a container using [Classless Inter-Domain Routing or CIDR notation](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing). 
@@ -25,15 +20,20 @@ You can manually change the IP of a container using [Classless Inter-Domain Rout
 For more information, refer to [Manually Specifying the IP Address of a Container](/site/using-weave/manual-ip-address.md). 
 
 
-**Q:Can I connect my existing 'legacy' network with a Weave container network?**
+**Q: How do I expose one of my containers to the outside world?**
+
+Exposing a container to the outside world is described in [Exporting Services](/site/using-weave/service-management.md#exporting).
+
+
+**Q: Can I connect my existing 'legacy' network with a Weave container network?**
 
 Yes you can. 
 
 For example, you have a Weave network that runs on hosts A, B, C. and you have an additional host, that we'll call P, where neither Weave nor Docker are running.  However, you need to connect from a process running on host P to a specific container running on host B on the Weave network.  Since the Weave network is completely separate from any network that P is connected to, you cannot connect the container using the container's IP address. 
 
-The simplest way to accomplish this would be to run Weave on the host and then run, `weave expose` to expose the network to any running containers.  See [Integrating a Network Host](/site/using-weave/host-network-integration.md) If this is not possible, you could also  expose a port on the container on host B and then connect to it. 
+A simple way to accomplish this would be to run Weave on the host and then run, `weave expose` to expose the network to any running containers. Or you set up a route from P to one of A, B or C. See [Integrating a Network Host](/site/using-weave/host-network-integration.md).
 
-You can read about exposing ports in [Service Exporting](/site/using-weave/service-export.md)
+Yet another option is to expose a port from the container on host B and then connect to it. You can read about exposing ports in [Exporting Services](/site/using-weave/service-management.md#exporting).
 
 
 **Q: Why am I seeing the same IP address assigned to two different containers on different hosts?**
@@ -46,7 +46,7 @@ Once the peers detect the inconsistency, they log the error and drop the connect
 
 Some peers may be able to communicate their claim to the others before they run `rmpeer` (i.e. it's a race), so what you can expect is a few cliques of peers that are still talking to each other, but repeatedly dropping attempted connections with peers in other cliques.
 
-For more information on see [Address Allocation with IP Address Management (IPAM)](/site/ipam.md) and also, [Starting, Stopping and Removing Peers](/site/ipam/stop-remove-peers-ipam.md)
+For more information on see [Allocating IP Addresses](/site/ipam.md) and also, [Starting, Stopping and Removing Peers](/site/ipam/stop-remove-peers-ipam.md).
 
 
 **Q: What is the best practice for resetting a node that goes out of service?**
@@ -62,14 +62,14 @@ All virtualization techniques have some overhead, and Weave's overhead is typica
 
 Weave Net also automatically uses the fastest datapath between two hosts. When Weave Net can't use the fast datapath between two hosts, it falls back to the slower packet forwarding approach. Selecting the fastest forwarding approach is automatic, and is determined on a connection-by-connection basis. For example, a Weave network spanning two data centers might use fast datapath within the data centers, but not for the more constrained network link between them.
 
-For more information about fast datapath see [How Fast Datapath Works](/site/how-it-works/fastdp-how-it-works.md)
+For more information about fast datapath see [How Fast Datapath Works](/site/how-it-works/fastdp-how-it-works.md).
 
 
-**Q:How can I tell if Weave is using fast datapath (fastdp) or not?**
+**Q: How can I tell if Weave is using fast datapath (fastdp) or not?**
 
 To view whether Weave is using fastdp or not, you can run, `weave status connections`
 
-For more information on this command, see [Using Fast Datapath](/site/using-weave/fastdp.md)
+For more information on this command, see [Using Fast Datapath](/site/using-weave/fastdp.md).
 
 
 **Q: Does encryption work with fastdp?**
@@ -78,16 +78,22 @@ Encryption does not work with fast datapath. If you enable encryption using the
 
 You can however have a mixture of fast datapath connections over trusted links, as well as, encrypted connections over untrusted links.
 
-See [Using Fast Datapath](/site/using-weave/fastdp.md) for more information
+See [Using Fast Datapath](/site/using-weave/fastdp.md) for more information.
 
 **Q: Can I create multiple networks where containers can communicate on one network, but are isolated from containers on other networks?**
 
-Yes, of course!  Weave allows you to run isolated networks and still allow open communications between individual containers from those isolated networks. You can find information on how to do this in [Application Isolation](/site/using-weave/application-isolation.md)
+Yes, of course!  Weave allows you to run isolated networks and still allow open communications between individual containers from those isolated networks. You can find information on how to do this in [Application Isolation](/site/using-weave/application-isolation.md).
+
+
+**Q: Which ports does Weave Net use (e.g. if I am configuring a firewall) ?**
+
+You must permit traffic to flow through TCP 6783 and UDP 6783/6784,
+which are Weave’s control and data ports.
 
 
 **See Also**
 
- * [Troubleshooting Weave](/site/troublehooting.md)
+ * [Troubleshooting Weave](/site/troubleshooting.md)
  * [Troubleshooting IPAM](/site/ipam.md)
  * [Troubleshooting the Proxy](/site/weave-docker-api/using-proxy.md)
  

site/features.md

@@ -111,7 +111,7 @@ Instead of allowing Weave to automatically allocate addresses, an IP address and
 specified. See [How to Manually Specify IP Addresses and Subnets](/site/using-weave/manual-ip-address.md) for instructions. 
 
 For a discussion on how Weave Net uses IPAM, see [Automatic IP Address Management](/site/ipam.md). And also review the 
-[the basics of IP addressing](/how-it-works/ip-addresses.md) for an explanation of addressing and private networks. 
+[the basics of IP addressing](/site/how-it-works/ip-addresses.md) for an explanation of addressing and private networks. 
 
 
 ###<a name="naming-and-discovery"></a>Naming and Discovery
@@ -178,7 +178,7 @@ running alongside Weave.
 
 Weave Net implements encryption and security using the Go version of [Daniel J. Bernstein's NaCl library](http://nacl.cr.yp.to/index.html).
 
-For information on how to secure your Docker network connections, see [Securing Connections Across Untrusted Networks](/site/using-weave/security-untrusted-networks.md) and for a more technical discussion on how Weave implements encryption see, [Weave Encryption](/how-it-works/encryption.md) and [How Weave Implements Encryption](/site/how-it-works/encryption-implementation.md).
+For information on how to secure your Docker network connections, see [Securing Connections Across Untrusted Networks](/site/using-weave/security-untrusted-networks.md) and for a more technical discussion on how Weave implements encryption see, [Weave Encryption](/site/how-it-works/encryption.md) and [How Weave Implements Encryption](/site/how-it-works/encryption-implementation.md).
 
 
 ###<a name="host-network-integration"></a>Host Network Integration
@@ -186,7 +186,7 @@ For information on how to secure your Docker network connections, see [Securing
 Weave Net application networks can be integrated with a host's network, and establish connectivity between the host and 
 application containers anywhere.
 
-See [Integrating a Host Network with Weave](/site/using-weave/host-network-integration.md).
+See [Integrating with the Host Network](/site/using-weave/host-network-integration.md).
 
 ###<a name="services"></a>Managing Services: Exporting, Importing, Binding and Routing
  
@@ -195,7 +195,7 @@ See [Integrating a Host Network with Weave](/site/using-weave/host-network-integ
  * **Binding Services** - A container can be bound to a particular IP and port without having to change your application code, while at the same time will maintain its original endpoint. 
  * **Routing Services** - By combining the importing and exporting features, you can connect to disjointed networks, even when separated by firewalls and where there may be overlapping IP addresses.  
 
-See [Managing Services in Weave: Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md) for instructions on how to manage services on a Weave container network. 
+See [Managing Services - Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md) for instructions on how to manage services on a Weave container network. 
 
 ###<a name="multi-cloud-networking"></a>Multi-Cloud Networking
 
@@ -232,7 +232,7 @@ reconfiguration or, in many cases, restarts of other containers.
 All that is required is for the migrated container to be started 
 with the same IP address as it was given originally.
 
-See [Managing Services in Weave: Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md), in particular, Routing Services for more information on container mobility. 
+See [Managing Services - Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md), in particular, Routing Services for more information on container mobility. 
 
 
 ###<a name="fault-tolerance"></a>Fault Tolerance

site/how-it-works.md

@@ -1,6 +1,5 @@
 ---
 title: Understanding how Weave Net Works
-layout: default
 menu_order: 90
 ---
 

site/how-it-works/encryption-implementation.md

@@ -1,11 +1,10 @@
 ---
 title: How Weave Net Implements Encryption
-layout: default
 menu_order: 70
 ---
 
 This section describes some details of Weave Net's built-in
-[encryption](/how-it-works/encryption.md):
+[encryption](/site/how-it-works/encryption.md):
 
  * [Establishing the Ephemeral Session Key](#ephemeral-key)
  * [Key Generation](#csprng)

site/how-it-works/ip-addresses.md

@@ -63,4 +63,4 @@ find a match, so uses the default rule.
 
 **See Also**
 
- * [Configuring Weave to Explicitly Use an IP Range](/site/ip-addresses/configuring-weave.md) 
+ * [Allocating IPs in a Specific Range](/site/ip-addresses/configuring-weave.md) 

site/how-it-works/router-encapsulation.md

@@ -45,7 +45,7 @@ the same peer. And in this instance, it will fit as many frames as possible into
 UDP packet.
 
 The meta data for each frame contains the names of the capturing and
-the destination peers. Since the name of the capturing peer name is
+the destination peers. Since the name of the capturing peer is
 associated with the source MAC of the captured payload, it allows
 receiving peers to build up their mappings of which client MAC
 addresses are local to which peers. 

site/installing-weave.md

@@ -24,8 +24,8 @@ With Weave Net downloaded onto your VMs or hosts, you are ready to launch a Weav
 
 ###Quick Start Screencast
 
-<a href="https://youtu.be/kihQCCT1ykE" alt="Click to watch the screencast" target="_blank">
-  <img src="hello-screencast.png" />
+<a href="https://youtu.be/kihQCCT1ykE" target="_blank">
+  <img src="hello-screencast.png" alt="Click to watch the screencast" />
 </a>
 
 ###Guides for specific platforms

site/ipam.md

@@ -28,10 +28,22 @@ suits your deployment best.
 
 #### <a name="seed"></a>Via Seed
 
-Configuration via seed requires you to provide a list of peer names
+Configuration via seed requires you to provide a list of _peer names_
 (via the `--ipalloc-init seed=` parameter) amongst which the address
-space will be shared initially - this means that you will also have to
-specify peer names explicitly on launch:
+space will be shared initially. Normally weave derives a unique peer
+name automatically at launch, but since you need to know them ahead of
+time in this case you will need to name each peer explicitly via the
+`--name` parameter.
+
+> Peers in the weave network are identified by a 48-bit value
+> formatted like an ethernet MAC address (e.g. 01:23:45:67:89:ab) -
+> you can either specify the name fully, or substitute a single run of
+> zero-octets using the `::` notation, similar to
+> [IPv6 address representation](https://en.wikipedia.org/wiki/IPv6_address#Recommended_representation_as_text):
+>
+> * `00:00:00:00:00:01` can be written `::1`
+> * `01:00:00:00:00:00` can be written `1::`
+> * `01:00:00:00:00:01` can be written `1::1`
 
     host1$ weave launch --name ::1 --ipalloc-init seed=::1,::2,::3
     host2$ weave launch --name ::2 --ipalloc-init seed=::1,::2,::3
@@ -43,10 +55,10 @@ outset even under conditions of partition - no consensus is required.
 
 #### <a name="consensus"></a>Via One-off Consensus
 
-Alternatively, you can let Weave Net determine the seed automatically via
-a consensus algorithm. Since you don't need to provide it with a list
-of peer names anymore, you can let Weave Net generate those randomly
-for you as well. 
+Alternatively, you can let Weave Net determine the seed automatically
+via a consensus algorithm. Since you don't need to provide it with a
+list of peer names anymore, you can let Weave Net derive those
+automatically for you as well.
 
 However, in order for Weave Net to form a single consensus
 reliably you must now instead tell each peer how many peers there are
@@ -191,5 +203,3 @@ named `weavedb` is used to store this data.
 
  * [Automatic Allocation Across Multiple Subnets](/site/ipam/allocation-multi-ipam.md)
  * [Plugin Command-line Arguments](/site/plugin/plug-in-command-line.md)
-
-

site/ipam/troubleshooting-ipam.md

@@ -43,3 +43,24 @@ the total allocation range and default subnet. Columns are as follows:
       for more
 * 'Range' - total allocation range set by `--ipalloc-range`
 * 'DefaultSubnet' - default subnet set by `--ipalloc-default-subnet`
+
+Information regarding the division of the IP allocation range amongst
+peers and their reachability can be obtained with
+
+```
+$ weave status ipam
+00:00:00:00:00:01(one)      349526 IPs (33.3% of total)
+00:00:00:00:00:02(two)      349525 IPs (33.3% of total)
+00:00:00:00:00:03(three)    349525 IPs (33.3% of total) - unreachable!
+```
+
+Columns are as follows:
+
+* Peer Name and Nickname
+* Absolute quantity/percentage of allocation range managed by peer
+* Indication of unreachability. This means that the peer is not
+  visible (directly or indirectly) to the peer on which `weave status
+  ipam` was run; whilst this could be a transient condition due to a
+  partition, it may be because the peer has failed and needs to be
+  removed administratively - see [Starting, Stopping and Removing
+  Peers](/site/ipam/stop-remove-peers-ipam.md) for more details.

site/plugin.md

@@ -3,6 +3,10 @@ title: Integrating Docker via the Network Plugin
 menu_order: 60
 ---
 
+ * [Launching Weave Net and Running Containers Using the Plugin](#launching)
+ * [Restarting the Plugin](#restarting)
+ * [Bypassing the Central Cluster Store When Building Docker Apps](#cluster-store)
+ * [Using other plugin command-line arguments](#plugin-args)
 
 Docker versions 1.9 and later have a plugin mechanism for adding
 different network providers. Weave Net installs itself as a network plugin
@@ -31,7 +35,7 @@ which a helper is provided in the Weave script:
     # ping foo
 
 
-###Launching Weave Net and Running Containers Using the Plugin
+###<a name="launching"></a>Launching Weave Net and Running Containers Using the Plugin
 
 Just launch the Weave Net router onto each host and make a peer connection with the other hosts:
 
@@ -52,7 +56,7 @@ then run your containers using the Docker command-line:
     64 bytes from 10.32.0.2: icmp_seq=2 ttl=64 time=0.052 ms
 
 
-### Restarting the Plugin
+###<a name="restarting"></a>Restarting the Plugin
 
 The plugin, like all Weave Net components, is started with a policy of `--restart=always`, so that it is always there after a restart or reboot. If you remove this container (for example, when using `weave reset`) before removing all endpoints created using `--net=weave`, Docker may hang for a long time when it subsequently tries to re-establish communications to the plugin.
 
@@ -62,7 +66,7 @@ If you are using `systemd` with Docker 1.9, it is advised that you modify the Do
 
     TimeoutStartSec=0
 
-###Bypassing the Central Cluster Store When Building Docker Apps
+###<a name="cluster-store"></a>Bypassing the Central Cluster Store When Building Docker Apps
 
 To run a Docker cluster without a central database, you need to ensure the following:
 
@@ -73,6 +77,32 @@ To run a Docker cluster without a central database, you need to ensure the follo
 All cross-host coordination is handled by Weave Net's "mesh" communication, using gossipDNS and eventual consistency to avoid the need for constant communication and dependency on a central cluster store.
 
 
+###<a name="plugin-args"></a>Using other plugin command-line arguments
+
+If you need to give additional arguments to the plugin independently, don't
+use `weave launch`, but instead run:
+
+    $ weave launch-router [other peers]
+    $ weave launch-plugin [plugin arguments]
+
+The plugin command-line arguments are:
+
+ * `--log-level=debug|info|warning|error` --tells the plugin
+   how much information to emit for debugging.
+ * `--no-restart` -- remove the default policy of `--restart=always`, if
+   you want to control start-up of the plugin yourself
+ * `--no-multicast-route` -- stops weave from adding a static IP route for
+   multicast traffic onto its interface
+
+By default, multicast traffic is routed over the weave network.
+To turn this off, e.g. you want to configure your own multicast
+route, add the `--no-multicast-route` flag to `weave launch-plugin`.
+
+
+>Note: When using the Docker Plugin, there is no need to run `eval
+ $(weave env)` to enable the Proxy. If you do, you may end up with two
+ weave network interfaces and two IP addresses for each container.
+
 **See Also**
 
  * [How the Weave Network Plugin Works](/site/plugin/plugin-how-it-works.md)

site/plugin/plug-in-command-line.md

@@ -1,33 +0,0 @@
----
-title: Plugin Command-line Arguments
-layout: default
----
-
-
-
-If you need to give additional arguments to the plugin independently, don't
-use `weave launch`, but instead run:
-
-    $ weave launch-router [other peers]
-    $ weave launch-plugin [plugin arguments]
-
-The plugin command-line arguments are:
-
- * `--log-level=debug|info|warning|error` --tells the plugin
-   how much information to emit for debugging.
- * `--mesh-network-name=<name>` -- set <name> to blank to disable creation
-   of a default network, or include a name of your own choice.
- * `--no-multicast-route` -- stops weave from adding a static IP route for
-   multicast traffic onto its interface
-
-By default, multicast traffic is routed over the weave network.
-To turn this off, e.g. you want to configure your own multicast
-route, add the `--no-multicast-route` flag to `weave launch-plugin`.
-
-
->Note: When using the Docker Plugin, there is no need to run eval $(weave env) to enable the Proxy. Because Weave is running as a plugin within Docker, the Weave Docker API Proxy, at present, cannot detect between networks.  
-
-**See Also**
-
- * [Using the Weave Net Docker Network Plugin](/site/plugin.md)
- * [How the Weave Network Plugin Works](/site/plugin/plugin-how-it-works.md)

site/plugin/plugin-how-it-works.md

@@ -1,6 +1,6 @@
 ---
 title: How the Weave Net Docker Network Plugin Works
-layout: default
+menu_order: 20
 ---
 
 

site/troubleshooting.md

@@ -80,11 +80,11 @@ $ weave status
 The terms used here are explained further at
 [How Weave Net Works](/site/how-it-works.md).
 
-  * **Version** - shows the Weave Net version. If checkpoint is enabled (i.e.
+ * **Version** - shows the Weave Net version. If checkpoint is enabled (i.e.
 `CHECKPOINT_DISABLE` is not set), information about existence of a new version
 will be shown.
 
-  * **Protocol**- indicates the Weave Router inter-peer
+ * **Protocol**- indicates the Weave Router inter-peer
 communication protocol name and supported versions (min..max).
 
  * **Name** - identifies the local Weave Router as a peer on the
@@ -94,7 +94,7 @@ can be overridden by using the `--nickname` argument at `weave
 launch`.
 
  * **Encryption** - indicates whether
-[encryption](/how-it-works/encryption.md) is in use for communication
+[encryption](/site/how-it-works/encryption.md) is in use for communication
 between peers.
 
  * **PeerDiscovery** - indicates whether
@@ -284,7 +284,7 @@ on reboot. This can be disabled via:
     weave launch --no-restart
 
 Note that the
-[Weave Net Docker API Proxy](/site/weave-docker-api/set-up-proxy.md)
+[Weave Net Docker API Proxy](/site/weave-docker-api.md)
 is responsible for reconfiguring the Weave router and re-attaching
 application containers to the Weave network at startup, so if you
 choose not to run it you must make arrangements for this

site/using-weave.md

@@ -55,7 +55,7 @@ As noted above, the same steps are repeated for `$HOST2`. The only difference, b
 
 You can also peer with other hosts by specifying the IP address, and a `:port` by which `$HOST2` can reach `$HOST1`. 
 
->**Note:** If there is a firewall between `$HOST1` and `$HOST2`,  you must permit traffic to flow through TCP 6783 and UDP 6783/6798, which are Weave’s control and data ports.
+>**Note:** If there is a firewall between `$HOST1` and `$HOST2`,  you must permit traffic to flow through TCP 6783 and UDP 6783/6784, which are Weave’s control and data ports.
 
 There are a number of different ways that you can specify peers on a Weave network. You can launch Weave Net on `$HOST1` and then peer with `$HOST2`, or you can launch on `$HOST2` and peer with `$HOST1` or you can tell both hosts about each other at launch. The order in which peers are specified is not important. Weave Net automatically (re)connects to peers when they become available. 
 

site/using-weave/application-isolation.md

@@ -66,5 +66,4 @@ isolation between application containers, that feature needs to be disabled by [
 **See Also** 
 
  * [Automatic Allocation Across Multiple Subnets](/site/ipam/allocation-multi-ipam.md)
- * [Managing Services in Weave Net: Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md)
- * [Exposing Services to the Outside World] (/site/using-weave/service-export.md)
+ * [Managing Services - Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md)

site/using-weave/configuring-weave.md

@@ -1,5 +1,5 @@
 ---
-title: Configuring Weave to Explicitly Use an IP Range
+title: Allocating IPs in a Specific Range
 menu_order: 15
 ---
 
@@ -39,4 +39,4 @@ Networks](https://en.wikipedia.org/wiki/Private_network).  For example
 
 **See Also**
 
- * [IP Addresses, Routes and Networks](/how-it-works/ip-addresses.md)
+ * [IP Addresses, Routes and Networks](/site/how-it-works/ip-addresses.md)

site/using-weave/fastdp.md

@@ -42,8 +42,26 @@ Where fastdp indicates that fast datapath is being used on a connection. If fast
     $ weave status connections
     <- 192.168.122.25:54782  established sleeve 8a:50:4c:23:11:ae(ubuntu1204)
 
+###Packet size (MTU)
+
+The Maximum Transmission Unit, or MTU, is the technical term for the
+limit on how big a single packet can be on the network. Weave Net
+defaults to 1410 bytes, but you can set a smaller size if your
+underlying network has a tighter limit, or set a larger size for
+better performance.
+
+The underlying network must be able to deliver packets of the size
+specified plus overheads of around 50 bytes.  This requirement applies
+to _every path_ between peers.
+
+To specify a different MTU, before launching Weave Net set the
+environment variable `WEAVE_MTU`.  For example, for a typical "jumbo
+frame" configuration:
+
+    $ WEAVE_MTU=8950 weave launch host2 host3
+
 **See Also**
 
  * [Using Weave Net](/site/using-weave.md)
  * [How Fastdp Works](/site/how-it-works/fastdp-how-it-works.md)
- 
+ * [Performance measurements](/blog/weave-docker-networking-performance-fast-data-path/)

site/using-weave/finding-adding-hosts-dynamically.md

@@ -1,6 +1,6 @@
 ---
 title: Adding and Removing Hosts Dynamically
-menu_order: 50
+menu_order: 90
 ---
 
 

site/using-weave/host-network-integration.md

@@ -1,5 +1,5 @@
 ---
-title: Integrating a Host Network with Weave Net
+title: Integrating with the Host Network
 menu_order: 60
 ---
 
@@ -38,8 +38,26 @@ Exposed addresses can also be added to weaveDNS by supplying fully qualified dom
     host2$ weave expose -h exposed.weave.local
     10.2.1.132
 
+###<a name="routing"></a>Routing from Another Host
+
+After running `weave expose`, you can use Linux routing to provide
+access to the Weave network from hosts that are not running Weave Net:
+
+    ip route add <network-cidr> via <exposing-host>
+
+Where,
+
+ * `<network-cidr>` is an IP address range in use on Weave Net,
+for example,  `10.2.0.0/16` or `10.32.0.0/12` and,
+ * `<exposing-host>` is the address of the machine on which you ran `weave expose`.
+
+>**Note:** You must ensure that the [IP subnet used by Weave
+Net](/site/ipam.md#range) does not clash with anything on those other
+hosts.
+
 
 **See Also**
 
  * [Using Weave Net](/site/using-weave.md)
- * [Managing Services in Weave: Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md)
+ * [General information on IP Addresses, Routes and Networks](/site/how-it-works/ip-addresses.md)
+ * [Managing Services - Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md)

site/using-weave/manual-ip-address.md

@@ -10,7 +10,7 @@ Containers are automatically allocated an IP address that is unique across the W
 
 Weave Net detects when a container has exited and releases its allocated addresses so they can be re-used by the network.
 
-See the [Automatic IP Address Management](/site/ipam.md) and also an explanation of [the basics of IP addressing](/how-it-works/ip-addresses.md) for further details.
+See the [Automatic IP Address Management](/site/ipam.md) and also an explanation of [the basics of IP addressing](/site/how-it-works/ip-addresses.md) for further details.
 
 Instead of allowing Weave Net to allocate IP addresses automatically (using IPAM), there may be instances where you need to control a particular container or a cluster by setting an IP address for it.  
 
@@ -51,7 +51,7 @@ Individual IP addresses given to containers must, of course, be unique. If you p
 
 **See Also**
 
- * [Managing Services: Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md)
- * [Configuring Weave to Explicitly Use an IP Range](/site/using-weave/configuring-weave.md) 
+ * [Managing Services - Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md)
+ * [Allocating IPs in a Specific Range](/site/using-weave/configuring-weave.md) 
  * [Automatic IP Address Management](/site/ipam.md)   
 

site/using-weave/security-untrusted-networks.md

@@ -45,5 +45,5 @@ Be aware that:
 
 **See Also**
 
- * [Weave Encryption](/how-it-works/encryption.md)
+ * [Weave Encryption](/site/how-it-works/encryption.md)
  * [Using Fast Datapath](/site/using-weave/fastdp.md)

site/using-weave/service-export.md

@@ -1,39 +0,0 @@
----
-title: Exposing Services to the Outside World
-menu_order: 70
----
-
-Services running in containers on a Weave network can be made
-accessible to the outside world (and, more generally, to other networks)
-from any Weave Net host, irrespective of where the service containers are
-located.
-
-Returning to the [netcat example service](/site/using-weave.md), 
-you can expose the netcat service running on `HOST1` and make it accessible to the outside world via `$HOST2`. 
-
-First, expose the application network to `$HOST2`, as explained in [Integrating a Host Network with a Weave Net](/site/using-weave/host-network-integration.md):
-
-    host2$ weave expose
-    10.2.1.132
-
-Then add a NAT rule that routes the traffic from the outside world to the destination container service.
-
-    host2$ iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 2211 \
-           -j DNAT --to-destination $(weave dns-lookup a1):4422
-
-In this example, it is assumed that the "outside world" is connecting to `$HOST2` via 'eth0'. The TCP traffic to port 2211 on the external IPs will be routed to the 'nc' service running on port 4422 in the container a1.
-
-With the above in place, you can connect to the 'nc' service from anywhere using:
-
-    echo 'Hello, world.' | nc $HOST2 2211
-
->**Note:** Due to the way routing is handled in the Linux kernel, this won't work when run *on* `$HOST2`.
-
-Similar NAT rules to the above can be used to expose services not just to the outside world but also to other, internal, networks.
-
-
-
-**See Also**
-
- * [Using Weave Net](/site/using-weave.md)
- * [Managing Services in Weave Net: Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md)

site/using-weave/service-management.md

@@ -1,6 +1,6 @@
 ---
-title: Managing Services in Weave Net - Exporting, Importing, Binding and Routing
-menu_order: 90
+title: Managing Services - Exporting, Importing, Binding and Routing
+menu_order: 70
 ---
 
 This section contains the following topics: 
@@ -13,31 +13,35 @@ This section contains the following topics:
 
 
 
-###<a name="exporting"></a> Exporting Services
+###<a name="exporting"></a>Exporting Services
 
-Services running in containers on a Weave network can be made accessible to the outside world (and, more generally, to other networks) from any Weave host, regardless of where the service containers are located.
+Services running in containers on a Weave network can be made
+accessible to the outside world (and, more generally, to other networks)
+from any Weave Net host, irrespective of where the service containers are
+located.
 
-Turning back to the [netcat example](/site/using-weave.md), say you want the netcat service, which is running in a container on `$HOST1`, to be accessible to the outside world via `$HOST2`.
+Returning to the [netcat example service](/site/using-weave.md), 
+you can expose the netcat service running on `HOST1` and make it accessible to the outside world via `$HOST2`. 
 
-To do this, expose the application network to `$HOST2`, as explained in [Host Network Integration](/site/using-weave/host-network-integration.md) by running:
+First, expose the application network to `$HOST2`, as explained in [Integrating with the Host Network](/site/using-weave/host-network-integration.md):
 
     host2$ weave expose
     10.2.1.132
 
-Then add a NAT rule to route from the outside world to the destination container service.
+Then add a NAT rule that routes the traffic from the outside world to the destination container service.
 
     host2$ iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 2211 \
            -j DNAT --to-destination $(weave dns-lookup a1):4422
 
-Here it is assumed that the "outside world" is connecting to `$HOST2` via 'eth0'. The TCP traffic to port 2211 on the external IPs will be routed to the 'nc' service, running in the a1 container and listening on port 4422.
-With the above in place, you are ready to connect to the 'nc' service from anywhere using:
+In this example, it is assumed that the "outside world" is connecting to `$HOST2` via 'eth0'. The TCP traffic to port 2211 on the external IPs will be routed to the 'nc' service running on port 4422 in the container a1.
 
-    $ echo 'Hello, world.' | nc $HOST2 2211
+With the above in place, you can connect to the 'nc' service from anywhere using:
+
+    echo 'Hello, world.' | nc $HOST2 2211
 
 >**Note:** Due to the way routing is handled in the Linux kernel, this won't work when run *on* `$HOST2`.
 
-Similar NAT rules to the above can used to expose services not just to the outside world but also other, internal, networks.
-
+Similar NAT rules to the above can be used to expose services not just to the outside world but also to other, internal, networks.
 
 ###<a name="importing"></a>Importing Services
 

site/weave-docker-api.md

@@ -71,7 +71,7 @@ to the original settings prior to stopping the proxy.
 **See Also**
 
  * [Using The Weave Docker API Proxy](/site/weave-docker-api/using-proxy.md)
- * [Securing Docker Communications With TLS](securing-proxy.md)
+ * [Securing Docker Communications With TLS](/site/weave-docker-api/securing-proxy.md)
  * [Launching Containers With Weave Run (without the Proxy)](/site/weave-docker-api/launching-without-proxy.md)
 
 

site/weave-docker-api/ipam-proxy.md

@@ -1,6 +1,6 @@
 ---
 title: Automatic IP Allocation and the Weave Proxy
-layout: default
+menu_order: 25
 ---
 
 

site/weave-docker-api/name-resolution-proxy.md

@@ -17,7 +17,7 @@ returns the Weave Net IP address. This is required for many cluster-aware
 applications to work.
 - unqualified names get resolved via DNS, for example typically via weaveDNS
 to Weave Net IP addresses. This is required so that in a typical setup
-one can simply "ping <container-name>", i.e. without having to
+one can simply "ping `<container-name>`", i.e. without having to
 specify a `.weave.local` suffix.
 
 If you prefer to keep `/etc/hosts` under Docker's control (for

site/weave-docker-api/using-proxy.md

@@ -53,9 +53,16 @@ By default, multicast traffic is routed over the Weave network.
 To turn this off, for example, because you want to configure your own multicast
 route, add the `--no-multicast-route` flag to `weave launch-proxy`.
 
+###Other Weave Proxy options
+
+ * `--without-dns` -- stop telling containers to use [WeaveDNS](/site/weavedns.md)
+ * `--log-level=debug|info|warning|error` -- controls how much
+   information to emit for debugging
+ * `--no-restart` -- remove the default policy of `--restart=always`, if
+   you want to control start-up of the proxy yourself
 
 **See Also**
 
  * [Setting Up The Weave Docker API Proxy](/site/weave-docker-api.md)
- * [Securing Docker Communications With TLS](securing-proxy.md)
+ * [Securing Docker Communications With TLS](/site/weave-docker-api/securing-proxy.md)
  * [Launching Containers With Weave Run (without the Proxy)](/site/weave-docker-api/launching-without-proxy.md)

site/weavedns/weavedns-design.md

@@ -1,83 +0,0 @@
----
-title: WeaveDNS (service discovery) Design Notes
-layout: default
----
-
-# WeaveDNS (service discovery) Design Notes
-
-The model is that each host has a service that is notified of
-hostnames and weave addresses for containers on the host.  Like IPAM,
-this service is embedded within the router.  It binds to
-the host bridge to answer DNS queries from local containers; for
-anything it can't answer, it uses the information in the host's
-/etc/resolv.conf to query an 'fallback' server.
-
-The service is comprised of a DNS server, which answers all DNS queries
-from containers, and a in-memory database of hostnames and IPs.  The
-database on each node contains a complete copy of the hostnames and IPs
-for every containers in the cluster.
-
-For hostname queries in the local domain (default weave.local), the DNS
-server will consult the in-memory database.  For reverse queries, we
-first consult the local database, and if not found we query the
-upstream server.  For all other queries, we consult the upstream
-server.
-
-Updates to the in-memory database are broadcast to other DNS servers
-within the cluster.  The in-memory database only contains entries from
-connected DNS servers; if a DNS server becomes partitioned from the
-cluster, entries belonging to that server are removed from each node in
-the cluster.  When the partitioned DNS server reconnects, the entries
-are re-broadcast around the cluster.
-
-The DNS server also listens to the Docker event stream, and removes
-entries for containers when they die.  Entries removed in this way are
-tombstoned, and the tombstone lazily broadcast around the cluster.
-After a short timeout the tombstones are independently removed from
-each host.
-
-
-## DNS server API
-
-The DNS server accepts HTTP requests on the following URL (patterns)
-and methods:
-
-`PUT /name/<identifier>/<ip-address>`
-
-Put a record for an IP, bound to a host-scoped identifier (e.g., a
-container ID), in the DNS database.  The request body must contain
-a `fqdn=foo.weave.local` key pair.
-
-`DELETE /name/<identifier>/<ip-address>`
-
-Remove a specific record for an IP and host-scoped identifier. The request
-body can optionally contain a `fqdn=foo.weave.local` key pair.
-
-`DELETE /name/<identifier>`
-
-Remove all records for the host-scoped identifier.
-
-`GET /name/<fqdn>`
-
-List of all IPs (in JSON format) for givne FQDN.
-
-## DNS updater
-
-The updater component uses the Docker remote API to monitor containers
-coming and going, and tells the DNS server to update its records via
-its HTTP interface. It does not need to be attached to the weave
-network.
-
-The updater starts by subscribing to the events, and getting a list of
-the current containers. Any containers given a domain ending with
-".weave" are considered for inclusion in the name database.
-
-When it sees a container start or stop, the updater checks the weave
-network attachment of the container, and updates the DNS server.
-
-> How does it check the network attachment from within a container?
-
-> Will it need to delay slightly so that `attach` has a chance to run?
-> Perhaps it could put containers on a watch list when it's noticed
-> them.
-

test/100_cross_hosts_2_test.sh

@@ -30,6 +30,8 @@ assert_raises "exec_on $HOST1 c5 $PING $C6"
 # check that restart retains the same IP, and reclaims it in IPAM
 weave_on $HOST2 restart c6
 assert_raises "exec_on $HOST1 c5 $PING $C6"
+# check that restart does not create any additional DNS entry
+assert "weave_on $HOST2 report -f \"{{len .DNS.Entries}}\"" "6"
 sleep 31 # past the IPAM timeout
 start_container $HOST2 net:$SUBNET_2 --name=c7
 C7=$(container_ip $HOST2 c7)

test/195_no_multicast_route_1_test.sh

@@ -30,4 +30,6 @@ proxy docker_on $HOST1 start c2
 
 assert "show_multicast_route_on $HOST1 c2" "224.0.0.0/4 dev ethwe "
 
+docker_on $HOST1 rm -f c3
+
 end_suite

test/250_dns_negative_test.sh

@@ -7,9 +7,8 @@ start_suite "Negative DNS queries"
 weave_on $HOST1 launch
 start_container_with_dns $HOST1 --name c1
 
-# unsupported query types, unknown names, and unknown domains should
-# all trigger NXDOMAIN
-assert_raises "exec_on $HOST1 c1 dig MX c1.weave.local | grep -q 'status: NXDOMAIN'"
+# unsupported query types, unknown names, and unknown domains
+assert_raises "exec_on $HOST1 c1 dig MX c1.weave.local | grep -q 'status: NOERROR'"
 assert_raises "exec_on $HOST1 c1 dig A  xx.weave.local | grep -q 'status: NXDOMAIN'"
 assert_raises "exec_on $HOST1 c1 dig A  xx.invalid     | grep -q 'status: NXDOMAIN'"
 

test/310_docker_restart_test.sh

@@ -22,10 +22,7 @@ wait_for_proxy() {
 
 start_suite "Containers get same IP address on restart"
 
-# Remove any persisted data from previous runs
-run_on $HOST1 "sudo rm -f /tmp/test163-*"
-
-WEAVE_DOCKER_ARGS="-v /tmp:/db" weave_on $HOST1 launch-router --db-prefix=/db/test163-
+weave_on $HOST1 launch-router
 weave_on $HOST1 launch-proxy
 
 # Use up first address with throwaway container

test/320_claim_3_test.sh

@@ -2,6 +2,15 @@
 
 . ./config.sh
 
+UNIVERSE=10.32.0.0/12
+
+delete_persistence() {
+    for host in "$@" ; do
+        docker_on $host rm -v weavedb >/dev/null
+        docker_on $host rm weave >/dev/null
+    done
+}
+
 start_suite "Claiming addresses"
 
 weave_on $HOST1 launch-router
@@ -15,6 +24,9 @@ assert_raises "exec_on $HOST2 c2 $PING $C1"
 stop_weave_on $HOST1
 stop_weave_on $HOST2
 
+# Delete persistence data so they form a blank ring
+delete_persistence $HOST1 $HOST2
+
 # Start hosts in reverse order so c1's address has to be claimed from host2
 weave_on $HOST2 launch-router
 weave_on $HOST1 launch-router $HOST2
@@ -31,12 +43,33 @@ assert_raises "exec_on $HOST1 c1 $PING $C3"
 stop_weave_on $HOST1
 stop_weave_on $HOST2
 
+delete_persistence $HOST1 $HOST2
+
 # Now make host1 attempt to claim from host2, when host2 is stopped
 # the point being to check whether host1 will hang trying to talk to host2
 weave_on $HOST2 launch-router
 # Introduce host3 to remember the IPAM CRDT when we stop host2
 weave_on $HOST3 launch-router $HOST2
+weave_on $HOST3 prime
 stop_weave_on $HOST2
 weave_on $HOST1 launch-router $HOST3
 
+stop_weave_on $HOST1
+stop_weave_on $HOST2
+stop_weave_on $HOST3
+delete_persistence $HOST1 $HOST2 $HOST3
+
+weave_on $HOST1 launch --ipalloc-range $UNIVERSE $HOST2
+# Start another container on host1. The starting should block, because host1 is
+# not able to establish the ring due to host2 being offline.
+CMD="proxy_start_container $HOST1 --name c4 -e WEAVE_CIDR=10.32.0.4/12"
+assert_raises "timeout 5 cat <( $CMD )" 124
+# However, allocation for an external subnet should not block.
+assert_raises "proxy_start_container $HOST1 -e WEAVE_CIDR=10.48.0.1/12"
+
+# Launch host2, so that host1 can establish the ring.
+weave_on $HOST2 launch --ipalloc-range $UNIVERSE $HOST1
+weave_on $HOST1 prime
+assert_raises "[ $(container_ip $HOST1 c4) == 10.32.0.4 ]"
+
 end_suite

test/362_ipalloc_range_change.sh

@@ -0,0 +1,14 @@
+#! /bin/bash
+
+. ./config.sh
+
+start_suite "IPAM alloc range change"
+
+weave_on $HOST1 launch --ipalloc-range 10.2.0.0/16
+weave_on $HOST1 prime
+weave_on $HOST1 stop
+weave_on $HOST1 launch --ipalloc-range 10.3.0.0/16
+# Ensure allocations can proceed
+assert_raises "timeout 10 cat <( start_container $HOST1 )"
+
+end_suite

test/380_ipam_seed_2_test.sh

@@ -19,4 +19,12 @@ weave_on $HOST2 connect $HOST1
 assert_raises "exec_on $HOST1 c1 $PING c2"
 assert_raises "exec_on $HOST2 c2 $PING c1"
 
+# Now restart one router with a different peername
+docker_on $HOST2 rm -f c2 >/dev/null
+weave_on $HOST2 forget $HOST1
+weave_on $HOST2 stop
+weave_on $HOST2 launch --name ::3
+# Check that this host has not retained the previous IPAM data
+assert "weave_on $HOST2 status ipam" ""
+
 end_suite

test/640_proxy_restart_reattaches_test.sh

@@ -60,8 +60,16 @@ check_attached
 
 assert_raises "timeout 90 cat <( run_many )"
 
+# Start a container that needs IPAM, then restart it when the router is stopped
+proxy_start_container $HOST1 -di --name=c3 --restart=always
+assert_raises "exec_on $HOST1 c3 $CHECK_ETHWE_UP"
+weave_on $HOST1 stop-router
+docker_on $HOST1 restart c3
+weave_on $HOST1 launch-router
+sleep 5
+assert_raises "exec_on $HOST1 c3 $CHECK_ETHWE_UP"
+
 # Restarting proxy shouldn't kill unattachable containers
-proxy_start_container $HOST1 -di --name=c3 --restart=always # Use ipam, so it won't be attachable w/o weave
 weave_on $HOST1 stop
 weave_on $HOST1 launch-proxy
 assert_raises "exec_on $HOST1 c3 $CHECK_ETHWE_UP"

test/800_plugin_cross_hosts_2_test.sh

@@ -18,4 +18,7 @@ start_container_local_plugin $HOST2 --name=c2 --hostname=$C2_NAME --dns=$DNS_IP
 assert_raises "exec_on $HOST1 c1 $PING $C2_NAME"
 assert_raises "exec_on $HOST2 c2 $PING $C1_NAME"
 
+docker_on $HOST1 rm -f c1
+docker_on $HOST2 rm -f c2
+
 end_suite

test/810_plugin_router_restart.sh

@@ -21,4 +21,6 @@ assert "weave_on $HOST1 dns-lookup $C1_NAME" "$c1ip"
 start_container $HOST1 --name=c2
 assert_raises "container_ip $HOST1 c2 | grep -v $c1ip"
 
+docker_on $HOST1 rm -f c1 c2
+
 end_suite

test/config.sh

@@ -226,9 +226,7 @@ assert_dns_ptr_record() {
 start_suite() {
     for host in $HOSTS; do
         [ -z "$DEBUG" ] || echo "Cleaning up on $host: removing all containers and resetting weave"
-        PLUGIN_ID=$(docker_on $host ps -aq --filter=name=weaveplugin)
-        PLUGIN_FILTER="cat"
-        [ -n "$PLUGIN_ID" ] && PLUGIN_FILTER="grep -v $PLUGIN_ID"
+        PLUGIN_FILTER=$(docker_on $host inspect -f 'grep -v {{.Id}}' weaveplugin 2>/dev/null) || PLUGIN_FILTER=cat
         rm_containers $host $(docker_on $host ps -aq 2>/dev/null | $PLUGIN_FILTER)
         weave_on $host reset 2>/dev/null
     done

weave

@@ -41,17 +41,20 @@ weave --help | help
 
 weave launch        <same arguments as 'weave launch-router'>
       launch-router [--password <pass>] [--name <mac>] [--nickname <nickname>]
+                      [--log-level=debug|info|warning|error]
                       [--no-restart] [--ipalloc-init <mode>]
                       [--ipalloc-range <cidr> [--ipalloc-default-subnet <cidr>]]
                       [--no-discovery] [--no-dns]
                       [--trusted-subnets <cidr>,...] <peer> ...
       launch-proxy  [-H <endpoint>] [--without-dns] [--no-multicast-route]
+                      [--log-level=debug|info|warning|error]
                       [--no-rewrite-hosts] [--no-default-ipalloc] [--no-restart]
                       [--hostname-from-label <labelkey>]
                       [--hostname-match <regexp>]
                       [--hostname-replacement <replacement>]
                       [--rewrite-inspect]
       launch-plugin [--no-restart] [--no-multicast-route]
+                      [--log-level=debug|info|warning|error]
 
 weave prime
 
@@ -825,9 +828,24 @@ with_container_netns() {
     return $STATUS
 }
 
+kill_container() {
+    docker kill $1 >/dev/null 2>&1 || true
+}
+
+kill_container_process() {
+    kill -9 $(docker inspect --format='{{.State.Pid}}' $1) >/dev/null 2>&1 || true
+}
+
 with_container_netns_or_die() {
     if ! with_container_netns "$@" >/dev/null ; then
-        docker kill $1 >/dev/null 2>&1 || true
+        kill_container $1
+        exit 1
+    fi
+}
+
+with_container_netns_or_kill() {
+    if ! with_container_netns "$@" >/dev/null ; then
+        kill_container_process $1
         exit 1
     fi
 }
@@ -1126,7 +1144,7 @@ wait_for_status() {
     while true ; do
         "$@" GET /status >/dev/null 2>&1 && return 0
         if ! check_running $container >/dev/null 2>&1 ; then
-            docker kill $container >/dev/null 2>&1 || true # stop it restarting
+            kill_container $container # stop it restarting
             echo $(death_msg $container) >&2
             return 1
         fi
@@ -1140,23 +1158,23 @@ with_container_addresses() {
     shift 1
 
     CONTAINER_ADDRS=$(util_op container-addrs $BRIDGE "$@") || return 1
-    echo "$CONTAINER_ADDRS" |  while read CONTAINER_ID CONTAINER_MAC CONTAINER_IPS; do
-        $COMMAND "$CONTAINER_ID" "$CONTAINER_MAC" "$CONTAINER_IPS"
+    echo "$CONTAINER_ADDRS" |  while read CONTAINER_ID CONTAINER_IFACE CONTAINER_MAC CONTAINER_IPS; do
+        $COMMAND "$CONTAINER_ID" "$CONTAINER_IFACE" "$CONTAINER_MAC" "$CONTAINER_IPS"
     done
 }
 
 echo_addresses() {
-    echo $1 $2 $3
+    echo $1 $3 $4
 }
 
 echo_ips() {
-    for CIDR in $3; do
-        echo ${3%/*}
+    for CIDR in $4; do
+        echo ${CIDR%/*}
     done
 }
 
 echo_cidrs() {
-    echo $3
+    echo $4
 }
 
 peer_args() {
@@ -1379,13 +1397,18 @@ check_overlap() {
 # Claim addresses for a container in IPAM.  Expects to be called from
 # with_container_addresses.
 ipam_reclaim() {
-    for CIDR in $3 ; do
-        http_call $HTTP_ADDR PUT "/ip/$1/$CIDR?noErrorOnUnknown=true&?check-alive=true" || true
+    CONTAINER_ID="$1"
+    # The weave IP addresses of containers attached by the plugin are
+    # recorded specially in IPAM, since the container id is not know
+    # at the time.
+    [ "$2" = "$CONTAINER_IFNAME" ] || CONTAINER_ID="_"
+    for CIDR in $4 ; do
+        http_call $HTTP_ADDR PUT "/ip/$CONTAINER_ID/$CIDR?noErrorOnUnknown=true&?check-alive=true" || true
     done
 }
 
 ipam_reclaim_no_check_alive() {
-    for CIDR in $3 ; do
+    for CIDR in $4 ; do
         http_call $HTTP_ADDR PUT /ip/$1/$CIDR?noErrorOnUnknown=true || true
     done
 }
@@ -1448,7 +1471,14 @@ ipam_cidrs() {
 
 ipam_cidrs_or_die() {
     if ! ipam_cidrs "$@" ; then
-        docker kill $2 >/dev/null 2>&1 || true
+        kill_container $2
+        exit 1
+    fi
+}
+
+ipam_cidrs_or_kill() {
+    if ! ipam_cidrs "$@" ; then
+        kill_container_process $2
         exit 1
     fi
 }
@@ -1937,7 +1967,7 @@ shift 1
 
 case "$COMMAND" in
     setup)
-        for img in $IMAGE $EXEC_IMAGE $PLUGIN_IMAGE ; do
+        for img in $IMAGE $EXEC_IMAGE $PLUGIN_IMAGE $WEAVEDB_IMAGE ; do
             docker pull $img
         done
         setup_cni
@@ -2129,6 +2159,9 @@ EOF
                 --or-die)
                     ATTACH_TYPE="_or_die"
                     ;;
+                --or-kill)
+                    ATTACH_TYPE="_or_kill"
+                    ;;
                 --rewrite-hosts)
                     REWRITE_HOSTS=1
                     ;;
@@ -2179,8 +2212,8 @@ EOF
         for CIDR in $ALL_CIDRS ; do
             call_weave PUT /ip/$CONTAINER/$CIDR?check-alive=true
         done
-        with_container_netns_or_die $1 attach $ALL_CIDRS
-        when_weave_running with_container_fqdn $1 put_dns_fqdn $ALL_CIDRS
+        with_container_netns_or_die $CONTAINER attach $ALL_CIDRS
+        when_weave_running with_container_fqdn $CONTAINER put_dns_fqdn $ALL_CIDRS
         echo $RES
         ;;
     dns-add)