Add release v2.1.3

Fix release script for plugin latest_release

.gitignore

@@ -33,11 +33,13 @@ coverage.html
 # Vim backup files
 *.swp
 
+# Sed backup files
+*.bak
+
 # Project specific
 prog/weaver/weaver
 prog/weavedns/weavedns
 prog/weavehosts/weavehosts
-prog/weaveproxy/weaveproxy
 prog/sigproxy/sigproxy
 prog/weavewait/weavewait
 prog/weavewait/weavewait_noop
@@ -55,7 +57,6 @@ releases
 weave*.tar.gz
 prog/weaver/weave
 prog/weaveexec/weavehosts
-prog/weaveexec/weaveproxy
 prog/weaveexec/weavewait
 prog/weaveexec/weavewait_noop
 prog/weaveexec/weavewait_nomcast

.gitmodules

@@ -133,3 +133,48 @@
 [submodule "vendor/golang.org/x/net"]
 	path = vendor/golang.org/x/net
 	url = https://github.com/golang/net
+[submodule "vendor/k8s.io/api"]
+	path = vendor/k8s.io/api
+	url = https://github.com/kubernetes/api
+[submodule "vendor/k8s.io/apimachinery"]
+	path = vendor/k8s.io/apimachinery
+	url = https://github.com/kubernetes/apimachinery
+[submodule "vendor/github.com/PuerkitoBio/purell"]
+	path = vendor/github.com/PuerkitoBio/purell
+	url = https://github.com/PuerkitoBio/purell
+[submodule "vendor/github.com/PuerkitoBio/urlesc"]
+	path = vendor/github.com/PuerkitoBio/urlesc
+	url = https://github.com/PuerkitoBio/urlesc
+[submodule "vendor/github.com/emicklei/go-restful"]
+	path = vendor/github.com/emicklei/go-restful
+	url = https://github.com/emicklei/go-restful
+[submodule "vendor/github.com/go-openapi/jsonpointer"]
+	path = vendor/github.com/go-openapi/jsonpointer
+	url = https://github.com/go-openapi/jsonpointer
+[submodule "vendor/github.com/go-openapi/jsonreference"]
+	path = vendor/github.com/go-openapi/jsonreference
+	url = https://github.com/go-openapi/jsonreference
+[submodule "vendor/github.com/go-openapi/spec"]
+	path = vendor/github.com/go-openapi/spec
+	url = https://github.com/go-openapi/spec
+[submodule "vendor/github.com/go-openapi/swag"]
+	path = vendor/github.com/go-openapi/swag
+	url = https://github.com/go-openapi/swag
+[submodule "vendor/github.com/golang/glog"]
+	path = vendor/github.com/golang/glog
+	url = https://github.com/golang/glog
+[submodule "vendor/github.com/google/gofuzz"]
+	path = vendor/github.com/google/gofuzz
+	url = https://github.com/google/gofuzz
+[submodule "vendor/github.com/mailru/easyjson"]
+	path = vendor/github.com/mailru/easyjson
+	url = https://github.com/mailru/easyjson
+[submodule "vendor/k8s.io/kube-openapi"]
+	path = vendor/k8s.io/kube-openapi
+	url = https://github.com/kubernetes/kube-openapi
+[submodule "vendor/golang.org/x/text"]
+	path = vendor/golang.org/x/text
+	url = https://go.googlesource.com/text
+[submodule "vendor/gopkg.in/yaml.v2"]
+	path = vendor/gopkg.in/yaml.v2
+	url = https://gopkg.in/yaml.v2

CHANGELOG.md

@@ -1,3 +1,143 @@
+## Release 2.1.3
+
+This release fixes a race-condition in the IP reclaim code for weave-kube
+where two nodes could end up fighting over the same space and break
+connectivity #3190, #3192
+
+## Release 2.1.2
+
+This release fixes a couple of bugs discovered since the release of Weave Net 2.1.0
+
+##Bug fixes
+
+* Fix crash seen when starting 10-15 nodes simultaneously #3184,#3186
+* Fix NetworkPolicy blocking traffic if updates come out of order from Kubernetes #3177,#3181
+
+Thanks to the following contributors:
+@zignig
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/66?closed=1)
+
+
+## Release 2.1.1
+
+As 2.1.0, but fixing a couple of installation glitches. #3175,#3176
+
+## Release 2.1.0
+
+##New Features
+
+Improved Kubernetes Network Policy - Weave Net now supports the
+'v1' policies introduced in Kubernetes 1.7 as well as the 'beta'
+policies supported previously. See [Kubernetes 1.7 changelog](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.7.md#network)
+for differences. To use old policies, `--use-legacy-netpol` argument
+should be passed to `weave-npc`. #3105,#3141,#3151,#3169
+
+Weave Net now reclaims IP addresses owned by Kubernetes nodes which
+have been deleted from the cluster - this avoids running out of IP
+addresses when many nodes are added and deleted over a long period.
+#2797,#3149,#3170,#3172
+
+##Other improvements
+
+* Export a Prometheus-style metric giving count of unreachable peers #3119
+* Update 'gopacket' library to reduce memory use by approx 15MB #3160
+* Replace bundling the 'docker' binary with our own code to avoid
+  security vulnerability alerts and save space #2957,#3110
+
+##Bug fixes
+
+* When `weave expose` is used, allow traffic into the Weave network -
+  up till version 1.12 Docker would do this for us, but in 1.13 they
+  stopped so now we do it. This change makes `weave expose` require
+  Weave Net to be running. #2758,#3122
+* Arm64 build now works on non-kubernetes installs #2832,#3110
+* TX offload was being disabled in 'awsvpc' mode, which slows down packet sending #3089
+* Removed spurious 'nil' in logs from CNI DEL operation #3143
+
+##Build and test
+
+* Images are now also built for the ppc64le platform #3129
+* Tweak build scripts to run on OSX as well as Linux #3135
+
+##External Contributors
+
+Thanks to the following contributors:
+@caarlos0
+@dtshepherd
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/59?closed=1)
+
+
+## Release 2.0.5
+
+Bug fixes
+
+* Fix /etc/hosts inside containers so the container's name resolves to
+  its Weave Net address #3136,#3138
+* New weave-kube config for Kubernetes 1.7 and 1.8 which resolves an
+  intermittent conflict with kube-proxy that could break Weave Net
+  until reboot #2998,#3134
+* Remove persistence file in `weave reset` when using Docker plugin V2 #3103,#3114
+
+Build and test
+
+* Some trivial changes to placate go-lint #3137
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/63?closed=1).
+
+
+## Release 2.0.4
+
+Bug fixes
+
+* weave-npc failed on Centos 7, due to older 'ipset' version in kernel #3099,#3100
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/62?closed=1).
+
+
+## Release 2.0.3
+
+Bug fixes
+
+* Weave-npc would crash on a policy with no 'from' part - regression introduced in 2.0.2 #3095,#3096,#3097
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/61?closed=1).
+
+
+## Release 2.0.2
+
+Bug fixes and minor improvements
+
+* Fix race condition in weave-npc which would intermittently block all traffic for a namespace  #3057,#3059
+* Ensure Fast Datapath works on machines that need to mount the kernel module dynamically #3080
+* Regression: weave-npc would block everything if `kubelet --hostname-override` was used #3049,#3051
+* Fix netfilter rules to block containers from accessing the Weave Net control endpoint #3093
+* If DNS server is off then disable proxy DNS registration, to avoid spurious errors #3054,#3088
+* Add comments to each iptables rule and ipset, to help when troubleshooting #3064
+* Remove code that checked for an outdated fallback address for Kubernetes api-server #3071
+* Add a label to the weavedb image so it can be filtered out by tools #3066
+* Fix various build and continuous-integration failures #3061
+* Print 'help' text faster in the weave script #3056
+* Add an option to create continuous integration hosts in different ways  #3060
+* Remove remnants of the pre-2.0 proxy and plugin from build and test #3035,#3036
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/60?closed=1).
+
+
+## Release 2.0.1
+
+Bug fixes and minor improvements
+
+* Fall back to slower data path (`sleeve`), rather than crashing, when the machine lacks VXLAN support (required for “fast data path”, `fastdp`)  #3043
+* Fix bug in processing of arguments when Docker has TLS enabled, rather than crashing with invalid peers list, e.g. `lookup --tlsverify: no such host` #3039
+* Add `kube-system` namespace back to `weave-kube`'s YAMLs, preventing omissions leading to errors like `error contacting APIServer: the server does not allow access to the requested resource` #3033,#3042
+* Fix release script to prevent ARM64 binaries to end up in AMD64 `net-plugin`, leading to `Error response from daemon: dial unix /run/docker/plugins/<id>/weave.sock: connect: no such file or directory` when installing `net-plugin` #3045
+* `weave reset` and `weave rmpeer` now only contact Weave Cloud when Weave Net is configured with a Weave Cloud token, preventing unnecessary requests and potentially confusing `401 Unauthorized` errors in Weave Net’s logs #3044
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/58?closed=1).
+
+
 ## Release 2.0.0
 
 New Features

DEPENDENCIES

@@ -1,3 +1,3 @@
-DOCKER_VERSION=1.13.1
-KUBERNETES_VERSION=1.6.2
+DOCKER_VERSION=17.06
+KUBERNETES_VERSION=1.7.8
 KUBERNETES_CNI_VERSION=0.5.1

LICENSE

@@ -176,7 +176,7 @@
 
    END OF TERMS AND CONDITIONS
 
-   Copyright 2014-2016 Weaveworks Ltd.
+   Copyright 2014-2017 Weaveworks Ltd.
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

Makefile

@@ -17,15 +17,15 @@ QEMU_VERSION=v2.7.0
 
 # A list of all supported architectures here. Should be named as Go is naming platforms
 # All supported architectures must have an "ifeq" block below that customizes the parameters
-ALL_ARCHITECTURES=amd64 arm arm64
-ML_PLATFORMS=linux/amd64,linux/arm,linux/arm64
+ALL_ARCHITECTURES=amd64 arm arm64 ppc64le
+ML_PLATFORMS=linux/amd64,linux/arm,linux/arm64,linux/ppc64le
 
 ifeq ($(ARCH),amd64)
 # The architecture to use when downloading the docker binary
 	WEAVEEXEC_DOCKER_ARCH?=x86_64
 
 # The name of the alpine baseimage to use as the base for weave images
-	ALPINE_BASEIMAGE?=alpine:3.4
+	ALPINE_BASEIMAGE?=alpine:3.6
 
 # The extension for the made images
 # Specifying none means for example weaveworks/weave:latest
@@ -42,7 +42,7 @@ ifeq ($(ARCH),arm)
 	WEAVEEXEC_DOCKER_ARCH?=armel
 
 # Using the (semi-)official alpine image
-	ALPINE_BASEIMAGE?=armhf/alpine:3.4
+	ALPINE_BASEIMAGE?=arm32v6/alpine:3.6
 
 # arm images have the -arm suffix, for instance weaveworks/weave-arm:latest
 	ARCH_EXT?=-arm
@@ -62,7 +62,7 @@ ifeq ($(ARCH),arm64)
 	WEAVEEXEC_DOCKER_ARCH?=armel
 
 # Using the (semi-)official alpine image
-	ALPINE_BASEIMAGE?=aarch64/alpine:3.5
+	ALPINE_BASEIMAGE?=arm64v8/alpine:3.6
 
 # arm64 images have the -arm64 suffix, for instance weaveworks/weave-arm64:latest
 	ARCH_EXT?=-arm64
@@ -77,6 +77,26 @@ ifeq ($(ARCH),arm64)
 # Tell the gcc linker to search for libpcap here
 	CGO_LDFLAGS="-L/usr/local/lib/$(CC)"
 endif
+ifeq ($(ARCH),ppc64le)
+# The architecture to use when downloading the docker binary
+	WEAVEEXEC_DOCKER_ARCH?=ppc64le
+
+# Using the (semi-)official alpine image
+	ALPINE_BASEIMAGE?=ppc64le/alpine:3.6
+
+# ppc64le images have the -ppc64le suffix, for instance weaveworks/weave-ppc64le:latest
+	ARCH_EXT?=-ppc64le
+
+# The name of the gcc binary
+	CC=powerpc64le-linux-gnu-gcc
+
+# The architecture name to use when downloading a prebuilt QEMU binary
+	QEMUARCH=ppc64le
+
+# In the weaveworks/build image; libpcap libraries for ppc64le are placed here
+# Tell the gcc linker to search for libpcap here
+	CGO_LDFLAGS="-L/usr/local/lib/$(CC)"
+endif
 
 # The name of the user that this Makefile should produce image artifacts for. Can/should be overridden
 DOCKERHUB_USER?=weaveworks
@@ -135,9 +155,6 @@ PUSH_ML=push_ml_weave push_ml_weaveexec push_ml_weave-kube push_ml_weave-npc
 
 WEAVE_EXPORT=weave$(ARCH_EXT).tar.gz
 
-DOCKER_VERSION=1.10.3
-DOCKER_DISTRIB=prog/weaveexec/docker-$(DOCKER_VERSION).tgz
-DOCKER_DISTRIB_URL=https://get.docker.com/builds/Linux/$(WEAVEEXEC_DOCKER_ARCH)/docker-$(DOCKER_VERSION).tgz
 NETGO_CHECK=@strings $@ | grep cgo_stub\\\.go >/dev/null || { \
 	rm $@; \
 	echo "\nYour go standard library was built without the 'netgo' build tag."; \
@@ -244,13 +261,13 @@ $(BUILD_UPTODATE): build/*
 	sed -e "s|DOCKERHUB_USER|$(DOCKERHUB_USER)|g;s|ARCH_EXT|$(ARCH_EXT)|g;s|ALPINE_BASEIMAGE|$(ALPINE_BASEIMAGE)|g;s|QEMUARCH|$(QEMUARCH)|g" $^ > $@
 ifeq ($(ARCH),amd64)
 # When building "normally" for amd64, remove the whole line, it has no part in the amd64 image
-	sed -i "/CROSS_BUILD_/d" $@
+	sed -i.bak "/CROSS_BUILD_/d" $@
 else
 # When cross-building, only the placeholder "CROSS_BUILD_" should be removed
-# Register /usr/bin/qemu-ARCH-static as the handler for ARM binaries in the kernel
+# Register /usr/bin/qemu-ARCH-static as the handler for ARM and ppc64le binaries in the kernel
 	curl -sSL https://github.com/multiarch/qemu-user-static/releases/download/$(QEMU_VERSION)/x86_64_qemu-$(QEMUARCH)-static.tar.gz | tar -xz -C $(shell dirname $@)
 	cd $(shell dirname $@) && sha256sum -c $(shell pwd)/build/shasums/qemu-$(QEMUARCH)-static.sha256sum
-	sed -i "s/CROSS_BUILD_//g" $@
+	sed -i.bak "s/CROSS_BUILD_//g" $@
 endif
 
 # The targets below builds the weave images
@@ -260,13 +277,11 @@ $(WEAVER_UPTODATE): prog/weaver/Dockerfile.$(DOCKERHUB_USER) $(WEAVER_EXE) weave
 	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker build -f prog/weaver/Dockerfile.$(DOCKERHUB_USER) -t $(WEAVER_IMAGE) prog/weaver
 	touch $@
 
-$(WEAVEEXEC_UPTODATE): prog/weaveexec/Dockerfile.$(DOCKERHUB_USER) prog/weaveexec/symlink $(DOCKER_DISTRIB) $(SIGPROXY_EXE) $(WEAVEWAIT_EXE) $(WEAVEWAIT_NOOP_EXE) $(WEAVEWAIT_NOMCAST_EXE) $(WEAVER_UPTODATE)
+$(WEAVEEXEC_UPTODATE): prog/weaveexec/Dockerfile.$(DOCKERHUB_USER) prog/weaveexec/symlink $(SIGPROXY_EXE) $(WEAVEWAIT_EXE) $(WEAVEWAIT_NOOP_EXE) $(WEAVEWAIT_NOMCAST_EXE) $(WEAVER_UPTODATE)
 	cp $(SIGPROXY_EXE) prog/weaveexec/sigproxy
 	cp $(WEAVEWAIT_EXE) prog/weaveexec/weavewait
 	cp $(WEAVEWAIT_NOOP_EXE) prog/weaveexec/weavewait_noop
 	cp $(WEAVEWAIT_NOMCAST_EXE) prog/weaveexec/weavewait_nomcast
-	tar -xf $(DOCKER_DISTRIB) usr/local/bin/docker -O > prog/weaveexec/docker
-	chmod +x prog/weaveexec/docker
 	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker build -f prog/weaveexec/Dockerfile.$(DOCKERHUB_USER) -t $(WEAVEEXEC_IMAGE) prog/weaveexec
 	touch $@
 
@@ -307,10 +322,6 @@ $(NETWORKTESTER_UPTODATE): test/images/network-tester/Dockerfile $(NETWORKTESTER
 $(WEAVE_EXPORT): $(IMAGES_UPTODATE) $(WEAVEDB_UPTODATE)
 	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker save $(addsuffix :latest,$(IMAGES)) | gzip > $@
 
-$(DOCKER_DISTRIB):
-	curl -o $(DOCKER_DISTRIB) $(DOCKER_DISTRIB_URL)
-	cd $(shell dirname $@) && sha256sum -c $(shell pwd)/build/shasums/docker-tgz-$(WEAVEEXEC_DOCKER_ARCH).sha256sum
-
 tools/.git $(MANIFEST_TOOL_DIR)/.git:
 	git submodule update --init
 
@@ -343,7 +354,7 @@ publish-one-arch: $(PUBLISH)
 $(PUBLISH): publish_%: $(IMAGES_UPTODATE)
 # Tag :latest with the real version
 	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker tag  $(DOCKERHUB_USER)/$*$(ARCH_EXT) $(DOCKERHUB_USER)/$*$(ARCH_EXT):$(WEAVE_VERSION)
-# Push the image with the arch suffix for arm and arm64, and without suffix for amd64
+# Push the image with the arch suffix for arm, arm64 and ppc64le, and without suffix for amd64
 	$(MAKE) DOCKER_HOST=$(DOCKER_HOST) DOCKERHUB_USER=$(DOCKERHUB_USER) WEAVE_VERSION=$(WEAVE_VERSION) UPDATE_LATEST=$(UPDATE_LATEST) push_$*$(ARCH_EXT)
 
 ifeq ($(ARCH),amd64)
@@ -383,7 +394,7 @@ clean-bin:
 	-$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker rmi $(IMAGES)
 	find prog -type f -name "Dockerfile.*" -not -name "Dockerfile.template" -print | xargs rm -f
 	find prog -type f -name "*qemu-*" -print | xargs rm -f
-	rm -rf $(EXES) $(IMAGES_UPTODATE) $(WEAVEDB_UPTODATE) weave*.tar.gz $(DOCKER_DISTRIB) prog/weaveexec/docker .pkg
+	rm -rf $(EXES) $(IMAGES_UPTODATE) $(WEAVEDB_UPTODATE) weave*.tar.gz .pkg
 
 clean: clean-bin clean-work-dir
 	-$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker rmi $(BUILD_IMAGE)

README.md

@@ -10,13 +10,13 @@
 [Weaveworks](https://www.weave.works) is the company that delivers the most productive way for developers to connect, observe and control
 Docker containers.
 
-This repository contains [Weave Net](https://www.weave.works/products/weave-net/), the first product developed by Weaveworks, and with over 8 million downloads to date, it enables you to get started with Docker clusters and portable apps in a fraction of the time compared with other solutions.
+This repository contains [Weave Net](https://www.weave.works/products/weave-net/), the first product developed by Weaveworks, and with over 8 million downloads to date, enables you to get started with Docker clusters and portable apps in a fraction of the time compared with other solutions.
 
 ## Weave Cloud: 
 
 [Weave Cloud](https://www.weave.works/product/cloud/) is software-as-a-service that simplifies deployment, monitoring and management for containers and microservices. It extends and complements popular orchestrators, and enables developers and DevOps to realize faster deployments, insightful monitoring, visualization and networking.
 
-Weave Cloud is built using these Open Source projects: [Weave Scope](https://www.weave.works/products/weave-scope/), a powerful container monitoring tool that automatically maps Docker containers and their interactions, [Weave Cortex](https://github.com/weaveworks/cortex), a horizontally-scalable version of Prometheus, and [Weave Flux](https://www.weave.works/products/weave-flux/), a continuous deployment tool that works with Kubernetes.
+Weave Cloud is built using these Open Source projects: [Weave Scope](https://www.weave.works/products/weave-scope/), a powerful container visualization tool that automatically maps Docker containers and their interactions, [Weave Cortex](https://github.com/weaveworks/cortex), a horizontally-scalable version of Prometheus, and [Weave Flux](https://www.weave.works/products/weave-flux/), a continuous deployment tool that works with Kubernetes.
 
 # Weave Net
 
@@ -28,7 +28,7 @@ Services provided by application containers on the Weave network can be exposed
 
 If you have any questions about, feedback for or a problem with Weave Net:
 
-- Read [the Weave Net docs](https://www.weave.works/docs/net/latest/introducing-weave/).
+- Read [the Weave Net docs](https://www.weave.works/docs/net/latest/overview).
 - Invite yourself to the <a href="https://weaveworks.github.io/community-slack/" target="_blank"> #weave-community </a> slack channel.
 - Ask a question on the <a href="https://weave-community.slack.com/messages/general/"> #weave-community</a> slack channel.
 - Join the <a href="https://www.meetup.com/pro/Weave/"> Weave User Group </a> and get invited to online talks, hands-on training and meetups in your area.
@@ -39,5 +39,5 @@ Your feedback is always welcome!
 
 ## Further Information
 
-* [Installing Weave Net](https://www.weave.works/docs/net/latest/installing-weave/)
-* [Weave Net Product Overview](https://www.weave.works/products/weave-net/)
+* [Installing Weave Net](https://www.weave.works/docs/net/latest/install/installing-weave)
+* [Weave Net Product Overview](https://www.weave.works/products/weave-net)

api/ipam.go

@@ -62,6 +62,12 @@ func (client *Client) ReleaseIPsFor(ID string) error {
 	return err
 }
 
+// release all IP space owned by a peer
+func (client *Client) RmPeer(peerName string) (string, error) {
+	result, err := client.httpVerb("DELETE", fmt.Sprintf("/peer/%s", peerName), nil)
+	return result, err
+}
+
 func (client *Client) DefaultSubnet() (*net.IPNet, error) {
 	cidr, err := client.httpVerb("GET", fmt.Sprintf("/ipinfo/defaultsubnet"), nil)
 	if err != nil {

api/router.go

@@ -0,0 +1,12 @@
+package api
+
+import (
+	"fmt"
+	"net"
+)
+
+// Expose calls the router to assign the given IP addr to the weave bridge.
+func (client *Client) Expose(ipAddr *net.IPNet) error {
+	_, err := client.httpVerb("POST", fmt.Sprintf("/expose/%s", ipAddr), nil)
+	return err
+}

bin/provision_test_vms.sh

@@ -17,7 +17,9 @@ function install_terraform() {
 
 function install_ansible() {
     sudo apt-get update || true
-    sudo apt-get install -qq -y python-pip python-dev libffi-dev libssl-dev && pip install --user -U cffi && pip install --user ansible
+    sudo apt-get install -qq -y python-pip python-dev libffi-dev libssl-dev \
+        && pip install --user -U setuptools cffi \
+        && pip install --user ansible
     export PATH="$PATH:$HOME/.local/bin"
 }
 

bin/release

@@ -88,9 +88,10 @@ build() {
 
     ## Inject the version numbers and build the distributables
     ## (library versions?)
-    sed -i "/SCRIPT_VERSION=/ c\SCRIPT_VERSION=\"$VERSION\"" ./weave
-    sed -i -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset.yaml
-    sed -i -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset-k8s-1.6.yaml
+    sed -i.bak "/SCRIPT_VERSION=/ c\SCRIPT_VERSION=\"$VERSION\"" ./weave
+    sed -i.bak -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset.yaml
+    sed -i.bak -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset-k8s-1.6.yaml
+    sed -i.bak -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset-k8s-1.7.yaml
     make SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER
 
     if make tests; then
@@ -173,6 +174,13 @@ draft() {
         --name "weave-daemonset-k8s-1.6.yaml" \
         --file "./prog/weave-kube/weave-daemonset-k8s-1.6.yaml"
 
+    github-release upload \
+        --user $GITHUB_USER \
+        --repo weave \
+        --tag $LATEST_TAG \
+        --name "weave-daemonset-k8s-1.7.yaml" \
+        --file "./prog/weave-kube/weave-daemonset-k8s-1.7.yaml"
+
     echo "** Draft $TYPE $RELEASE_NAME $VERSION created at"
     echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/$LATEST_TAG"
 }
@@ -187,14 +195,14 @@ publish() {
     fi
 
     if [ "$RELEASE_TYPE" = 'PRERELEASE' ] ; then
+        echo "== Building and pushing plugin"
+        make UPDATE_LATEST=false SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER plugin_publish
+        echo "** Plugin built and pushed"
+
         echo "== Tagging and pushing images on docker hub as user $DOCKERHUB_USER"
         make UPDATE_LATEST=$IS_MAINLINE SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER publish
         echo "** Docker images tagged and pushed"
 
-        echo "== Building and pushing plugin"
-        make UPDATE_LATEST=$IS_MAINLINE SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER plugin_publish
-        echo "** Plugin built and pushed"
-
         echo "== Publishing pre-release on GitHub"
 
         github-release publish \
@@ -223,14 +231,14 @@ publish() {
         fi
         echo '** Sanity checks OK for publishing tag' $LATEST_TAG as $DOCKERHUB_USER/weave:$VERSION
 
+        echo "== Building and pushing plugin"
+        make UPDATE_LATEST=true SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER plugin_publish
+        echo "** Plugin built and pushed"
+
         echo "== Tagging and pushing images on docker hub as user $DOCKERHUB_USER"
         make PUBLISH_WEAVEDB=$IS_MAINLINE UPDATE_LATEST=$IS_MAINLINE SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER publish
         echo "** Docker images tagged and pushed"
 
-        echo "== Building and pushing plugin"
-        make UPDATE_LATEST=$IS_MAINLINE SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER plugin_publish
-        echo "** Plugin built and pushed"
-
         echo "== Publishing release on GitHub"
 
         github-release publish \
@@ -274,6 +282,13 @@ publish() {
             --name "weave-daemonset-k8s-1.6.yaml" \
             --file "./prog/weave-kube/weave-daemonset-k8s-1.6.yaml"
 
+        github-release upload \
+            --user $GITHUB_USER \
+            --repo weave \
+            --tag latest_release \
+            --name "weave-daemonset-k8s-1.7.yaml" \
+            --file "./prog/weave-kube/weave-daemonset-k8s-1.7.yaml"
+
         echo "** Release $RELEASE_NAME $VERSION published at"
         echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/$LATEST_TAG"
         echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/latest_release"

build/Dockerfile

@@ -6,17 +6,20 @@ ENV GOARM 7
 # The names of the architectures that we're fetching cross-compilers for from the Debian packages
 ENV DEB_CROSSPLATFORMS \
 	armhf \
-	arm64
+	arm64 \
+	ppc64el
 
 # The go platforms we are supporting to cross-compile
 ENV GO_CROSSPLATFORMS \
 	linux/arm \
-	linux/arm64
+	linux/arm64 \
+	linux/ppc64le
 
 # The names of the gcc cross-compilers we have installed
 ENV GCC_CROSSCOMPILERS \
 	arm-linux-gnueabihf \
-	aarch64-linux-gnu
+	aarch64-linux-gnu \
+	powerpc64le-linux-gnu
 
 # Temporarily add the Debian repositories, because we're going to install gcc cross-compilers from there
 # Install the build-essential and crossbuild-essential-ARCH packages

build/shasums/docker-tgz-armel.sha256sum

@@ -1 +0,0 @@
-8f313a9df867c207ae9ceac6e0935d1d2a0360aafce780dd5d1e0687ee98e0c3 docker-1.10.3.tgz

build/shasums/docker-tgz-x86_64.sha256sum

@@ -1 +0,0 @@
-7541b5490cee1183d911f909c0e91d3d9d57b2b775e40f14a1a7a6de216ca4a8 docker-1.10.3.tgz

build/shasums/qemu-ppc64le-static.sha256sum

@@ -0,0 +1 @@
+926f7f265332076ea4abf6e9586288376dbed2322c155b8d0cafbee614aaa7f2  qemu-ppc64le-static

net/arp.go

@@ -17,10 +17,7 @@ func ConfigureARPCache(procPath, name string) error {
 	if err := sysctl(procPath, fmt.Sprintf("net/ipv4/neigh/%s/delay_first_probe_time", name), "2"); err != nil {
 		return err
 	}
-	if err := sysctl(procPath, fmt.Sprintf("net/ipv4/neigh/%s/ucast_solicit", name), "1"); err != nil {
-		return err
-	}
-	return nil
+	return sysctl(procPath, fmt.Sprintf("net/ipv4/neigh/%s/ucast_solicit", name), "1")
 }
 
 func sysctl(procPath, variable, value string) error {

net/bridge.go

@@ -8,7 +8,6 @@ import (
 
 	"github.com/Sirupsen/logrus"
 	"github.com/coreos/go-iptables/iptables"
-	"github.com/j-keck/arping"
 	"github.com/pkg/errors"
 	"github.com/vishvananda/netlink"
 
@@ -62,7 +61,8 @@ type Bridge interface {
 	String() string                  // human-readable type string
 }
 
-var errBridgeFallback = errors.New("special error value used to trigger a fallback to bridge type")
+// Used to indicate a fallback to the Bridge type
+var errBridgeNotSupported = errors.New("bridge not supported")
 
 type bridgeImpl struct{ bridge netlink.Link }
 type fastdpImpl struct{ datapathName string }
@@ -226,7 +226,7 @@ func (config *BridgeConfig) configuredBridgeType() Bridge {
 	}
 }
 
-func EnsureBridge(procPath string, config *BridgeConfig) (Bridge, error) {
+func EnsureBridge(procPath string, config *BridgeConfig, log *logrus.Logger) (Bridge, error) {
 	bridgeType, err := ExistingBridgeType(config.WeaveBridgeName, config.DatapathName)
 	if bridgeType != nil || err != nil {
 		return bridgeType, err
@@ -235,7 +235,8 @@ func EnsureBridge(procPath string, config *BridgeConfig) (Bridge, error) {
 	bridgeType = config.configuredBridgeType()
 	for {
 		if err := bridgeType.init(config); err != nil {
-			if err == errBridgeFallback {
+			if errors.Cause(err) == errBridgeNotSupported {
+				log.Warnf("Skipping bridge creation of %q due to: %s", bridgeType, err)
 				bridgeType = bridgeImpl{}
 				continue
 			}
@@ -333,12 +334,16 @@ func (b bridgeImpl) init(config *BridgeConfig) error {
 
 func (f fastdpImpl) init(config *BridgeConfig) error {
 	odpSupported, err := odp.CreateDatapath(f.datapathName)
+	if !odpSupported {
+		msg := ""
+		if err != nil {
+			msg = err.Error()
+		}
+		return errors.Wrap(errBridgeNotSupported, msg)
+	}
 	if err != nil {
 		return errors.Wrapf(err, "creating datapath %q", f.datapathName)
 	}
-	if !odpSupported {
-		return errBridgeFallback
-	}
 	datapath, err := netlink.LinkByName(f.datapathName)
 	if err != nil {
 		return errors.Wrapf(err, "finding datapath %q", f.datapathName)
@@ -377,11 +382,7 @@ func (bf bridgedFastdpImpl) init(config *BridgeConfig) error {
 		return errors.Wrap(err, "creating bridged fastdp veth pair")
 	}
 
-	if err := linkSetUpByName(bf.datapathName); err != nil {
-		return err
-	}
-
-	return nil
+	return linkSetUpByName(bf.datapathName)
 }
 
 func (b bridgeImpl) attach(veth *netlink.Veth) error {
@@ -454,6 +455,15 @@ func configureIPTables(config *BridgeConfig) error {
 			return err
 		}
 	}
+
+	if !config.NPC {
+		// Create a chain for allowing ingress traffic when the bridge is exposed
+		_ = ipt.NewChain("filter", "WEAVE-EXPOSE")
+		if err = ipt.AppendUnique("filter", "FORWARD", "-o", config.WeaveBridgeName, "-j", "WEAVE-EXPOSE"); err != nil {
+			return err
+		}
+	}
+
 	// Forward from weave to the rest of the world
 	if err = ipt.AppendUnique("filter", "FORWARD", "-i", config.WeaveBridgeName, "!", "-o", config.WeaveBridgeName, "-j", "ACCEPT"); err != nil {
 		return err
@@ -467,11 +477,7 @@ func configureIPTables(config *BridgeConfig) error {
 	if err = ipt.ClearChain("nat", "WEAVE"); err != nil {
 		return errors.Wrap(err, "clearing WEAVE chain")
 	}
-	if err = ipt.AppendUnique("nat", "POSTROUTING", "-j", "WEAVE"); err != nil {
-		return err
-	}
-
-	return nil
+	return ipt.AppendUnique("nat", "POSTROUTING", "-j", "WEAVE")
 }
 
 func linkSetUpByName(linkName string) error {
@@ -481,35 +487,3 @@ func linkSetUpByName(linkName string) error {
 	}
 	return netlink.LinkSetUp(link)
 }
-
-func AddBridgeAddr(bridgeName string, addr *net.IPNet, removeDefaultRoute bool) error {
-	link, err := netlink.LinkByName(bridgeName)
-	if err != nil {
-		return errors.Wrapf(err, "AddBridgeAddr finding bridge %q", bridgeName)
-	}
-	newAddresses, err := AddAddresses(link, []*net.IPNet{addr})
-	if err != nil {
-		return errors.Wrapf(err, "adding address %v to %q", addr, bridgeName)
-	}
-	for _, ipnet := range newAddresses {
-		arping.GratuitousArpOverIfaceByName(ipnet.IP, bridgeName)
-	}
-	if removeDefaultRoute {
-		routeFilter := &netlink.Route{
-			LinkIndex: link.Attrs().Index,
-			Dst:       &net.IPNet{IP: addr.IP.Mask(addr.Mask), Mask: addr.Mask},
-			Protocol:  2, // RTPROT_KERNEL
-		}
-		filterMask := netlink.RT_FILTER_OIF | netlink.RT_FILTER_DST | netlink.RT_FILTER_PROTOCOL
-		routes, err := netlink.RouteListFiltered(netlink.FAMILY_V4, routeFilter, filterMask)
-		if err != nil {
-			return errors.Wrapf(err, "failed to get route list for bridge %q", bridgeName)
-		}
-		for _, r := range routes {
-			if err = netlink.RouteDel(&r); err != nil {
-				return errors.Wrapf(err, "failed to delete default route %+v", r)
-			}
-		}
-	}
-	return nil
-}

net/expose.go

@@ -0,0 +1,115 @@
+package net
+
+import (
+	"net"
+	"syscall"
+
+	"github.com/coreos/go-iptables/iptables"
+	"github.com/j-keck/arping"
+	"github.com/pkg/errors"
+	"github.com/vishvananda/netlink"
+)
+
+// Expose makes the network accessible from a host by assigning a given IP address
+// to the weave bridge.
+//
+// List of params:
+// * "bridgeName" - a name of the weave bridge.
+// * "ipAddr" - IP addr to be assigned to the bridge.
+// * "removeDefaultRoute" - whether to remove a default route installed by the kernel (used only in the AWSVPC mode).
+// * "npc" - whether is Weave NPC running.
+func Expose(bridgeName string, ipAddr *net.IPNet, removeDefaultRoute, npc bool) error {
+	ipt, err := iptables.New()
+	if err != nil {
+		return errors.Wrap(err, "iptables.New")
+	}
+	cidr := ipAddr.String()
+
+	if err := addBridgeIPAddr(bridgeName, ipAddr, removeDefaultRoute); err != nil {
+		return errors.Wrap(err, "addBridgeIPAddr")
+	}
+
+	if err := exposeNAT(ipt, cidr); err != nil {
+		return errors.Wrap(err, "exposeNAT")
+	}
+
+	if !npc {
+		// Docker 1.13 has changed a default policy of FORWARD chain to DROP
+		// (https://github.com/moby/moby/pull/28257) which makes containers
+		// inaccessible from a remote host when the bridge is exposed.
+		//
+		// The change breaks e.g. the AWSVPC mode. To overcome this we install
+		// an explicit rule for accepting forwarded ingress traffic to an
+		// exposed subnet.
+		if err := ipt.AppendUnique("filter", "WEAVE-EXPOSE", "-d", cidr, "-j", "ACCEPT"); err != nil {
+			return errors.Wrap(err, "ipt.AppendUnique")
+		}
+	}
+
+	return nil
+}
+
+func addBridgeIPAddr(bridgeName string, addr *net.IPNet, removeDefaultRoute bool) error {
+	link, err := netlink.LinkByName(bridgeName)
+	if err != nil {
+		return errors.Wrapf(err, "addBridgeIPAddr finding bridge %q", bridgeName)
+	}
+	err = netlink.AddrAdd(link, &netlink.Addr{IPNet: addr})
+	// The IP addr might have been already set by a concurrent request, just ignore then
+	if err != nil && err != syscall.Errno(syscall.EEXIST) {
+		return errors.Wrapf(err, "adding address %v to %q", addr, bridgeName)
+	}
+
+	// Sending multiple ARP REQUESTs in the case of EEXIST above does not hurt
+	arping.GratuitousArpOverIfaceByName(addr.IP, bridgeName)
+
+	// Remove a default route installed by the kernel. Required by the AWSVPC mode.
+	if removeDefaultRoute {
+		routeFilter := &netlink.Route{
+			LinkIndex: link.Attrs().Index,
+			Dst:       &net.IPNet{IP: addr.IP.Mask(addr.Mask), Mask: addr.Mask},
+			Protocol:  2, // RTPROT_KERNEL
+		}
+		filterMask := netlink.RT_FILTER_OIF | netlink.RT_FILTER_DST | netlink.RT_FILTER_PROTOCOL
+		routes, err := netlink.RouteListFiltered(netlink.FAMILY_V4, routeFilter, filterMask)
+		if err != nil {
+			return errors.Wrapf(err, "failed to get route list for bridge %q", bridgeName)
+		}
+		for _, r := range routes {
+			err = netlink.RouteDel(&r)
+			// Again, there might be a concurrent request for removing routes
+			if err != nil && err != syscall.Errno(syscall.ESRCH) {
+				return errors.Wrapf(err, "failed to delete default route %+v", r)
+			}
+		}
+	}
+	return nil
+}
+
+func exposeNAT(ipt *iptables.IPTables, cidr string) error {
+	if err := addNatRule(ipt, "-s", cidr, "-d", "224.0.0.0/4", "-j", "RETURN"); err != nil {
+		return err
+	}
+	if err := addNatRule(ipt, "-d", cidr, "!", "-s", cidr, "-j", "MASQUERADE"); err != nil {
+		return err
+	}
+	return addNatRule(ipt, "-s", cidr, "!", "-d", cidr, "-j", "MASQUERADE")
+}
+
+func addNatRule(ipt *iptables.IPTables, rulespec ...string) error {
+	// Loop until we get an exit code other than "temporarily unavailable"
+	for {
+		if err := ipt.AppendUnique("nat", "WEAVE", rulespec...); err != nil {
+			if ierr, ok := err.(*iptables.Error); ok {
+				if status, ok := ierr.ExitError.Sys().(syscall.WaitStatus); ok {
+					// (magic exit code 4 found in iptables source code; undocumented)
+					if status.ExitStatus() == 4 {
+						continue
+					}
+				}
+			}
+			return err
+		}
+		return nil
+	}
+}

net/ipsec/ipsec.go

@@ -421,10 +421,7 @@ func (ipsec *IPSec) installDropNonEncrypted(srcIP, dstIP net.IP, udpPort int, in
 
 func (ipsec *IPSec) removeDropNonEncrypted(srcIP, dstIP net.IP, udpPort int, inSPI SPI) error {
 	rules := rulesDropNonEncrypted(srcIP, dstIP, udpPort, inSPI)
-	if err := ipsec.resetRules(rules, true); err != nil {
-		return err
-	}
-	return nil
+	return ipsec.resetRules(rules, true)
 }
 
 func (ipsec *IPSec) removeDropNonEncryptedInbound(srcIP, dstIP net.IP, inSPI SPI) error {

net/veth.go

@@ -4,7 +4,6 @@ import (
 	"fmt"
 	"net"
 	"strings"
-	"syscall"
 	"time"
 
 	"github.com/coreos/go-iptables/iptables"
@@ -218,11 +217,7 @@ func SetupIface(ifaceName, newIfName string) error {
 	if err := ConfigureARPCache("/proc", newIfName); err != nil {
 		return err
 	}
-	if err := ipt.Append("filter", "INPUT", "-i", newIfName, "-d", "224.0.0.0/4", "-j", "DROP"); err != nil {
-		return err
-	}
-
-	return nil
+	return ipt.Append("filter", "INPUT", "-i", newIfName, "-d", "224.0.0.0/4", "-j", "DROP")
 }
 
 // NB: This function can be used only by a process that terminates immediately
@@ -301,39 +296,3 @@ func subnets(addrs []netlink.Addr) map[string]struct{} {
 	}
 	return subnets
 }
-
-func addNatRule(ipt *iptables.IPTables, rulespec ...string) error {
-	// Loop until we get an exit code other than "temporarily unavailable"
-	for {
-		if err := ipt.AppendUnique("nat", "WEAVE", rulespec...); err != nil {
-			if ierr, ok := err.(*iptables.Error); ok {
-				if status, ok := ierr.ExitError.Sys().(syscall.WaitStatus); ok {
-					// (magic exit code 4 found in iptables source code; undocumented)
-					if status.ExitStatus() == 4 {
-						continue
-					}
-				}
-			}
-			return err
-		}
-		return nil
-	}
-}
-
-func ExposeNAT(ipnet net.IPNet) error {
-	ipt, err := iptables.New()
-	if err != nil {
-		return err
-	}
-	cidr := ipnet.String()
-	if err := addNatRule(ipt, "-s", cidr, "-d", "224.0.0.0/4", "-j", "RETURN"); err != nil {
-		return err
-	}
-	if err := addNatRule(ipt, "-d", cidr, "!", "-s", cidr, "-j", "MASQUERADE"); err != nil {
-		return err
-	}
-	if err := addNatRule(ipt, "-s", cidr, "!", "-d", cidr, "-j", "MASQUERADE"); err != nil {
-		return err
-	}
-	return nil
-}

npc/analyser.go

@@ -3,14 +3,19 @@ package npc
 import (
 	"fmt"
 
-	"k8s.io/client-go/pkg/api"
-	extnapi "k8s.io/client-go/pkg/apis/extensions/v1beta1"
-	"k8s.io/client-go/pkg/util/intstr"
+	apiv1 "k8s.io/api/core/v1"
+	extnapi "k8s.io/api/extensions/v1beta1"
+	networkingv1 "k8s.io/api/networking/v1"
+	"k8s.io/apimachinery/pkg/util/intstr"
 
 	"github.com/weaveworks/weave/npc/ipset"
 )
 
-func (ns *ns) analysePolicy(policy *extnapi.NetworkPolicy) (
+// analysePolicyLegacy is used to analyse extensions/v1beta1/NetworkPolicy (legacy) and
+// implements pre-1.7 k8s netpol semantics, whilst analysePolicy - networking.k8s.io/v1/NetworkPolicy
+// and 1.7 semantics.
+
+func (ns *ns) analysePolicyLegacy(policy *extnapi.NetworkPolicy) (
 	rules map[string]*ruleSpec,
 	nsSelectors, podSelectors map[string]*selectorSpec,
 	err error) {
@@ -19,7 +24,7 @@ func (ns *ns) analysePolicy(policy *extnapi.NetworkPolicy) (
 	podSelectors = make(map[string]*selectorSpec)
 	rules = make(map[string]*ruleSpec)
 
-	dstSelector, err := newSelectorSpec(&policy.Spec.PodSelector, ns.name, ipset.HashIP)
+	dstSelector, err := newSelectorSpec(&policy.Spec.PodSelector, true, ns.name, ipset.HashIP)
 	if err != nil {
 		return nil, nil, nil, err
 	}
@@ -45,7 +50,7 @@ func (ns *ns) analysePolicy(policy *extnapi.NetworkPolicy) (
 			} else {
 				// Ports is present and contains at least one item, then this rule allows traffic
 				// only if the traffic matches at least one port in the ports list.
-				withNormalisedProtoAndPort(ingressRule.Ports, func(proto, port string) {
+				withNormalisedProtoAndPortLegacy(ingressRule.Ports, func(proto, port string) {
 					rule := newRuleSpec(&proto, nil, dstSelector, &port)
 					rules[rule.key] = rule
 				})
@@ -56,14 +61,14 @@ func (ns *ns) analysePolicy(policy *extnapi.NetworkPolicy) (
 			for _, peer := range ingressRule.From {
 				var srcSelector *selectorSpec
 				if peer.PodSelector != nil {
-					srcSelector, err = newSelectorSpec(peer.PodSelector, ns.name, ipset.HashIP)
+					srcSelector, err = newSelectorSpec(peer.PodSelector, false, ns.name, ipset.HashIP)
 					if err != nil {
 						return nil, nil, nil, err
 					}
 					podSelectors[srcSelector.key] = srcSelector
 				}
 				if peer.NamespaceSelector != nil {
-					srcSelector, err = newSelectorSpec(peer.NamespaceSelector, "", ipset.ListSet)
+					srcSelector, err = newSelectorSpec(peer.NamespaceSelector, false, "", ipset.ListSet)
 					if err != nil {
 						return nil, nil, nil, err
 					}
@@ -77,6 +82,80 @@ func (ns *ns) analysePolicy(policy *extnapi.NetworkPolicy) (
 				} else {
 					// Ports is present and contains at least one item, then this rule allows traffic
 					// only if the traffic matches at least one port in the ports list.
+					withNormalisedProtoAndPortLegacy(ingressRule.Ports, func(proto, port string) {
+						rule := newRuleSpec(&proto, srcSelector, dstSelector, &port)
+						rules[rule.key] = rule
+					})
+				}
+			}
+		}
+	}
+
+	return rules, nsSelectors, podSelectors, nil
+}
+
+func (ns *ns) analysePolicy(policy *networkingv1.NetworkPolicy) (
+	rules map[string]*ruleSpec,
+	nsSelectors, podSelectors map[string]*selectorSpec,
+	err error) {
+
+	nsSelectors = make(map[string]*selectorSpec)
+	podSelectors = make(map[string]*selectorSpec)
+	rules = make(map[string]*ruleSpec)
+
+	// If empty, matches all pods in a namespace
+	dstSelector, err := newSelectorSpec(&policy.Spec.PodSelector, true, ns.name, ipset.HashIP)
+	if err != nil {
+		return nil, nil, nil, err
+	}
+	podSelectors[dstSelector.key] = dstSelector
+
+	// If ingress is empty then this NetworkPolicy does not allow any traffic
+	if policy.Spec.Ingress == nil || len(policy.Spec.Ingress) == 0 {
+		return
+	}
+
+	for _, ingressRule := range policy.Spec.Ingress {
+		// If Ports is empty or missing, this rule matches all ports
+		allPorts := ingressRule.Ports == nil || len(ingressRule.Ports) == 0
+		// If From is empty or missing, this rule matches all sources
+		allSources := ingressRule.From == nil || len(ingressRule.From) == 0
+
+		if allSources {
+			if allPorts {
+				rule := newRuleSpec(nil, nil, dstSelector, nil)
+				rules[rule.key] = rule
+			} else {
+				withNormalisedProtoAndPort(ingressRule.Ports, func(proto, port string) {
+					rule := newRuleSpec(&proto, nil, dstSelector, &port)
+					rules[rule.key] = rule
+				})
+			}
+		} else {
+			for _, peer := range ingressRule.From {
+				var srcSelector *selectorSpec
+
+				// NetworkPolicyPeer describes a peer to allow traffic from.
+				// Exactly one of its fields must be specified.
+				if peer.PodSelector != nil {
+					srcSelector, err = newSelectorSpec(peer.PodSelector, false, ns.name, ipset.HashIP)
+					if err != nil {
+						return nil, nil, nil, err
+					}
+					podSelectors[srcSelector.key] = srcSelector
+
+				} else if peer.NamespaceSelector != nil {
+					srcSelector, err = newSelectorSpec(peer.NamespaceSelector, false, "", ipset.ListSet)
+					if err != nil {
+						return nil, nil, nil, err
+					}
+					nsSelectors[srcSelector.key] = srcSelector
+				}
+
+				if allPorts {
+					rule := newRuleSpec(nil, srcSelector, dstSelector, nil)
+					rules[rule.key] = rule
+				} else {
 					withNormalisedProtoAndPort(ingressRule.Ports, func(proto, port string) {
 						rule := newRuleSpec(&proto, srcSelector, dstSelector, &port)
 						rules[rule.key] = rule
@@ -89,26 +168,40 @@ func (ns *ns) analysePolicy(policy *extnapi.NetworkPolicy) (
 	return rules, nsSelectors, podSelectors, nil
 }
 
-func withNormalisedProtoAndPort(npps []extnapi.NetworkPolicyPort, f func(proto, port string)) {
+func withNormalisedProtoAndPortLegacy(npps []extnapi.NetworkPolicyPort, f func(proto, port string)) {
 	for _, npp := range npps {
-		// If no proto is specified, default to TCP
-		proto := string(api.ProtocolTCP)
-		if npp.Protocol != nil {
-			proto = string(*npp.Protocol)
-		}
-
-		// If no port is specified, match any port. Let iptables executable handle
-		// service name resolution
-		port := "0:65535"
-		if npp.Port != nil {
-			switch npp.Port.Type {
-			case intstr.Int:
-				port = fmt.Sprintf("%d", npp.Port.IntVal)
-			case intstr.String:
-				port = npp.Port.StrVal
-			}
-		}
-
-		f(proto, port)
+		f(proto(npp.Protocol), port(npp.Port))
 	}
 }
+
+func withNormalisedProtoAndPort(npps []networkingv1.NetworkPolicyPort, f func(proto, port string)) {
+	for _, npp := range npps {
+		f(proto(npp.Protocol), port(npp.Port))
+	}
+}
+
+func proto(p *apiv1.Protocol) string {
+	// If no proto is specified, default to TCP
+	proto := string(apiv1.ProtocolTCP)
+	if p != nil {
+		proto = string(*p)
+	}
+
+	return proto
+}
+
+func port(p *intstr.IntOrString) string {
+	// If no port is specified, match any port. Let iptables executable handle
+	// service name resolution
+	port := "0:65535"
+	if p != nil {
+		switch p.Type {
+		case intstr.Int:
+			port = fmt.Sprintf("%d", p.IntVal)
+		case intstr.String:
+			port = p.StrVal
+		}
+	}
+
+	return port
+}

npc/controller.go

@@ -3,13 +3,14 @@ package npc
 import (
 	"sync"
 
-	"github.com/coreos/go-iptables/iptables"
 	"github.com/pkg/errors"
-	coreapi "k8s.io/client-go/pkg/api/v1"
-	extnapi "k8s.io/client-go/pkg/apis/extensions/v1beta1"
+	coreapi "k8s.io/api/core/v1"
+	extnapi "k8s.io/api/extensions/v1beta1"
+	networkingv1 "k8s.io/api/networking/v1"
 
 	"github.com/weaveworks/weave/common"
 	"github.com/weaveworks/weave/npc/ipset"
+	"github.com/weaveworks/weave/npc/iptables"
 )
 
 type NetworkPolicyController interface {
@@ -21,9 +22,9 @@ type NetworkPolicyController interface {
 	UpdatePod(oldObj, newObj *coreapi.Pod) error
 	DeletePod(obj *coreapi.Pod) error
 
-	AddNetworkPolicy(obj *extnapi.NetworkPolicy) error
-	UpdateNetworkPolicy(oldObj, newObj *extnapi.NetworkPolicy) error
-	DeleteNetworkPolicy(obj *extnapi.NetworkPolicy) error
+	AddNetworkPolicy(obj interface{}) error
+	UpdateNetworkPolicy(oldObj, newObj interface{}) error
+	DeleteNetworkPolicy(obj interface{}) error
 }
 
 type controller struct {
@@ -31,21 +32,25 @@ type controller struct {
 
 	nodeName string // my node name
 
-	ipt *iptables.IPTables
+	ipt iptables.Interface
 	ips ipset.Interface
 
 	nss         map[string]*ns // ns name -> ns struct
 	nsSelectors *selectorSet   // selector string -> nsSelector
+
+	legacy bool // use legacy network policy semantics (k8s pre-1.7)
 }
 
-func New(nodeName string, ipt *iptables.IPTables, ips ipset.Interface) NetworkPolicyController {
+func New(nodeName string, legacy bool, ipt iptables.Interface, ips ipset.Interface) NetworkPolicyController {
 	c := &controller{
 		nodeName: nodeName,
+		legacy:   legacy,
 		ipt:      ipt,
 		ips:      ips,
 		nss:      make(map[string]*ns)}
 
-	c.nsSelectors = newSelectorSet(ips, c.onNewNsSelector)
+	doNothing := func(*selector) error { return nil }
+	c.nsSelectors = newSelectorSet(ips, c.onNewNsSelector, doNothing, doNothing)
 
 	return c
 }
@@ -54,7 +59,7 @@ func (npc *controller) onNewNsSelector(selector *selector) error {
 	for _, ns := range npc.nss {
 		if ns.namespace != nil {
 			if selector.matches(ns.namespace.ObjectMeta.Labels) {
-				if err := ns.ips.AddEntry(selector.spec.ipsetName, string(ns.allPods.ipsetName)); err != nil {
+				if err := selector.addEntry(ns.namespace.ObjectMeta.UID, string(ns.allPods.ipsetName), namespaceComment(ns)); err != nil {
 					return err
 				}
 			}
@@ -66,7 +71,7 @@ func (npc *controller) onNewNsSelector(selector *selector) error {
 func (npc *controller) withNS(name string, f func(ns *ns) error) error {
 	ns, found := npc.nss[name]
 	if !found {
-		newNs, err := newNS(name, npc.nodeName, npc.ipt, npc.ips, npc.nsSelectors)
+		newNs, err := newNS(name, npc.nodeName, npc.legacy, npc.ipt, npc.ips, npc.nsSelectors)
 		if err != nil {
 			return err
 		}
@@ -115,32 +120,47 @@ func (npc *controller) DeletePod(obj *coreapi.Pod) error {
 	})
 }
 
-func (npc *controller) AddNetworkPolicy(obj *extnapi.NetworkPolicy) error {
+func (npc *controller) AddNetworkPolicy(obj interface{}) error {
 	npc.Lock()
 	defer npc.Unlock()
 
+	nsName, err := nsName(obj)
+	if err != nil {
+		return err
+	}
+
 	common.Log.Infof("EVENT AddNetworkPolicy %s", js(obj))
-	return npc.withNS(obj.ObjectMeta.Namespace, func(ns *ns) error {
+	return npc.withNS(nsName, func(ns *ns) error {
 		return errors.Wrap(ns.addNetworkPolicy(obj), "add network policy")
 	})
 }
 
-func (npc *controller) UpdateNetworkPolicy(oldObj, newObj *extnapi.NetworkPolicy) error {
+func (npc *controller) UpdateNetworkPolicy(oldObj, newObj interface{}) error {
 	npc.Lock()
 	defer npc.Unlock()
 
+	nsName, err := nsName(oldObj)
+	if err != nil {
+		return err
+	}
+
 	common.Log.Infof("EVENT UpdateNetworkPolicy %s %s", js(oldObj), js(newObj))
-	return npc.withNS(oldObj.ObjectMeta.Namespace, func(ns *ns) error {
+	return npc.withNS(nsName, func(ns *ns) error {
 		return errors.Wrap(ns.updateNetworkPolicy(oldObj, newObj), "update network policy")
 	})
 }
 
-func (npc *controller) DeleteNetworkPolicy(obj *extnapi.NetworkPolicy) error {
+func (npc *controller) DeleteNetworkPolicy(obj interface{}) error {
 	npc.Lock()
 	defer npc.Unlock()
 
+	nsName, err := nsName(obj)
+	if err != nil {
+		return err
+	}
+
 	common.Log.Infof("EVENT DeleteNetworkPolicy %s", js(obj))
-	return npc.withNS(obj.ObjectMeta.Namespace, func(ns *ns) error {
+	return npc.withNS(nsName, func(ns *ns) error {
 		return errors.Wrap(ns.deleteNetworkPolicy(obj), "delete network policy")
 	})
 }
@@ -174,3 +194,14 @@ func (npc *controller) DeleteNamespace(obj *coreapi.Namespace) error {
 		return errors.Wrap(ns.deleteNamespace(obj), "delete namespace")
 	})
 }
+
+func nsName(obj interface{}) (string, error) {
+	switch obj := obj.(type) {
+	case *networkingv1.NetworkPolicy:
+		return obj.ObjectMeta.Namespace, nil
+	case *extnapi.NetworkPolicy:
+		return obj.ObjectMeta.Namespace, nil
+	}
+
+	return "", errInvalidNetworkPolicyObjType
+}

npc/controller_test.go

@@ -0,0 +1,378 @@
+package npc
+
+import (
+	"log"
+	"testing"
+
+	"github.com/pkg/errors"
+	"github.com/stretchr/testify/require"
+	"github.com/weaveworks/weave/npc/ipset"
+	coreapi "k8s.io/api/core/v1"
+	networkingv1 "k8s.io/api/networking/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/types"
+	"k8s.io/apimachinery/pkg/util/intstr"
+)
+
+type mockSet struct {
+	name    ipset.Name
+	setType ipset.Type
+	subSets map[string]map[types.UID]bool
+}
+
+type mockIPSet struct {
+	sets map[string]mockSet
+}
+
+func newMockIPSet() mockIPSet {
+	return mockIPSet{sets: make(map[string]mockSet)}
+}
+
+func (i *mockIPSet) Create(ipsetName ipset.Name, ipsetType ipset.Type) error {
+	if _, ok := i.sets[string(ipsetName)]; ok {
+		return errors.Errorf("ipset %s already exists", ipsetName)
+	}
+	i.sets[string(ipsetName)] = mockSet{name: ipsetName, setType: ipsetType, subSets: make(map[string]map[types.UID]bool)}
+	return nil
+}
+
+func (i *mockIPSet) AddEntry(user types.UID, ipsetName ipset.Name, entry string, comment string) error {
+	log.Printf("adding entry %s to %s for %s", entry, ipsetName, user)
+	if _, ok := i.sets[string(ipsetName)]; !ok {
+		return errors.Errorf("%s does not exist", entry)
+	}
+	if i.sets[string(ipsetName)].subSets[entry] == nil {
+		i.sets[string(ipsetName)].subSets[entry] = make(map[types.UID]bool)
+	}
+	if _, ok := i.sets[string(ipsetName)].subSets[entry][user]; ok {
+		return errors.Errorf("user %s already owns entry %s", user, entry)
+	}
+	i.sets[string(ipsetName)].subSets[entry][user] = true
+
+	return nil
+}
+
+func (i *mockIPSet) DelEntry(user types.UID, ipsetName ipset.Name, entry string) error {
+	log.Printf("deleting entry %s from %s for %s", entry, ipsetName, user)
+	if _, ok := i.sets[string(ipsetName)]; !ok {
+		return errors.Errorf("ipset %s does not exist", ipsetName)
+	}
+	if _, ok := i.sets[string(ipsetName)].subSets[entry][user]; !ok {
+		return errors.Errorf("user %s does not own entry %s", user, entry)
+	}
+	delete(i.sets[string(ipsetName)].subSets[entry], user)
+
+	if len(i.sets[string(ipsetName)].subSets[entry]) == 0 {
+		delete(i.sets[string(ipsetName)].subSets, entry)
+	}
+
+	return nil
+}
+
+func (i *mockIPSet) Exist(user types.UID, ipsetName ipset.Name, entry string) bool {
+	_, ok := i.sets[string(ipsetName)].subSets[entry][user]
+	return ok
+}
+
+func (i *mockIPSet) entryExists(ipsetName ipset.Name, entry string) bool {
+	return len(i.sets[string(ipsetName)].subSets[entry]) > 0
+}
+
+func (i *mockIPSet) Flush(ipsetName ipset.Name) error {
+	return errors.New("Not Implemented")
+}
+
+func (i *mockIPSet) FlushAll() error {
+	return errors.New("Not Implemented")
+}
+
+func (i *mockIPSet) Destroy(ipsetName ipset.Name) error {
+	if _, ok := i.sets[string(ipsetName)]; !ok {
+		return errors.Errorf("ipset %s does not exist", ipsetName)
+	}
+	delete(i.sets, string(ipsetName))
+	return nil
+}
+
+func (i *mockIPSet) DestroyAll() error {
+	return errors.New("Not Implemented")
+}
+
+func (i *mockIPSet) List(prefix string) ([]ipset.Name, error) {
+	return []ipset.Name{}, errors.New("Not Implemented")
+}
+
+type mockIPTables struct {
+}
+
+func (ipt *mockIPTables) Append(table, chain string, rulespec ...string) error {
+	return nil
+}
+
+func (ipt *mockIPTables) Delete(table, chain string, rulespec ...string) error {
+	return nil
+}
+
+func (ipt *mockIPTables) Insert(table, chain string, pos int, rulespec ...string) error {
+	return nil
+}
+
+func TestRegressionPolicyNamespaceOrdering3059(t *testing.T) {
+	// Test for race condition between namespace and networkpolicy events
+	// https://github.com/weaveworks/weave/issues/3059
+
+	sourceNamespace := &coreapi.Namespace{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "source",
+			Labels: map[string]string{
+				"app": "source",
+			},
+		},
+	}
+
+	destinationNamespace := &coreapi.Namespace{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "destination",
+		},
+	}
+
+	port := intstr.FromInt(12345)
+
+	networkPolicy := &networkingv1.NetworkPolicy{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "network-policy",
+			Namespace: "destination",
+		},
+		Spec: networkingv1.NetworkPolicySpec{
+			Ingress: []networkingv1.NetworkPolicyIngressRule{
+				{
+					From: []networkingv1.NetworkPolicyPeer{
+						{
+							NamespaceSelector: &metav1.LabelSelector{
+								MatchLabels: map[string]string{
+									"app": "source",
+								},
+							},
+						},
+					},
+					Ports: []networkingv1.NetworkPolicyPort{
+						{
+							Port: &port,
+						},
+					},
+				},
+			},
+		},
+	}
+
+	// Namespaces first
+	m := newMockIPSet()
+	controller := New("foo", false, &mockIPTables{}, &m)
+
+	const (
+		selectorIPSetName = "weave-I239Zp%sCvoVt*D6u=A!2]YEk"
+		sourceIPSetName   = "weave-HboJG1fGgG]/SR%k9H#hv5e96"
+	)
+
+	controller.AddNamespace(sourceNamespace)
+	controller.AddNamespace(destinationNamespace)
+
+	controller.AddNetworkPolicy(networkPolicy)
+
+	require.Equal(t, true, len(m.sets[selectorIPSetName].subSets[sourceIPSetName]) > 0)
+
+	// NetworkPolicy first
+	m = newMockIPSet()
+	controller = New("foo", false, &mockIPTables{}, &m)
+
+	controller.AddNetworkPolicy(networkPolicy)
+
+	controller.AddNamespace(sourceNamespace)
+	controller.AddNamespace(destinationNamespace)
+
+	require.Equal(t, true, len(m.sets[selectorIPSetName].subSets[sourceIPSetName]) > 0)
+}
+
+// Tests default-allow ipset behavior when running in non-legacy mode.
+func TestDefaultAllow(t *testing.T) {
+	const (
+		defaultAllowIPSetName = "weave-E.1.0W^NGSp]0_t5WwH/]gX@L"
+		fooPodIP              = "10.32.0.10"
+		barPodIP              = "10.32.0.11"
+		barPodNewIP           = "10.32.0.12"
+	)
+
+	m := newMockIPSet()
+	controller := New("bar", false, &mockIPTables{}, &m)
+
+	defaultNamespace := &coreapi.Namespace{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "default",
+		},
+	}
+	controller.AddNamespace(defaultNamespace)
+
+	// Should create an ipset for default-allow
+	require.Contains(t, m.sets, defaultAllowIPSetName)
+
+	podFoo := &coreapi.Pod{
+		ObjectMeta: metav1.ObjectMeta{
+			UID:       "foo",
+			Namespace: "default",
+			Name:      "foo",
+			Labels:    map[string]string{"run": "foo"}},
+		Status: coreapi.PodStatus{PodIP: fooPodIP}}
+	controller.AddPod(podFoo)
+
+	// Should add the foo pod to default-allow
+	require.True(t, m.entryExists(defaultAllowIPSetName, fooPodIP))
+
+	podBar := &coreapi.Pod{
+		ObjectMeta: metav1.ObjectMeta{
+			UID:       "bar",
+			Namespace: "default",
+			Name:      "bar",
+			Labels:    map[string]string{"run": "bar"}},
+		Status: coreapi.PodStatus{PodIP: barPodIP}}
+	podBarNoIP := &coreapi.Pod{ObjectMeta: podBar.ObjectMeta}
+	controller.AddPod(podBarNoIP)
+	controller.UpdatePod(podBarNoIP, podBar)
+
+	// Should add the bar pod to default-allow
+	require.True(t, m.entryExists(defaultAllowIPSetName, barPodIP))
+
+	// Allow access from the bar pod to the foo pod
+	netpol := &networkingv1.NetworkPolicy{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "allow-from-bar-to-foo",
+			Namespace: "default",
+		},
+		Spec: networkingv1.NetworkPolicySpec{
+			PodSelector: metav1.LabelSelector{MatchLabels: map[string]string{"run": "foo"}},
+			Ingress: []networkingv1.NetworkPolicyIngressRule{{
+				From: []networkingv1.NetworkPolicyPeer{{
+					PodSelector: &metav1.LabelSelector{
+						MatchLabels: map[string]string{"run": "bar"},
+					},
+				}},
+			}},
+		},
+	}
+	controller.AddNetworkPolicy(netpol)
+
+	// Should remove the foo pod from default-allow as the netpol selects it
+	require.False(t, m.entryExists(defaultAllowIPSetName, fooPodIP))
+	require.True(t, m.entryExists(defaultAllowIPSetName, barPodIP))
+
+	podBarWithNewIP := *podBar
+	podBarWithNewIP.Status.PodIP = barPodNewIP
+	controller.UpdatePod(podBar, &podBarWithNewIP)
+
+	// Should update IP addr of the bar pod in default-allow
+	require.False(t, m.entryExists(defaultAllowIPSetName, barPodIP))
+	require.True(t, m.entryExists(defaultAllowIPSetName, barPodNewIP))
+
+	controller.UpdatePod(&podBarWithNewIP, podBarNoIP)
+	// Should remove the bar pod from default-allow as it does not have any IP addr
+	require.False(t, m.entryExists(defaultAllowIPSetName, barPodNewIP))
+
+	podFooWithNewLabel := *podFoo
+	podFooWithNewLabel.ObjectMeta.Labels = map[string]string{"run": "new-foo"}
+	controller.UpdatePod(podFoo, &podFooWithNewLabel)
+
+	// Should bring back the foo pod to default-allow as it does not match dst of any netpol
+	require.True(t, m.entryExists(defaultAllowIPSetName, fooPodIP))
+
+	controller.UpdatePod(&podFooWithNewLabel, podFoo)
+	// Should remove from default-allow as it matches the netpol after the update
+	require.False(t, m.entryExists(defaultAllowIPSetName, fooPodIP))
+
+	controller.DeleteNetworkPolicy(netpol)
+	// Should bring back the foo pod to default-allow as no netpol selects it
+	require.True(t, m.entryExists(defaultAllowIPSetName, fooPodIP))
+
+	controller.DeletePod(podFoo)
+	// Should remove foo pod from default-allow
+	require.False(t, m.entryExists(defaultAllowIPSetName, fooPodIP))
+}
+
+func TestOutOfOrderPodEvents(t *testing.T) {
+	const (
+		defaultAllowIPSetName = "weave-E.1.0W^NGSp]0_t5WwH/]gX@L"
+		runBarIPSetName       = "weave-bZ~x=yBgzH)Ht()K*Uv3z{M]Y"
+		podIP                 = "10.32.0.10"
+	)
+
+	m := newMockIPSet()
+	controller := New("qux", false, &mockIPTables{}, &m)
+
+	defaultNamespace := &coreapi.Namespace{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "default",
+		},
+	}
+	controller.AddNamespace(defaultNamespace)
+
+	podFoo := &coreapi.Pod{
+		ObjectMeta: metav1.ObjectMeta{
+			UID:       "foo",
+			Namespace: "default",
+			Name:      "foo",
+			Labels:    map[string]string{"run": "foo"}},
+		Status: coreapi.PodStatus{PodIP: podIP}}
+	controller.AddPod(podFoo)
+
+	// Should be in default-allow as no netpol selects podFoo
+	require.True(t, m.entryExists(defaultAllowIPSetName, podIP))
+
+	netpol := &networkingv1.NetworkPolicy{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "allow-from-bar-to-foo",
+			Namespace: "default",
+		},
+		Spec: networkingv1.NetworkPolicySpec{
+			PodSelector: metav1.LabelSelector{MatchLabels: map[string]string{"run": "foo"}},
+			Ingress: []networkingv1.NetworkPolicyIngressRule{{
+				From: []networkingv1.NetworkPolicyPeer{{
+					PodSelector: &metav1.LabelSelector{
+						MatchLabels: map[string]string{"run": "bar"},
+					},
+				}},
+			}},
+		},
+	}
+	controller.AddNetworkPolicy(netpol)
+
+	// Shouldn't be in default-allow as netpol above selects podFoo
+	require.False(t, m.entryExists(defaultAllowIPSetName, podIP))
+
+	podBar := &coreapi.Pod{
+		ObjectMeta: metav1.ObjectMeta{
+			UID:       "bar",
+			Namespace: "default",
+			Name:      "bar",
+			Labels:    map[string]string{"run": "bar"}},
+		Status: coreapi.PodStatus{PodIP: podIP}}
+	controller.AddPod(podBar)
+
+	// Should be in default-allow as no netpol selects podBar
+	require.True(t, m.entryExists(defaultAllowIPSetName, podIP))
+	require.True(t, m.Exist(podBar.ObjectMeta.UID, defaultAllowIPSetName, podIP))
+	// Should be in run=bar ipset
+	require.True(t, m.entryExists(runBarIPSetName, podIP))
+
+	controller.DeletePod(podFoo)
+	// Multiple duplicate events should not affect npc state
+	controller.DeletePod(podFoo)
+	controller.DeletePod(podFoo)
+
+	// Should be in default-allow as no netpol selects podBar and podFoo removal
+	// should not affect podBar in default-allow
+	require.True(t, m.entryExists(defaultAllowIPSetName, podIP))
+
+	controller.DeletePod(podBar)
+
+	// Should remove from default-allow and run=bar ipsets
+	require.Equal(t, 0, len(m.sets[defaultAllowIPSetName].subSets))
+	require.False(t, m.entryExists(runBarIPSetName, podIP))
+}

npc/ipset/ipset.go

@@ -6,6 +6,7 @@ import (
 	"strings"
 
 	"github.com/pkg/errors"
+	"k8s.io/apimachinery/pkg/types"
 )
 
 type Name string
@@ -19,8 +20,9 @@ const (
 
 type Interface interface {
 	Create(ipsetName Name, ipsetType Type) error
-	AddEntry(ipsetName Name, entry string) error
-	DelEntry(ipsetName Name, entry string) error
+	AddEntry(user types.UID, ipsetName Name, entry string, comment string) error
+	DelEntry(user types.UID, ipsetName Name, entry string) error
+	Exist(user types.UID, ipsetName Name, entry string) bool
 	Flush(ipsetName Name) error
 	Destroy(ipsetName Name) error
 
@@ -30,52 +32,100 @@ type Interface interface {
 	DestroyAll() error
 }
 
+type entryKey struct {
+	ipsetName Name
+	entry     string
+}
+
 type ipset struct {
-	refCount
 	*log.Logger
+	enableComments bool
+	// List of users per ipset entry. User is either a namespace or a pod.
+	// There might be multiple users for the same ipset & entry pair because
+	// events from k8s API server might be out of order causing duplicate IPs:
+	// https://github.com/weaveworks/weave/issues/2792.
+	users map[entryKey]map[types.UID]struct{}
 }
 
 func New(logger *log.Logger) Interface {
-	return &ipset{refCount: newRefCount(), Logger: logger}
+	ips := &ipset{
+		Logger:         logger,
+		enableComments: true,
+		users:          make(map[entryKey]map[types.UID]struct{}),
+	}
+
+	// Check for comment support
+	testIpsetName := Name("weave-test-comment")
+	// Clear it out if it already exists
+	_ = ips.Destroy(testIpsetName)
+	// Test for comment support
+	if err := ips.Create(testIpsetName, HashIP); err != nil {
+		ips.Logger.Printf("failed to create %s; disabling comment support", testIpsetName)
+		ips.enableComments = false
+	}
+	// If it was created, destroy it
+	_ = ips.Destroy(testIpsetName)
+
+	return ips
 }
 
 func (i *ipset) Create(ipsetName Name, ipsetType Type) error {
-	return doExec("create", string(ipsetName), string(ipsetType))
+	args := []string{"create", string(ipsetName), string(ipsetType)}
+	if i.enableComments {
+		args = append(args, "comment")
+	}
+	return doExec(args...)
 }
 
-func (i *ipset) AddEntry(ipsetName Name, entry string) error {
-	i.Logger.Printf("adding entry %s to %s", entry, ipsetName)
-	if i.inc(ipsetName, entry) > 1 { // already in the set
+func (i *ipset) AddEntry(user types.UID, ipsetName Name, entry string, comment string) error {
+	i.Logger.Printf("adding entry %s to %s of %s", entry, ipsetName, user)
+
+	if !i.addUser(user, ipsetName, entry) { // already in the set
 		return nil
 	}
-	return doExec("add", string(ipsetName), entry)
+
+	i.Logger.Printf("added entry %s to %s of %s", entry, ipsetName, user)
+
+	args := []string{"add", string(ipsetName), entry}
+	if i.enableComments {
+		args = append(args, "comment", comment)
+	}
+	return doExec(args...)
 }
 
-func (i *ipset) DelEntry(ipsetName Name, entry string) error {
-	i.Logger.Printf("deleting entry %s from %s", entry, ipsetName)
-	if i.dec(ipsetName, entry) > 0 { // still needed
+func (i *ipset) DelEntry(user types.UID, ipsetName Name, entry string) error {
+	i.Logger.Printf("deleting entry %s from %s of %s", entry, ipsetName, user)
+
+	if !i.delUser(user, ipsetName, entry) { // still needed
 		return nil
 	}
+
+	i.Logger.Printf("deleted entry %s from %s of %s", entry, ipsetName, user)
+
 	return doExec("del", string(ipsetName), entry)
 }
 
+func (i *ipset) Exist(user types.UID, ipsetName Name, entry string) bool {
+	return i.existUser(user, ipsetName, entry)
+}
+
 func (i *ipset) Flush(ipsetName Name) error {
-	i.removeSet(ipsetName)
+	i.removeSetFromUsers(ipsetName)
 	return doExec("flush", string(ipsetName))
 }
 
 func (i *ipset) FlushAll() error {
-	i.refCount = newRefCount()
+	i.users = make(map[entryKey]map[types.UID]struct{})
 	return doExec("flush")
 }
 
 func (i *ipset) Destroy(ipsetName Name) error {
-	i.removeSet(ipsetName)
+	i.removeSetFromUsers(ipsetName)
 	return doExec("destroy", string(ipsetName))
 }
 
 func (i *ipset) DestroyAll() error {
-	i.refCount = newRefCount()
+	i.users = make(map[entryKey]map[types.UID]struct{})
 	return doExec("destroy")
 }
 
@@ -97,44 +147,52 @@ func (i *ipset) List(prefix string) ([]Name, error) {
 	return selected, err
 }
 
+// Returns true if entry does not exist in ipset (entry has to be inserted into ipset).
+func (i *ipset) addUser(user types.UID, ipsetName Name, entry string) bool {
+	k := entryKey{ipsetName, entry}
+	add := false
+
+	if i.users[k] == nil {
+		i.users[k] = make(map[types.UID]struct{})
+	}
+	if len(i.users[k]) == 0 {
+		add = true
+	}
+	i.users[k][user] = struct{}{}
+
+	return add
+}
+
+// Returns true if user is the last owner of entry (entry has to be removed from ipset).
+func (i *ipset) delUser(user types.UID, ipsetName Name, entry string) bool {
+	k := entryKey{ipsetName, entry}
+
+	oneLeft := len(i.users[k]) == 1
+	delete(i.users[k], user)
+
+	if len(i.users[k]) == 0 {
+		delete(i.users, k)
+	}
+
+	return oneLeft && (len(i.users[k]) == 0)
+}
+
+func (i *ipset) existUser(user types.UID, ipsetName Name, entry string) bool {
+	_, ok := i.users[entryKey{ipsetName, entry}][user]
+	return ok
+}
+
+func (i *ipset) removeSetFromUsers(ipsetName Name) {
+	for k := range i.users {
+		if k.ipsetName == ipsetName {
+			delete(i.users, k)
+		}
+	}
+}
+
 func doExec(args ...string) error {
 	if output, err := exec.Command("ipset", args...).CombinedOutput(); err != nil {
 		return errors.Wrapf(err, "ipset %v failed: %s", args, output)
 	}
 	return nil
 }
-
-// Reference-counting
-type key struct {
-	ipsetName Name
-	entry     string
-}
-
-// note no locking is required as all operations are serialised in the controller
-type refCount struct {
-	ref map[key]int
-}
-
-func newRefCount() refCount {
-	return refCount{ref: make(map[key]int)}
-}
-
-func (rc *refCount) inc(ipsetName Name, entry string) int {
-	k := key{ipsetName, entry}
-	rc.ref[k]++
-	return rc.ref[k]
-}
-
-func (rc *refCount) dec(ipsetName Name, entry string) int {
-	k := key{ipsetName, entry}
-	rc.ref[k]--
-	return rc.ref[k]
-}
-
-func (rc *refCount) removeSet(ipsetName Name) {
-	for k := range rc.ref {
-		if k.ipsetName == ipsetName {
-			delete(rc.ref, k)
-		}
-	}
-}

npc/iptables/iptables.go

@@ -0,0 +1,7 @@
+package iptables
+
+type Interface interface {
+	Append(table, chain string, rulespec ...string) error
+	Delete(table, chain string, rulespec ...string) error
+	Insert(table, chain string, pos int, rulespec ...string) error
+}

npc/namespace.go

@@ -2,38 +2,50 @@ package npc
 
 import (
 	"encoding/json"
+	"errors"
+	"fmt"
 
-	"github.com/coreos/go-iptables/iptables"
-	"k8s.io/client-go/pkg/api/unversioned"
-	coreapi "k8s.io/client-go/pkg/api/v1"
-	extnapi "k8s.io/client-go/pkg/apis/extensions/v1beta1"
-	"k8s.io/client-go/pkg/types"
-	"k8s.io/client-go/pkg/util/uuid"
+	coreapi "k8s.io/api/core/v1"
+	extnapi "k8s.io/api/extensions/v1beta1"
+	networkingv1 "k8s.io/api/networking/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/types"
+	"k8s.io/apimachinery/pkg/util/uuid"
 
 	"github.com/weaveworks/weave/common"
 	"github.com/weaveworks/weave/npc/ipset"
+	"github.com/weaveworks/weave/npc/iptables"
 )
 
+var errInvalidNetworkPolicyObjType = errors.New("invalid NetworkPolicy object type")
+
 type ns struct {
-	ipt *iptables.IPTables // interface to iptables
+	ipt iptables.Interface // interface to iptables
 	ips ipset.Interface    // interface to ipset
 
-	name      string                               // k8s Namespace name
-	nodeName  string                               // my node name
-	namespace *coreapi.Namespace                   // k8s Namespace object
-	pods      map[types.UID]*coreapi.Pod           // k8s Pod objects by UID
-	policies  map[types.UID]*extnapi.NetworkPolicy // k8s NetworkPolicy objects by UID
+	name      string                     // k8s Namespace name
+	nodeName  string                     // my node name
+	namespace *coreapi.Namespace         // k8s Namespace object
+	pods      map[types.UID]*coreapi.Pod // k8s Pod objects by UID
+	policies  map[types.UID]interface{}  // k8s NetworkPolicy objects by UID
 
 	uid     types.UID     // surrogate UID to own allPods selector
 	allPods *selectorSpec // hash:ip ipset of all pod IPs in this namespace
 
+	// stores IP addrs of pods which are not selected by any dst podSelector of
+	// any netpol; used only in non-legacy mode and is used as a dst in
+	// the WEAVE-NPC-DEFAULT iptables chain.
+	defaultAllowIPSet ipset.Name
+
 	nsSelectors  *selectorSet
 	podSelectors *selectorSet
 	rules        *ruleSet
+
+	legacy bool
 }
 
-func newNS(name, nodeName string, ipt *iptables.IPTables, ips ipset.Interface, nsSelectors *selectorSet) (*ns, error) {
-	allPods, err := newSelectorSpec(&unversioned.LabelSelector{}, name, ipset.HashIP)
+func newNS(name, nodeName string, legacy bool, ipt iptables.Interface, ips ipset.Interface, nsSelectors *selectorSet) (*ns, error) {
+	allPods, err := newSelectorSpec(&metav1.LabelSelector{}, false, name, ipset.HashIP)
 	if err != nil {
 		return nil, err
 	}
@@ -44,13 +56,22 @@ func newNS(name, nodeName string, ipt *iptables.IPTables, ips ipset.Interface, n
 		name:        name,
 		nodeName:    nodeName,
 		pods:        make(map[types.UID]*coreapi.Pod),
-		policies:    make(map[types.UID]*extnapi.NetworkPolicy),
+		policies:    make(map[types.UID]interface{}),
 		uid:         uuid.NewUUID(),
 		allPods:     allPods,
 		nsSelectors: nsSelectors,
-		rules:       newRuleSet(ipt)}
+		rules:       newRuleSet(ipt),
+		legacy:      legacy}
 
-	ns.podSelectors = newSelectorSet(ips, ns.onNewPodSelector)
+	ns.podSelectors = newSelectorSet(ips, ns.onNewPodSelector, ns.onNewDstPodSelector, ns.onDestroyDstPodSelector)
+
+	if !legacy {
+		defaultAllowIPSet := ipset.Name(IpsetNamePrefix + shortName("default-allow:"+name))
+		if err := ips.Create(defaultAllowIPSet, ipset.HashIP); err != nil {
+			return nil, err
+		}
+		ns.defaultAllowIPSet = defaultAllowIPSet
+	}
 
 	if err := ns.podSelectors.provision(ns.uid, nil, map[string]*selectorSpec{ns.allPods.key: ns.allPods}); err != nil {
 		return nil, err
@@ -64,22 +85,75 @@ func (ns *ns) empty() bool {
 }
 
 func (ns *ns) destroy() error {
-	if err := ns.podSelectors.deprovision(ns.uid, map[string]*selectorSpec{ns.allPods.key: ns.allPods}, nil); err != nil {
-		return err
-	}
-	return nil
+	return ns.podSelectors.deprovision(ns.uid, map[string]*selectorSpec{ns.allPods.key: ns.allPods}, nil)
 }
 
 func (ns *ns) onNewPodSelector(selector *selector) error {
 	for _, pod := range ns.pods {
 		if hasIP(pod) {
 			if selector.matches(pod.ObjectMeta.Labels) {
-				if err := selector.addEntry(pod.Status.PodIP); err != nil {
+				if err := selector.addEntry(pod.ObjectMeta.UID, pod.Status.PodIP, podComment(pod)); err != nil {
+					return err
+				}
+
+			}
+		}
+	}
+	return nil
+}
+
+func (ns *ns) onNewDstPodSelector(selector *selector) error {
+	if ns.legacy {
+		return nil
+	}
+
+	for _, pod := range ns.pods {
+		if hasIP(pod) {
+			// Remove the pod from default-allow if dst podselector matches the pod
+			if selector.matches(pod.ObjectMeta.Labels) {
+				if err := ns.ips.DelEntry(pod.ObjectMeta.UID, ns.defaultAllowIPSet, pod.Status.PodIP); err != nil {
 					return err
 				}
 			}
 		}
 	}
+
+	return nil
+}
+
+func (ns *ns) onDestroyDstPodSelector(selector *selector) error {
+	if ns.legacy {
+		return nil
+	}
+
+	for _, pod := range ns.pods {
+		if hasIP(pod) {
+			if selector.matches(pod.ObjectMeta.Labels) {
+				if err := ns.addToDefaultAllowIfNoMatching(pod); err != nil {
+					return err
+				}
+			}
+		}
+	}
+
+	return nil
+}
+
+// Add pod IP addr to default-allow ipset if there are no matching dst selectors
+func (ns *ns) addToDefaultAllowIfNoMatching(pod *coreapi.Pod) error {
+	found := false
+	// TODO(mp) optimize (avoid iterating over selectors) by ref counting IP addrs.
+	for _, s := range ns.podSelectors.entries {
+		if ns.podSelectors.dstSelectorExist(s) && s.matches(pod.ObjectMeta.Labels) {
+			found = true
+			break
+		}
+	}
+	if !found {
+		if err := ns.ips.AddEntry(pod.ObjectMeta.UID, ns.defaultAllowIPSet, pod.Status.PodIP, podComment(pod)); err != nil {
+			return err
+		}
+	}
 	return nil
 }
 
@@ -98,9 +172,21 @@ func (ns *ns) addPod(obj *coreapi.Pod) error {
 	}
 
 	if ns.checkLocalPod(obj) {
-		ns.ips.AddEntry(LocalIpset, obj.Status.PodIP)
+		ns.ips.AddEntry(obj.ObjectMeta.UID, LocalIpset, obj.Status.PodIP, podComment(obj))
 	}
-	return ns.podSelectors.addToMatching(obj.ObjectMeta.Labels, obj.Status.PodIP)
+
+	found, err := ns.podSelectors.addToMatching(obj.ObjectMeta.UID, obj.ObjectMeta.Labels, obj.Status.PodIP, podComment(obj))
+	if err != nil {
+		return err
+	}
+	// If there are no matching dst selectors, add the pod to default-allow
+	if !ns.legacy && !found {
+		if err := ns.ips.AddEntry(obj.ObjectMeta.UID, ns.defaultAllowIPSet, obj.Status.PodIP, podComment(obj)); err != nil {
+			return err
+		}
+	}
+
+	return nil
 }
 
 func (ns *ns) updatePod(oldObj, newObj *coreapi.Pod) error {
@@ -113,16 +199,49 @@ func (ns *ns) updatePod(oldObj, newObj *coreapi.Pod) error {
 
 	if hasIP(oldObj) && !hasIP(newObj) {
 		if ns.checkLocalPod(oldObj) {
-			ns.ips.DelEntry(LocalIpset, oldObj.Status.PodIP)
+			ns.ips.DelEntry(oldObj.ObjectMeta.UID, LocalIpset, oldObj.Status.PodIP)
 		}
-		return ns.podSelectors.delFromMatching(oldObj.ObjectMeta.Labels, oldObj.Status.PodIP)
+
+		if !ns.legacy {
+			if err := ns.ips.DelEntry(oldObj.ObjectMeta.UID, ns.defaultAllowIPSet, oldObj.Status.PodIP); err != nil {
+				return err
+			}
+		}
+
+		return ns.podSelectors.delFromMatching(oldObj.ObjectMeta.UID, oldObj.ObjectMeta.Labels, oldObj.Status.PodIP)
 	}
 
 	if !hasIP(oldObj) && hasIP(newObj) {
 		if ns.checkLocalPod(newObj) {
-			ns.ips.AddEntry(LocalIpset, newObj.Status.PodIP)
+			ns.ips.AddEntry(newObj.ObjectMeta.UID, LocalIpset, newObj.Status.PodIP, podComment(newObj))
+		}
+		found, err := ns.podSelectors.addToMatching(newObj.ObjectMeta.UID, newObj.ObjectMeta.Labels, newObj.Status.PodIP, podComment(newObj))
+		if err != nil {
+			return err
+		}
+
+		if !ns.legacy && !found {
+			if err := ns.ips.AddEntry(newObj.ObjectMeta.UID, ns.defaultAllowIPSet, newObj.Status.PodIP, podComment(newObj)); err != nil {
+				return err
+			}
+		}
+
+		return nil
+	}
+
+	if !ns.legacy &&
+		oldObj.Status.PodIP != newObj.Status.PodIP &&
+		ns.ips.Exist(oldObj.ObjectMeta.UID, ns.defaultAllowIPSet, oldObj.Status.PodIP) {
+		// Instead of iterating over all selectors we check whether old pod IP
+		// has been inserted into default-allow ipset to decide whether the IP
+		// in the ipset has to be updated.
+
+		if err := ns.ips.DelEntry(oldObj.ObjectMeta.UID, ns.defaultAllowIPSet, oldObj.Status.PodIP); err != nil {
+			return err
+		}
+		if err := ns.ips.AddEntry(newObj.ObjectMeta.UID, ns.defaultAllowIPSet, newObj.Status.PodIP, podComment(newObj)); err != nil {
+			return err
 		}
-		return ns.podSelectors.addToMatching(newObj.ObjectMeta.Labels, newObj.Status.PodIP)
 	}
 
 	if !equals(oldObj.ObjectMeta.Labels, newObj.ObjectMeta.Labels) ||
@@ -135,15 +254,28 @@ func (ns *ns) updatePod(oldObj, newObj *coreapi.Pod) error {
 				continue
 			}
 			if oldMatch {
-				if err := ps.delEntry(oldObj.Status.PodIP); err != nil {
+				if err := ps.delEntry(oldObj.ObjectMeta.UID, oldObj.Status.PodIP); err != nil {
 					return err
 				}
 			}
 			if newMatch {
-				if err := ps.addEntry(newObj.Status.PodIP); err != nil {
+				if err := ps.addEntry(newObj.ObjectMeta.UID, newObj.Status.PodIP, podComment(newObj)); err != nil {
 					return err
 				}
 			}
+
+			if !ns.legacy && ns.podSelectors.dstSelectorExist(ps) {
+				switch {
+				case !oldMatch && newMatch:
+					if err := ns.ips.DelEntry(oldObj.ObjectMeta.UID, ns.defaultAllowIPSet, oldObj.Status.PodIP); err != nil {
+						return err
+					}
+				case oldMatch && !newMatch:
+					if err := ns.addToDefaultAllowIfNoMatching(newObj); err != nil {
+						return err
+					}
+				}
+			}
 		}
 	}
 
@@ -158,134 +290,149 @@ func (ns *ns) deletePod(obj *coreapi.Pod) error {
 	}
 
 	if ns.checkLocalPod(obj) {
-		ns.ips.DelEntry(LocalIpset, obj.Status.PodIP)
+		ns.ips.DelEntry(obj.ObjectMeta.UID, LocalIpset, obj.Status.PodIP)
 	}
-	return ns.podSelectors.delFromMatching(obj.ObjectMeta.Labels, obj.Status.PodIP)
+
+	if !ns.legacy {
+		if err := ns.ips.DelEntry(obj.ObjectMeta.UID, ns.defaultAllowIPSet, obj.Status.PodIP); err != nil {
+			return err
+		}
+	}
+
+	return ns.podSelectors.delFromMatching(obj.ObjectMeta.UID, obj.ObjectMeta.Labels, obj.Status.PodIP)
 }
 
-func (ns *ns) addNetworkPolicy(obj *extnapi.NetworkPolicy) error {
-	ns.policies[obj.ObjectMeta.UID] = obj
-
+func (ns *ns) addNetworkPolicy(obj interface{}) error {
 	// Analyse policy, determine which rules and ipsets are required
-	rules, nsSelectors, podSelectors, err := ns.analysePolicy(obj)
+
+	uid, rules, nsSelectors, podSelectors, err := ns.analyse(obj)
 	if err != nil {
 		return err
 	}
 
 	// Provision required resources in dependency order
-	if err := ns.nsSelectors.provision(obj.ObjectMeta.UID, nil, nsSelectors); err != nil {
+	if err := ns.nsSelectors.provision(uid, nil, nsSelectors); err != nil {
 		return err
 	}
-	if err := ns.podSelectors.provision(obj.ObjectMeta.UID, nil, podSelectors); err != nil {
+	if err := ns.podSelectors.provision(uid, nil, podSelectors); err != nil {
 		return err
 	}
-	if err := ns.rules.provision(obj.ObjectMeta.UID, nil, rules); err != nil {
-		return err
-	}
-
-	return nil
+	return ns.rules.provision(uid, nil, rules)
 }
 
-func (ns *ns) updateNetworkPolicy(oldObj, newObj *extnapi.NetworkPolicy) error {
-	delete(ns.policies, oldObj.ObjectMeta.UID)
-	ns.policies[newObj.ObjectMeta.UID] = newObj
-
+func (ns *ns) updateNetworkPolicy(oldObj, newObj interface{}) error {
 	// Analyse the old and the new policy so we can determine differences
-	oldRules, oldNsSelectors, oldPodSelectors, err := ns.analysePolicy(oldObj)
+	oldUID, oldRules, oldNsSelectors, oldPodSelectors, err := ns.analyse(oldObj)
 	if err != nil {
 		return err
 	}
-	newRules, newNsSelectors, newPodSelectors, err := ns.analysePolicy(newObj)
+	newUID, newRules, newNsSelectors, newPodSelectors, err := ns.analyse(newObj)
 	if err != nil {
 		return err
 	}
 
+	delete(ns.policies, oldUID)
+	ns.policies[newUID] = newObj
+
 	// Deprovision unused and provision newly required resources in dependency order
-	if err := ns.rules.deprovision(oldObj.ObjectMeta.UID, oldRules, newRules); err != nil {
+	if err := ns.rules.deprovision(oldUID, oldRules, newRules); err != nil {
 		return err
 	}
-	if err := ns.nsSelectors.deprovision(oldObj.ObjectMeta.UID, oldNsSelectors, newNsSelectors); err != nil {
+	if err := ns.nsSelectors.deprovision(oldUID, oldNsSelectors, newNsSelectors); err != nil {
 		return err
 	}
-	if err := ns.podSelectors.deprovision(oldObj.ObjectMeta.UID, oldPodSelectors, newPodSelectors); err != nil {
+	if err := ns.podSelectors.deprovision(oldUID, oldPodSelectors, newPodSelectors); err != nil {
 		return err
 	}
-	if err := ns.nsSelectors.provision(oldObj.ObjectMeta.UID, oldNsSelectors, newNsSelectors); err != nil {
+	if err := ns.nsSelectors.provision(oldUID, oldNsSelectors, newNsSelectors); err != nil {
 		return err
 	}
-	if err := ns.podSelectors.provision(oldObj.ObjectMeta.UID, oldPodSelectors, newPodSelectors); err != nil {
+	if err := ns.podSelectors.provision(oldUID, oldPodSelectors, newPodSelectors); err != nil {
 		return err
 	}
-	if err := ns.rules.provision(oldObj.ObjectMeta.UID, oldRules, newRules); err != nil {
-		return err
-	}
-
-	return nil
+	return ns.rules.provision(oldUID, oldRules, newRules)
 }
 
-func (ns *ns) deleteNetworkPolicy(obj *extnapi.NetworkPolicy) error {
-	delete(ns.policies, obj.ObjectMeta.UID)
-
+func (ns *ns) deleteNetworkPolicy(obj interface{}) error {
 	// Analyse network policy to free resources
-	rules, nsSelectors, podSelectors, err := ns.analysePolicy(obj)
+	uid, rules, nsSelectors, podSelectors, err := ns.analyse(obj)
 	if err != nil {
 		return err
 	}
 
+	delete(ns.policies, uid)
+
 	// Deprovision unused resources in dependency order
-	if err := ns.rules.deprovision(obj.ObjectMeta.UID, rules, nil); err != nil {
+	if err := ns.rules.deprovision(uid, rules, nil); err != nil {
 		return err
 	}
-	if err := ns.nsSelectors.deprovision(obj.ObjectMeta.UID, nsSelectors, nil); err != nil {
+	if err := ns.nsSelectors.deprovision(uid, nsSelectors, nil); err != nil {
 		return err
 	}
-	if err := ns.podSelectors.deprovision(obj.ObjectMeta.UID, podSelectors, nil); err != nil {
-		return err
+	return ns.podSelectors.deprovision(uid, podSelectors, nil)
+}
+
+func bypassRule(nsIpsetName ipset.Name, namespace string) []string {
+	return []string{"-m", "set", "--match-set", string(nsIpsetName), "dst", "-j", "ACCEPT", "-m", "comment", "--comment", "DefaultAllow isolation for namespace: " + namespace}
+}
+
+func (ns *ns) ensureBypassRule() error {
+	var ipset ipset.Name
+	if ns.legacy {
+		ipset = ns.allPods.ipsetName
+	} else {
+		ipset = ns.defaultAllowIPSet
 	}
 
-	return nil
+	common.Log.Debugf("ensuring rule for DefaultAllow in namespace: %s, set %s", ns.name, ipset)
+	return ns.ipt.Append(TableFilter, DefaultChain, bypassRule(ipset, ns.name)...)
 }
 
-func bypassRule(nsIpsetName ipset.Name) []string {
-	return []string{"-m", "set", "--match-set", string(nsIpsetName), "dst", "-j", "ACCEPT"}
-}
+func (ns *ns) deleteBypassRule() error {
+	var ipset ipset.Name
+	if ns.legacy {
+		ipset = ns.allPods.ipsetName
+	} else {
+		ipset = ns.defaultAllowIPSet
+	}
 
-func (ns *ns) ensureBypassRule(nsIpsetName ipset.Name) error {
-	common.Log.Debugf("ensuring rule for DefaultAllow in namespace: %s, set %s", ns.name, nsIpsetName)
-	return ns.ipt.Append(TableFilter, DefaultChain, bypassRule(ns.allPods.ipsetName)...)
-}
-
-func (ns *ns) deleteBypassRule(nsIpsetName ipset.Name) error {
-	common.Log.Debugf("removing default rule in namespace: %s, set %s", ns.name, nsIpsetName)
-	return ns.ipt.Delete(TableFilter, DefaultChain, bypassRule(ns.allPods.ipsetName)...)
+	common.Log.Debugf("removing default rule in namespace: %s, set %s", ns.name, ipset)
+	return ns.ipt.Delete(TableFilter, DefaultChain, bypassRule(ipset, ns.name)...)
 }
 
 func (ns *ns) addNamespace(obj *coreapi.Namespace) error {
 	ns.namespace = obj
 
 	// Insert a rule to bypass policies if namespace is DefaultAllow
-	if !isDefaultDeny(obj) {
-		return ns.ensureBypassRule(ns.allPods.ipsetName)
+	if !ns.isDefaultDeny(obj) {
+		if err := ns.ensureBypassRule(); err != nil {
+			return err
+		}
 	}
 
 	// Add namespace ipset to matching namespace selectors
-	return ns.nsSelectors.addToMatching(obj.ObjectMeta.Labels, string(ns.allPods.ipsetName))
+	_, err := ns.nsSelectors.addToMatching(obj.ObjectMeta.UID, obj.ObjectMeta.Labels, string(ns.allPods.ipsetName), namespaceComment(ns))
+	return err
 }
 
 func (ns *ns) updateNamespace(oldObj, newObj *coreapi.Namespace) error {
 	ns.namespace = newObj
 
 	// Update bypass rule if ingress default has changed
-	oldDefaultDeny := isDefaultDeny(oldObj)
-	newDefaultDeny := isDefaultDeny(newObj)
+	oldDefaultDeny := ns.isDefaultDeny(oldObj)
+	newDefaultDeny := ns.isDefaultDeny(newObj)
 
 	if oldDefaultDeny != newDefaultDeny {
 		common.Log.Infof("namespace DefaultDeny changed from %t to %t", oldDefaultDeny, newDefaultDeny)
 		if oldDefaultDeny {
-			return ns.ensureBypassRule(ns.allPods.ipsetName)
+			if err := ns.ensureBypassRule(); err != nil {
+				return err
+			}
 		}
 		if newDefaultDeny {
-			return ns.deleteBypassRule(ns.allPods.ipsetName)
+			if err := ns.deleteBypassRule(); err != nil {
+				return err
+			}
 		}
 	}
 
@@ -298,12 +445,12 @@ func (ns *ns) updateNamespace(oldObj, newObj *coreapi.Namespace) error {
 				continue
 			}
 			if oldMatch {
-				if err := selector.delEntry(string(ns.allPods.ipsetName)); err != nil {
+				if err := selector.delEntry(ns.namespace.ObjectMeta.UID, string(ns.allPods.ipsetName)); err != nil {
 					return err
 				}
 			}
 			if newMatch {
-				if err := selector.addEntry(string(ns.allPods.ipsetName)); err != nil {
+				if err := selector.addEntry(ns.namespace.ObjectMeta.UID, string(ns.allPods.ipsetName), namespaceComment(ns)); err != nil {
 					return err
 				}
 			}
@@ -317,12 +464,37 @@ func (ns *ns) deleteNamespace(obj *coreapi.Namespace) error {
 	ns.namespace = nil
 
 	// Remove bypass rule
-	if !isDefaultDeny(obj) {
-		return ns.deleteBypassRule(ns.allPods.ipsetName)
+	if !ns.isDefaultDeny(obj) {
+		if err := ns.deleteBypassRule(); err != nil {
+			return err
+		}
 	}
 
 	// Remove namespace ipset from any matching namespace selectors
-	return ns.nsSelectors.delFromMatching(obj.ObjectMeta.Labels, string(ns.allPods.ipsetName))
+	return ns.nsSelectors.delFromMatching(obj.ObjectMeta.UID, obj.ObjectMeta.Labels, string(ns.allPods.ipsetName))
+}
+
+func (ns *ns) isDefaultDeny(namespace *coreapi.Namespace) bool {
+	nnpJSON, found := namespace.ObjectMeta.Annotations["net.beta.kubernetes.io/network-policy"]
+	if !found {
+		return false
+	}
+
+	if !ns.legacy {
+		common.Log.Warn("DefaultDeny annotation is supported only in legacy mode (--use-legacy-netpol)")
+		return false
+	}
+
+	var nnp NamespaceNetworkPolicy
+	if err := json.Unmarshal([]byte(nnpJSON), &nnp); err != nil {
+		common.Log.Warn("Ignoring network policy annotation: unmarshal failed:", err)
+		// If we can't understand the annotation, behave as if it isn't present
+		return false
+	}
+
+	return nnp.Ingress != nil &&
+		nnp.Ingress.Isolation != nil &&
+		*(nnp.Ingress.Isolation) == DefaultDeny
 }
 
 func hasIP(pod *coreapi.Pod) bool {
@@ -343,20 +515,43 @@ func equals(a, b map[string]string) bool {
 	return true
 }
 
-func isDefaultDeny(namespace *coreapi.Namespace) bool {
-	nnpJSON, found := namespace.ObjectMeta.Annotations["net.beta.kubernetes.io/network-policy"]
-	if !found {
-		return false
+func namespaceComment(namespace *ns) string {
+	return "namespace: " + namespace.name
+}
+
+func podComment(pod *coreapi.Pod) string {
+	return fmt.Sprintf("namespace: %s, pod: %s", pod.ObjectMeta.Namespace, pod.ObjectMeta.Name)
+}
+
+func (ns *ns) analyse(obj interface{}) (
+	uid types.UID,
+	rules map[string]*ruleSpec,
+	nsSelectors, podSelectors map[string]*selectorSpec,
+	err error) {
+
+	switch p := obj.(type) {
+	case *extnapi.NetworkPolicy:
+		uid = p.ObjectMeta.UID
+	case *networkingv1.NetworkPolicy:
+		uid = p.ObjectMeta.UID
+	default:
+		err = errInvalidNetworkPolicyObjType
+		return
 	}
+	ns.policies[uid] = obj
 
-	var nnp NamespaceNetworkPolicy
-	if err := json.Unmarshal([]byte(nnpJSON), &nnp); err != nil {
-		common.Log.Warn("Ignoring network policy annotation: unmarshal failed:", err)
-		// If we can't understand the annotation, behave as if it isn't present
-		return false
+	// Analyse policy, determine which rules and ipsets are required
+	if ns.legacy {
+		rules, nsSelectors, podSelectors, err = ns.analysePolicyLegacy(obj.(*extnapi.NetworkPolicy))
+		if err != nil {
+			return
+		}
+	} else {
+		rules, nsSelectors, podSelectors, err = ns.analysePolicy(obj.(*networkingv1.NetworkPolicy))
+		if err != nil {
+			return
+		}
 	}
 
-	return nnp.Ingress != nil &&
-		nnp.Ingress.Isolation != nil &&
-		*(nnp.Ingress.Isolation) == DefaultDeny
+	return
 }

npc/rule.go

@@ -1,12 +1,13 @@
 package npc
 
 import (
+	"fmt"
 	"strings"
 
-	"github.com/coreos/go-iptables/iptables"
-	"k8s.io/client-go/pkg/types"
+	"k8s.io/apimachinery/pkg/types"
 
 	"github.com/weaveworks/weave/common"
+	"github.com/weaveworks/weave/npc/iptables"
 )
 
 type ruleSpec struct {
@@ -19,27 +20,36 @@ func newRuleSpec(proto *string, srcHost *selectorSpec, dstHost *selectorSpec, ds
 	if proto != nil {
 		args = append(args, "-p", *proto)
 	}
+	srcComment := "anywhere"
 	if srcHost != nil {
 		args = append(args, "-m", "set", "--match-set", string(srcHost.ipsetName), "src")
+		if srcHost.nsName != "" {
+			srcComment = fmt.Sprintf("pods: namespace: %s, selector: %s", srcHost.nsName, srcHost.key)
+		} else {
+			srcComment = fmt.Sprintf("namespaces: selector: %s", srcHost.key)
+		}
 	}
+	dstComment := "anywhere"
 	if dstHost != nil {
 		args = append(args, "-m", "set", "--match-set", string(dstHost.ipsetName), "dst")
+		dstComment = fmt.Sprintf("pods: namespace: %s, selector: %s", dstHost.nsName, dstHost.key)
 	}
 	if dstPort != nil {
 		args = append(args, "--dport", *dstPort)
 	}
 	args = append(args, "-j", "ACCEPT")
+	args = append(args, "-m", "comment", "--comment", fmt.Sprintf("%s -> %s", srcComment, dstComment))
 	key := strings.Join(args, " ")
 
 	return &ruleSpec{key, args}
 }
 
 type ruleSet struct {
-	ipt   *iptables.IPTables
+	ipt   iptables.Interface
 	users map[string]map[types.UID]struct{}
 }
 
-func newRuleSet(ipt *iptables.IPTables) *ruleSet {
+func newRuleSet(ipt iptables.Interface) *ruleSet {
 	return &ruleSet{ipt, make(map[string]map[types.UID]struct{})}
 }
 

npc/rule_test.go

@@ -0,0 +1,16 @@
+package npc
+
+import (
+	"github.com/stretchr/testify/require"
+	"testing"
+)
+
+func TestRegressionHandleNil3095(t *testing.T) {
+	// Test for handling nil values
+	// https://github.com/weaveworks/weave/issues/3095
+
+	rs := newRuleSpec(nil, nil, nil, nil)
+
+	require.NotEqual(t, rs.key, "")
+
+}

npc/selector.go

@@ -1,9 +1,9 @@
 package npc
 
 import (
-	"k8s.io/client-go/pkg/api/unversioned"
-	"k8s.io/client-go/pkg/labels"
-	"k8s.io/client-go/pkg/types"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/labels"
+	"k8s.io/apimachinery/pkg/types"
 
 	"github.com/weaveworks/weave/common"
 	"github.com/weaveworks/weave/npc/ipset"
@@ -12,13 +12,15 @@ import (
 type selectorSpec struct {
 	key      string          // string representation (for hash keying/equality comparison)
 	selector labels.Selector // k8s Selector object (for matching)
+	dst      bool            // destination selector
 
 	ipsetType ipset.Type // type of ipset to provision
 	ipsetName ipset.Name // generated ipset name
+	nsName    string     // Namespace name
 }
 
-func newSelectorSpec(json *unversioned.LabelSelector, nsName string, ipsetType ipset.Type) (*selectorSpec, error) {
-	selector, err := unversioned.LabelSelectorAsSelector(json)
+func newSelectorSpec(json *metav1.LabelSelector, dst bool, nsName string, ipsetType ipset.Type) (*selectorSpec, error) {
+	selector, err := metav1.LabelSelectorAsSelector(json)
 	if err != nil {
 		return nil, err
 	}
@@ -26,11 +28,13 @@ func newSelectorSpec(json *unversioned.LabelSelector, nsName string, ipsetType i
 	return &selectorSpec{
 		key:      key,
 		selector: selector,
+		dst:      dst,
 		// We prefix the selector string with the namespace name when generating
 		// the shortname because you can specify the same selector in multiple
 		// namespaces - we need those to map to distinct ipsets
 		ipsetName: ipset.Name(IpsetNamePrefix + shortName(nsName+":"+key)),
-		ipsetType: ipsetType}, nil
+		ipsetType: ipsetType,
+		nsName:    nsName}, nil
 }
 
 type selector struct {
@@ -42,12 +46,12 @@ func (s *selector) matches(labelMap map[string]string) bool {
 	return s.spec.selector.Matches(labels.Set(labelMap))
 }
 
-func (s *selector) addEntry(entry string) error {
-	return s.ips.AddEntry(s.spec.ipsetName, entry)
+func (s *selector) addEntry(user types.UID, entry string, comment string) error {
+	return s.ips.AddEntry(user, s.spec.ipsetName, entry, comment)
 }
 
-func (s *selector) delEntry(entry string) error {
-	return s.ips.DelEntry(s.spec.ipsetName, entry)
+func (s *selector) delEntry(user types.UID, entry string) error {
+	return s.ips.DelEntry(user, s.spec.ipsetName, entry)
 }
 
 type selectorFn func(selector *selector) error
@@ -55,22 +59,51 @@ type selectorFn func(selector *selector) error
 type selectorSet struct {
 	ips           ipset.Interface
 	onNewSelector selectorFn
-	users         map[string]map[types.UID]struct{} // list of users per selector
-	entries       map[string]*selector
+
+	// invoked after dst selector has been provisioned for the first time
+	onNewDstSelector selectorFn
+	// invoked after the last instance of dst selector has been deprovisioned
+	onDestroyDstSelector selectorFn
+
+	users   map[string]map[types.UID]struct{} // list of users per selector
+	entries map[string]*selector
+
+	// We need to keep track of dst selector instances to be able to invoke
+	// onNewDstSelector and onDestroyDstSelector callbacks at the right time;
+	// selectorSpec.Key -> count
+	dstSelectorsCount map[string]int
 }
 
-func newSelectorSet(ips ipset.Interface, onNewSelector selectorFn) *selectorSet {
+func newSelectorSet(ips ipset.Interface, onNewSelector, onNewDstSelector selectorFn, onDestroyDstSelector selectorFn) *selectorSet {
 	return &selectorSet{
-		ips:           ips,
-		onNewSelector: onNewSelector,
-		users:         make(map[string]map[types.UID]struct{}),
-		entries:       make(map[string]*selector)}
+		ips:                  ips,
+		onNewSelector:        onNewSelector,
+		onNewDstSelector:     onNewDstSelector,
+		onDestroyDstSelector: onDestroyDstSelector,
+		users:                make(map[string]map[types.UID]struct{}),
+		entries:              make(map[string]*selector),
+		dstSelectorsCount:    make(map[string]int)}
 }
 
-func (ss *selectorSet) addToMatching(labelMap map[string]string, entry string) error {
+func (ss *selectorSet) addToMatching(user types.UID, labelMap map[string]string, entry string, comment string) (bool, error) {
+	found := false
 	for _, s := range ss.entries {
 		if s.matches(labelMap) {
-			if err := s.addEntry(entry); err != nil {
+			if ss.dstSelectorExist(s) {
+				found = true
+			}
+			if err := s.addEntry(user, entry, comment); err != nil {
+				return found, err
+			}
+		}
+	}
+	return found, nil
+}
+
+func (ss *selectorSet) delFromMatching(user types.UID, labelMap map[string]string, entry string) error {
+	for _, s := range ss.entries {
+		if s.matches(labelMap) {
+			if err := s.delEntry(user, entry); err != nil {
 				return err
 			}
 		}
@@ -78,15 +111,8 @@ func (ss *selectorSet) addToMatching(labelMap map[string]string, entry string) e
 	return nil
 }
 
-func (ss *selectorSet) delFromMatching(labelMap map[string]string, entry string) error {
-	for _, s := range ss.entries {
-		if s.matches(labelMap) {
-			if err := s.delEntry(entry); err != nil {
-				return err
-			}
-		}
-	}
-	return nil
+func (ss *selectorSet) dstSelectorExist(s *selector) bool {
+	return ss.dstSelectorsCount[s.spec.key] > 0
 }
 
 func (ss *selectorSet) deprovision(user types.UID, current, desired map[string]*selectorSpec) error {
@@ -98,9 +124,19 @@ func (ss *selectorSet) deprovision(user types.UID, current, desired map[string]*
 				if err := ss.ips.Destroy(spec.ipsetName); err != nil {
 					return err
 				}
+
 				delete(ss.entries, key)
 				delete(ss.users, key)
 			}
+
+			if spec.dst {
+				ss.dstSelectorsCount[key]--
+				if ss.dstSelectorsCount[key] == 0 {
+					if err := ss.onDestroyDstSelector(&selector{ss.ips, spec}); err != nil {
+						return err
+					}
+				}
+			}
 		}
 	}
 	return nil
@@ -109,12 +145,13 @@ func (ss *selectorSet) deprovision(user types.UID, current, desired map[string]*
 func (ss *selectorSet) provision(user types.UID, current, desired map[string]*selectorSpec) error {
 	for key, spec := range desired {
 		if _, found := current[key]; !found {
+			selector := &selector{ss.ips, spec}
+
 			if _, found := ss.users[key]; !found {
 				common.Log.Infof("creating ipset: %#v", spec)
 				if err := ss.ips.Create(spec.ipsetName, spec.ipsetType); err != nil {
 					return err
 				}
-				selector := &selector{ss.ips, spec}
 				if err := ss.onNewSelector(selector); err != nil {
 					return err
 				}
@@ -122,6 +159,15 @@ func (ss *selectorSet) provision(user types.UID, current, desired map[string]*se
 				ss.entries[key] = selector
 			}
 			ss.users[key][user] = struct{}{}
+
+			if spec.dst {
+				ss.dstSelectorsCount[key]++
+				if ss.dstSelectorsCount[key] == 1 {
+					if err := ss.onNewDstSelector(selector); err != nil {
+						return err
+					}
+				}
+			}
 		}
 	}
 	return nil

plugin/net/cni.go

@@ -6,7 +6,6 @@ import (
 	"fmt"
 	"net"
 	"os"
-	"syscall"
 
 	"github.com/containernetworking/cni/pkg/ipam"
 	"github.com/containernetworking/cni/pkg/skel"
@@ -100,12 +99,11 @@ func (c *CNIPlugin) CmdAdd(args *skel.CmdArgs) error {
 				return fmt.Errorf("unable to allocate IP address for bridge: %s", err)
 			}
 			bridgeCIDR := bridgeIPResult.IPs[0].Address
-			if err := assignBridgeIP(conf.BrName, bridgeCIDR); err != nil {
-				return fmt.Errorf("unable to assign IP address to bridge: %s", err)
-			}
-			if err := weavenet.ExposeNAT(bridgeCIDR); err != nil {
-				return fmt.Errorf("unable to create NAT rules: %s", err)
+
+			if err := c.weave.Expose(&bridgeCIDR); err != nil {
+				return fmt.Errorf("unable to expose bridge %q: %s", conf.BrName, err)
 			}
+
 			bridgeIP = bridgeCIDR.IP
 		} else if err != nil {
 			return err
@@ -167,22 +165,6 @@ func setupRoutes(link netlink.Link, name string, ipnet net.IPNet, gw net.IP, rou
 	return nil
 }
 
-func assignBridgeIP(bridgeName string, ipnet net.IPNet) error {
-	link, err := netlink.LinkByName(bridgeName)
-	if err != nil {
-		return err
-	}
-	if err := netlink.AddrAdd(link, &netlink.Addr{IPNet: &ipnet}); err != nil {
-		// Treat as non-error if this address is already there
-		// - maybe another copy of this program just added it
-		if err == syscall.Errno(syscall.EEXIST) {
-			return nil
-		}
-		return fmt.Errorf("failed to add IP address to %q: %v", bridgeName, err)
-	}
-	return nil
-}
-
 // As of CNI 0.5 spec:
 //   "Plugins should generally complete a DEL action without error even if some resources are missing"
 // this method should therefore return nil in most, if not all, cases.
@@ -207,7 +189,9 @@ func (c *CNIPlugin) CmdDel(args *skel.CmdArgs) error {
 	} else {
 		err = ipam.ExecDel(conf.IPAM.Type, args.StdinData)
 	}
-	logOnStderr(fmt.Errorf("unable to release IP address: %s", err))
+	if err != nil {
+		logOnStderr(fmt.Errorf("unable to release IP address: %s", err))
+	}
 	return nil
 }
 

plugin/net/watcher.go

@@ -8,10 +8,6 @@ import (
 	weavenet "github.com/weaveworks/weave/net"
 )
 
-const (
-	WeaveDomain = "weave.local"
-)
-
 type watcher struct {
 	client *docker.Client
 	weave  *weaveapi.Client

prog/kube-peers/annotations.go

@@ -0,0 +1,210 @@
+/*
+In order to keep track of active weave peers, we use annotations on the Kubernetes cluster.
+
+Kubernetes uses etcd to distribute and synchronise these annotations so we don't have to.
+
+This module deals with operations on the peerlist backed by Kubernetes' annotation mechanism.
+*/
+package main
+
+import (
+	"encoding/json"
+	"log"
+	"time"
+
+	"github.com/pkg/errors"
+
+	v1 "k8s.io/api/core/v1"
+	kubeErrors "k8s.io/apimachinery/pkg/api/errors"
+	api "k8s.io/apimachinery/pkg/apis/meta/v1"
+	wait "k8s.io/apimachinery/pkg/util/wait"
+	kubernetes "k8s.io/client-go/kubernetes"
+	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
+)
+
+type configMapAnnotations struct {
+	ConfigMapName string
+	Namespace     string
+	Client        corev1client.ConfigMapsGetter
+	cm            *v1.ConfigMap
+}
+
+func newConfigMapAnnotations(ns string, configMapName string, clientset *kubernetes.Clientset) *configMapAnnotations {
+	return &configMapAnnotations{
+		Namespace:     ns,
+		ConfigMapName: configMapName,
+		Client:        clientset.CoreV1(),
+	}
+}
+
+type peerList struct {
+	Peers []peerInfo
+}
+
+type peerInfo struct {
+	PeerName string // Weave internal unique ID
+	NodeName string // Kubernetes node name
+}
+
+func (pl peerList) contains(peerName string) bool {
+	for _, peer := range pl.Peers {
+		if peer.PeerName == peerName {
+			return true
+		}
+	}
+	return false
+}
+
+func (pl *peerList) add(peerName string, name string) {
+	pl.Peers = append(pl.Peers, peerInfo{PeerName: peerName, NodeName: name})
+}
+
+func (pl *peerList) remove(peerNameToRemove string) {
+	for i := 0; i < len(pl.Peers); {
+		if pl.Peers[i].PeerName == peerNameToRemove {
+			pl.Peers = append(pl.Peers[:i], pl.Peers[i+1:]...)
+		} else {
+			i++
+		}
+	}
+}
+
+const (
+	retryPeriod  = time.Second * 2
+	jitterFactor = 1.0
+
+	// Prefix all our annotation keys with this string so they don't clash with anyone else's
+	KubePeersPrefix = "kube-peers.weave.works/"
+	// KubePeersAnnotationKey is the default annotation key
+	KubePeersAnnotationKey = KubePeersPrefix + "peers"
+)
+
+func (cml *configMapAnnotations) Init() error {
+	for {
+		// Since it's potentially racy to GET, then CREATE if not found, we wrap in a check loop
+		// so that if the configmap is created after our GET but before or CREATE, we'll gracefully
+		// re-try to get the configmap.
+		var err error
+		cml.cm, err = cml.Client.ConfigMaps(cml.Namespace).Get(cml.ConfigMapName, api.GetOptions{})
+		if err != nil {
+			if !kubeErrors.IsNotFound(err) {
+				return errors.Wrapf(err, "Unable to fetch ConfigMap %s/%s", cml.Namespace, cml.ConfigMapName)
+			}
+			cml.cm, err = cml.Client.ConfigMaps(cml.Namespace).Create(&v1.ConfigMap{
+				ObjectMeta: api.ObjectMeta{
+					Name:      cml.ConfigMapName,
+					Namespace: cml.Namespace,
+				},
+			})
+			if err != nil {
+				if kubeErrors.IsAlreadyExists(err) {
+					continue
+				}
+				return errors.Wrapf(err, "Unable to create ConfigMap %s/%s", cml.Namespace, cml.ConfigMapName)
+			}
+		}
+		break
+	}
+	if cml.cm.Annotations == nil {
+		cml.cm.Annotations = make(map[string]string)
+	}
+	return nil
+}
+
+func (cml *configMapAnnotations) GetPeerList() (*peerList, error) {
+	var record peerList
+	if cml.cm == nil {
+		return nil, errors.New("endpoint not initialized, call Init first")
+	}
+	if recordBytes, found := cml.cm.Annotations[KubePeersAnnotationKey]; found {
+		if err := json.Unmarshal([]byte(recordBytes), &record); err != nil {
+			return nil, err
+		}
+	}
+	return &record, nil
+}
+
+func (cml *configMapAnnotations) UpdatePeerList(list peerList) error {
+	recordBytes, err := json.Marshal(list)
+	if err != nil {
+		return err
+	}
+	return cml.UpdateAnnotation(KubePeersAnnotationKey, string(recordBytes))
+}
+
+// Clean up a string so it meets the Kubernetes requiremements for Annotation keys:
+// name part must consist of alphanumeric characters, '-', '_' or '.', and must
+// start and end with an alphanumeric character (e.g. 'MyName', or 'my.name', or '123-abc')
+func cleanKey(key string) string {
+	buf := []byte(key)
+	for i, c := range buf {
+		if (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-' || c == '_' || c == '.' || c == '/' {
+			continue
+		}
+		buf[i] = '_'
+	}
+	return string(buf)
+}
+
+func (cml *configMapAnnotations) GetAnnotation(key string) (string, bool) {
+	value, ok := cml.cm.Annotations[cleanKey(key)]
+	return value, ok
+}
+
+func (cml *configMapAnnotations) UpdateAnnotation(key, value string) (err error) {
+	if cml.cm == nil {
+		return errors.New("endpoint not initialized, call Init first")
+	}
+	// speculatively change the state, then replace with whatever comes back
+	// from Update(), which will be the latest on the server whatever happened
+	cml.cm.Annotations[cleanKey(key)] = value
+	cml.cm, err = cml.Client.ConfigMaps(cml.Namespace).Update(cml.cm)
+	return err
+}
+
+func (cml *configMapAnnotations) RemoveAnnotation(key string) (err error) {
+	if cml.cm == nil {
+		return errors.New("endpoint not initialized, call Init first")
+	}
+	// speculatively change the state, then replace with whatever comes back
+	// from Update(), which will be the latest on the server whatever happened
+	delete(cml.cm.Annotations, cleanKey(key))
+	cml.cm, err = cml.Client.ConfigMaps(cml.Namespace).Update(cml.cm)
+	return err
+}
+
+func (cml *configMapAnnotations) RemoveAnnotationsWithValue(valueToRemove string) (err error) {
+	if cml.cm == nil {
+		return errors.New("endpoint not initialized, call Init first")
+	}
+	// speculatively change the state, then replace with whatever comes back
+	// from Update(), which will be the latest on the server whatever happened
+	for key, value := range cml.cm.Annotations {
+		if value == valueToRemove {
+			delete(cml.cm.Annotations, key) // don't need to clean this key as it came from the map
+		}
+	}
+	cml.cm, err = cml.Client.ConfigMaps(cml.Namespace).Update(cml.cm)
+	return err
+}
+
+// Loop with jitter, fetching the cml data and calling f() until it
+// doesn't get an optimistic locking conflict.
+// If it succeeds or gets any other kind of error, stop the loop.
+func (cml *configMapAnnotations) LoopUpdate(f func() error) error {
+	stop := make(chan struct{})
+	var err error
+	wait.JitterUntil(func() {
+		if err = cml.Init(); err != nil {
+			close(stop)
+			return
+		}
+		err = f()
+		if err != nil && kubeErrors.IsConflict(err) {
+			log.Printf("Optimistic locking conflict: trying again: %s", err)
+			return
+		}
+		close(stop)
+	}, retryPeriod, jitterFactor, true, stop)
+	return err
+}

prog/kube-peers/main.go

@@ -1,40 +1,36 @@
+/*
+Package main deals with weave-net peers on the cluster.
+
+This involves peer management, such as getting the latest peers or removing defunct peers from the cluster
+*/
 package main
 
 import (
+	"flag"
 	"fmt"
-	"log"
 	"net"
+	"os"
 
+	api "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/client-go/kubernetes"
-	api "k8s.io/client-go/pkg/api/v1"
 	"k8s.io/client-go/rest"
+
+	weaveapi "github.com/weaveworks/weave/api"
+	"github.com/weaveworks/weave/common"
 )
 
-func getKubePeers() ([]string, error) {
-	config, err := rest.InClusterConfig()
-	if err != nil {
-		return nil, err
-	}
-	c, err := kubernetes.NewForConfig(config)
-	if err != nil {
-		return nil, err
-	}
-	nodeList, err := c.Nodes().List(api.ListOptions{})
-	if err != nil {
-		// Fallback for cases (e.g. from kube-up.sh) where kube-proxy is not running on master
-		config.Host = "http://localhost:8080"
-		log.Print("error contacting APIServer: ", err, "; trying with fallback: ", config.Host)
-		c, err = kubernetes.NewForConfig(config)
-		if err != nil {
-			return nil, err
-		}
-		nodeList, err = c.Nodes().List(api.ListOptions{})
-	}
+type nodeInfo struct {
+	name string
+	addr string
+}
 
+// return the IP addresses of all nodes in the cluster
+func getKubePeers(c *kubernetes.Clientset) ([]nodeInfo, error) {
+	nodeList, err := c.CoreV1().Nodes().List(api.ListOptions{})
 	if err != nil {
 		return nil, err
 	}
-	addresses := make([]string, 0, len(nodeList.Items))
+	addresses := make([]nodeInfo, 0, len(nodeList.Items))
 	for _, peer := range nodeList.Items {
 		var internalIP, externalIP string
 		for _, addr := range peer.Status.Addresses {
@@ -52,20 +48,189 @@ func getKubePeers() ([]string, error) {
 
 		// Fallback for cases where a Node has an ExternalIP but no InternalIP
 		if internalIP != "" {
-			addresses = append(addresses, internalIP)
+			addresses = append(addresses, nodeInfo{name: peer.Name, addr: internalIP})
 		} else if externalIP != "" {
-			addresses = append(addresses, externalIP)
+			addresses = append(addresses, nodeInfo{name: peer.Name, addr: externalIP})
 		}
 	}
 	return addresses, nil
 }
 
-func main() {
-	peers, err := getKubePeers()
-	if err != nil {
-		log.Fatalf("Could not get peers: %v", err)
+const (
+	configMapName      = "weave-net"
+	configMapNamespace = "kube-system"
+)
+
+// update the list of all peers that have gone through this code path
+func addMyselfToPeerList(cml *configMapAnnotations, c *kubernetes.Clientset, peerName, name string) (*peerList, error) {
+	var list *peerList
+	err := cml.LoopUpdate(func() error {
+		var err error
+		list, err = cml.GetPeerList()
+		if err != nil {
+			return err
+		}
+		if !list.contains(peerName) {
+			list.add(peerName, name)
+			err = cml.UpdatePeerList(*list)
+			if err != nil {
+				return err
+			}
+		}
+		return nil
+	})
+	return list, err
+}
+
+func checkIamInPeerList(cml *configMapAnnotations, c *kubernetes.Clientset, peerName string) (bool, error) {
+	if err := cml.Init(); err != nil {
+		return false, err
 	}
-	for _, addr := range peers {
-		fmt.Println(addr)
+	list, err := cml.GetPeerList()
+	if err != nil {
+		return false, err
+	}
+	common.Log.Debugf("[kube-peers] Checking peer %q against list %v", peerName, list)
+	return list.contains(peerName), nil
+}
+
+// For each of those peers that is no longer listed as a node by
+// Kubernetes, remove it from Weave IPAM
+func reclaimRemovedPeers(weave *weaveapi.Client, cml *configMapAnnotations, nodes []nodeInfo, myPeerName string) error {
+	for {
+		if err := cml.Init(); err != nil {
+			return err
+		}
+		// 1. Compare peers stored in the peerList against all peers reported by k8s now.
+		storedPeerList, err := cml.GetPeerList()
+		if err != nil {
+			return err
+		}
+		peerMap := make(map[string]peerInfo, len(storedPeerList.Peers))
+		for _, peer := range storedPeerList.Peers {
+			peerMap[peer.NodeName] = peer
+		}
+		for _, node := range nodes {
+			delete(peerMap, node.name)
+		}
+		common.Log.Debugln("[kube-peers] Nodes that have disappeared:", peerMap)
+		if len(peerMap) == 0 {
+			break
+		}
+		// 2. Loop for each X in the first set and not in the second - we wish to remove X from our data structures
+		for _, peer := range peerMap {
+			common.Log.Debugln("[kube-peers] Preparing to remove disappeared peer", peer)
+			okToRemove := false
+			// 3. Check if there is an existing annotation with key X
+			if existingAnnotation, found := cml.GetAnnotation(KubePeersPrefix + peer.PeerName); found {
+				common.Log.Debugln("[kube-peers] Existing annotation", existingAnnotation)
+				// 4.   If annotation already contains my identity, ok;
+				if existingAnnotation == myPeerName {
+					okToRemove = true
+				}
+			} else {
+				// 5.   If non-existent, write an annotation with key X and contents "my identity"
+				common.Log.Debugln("[kube-peers] Noting I plan to remove ", peer.PeerName)
+				if err := cml.UpdateAnnotation(KubePeersPrefix+peer.PeerName, myPeerName); err == nil {
+					okToRemove = true
+				} else {
+					common.Log.Debugln("[kube-peers] error from UpdateAnnotation: ", err)
+				}
+			}
+			if okToRemove {
+				// 6.   If step 4 or 5 succeeded, rmpeer X
+				result, err := weave.RmPeer(peer.PeerName)
+				common.Log.Infof("[kube-peers] rmpeer of %s: %s", peer.PeerName, result)
+				if err != nil {
+					return err
+				}
+				cml.LoopUpdate(func() error {
+					// 7aa.   Remove any annotations Z* that have contents X
+					if err := cml.RemoveAnnotationsWithValue(peer.PeerName); err != nil {
+						return err
+					}
+					// 7a.    Remove X from peerList
+					storedPeerList.remove(peer.PeerName)
+					if err := cml.UpdatePeerList(*storedPeerList); err != nil {
+						return err
+					}
+					// 7b.    Remove annotation with key X
+					return cml.RemoveAnnotation(KubePeersPrefix + peer.PeerName)
+				})
+			}
+			// 8.   If step 5 failed due to optimistic lock conflict, stop: someone else is handling X
+
+			// Step 3-5 is to protect against two simultaneous rmpeers of X
+			// Step 4 is to pick up again after a restart between step 5 and step 7b
+			// If the peer doing the reclaim disappears between steps 5 and 7a, then someone will clean it up in step 7aa
+			// If peer doing the reclaim disappears forever between 7a and 7b then we get a dangling annotation
+			// This should be sufficiently rare that we don't care.
+		}
+
+		// 9. Go back to step 1 until there is no difference between the two sets
+	}
+	// Question: Should we narrow step 2 by checking against Weave Net IPAM?
+	// i.e. If peer X owns any address space and is marked unreachable, we want to rmpeer X
+	return nil
+}
+
+func main() {
+	var (
+		justReclaim bool
+		justCheck   bool
+		peerName    string
+		nodeName    string
+		logLevel    string
+	)
+	flag.BoolVar(&justReclaim, "reclaim", false, "reclaim IP space from dead peers")
+	flag.BoolVar(&justCheck, "check-peer-new", false, "return success if peer name is not stored in annotation")
+	flag.StringVar(&peerName, "peer-name", "unknown", "name of this Weave Net peer")
+	flag.StringVar(&nodeName, "node-name", "unknown", "name of this Kubernetes node")
+	flag.StringVar(&logLevel, "log-level", "info", "logging level (debug, info, warning, error)")
+	flag.Parse()
+
+	common.SetLogLevel(logLevel)
+	config, err := rest.InClusterConfig()
+	if err != nil {
+		common.Log.Fatalf("[kube-peers] Could not get cluster config: %v", err)
+	}
+	c, err := kubernetes.NewForConfig(config)
+	if err != nil {
+		common.Log.Fatalf("[kube-peers] Could not make Kubernetes connection: %v", err)
+	}
+	if justCheck {
+		cml := newConfigMapAnnotations(configMapNamespace, configMapName, c)
+		exists, err := checkIamInPeerList(cml, c, peerName)
+		if err != nil {
+			common.Log.Fatalf("[kube-peers] Could not check peer list: %v", err)
+		}
+		if exists {
+			os.Exit(9)
+		} else {
+			os.Exit(0)
+		}
+	}
+	peers, err := getKubePeers(c)
+	if err != nil {
+		common.Log.Fatalf("[kube-peers] Could not get peers: %v", err)
+	}
+	if justReclaim {
+		cml := newConfigMapAnnotations(configMapNamespace, configMapName, c)
+
+		list, err := addMyselfToPeerList(cml, c, peerName, nodeName)
+		if err != nil {
+			common.Log.Fatalf("[kube-peers] Could not update peer list: %v", err)
+		}
+		common.Log.Infoln("[kube-peers] Added myself to peer list", list)
+
+		weave := weaveapi.NewClient(os.Getenv("WEAVE_HTTP_ADDR"), common.Log)
+		err = reclaimRemovedPeers(weave, cml, peers, peerName)
+		if err != nil {
+			common.Log.Fatalf("[kube-peers] Error while reclaiming space: %v", err)
+		}
+		return
+	}
+	for _, node := range peers {
+		fmt.Println(node.addr)
 	}
 }

prog/net-plugin/config.json

@@ -13,7 +13,8 @@
   "linux": {
     "capabilities": [
       "CAP_SYS_ADMIN",
-      "CAP_NET_ADMIN"
+      "CAP_NET_ADMIN",
+      "CAP_SYS_MODULE"
     ]
   },
   "mounts": [
@@ -40,6 +41,12 @@
       "source": "/etc/",
       "type": "bind",
       "options": ["rbind"]
+    },
+    {
+      "destination": "/lib/modules/",
+      "source": "/lib/modules/",
+      "type": "bind",
+      "options": ["rbind"]
     }
   ],
   "env": [

prog/weave-kube/launch.sh

@@ -30,6 +30,7 @@ HTTP_ADDR=${WEAVE_HTTP_ADDR:-127.0.0.1:6784}
 STATUS_ADDR=${WEAVE_STATUS_ADDR:-0.0.0.0:6782}
 HOST_ROOT=${HOST_ROOT:-/host}
 CONN_LIMIT=${CONN_LIMIT:-30}
+DB_PREFIX=${DB_PREFIX:-/weavedb/weave-net}
 
 # Check if the IP range overlaps anything existing on the host
 /usr/bin/weaveutil netcheck $IPALLOC_RANGE weave
@@ -87,6 +88,29 @@ if [ -z "$IPALLOC_INIT" ]; then
     IPALLOC_INIT="consensus=$(peer_count $KUBE_PEERS)"
 fi
 
+# Find out what peer name we will be using
+PEERNAME=$(/home/weave/weaver $EXTRA_ARGS --print-peer-name --host-root=$HOST_ROOT --db-prefix="$DB_PREFIX")
+if [ -z "$PEERNAME" ]; then
+    echo "Unable to get peer name" >&2
+    exit 2
+fi
+
+# If we have persisted data and peer reclaim is in use
+if [ -f ${DB_PREFIX}data.db ]; then
+    if /usr/bin/weaveutil get-db-flag "$DB_PREFIX" peer-reclaim >/dev/null ; then
+        # If this peer name is not stored in the list then we were removed by another peer.
+        if /home/weave/kube-peers -check-peer-new -peer-name="$PEERNAME" -log-level=debug ; then
+            # In order to avoid a CRDT clash, clean up
+            echo "Peer not in list; removing persisted data" >&2
+            rm -f ${DB_PREFIX}data.db
+        fi
+    fi
+fi
+
+# flag that peer reclaim is now in use - note we have to do this before
+# weaver is running as the DB can only be opened by one process at a time
+/usr/bin/weaveutil set-db-flag "$DB_PREFIX" peer-reclaim ok
+
 post_start_actions() {
     # Wait for weave process to become responsive
     while true ; do
@@ -108,6 +132,9 @@ post_start_actions() {
     export HOST_ROOT
     /home/weave/weave --local setup-cni
 
+    # Attempt to run the reclaim process, but don't halt the script if it fails
+    /home/weave/kube-peers -reclaim -node-name="$HOSTNAME" -peer-name="$PEERNAME" -log-level=debug || true
+
     # Expose the weave network so host processes can communicate with pods
     /home/weave/weave --local expose $WEAVE_EXPOSE_IP
 }
@@ -115,9 +142,10 @@ post_start_actions() {
 post_start_actions &
 
 /home/weave/weaver $EXTRA_ARGS --port=6783 $(router_bridge_opts) \
+     --name="$PEERNAME" \
      --host-root=$HOST_ROOT \
      --http-addr=$HTTP_ADDR --status-addr=$STATUS_ADDR --docker-api='' --no-dns \
-     --db-prefix="/weavedb/weave-net" \
+     --db-prefix="$DB_PREFIX" \
      --ipalloc-range=$IPALLOC_RANGE $NICKNAME_ARG \
      --ipalloc-init $IPALLOC_INIT \
      --conn-limit=$CONN_LIMIT \

prog/weave-kube/weave-daemonset-k8s-1.6.yaml

@@ -7,6 +7,7 @@ items:
       name: weave-net
       labels:
         name: weave-net
+      namespace: kube-system
   - apiVersion: rbac.authorization.k8s.io/v1beta1
     kind: ClusterRole
     metadata:
@@ -46,12 +47,51 @@ items:
       - kind: ServiceAccount
         name: weave-net
         namespace: kube-system
+  - apiVersion: rbac.authorization.k8s.io/v1beta1
+    kind: Role
+    metadata:
+      name: weave-net-kube-peer
+      namespace: kube-system
+      labels:
+        name: weave-net-kube-peer
+    rules:
+      - apiGroups:
+          - ''
+        resources:
+          - configmaps
+        resourceNames:
+          - weave-net
+        verbs:
+          - get
+          - update
+      - apiGroups:
+          - ''
+        resources:
+          - configmaps
+        verbs:
+          - create
+  - apiVersion: rbac.authorization.k8s.io/v1beta1
+    kind: RoleBinding
+    metadata:
+      name: weave-net-kube-peer
+      namespace: kube-system
+      labels:
+        name: weave-net-kube-peer
+    roleRef:
+      kind: Role
+      name: weave-net-kube-peer
+      apiGroup: rbac.authorization.k8s.io
+    subjects:
+      - kind: ServiceAccount
+        name: weave-net
+        namespace: kube-system
   - apiVersion: extensions/v1beta1
     kind: DaemonSet
     metadata:
       name: weave-net
       labels:
         name: weave-net
+      namespace: kube-system
     spec:
       template:
         metadata:
@@ -95,6 +135,12 @@ items:
                 - name: lib-modules
                   mountPath: /lib/modules
             - name: weave-npc
+              env:
+                - name: HOSTNAME
+                  valueFrom:
+                    fieldRef:
+                      apiVersion: v1
+                      fieldPath: spec.nodeName
               image: 'weaveworks/weave-npc:latest'
               imagePullPolicy: Always
               resources:

prog/weave-kube/weave-daemonset-k8s-1.7.yaml

@@ -0,0 +1,199 @@
+apiVersion: v1
+kind: List
+items:
+  - apiVersion: v1
+    kind: ServiceAccount
+    metadata:
+      name: weave-net
+      labels:
+        name: weave-net
+      namespace: kube-system
+  - apiVersion: rbac.authorization.k8s.io/v1beta1
+    kind: ClusterRole
+    metadata:
+      name: weave-net
+      labels:
+        name: weave-net
+    rules:
+      - apiGroups:
+          - ''
+        resources:
+          - pods
+          - namespaces
+          - nodes
+        verbs:
+          - get
+          - list
+          - watch
+      - apiGroups:
+          - extensions
+        resources:
+          - networkpolicies
+        verbs:
+          - get
+          - list
+          - watch
+      - apiGroups:
+          - 'networking.k8s.io'
+        resources:
+          - networkpolicies
+        verbs:
+          - get
+          - list
+          - watch
+  - apiVersion: rbac.authorization.k8s.io/v1beta1
+    kind: ClusterRoleBinding
+    metadata:
+      name: weave-net
+      labels:
+        name: weave-net
+    roleRef:
+      kind: ClusterRole
+      name: weave-net
+      apiGroup: rbac.authorization.k8s.io
+    subjects:
+      - kind: ServiceAccount
+        name: weave-net
+        namespace: kube-system
+  - apiVersion: rbac.authorization.k8s.io/v1beta1
+    kind: Role
+    metadata:
+      name: weave-net
+      namespace: kube-system
+      labels:
+        name: weave-net
+    rules:
+      - apiGroups:
+          - ''
+        resources:
+          - configmaps
+        resourceNames:
+          - weave-net
+        verbs:
+          - get
+          - update
+      - apiGroups:
+          - ''
+        resources:
+          - configmaps
+        verbs:
+          - create
+  - apiVersion: rbac.authorization.k8s.io/v1beta1
+    kind: RoleBinding
+    metadata:
+      name: weave-net
+      namespace: kube-system
+      labels:
+        name: weave-net
+    roleRef:
+      kind: Role
+      name: weave-net
+      apiGroup: rbac.authorization.k8s.io
+    subjects:
+      - kind: ServiceAccount
+        name: weave-net
+        namespace: kube-system
+  - apiVersion: extensions/v1beta1
+    kind: DaemonSet
+    metadata:
+      name: weave-net
+      labels:
+        name: weave-net
+      namespace: kube-system
+    spec:
+      template:
+        metadata:
+          labels:
+            name: weave-net
+        spec:
+          containers:
+            - name: weave
+              command:
+                - /home/weave/launch.sh
+              env:
+                - name: HOSTNAME
+                  valueFrom:
+                    fieldRef:
+                      apiVersion: v1
+                      fieldPath: spec.nodeName
+              image: 'weaveworks/weave-kube:latest'
+              imagePullPolicy: Always
+              livenessProbe:
+                httpGet:
+                  host: 127.0.0.1
+                  path: /status
+                  port: 6784
+                initialDelaySeconds: 30
+              resources:
+                requests:
+                  cpu: 10m
+              securityContext:
+                privileged: true
+              volumeMounts:
+                - name: weavedb
+                  mountPath: /weavedb
+                - name: cni-bin
+                  mountPath: /host/opt
+                - name: cni-bin2
+                  mountPath: /host/home
+                - name: cni-conf
+                  mountPath: /host/etc
+                - name: dbus
+                  mountPath: /host/var/lib/dbus
+                - name: lib-modules
+                  mountPath: /lib/modules
+                - name: xtables-lock
+                  mountPath: /run/xtables.lock
+                  readOnly: false
+            - name: weave-npc
+              env:
+                - name: HOSTNAME
+                  valueFrom:
+                    fieldRef:
+                      apiVersion: v1
+                      fieldPath: spec.nodeName
+              image: 'weaveworks/weave-npc:latest'
+              imagePullPolicy: Always
+#npc-args
+              resources:
+                requests:
+                  cpu: 10m
+              securityContext:
+                privileged: true
+              volumeMounts:
+                - name: xtables-lock
+                  mountPath: /run/xtables.lock
+                  readOnly: false
+          hostNetwork: true
+          hostPID: true
+          restartPolicy: Always
+          securityContext:
+            seLinuxOptions: {}
+          serviceAccountName: weave-net
+          tolerations:
+            - effect: NoSchedule
+              operator: Exists
+          volumes:
+            - name: weavedb
+              hostPath:
+                path: /var/lib/weave
+            - name: cni-bin
+              hostPath:
+                path: /opt
+            - name: cni-bin2
+              hostPath:
+                path: /home
+            - name: cni-conf
+              hostPath:
+                path: /etc
+            - name: dbus
+              hostPath:
+                path: /var/lib/dbus
+            - name: lib-modules
+              hostPath:
+                path: /lib/modules
+            - name: xtables-lock
+              hostPath:
+                path: /run/xtables.lock
+      updateStrategy:
+        type: RollingUpdate

prog/weave-kube/weave-daemonset.yaml

@@ -7,12 +7,14 @@ items:
       name: weave-net
       labels:
         name: weave-net
+      namespace: kube-system
   - apiVersion: extensions/v1beta1
     kind: DaemonSet
     metadata:
       name: weave-net
       labels:
         name: weave-net
+      namespace: kube-system
     spec:
       template:
         metadata:
@@ -59,6 +61,12 @@ items:
                 - name: lib-modules
                   mountPath: /lib/modules
             - name: weave-npc
+              env:
+                - name: HOSTNAME
+                  valueFrom:
+                    fieldRef:
+                      apiVersion: v1
+                      fieldPath: spec.nodeName
               image: 'weaveworks/weave-npc:latest'
               imagePullPolicy: Always
               resources:

prog/weave-npc/main.go

@@ -7,12 +7,13 @@ import (
 
 	"github.com/coreos/go-iptables/iptables"
 	"github.com/spf13/cobra"
+	coreapi "k8s.io/api/core/v1"
+	extnapi "k8s.io/api/extensions/v1beta1"
+	networkingv1 "k8s.io/api/networking/v1"
+	"k8s.io/apimachinery/pkg/fields"
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/util/wait"
 	"k8s.io/client-go/kubernetes"
-	coreapi "k8s.io/client-go/pkg/api/v1"
-	extnapi "k8s.io/client-go/pkg/apis/extensions/v1beta1"
-	"k8s.io/client-go/pkg/fields"
-	"k8s.io/client-go/pkg/runtime"
-	"k8s.io/client-go/pkg/util/wait"
 	"k8s.io/client-go/rest"
 	"k8s.io/client-go/tools/cache"
 
@@ -29,12 +30,13 @@ var (
 	logLevel    string
 	allowMcast  bool
 	nodeName    string
+	legacy      bool
 )
 
 func handleError(err error) { common.CheckFatal(err) }
 
 func makeController(getter cache.Getter, resource string,
-	objType runtime.Object, handlers cache.ResourceEventHandlerFuncs) *cache.Controller {
+	objType runtime.Object, handlers cache.ResourceEventHandlerFuncs) cache.Controller {
 	listWatch := cache.NewListWatchFromClient(getter, resource, "", fields.Everything())
 	_, controller := cache.NewInformer(listWatch, objType, 0, handlers)
 	return controller
@@ -50,11 +52,7 @@ func resetIPTables(ipt *iptables.IPTables) error {
 		return err
 	}
 
-	if err := ipt.ClearChain(npc.TableFilter, npc.MainChain); err != nil {
-		return err
-	}
-
-	return nil
+	return ipt.ClearChain(npc.TableFilter, npc.MainChain)
 }
 
 func resetIPSets(ips ipset.Interface) error {
@@ -116,15 +114,13 @@ func createBaseRules(ipt *iptables.IPTables, ips ipset.Interface) error {
 	if err := ips.Create(npc.LocalIpset, ipset.HashIP); err != nil {
 		return err
 	}
-	if err := ipt.Append(npc.TableFilter, npc.MainChain,
-		"-m", "set", "!", "--match-set", npc.LocalIpset, "dst", "-j", "ACCEPT"); err != nil {
-		return err
-	}
-
-	return nil
+	return ipt.Append(npc.TableFilter, npc.MainChain,
+		"-m", "set", "!", "--match-set", npc.LocalIpset, "dst", "-j", "ACCEPT")
 }
 
 func root(cmd *cobra.Command, args []string) {
+	var npController cache.Controller
+
 	common.SetLogLevel(logLevel)
 	if nodeName == "" {
 		// HOSTNAME is set by Kubernetes for pods in the host network namespace
@@ -135,6 +131,10 @@ func root(cmd *cobra.Command, args []string) {
 	}
 	common.Log.Infof("Starting Weaveworks NPC %s; node name %q", version, nodeName)
 
+	if legacy {
+		common.Log.Info("Running in legacy mode (k8s pre-1.7 network policy semantics)")
+	}
+
 	if err := metrics.Start(metricsAddr); err != nil {
 		common.Log.Fatalf("Failed to start metrics: %v", err)
 	}
@@ -158,7 +158,7 @@ func root(cmd *cobra.Command, args []string) {
 	handleError(resetIPSets(ips))
 	handleError(createBaseRules(ipt, ips))
 
-	npc := npc.New(nodeName, ipt, ips)
+	npc := npc.New(nodeName, legacy, ipt, ips)
 
 	nsController := makeController(client.Core().RESTClient(), "namespaces", &coreapi.Namespace{},
 		cache.ResourceEventHandlerFuncs{
@@ -200,25 +200,30 @@ func root(cmd *cobra.Command, args []string) {
 				handleError(npc.UpdatePod(old.(*coreapi.Pod), new.(*coreapi.Pod)))
 			}})
 
-	npController := makeController(client.Extensions().RESTClient(), "networkpolicies", &extnapi.NetworkPolicy{},
-		cache.ResourceEventHandlerFuncs{
-			AddFunc: func(obj interface{}) {
-				handleError(npc.AddNetworkPolicy(obj.(*extnapi.NetworkPolicy)))
-			},
-			DeleteFunc: func(obj interface{}) {
-				switch obj := obj.(type) {
-				case *extnapi.NetworkPolicy:
-					handleError(npc.DeleteNetworkPolicy(obj))
-				case cache.DeletedFinalStateUnknown:
-					// We know this object has gone away, but its final state is no longer
-					// available from the API server. Instead we use the last copy of it
-					// that we have, which is good enough for our cleanup.
-					handleError(npc.DeleteNetworkPolicy(obj.Obj.(*extnapi.NetworkPolicy)))
-				}
-			},
-			UpdateFunc: func(old, new interface{}) {
-				handleError(npc.UpdateNetworkPolicy(old.(*extnapi.NetworkPolicy), new.(*extnapi.NetworkPolicy)))
-			}})
+	npHandlers := cache.ResourceEventHandlerFuncs{
+		AddFunc: func(obj interface{}) {
+			handleError(npc.AddNetworkPolicy(obj))
+		},
+		DeleteFunc: func(obj interface{}) {
+			switch obj := obj.(type) {
+			case cache.DeletedFinalStateUnknown:
+				// We know this object has gone away, but its final state is no longer
+				// available from the API server. Instead we use the last copy of it
+				// that we have, which is good enough for our cleanup.
+				handleError(npc.DeleteNetworkPolicy(obj.Obj))
+			default:
+				handleError(npc.DeleteNetworkPolicy(obj))
+			}
+		},
+		UpdateFunc: func(old, new interface{}) {
+			handleError(npc.UpdateNetworkPolicy(old, new))
+		},
+	}
+	if legacy {
+		npController = makeController(client.Extensions().RESTClient(), "networkpolicies", &extnapi.NetworkPolicy{}, npHandlers)
+	} else {
+		npController = makeController(client.NetworkingV1().RESTClient(), "networkpolicies", &networkingv1.NetworkPolicy{}, npHandlers)
+	}
 
 	go nsController.Run(wait.NeverStop)
 	go podController.Run(wait.NeverStop)
@@ -239,6 +244,7 @@ func main() {
 	rootCmd.PersistentFlags().StringVar(&logLevel, "log-level", "debug", "logging level (debug, info, warning, error)")
 	rootCmd.PersistentFlags().BoolVar(&allowMcast, "allow-mcast", true, "allow all multicast traffic")
 	rootCmd.PersistentFlags().StringVar(&nodeName, "node-name", "", "only generate rules that apply to this node")
+	rootCmd.PersistentFlags().BoolVar(&legacy, "use-legacy-netpol", false, "use legacy network policies (pre k8s 1.7 vsn)")
 
 	handleError(rootCmd.Execute())
 }

prog/weavedb/Dockerfile

@@ -1,6 +1,7 @@
 # This is a nearly-empty image that we use to create a data-only container for persistence
 FROM scratch
 LABEL maintainer "Weaveworks Inc <help@weave.works>"
+LABEL works.weave.role=system
 ENTRYPOINT ["data-only"]
 # Work round Docker refusing to save an empty image
 COPY Dockerfile /

prog/weaveexec/Dockerfile.template

@@ -3,7 +3,6 @@ FROM DOCKERHUB_USER/weaveARCH_EXT
 ENTRYPOINT ["/home/weave/sigproxy", "/home/weave/weave"]
 
 ADD ./sigproxy ./symlink /home/weave/
-ADD ./docker /usr/bin/
 ADD ./weavewait /w/w
 ADD ./weavewait_noop /w-noop/w
 ADD ./weavewait_nomcast /w-nomcast/w

prog/weaver/discovery.go

@@ -77,7 +77,7 @@ func peerDiscoveryDelete(discoveryEndpoint, token, peername string) error {
 
 func HandleHTTPPeer(router *mux.Router, alloc *ipam.Allocator, discoveryEndpoint, token, peername string) {
 	router.Methods("DELETE").Path("/peer").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		if discoveryEndpoint != "" {
+		if discoveryEndpoint != "" && token != "" {
 			if err := peerDiscoveryDelete(discoveryEndpoint, token, peername); err != nil {
 				Log.Errorf("Error while deleting self from peer discovery: %s", err)
 			}
@@ -90,7 +90,7 @@ func HandleHTTPPeer(router *mux.Router, alloc *ipam.Allocator, discoveryEndpoint
 
 	router.Methods("DELETE").Path("/peer/{id}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ident := mux.Vars(r)["id"]
-		if discoveryEndpoint != "" {
+		if discoveryEndpoint != "" && token != "" {
 			// TODO: deal with this being either a peername or a nickname
 			if err := peerDiscoveryDelete(discoveryEndpoint, token, ident); err != nil {
 				Log.Errorf("Error while deleting self from peer discovery: %s", err)

prog/weaver/http.go

@@ -34,23 +34,7 @@ var rootTemplate = template.New("root").Funcs(map[string]interface{}{
 	},
 	"printIPAMRanges": func(router weave.NetworkRouterStatus, status ipam.Status) string {
 		var buffer bytes.Buffer
-
-		type stats struct {
-			ips       uint32
-			nickname  string
-			reachable bool
-		}
-
-		peerStats := make(map[string]*stats)
-
-		for _, entry := range status.Entries {
-			s, found := peerStats[entry.Peer]
-			if !found {
-				s = &stats{nickname: entry.Nickname, reachable: entry.IsKnownPeer}
-				peerStats[entry.Peer] = s
-			}
-			s.ips += entry.Size
-		}
+		peerStats := summariseIpamStats(&status)
 
 		printOwned := func(name string, nickName string, info string, ips uint32) {
 			percentageRanges := float32(ips) * 100.0 / float32(status.RangeNumIPs)
@@ -116,6 +100,25 @@ var rootTemplate = template.New("root").Funcs(map[string]interface{}{
 	"trimSuffix": strings.TrimSuffix,
 })
 
+type ipamStats struct {
+	ips       uint32
+	nickname  string
+	reachable bool
+}
+
+func summariseIpamStats(status *ipam.Status) map[string]*ipamStats {
+	peerStats := make(map[string]*ipamStats)
+	for _, entry := range status.Entries {
+		s, found := peerStats[entry.Peer]
+		if !found {
+			s = &ipamStats{nickname: entry.Nickname, reachable: entry.IsKnownPeer}
+			peerStats[entry.Peer] = s
+		}
+		s.ips += entry.Size
+	}
+	return peerStats
+}
+
 func countDNSEntries(entries []nameserver.EntryStatus) int {
 	count := 0
 	for _, entry := range entries {

prog/weaver/main.go

@@ -131,6 +131,7 @@ func main() {
 
 	var (
 		justVersion        bool
+		justPeerName       bool
 		config             mesh.Config
 		bridgeConfig       weavenet.BridgeConfig
 		networkConfig      weave.NetworkConfig
@@ -163,6 +164,7 @@ func main() {
 	)
 
 	mflag.BoolVar(&justVersion, []string{"-version"}, false, "print version and exit")
+	mflag.BoolVar(&justPeerName, []string{"-print-peer-name"}, false, "print peer name and exit")
 	mflag.StringVar(&config.Host, []string{"-host"}, "", "router host")
 	mflag.IntVar(&config.Port, []string{"-port"}, mesh.Port, "router port")
 	mflag.IntVar(&protocolMinVersion, []string{"-min-protocol-version"}, mesh.ProtocolMinVersion, "minimum weave protocol version")
@@ -210,11 +212,7 @@ func main() {
 	mflag.StringVar(&pluginConfig.Socket, []string{"-plugin-socket"}, "/run/docker/plugins/weave.sock", "plugin socket on which to listen")
 	mflag.StringVar(&pluginConfig.MeshSocket, []string{"-plugin-mesh-socket"}, "/run/docker/plugins/weavemesh.sock", "plugin socket on which to listen in mesh mode")
 
-	proxyConfig := configureProxy(version, defaultDockerHost)
-	if bridgeConfig.AWSVPC {
-		proxyConfig.NoMulticastRoute = true
-		proxyConfig.KeepTXOn = true
-	}
+	proxyConfig := newProxyConfig()
 
 	// crude way of detecting that we probably have been started in a
 	// container, with `weave launch` --> suppress misleading paths in
@@ -230,6 +228,17 @@ func main() {
 		fmt.Printf("weave %s\n", version)
 		os.Exit(0)
 	}
+	name := peerName(routerName, bridgeConfig.WeaveBridgeName, dbPrefix, hostRoot)
+	if justPeerName {
+		fmt.Printf("%s\n", name)
+		os.Exit(0)
+	}
+
+	proxyConfig.DockerHost = dockerAPI
+	if bridgeConfig.AWSVPC {
+		proxyConfig.NoMulticastRoute = true
+		proxyConfig.KeepTXOn = true
+	}
 
 	peers = mflag.Args()
 	if resume && len(peers) > 0 {
@@ -254,6 +263,9 @@ func main() {
 	var proxy *weaveproxy.Proxy
 	var err error
 	if proxyConfig.Enabled {
+		if noDNS {
+			proxyConfig.WithoutDNS = true
+		}
 		// Start Weave Proxy:
 		proxy, err = weaveproxy.NewProxy(*proxyConfig)
 		if err != nil {
@@ -279,10 +291,9 @@ func main() {
 		}
 	}
 
-	name := peerName(routerName, bridgeConfig.WeaveBridgeName, dbPrefix, hostRoot)
-
 	bridgeConfig.Mac = name.String()
-	bridgeType, err := weavenet.EnsureBridge(procPath, &bridgeConfig)
+	bridgeConfig.Port = config.Port
+	bridgeType, err := weavenet.EnsureBridge(procPath, &bridgeConfig, Log)
 	checkFatal(err)
 	Log.Println("Bridge type is", bridgeType)
 
@@ -318,7 +329,7 @@ func main() {
 	checkFatal(err)
 	defer db.Close()
 
-	router, err := weave.NewNetworkRouter(config, networkConfig, name, nickName, overlay, db)
+	router, err := weave.NewNetworkRouter(config, networkConfig, bridgeConfig, name, nickName, overlay, db)
 	checkFatal(err)
 	Log.Println("Our name is", router.Ourself)
 
@@ -465,17 +476,17 @@ func main() {
 		// Run this on its own goroutine because the allocator can block
 		// We remove the default route installed by the kernel,
 		// because awsvpc has installed it as well
-		go expose(allocator, defaultSubnet, bridgeConfig.WeaveBridgeName, bridgeConfig.AWSVPC, waitReady.Add())
+		go exposeForAWSVPC(allocator, defaultSubnet, bridgeConfig.WeaveBridgeName, waitReady.Add())
 	}
 
 	signals.SignalHandlerLoop(common.Log, router)
 }
 
-func expose(alloc *ipam.Allocator, subnet address.CIDR, bridgeName string, removeDefaultRoute bool, ready func()) {
+func exposeForAWSVPC(alloc *ipam.Allocator, subnet address.CIDR, bridgeName string, ready func()) {
 	addr, err := alloc.Allocate("weave:expose", subnet, false, func() bool { return false })
 	checkFatal(err)
 	cidr := address.MakeCIDR(subnet, addr)
-	err = weavenet.AddBridgeAddr(bridgeName, cidr.IPNet(), removeDefaultRoute)
+	err = weavenet.Expose(bridgeName, cidr.IPNet(), true, false)
 	checkFatal(err)
 	Log.Printf("Bridge %q exposed on address %v", bridgeName, cidr)
 	ready()
@@ -521,12 +532,10 @@ func (nopPacketLogging) LogPacket(string, weave.PacketKey) {
 func (nopPacketLogging) LogForwardPacket(string, weave.ForwardPacketKey) {
 }
 
-func configureProxy(version string, defaultDockerHost string) *weaveproxy.Config {
+func newProxyConfig() *weaveproxy.Config {
 	proxyConfig := weaveproxy.Config{
-		Version:      version,
 		Image:        getenvOrDefault("EXEC_IMAGE", "weaveworks/weaveexec"),
 		DockerBridge: getenvOrDefault("DOCKER_BRIDGE", "docker0"),
-		DockerHost:   defaultDockerHost,
 	}
 	mflag.BoolVar(&proxyConfig.Enabled, []string{"-proxy"}, false, "instruct Weave Net to start its Docker proxy")
 	mflagext.ListVar(&proxyConfig.ListenAddrs, []string{"H"}, nil, "addresses on which to listen for Docker proxy")

prog/weaver/main_test.go

@@ -7,6 +7,7 @@ import (
 )
 
 func TestMain(t *testing.T) {
-	weavetest.TrimTestArgs()
-	main()
+	if weavetest.TrimTestArgs() {
+		main()
+	}
 }

prog/weaver/metrics.go

@@ -82,6 +82,27 @@ var metrics = []metric{
 				ch <- intGauge(desc, metrics.Flows)
 			}
 		}},
+	{desc("weave_ipam_unreachable_count", "Number of unreachable peers."),
+		func(s WeaveStatus, desc *prometheus.Desc, ch chan<- prometheus.Metric) {
+			var count int
+			for _, entry := range summariseIpamStats(s.IPAM) {
+				if !entry.reachable {
+					count++
+				}
+			}
+			ch <- intGauge(desc, count)
+		}},
+	{desc("weave_ipam_unreachable_percentage", "Percentage of IP addresses owned  by unreachable peers."),
+		func(s WeaveStatus, desc *prometheus.Desc, ch chan<- prometheus.Metric) {
+			var totalUnreachable uint32
+			for _, entry := range summariseIpamStats(s.IPAM) {
+				if !entry.reachable {
+					totalUnreachable += entry.ips
+				}
+			}
+			percentage := float64(totalUnreachable) * 100.0 / float64(s.IPAM.RangeNumIPs)
+			ch <- prometheus.MustNewConstMetric(desc, prometheus.GaugeValue, percentage)
+		}},
 	{desc("weave_ipam_pending_allocates", "Number of pending allocates."),
 		func(s WeaveStatus, desc *prometheus.Desc, ch chan<- prometheus.Metric) {
 			if s.IPAM != nil {

prog/weaveutil/attach.go

@@ -8,7 +8,6 @@ import (
 	"strings"
 	"syscall"
 
-	docker "github.com/fsouza/go-dockerclient"
 	"github.com/vishvananda/netns"
 
 	weavenet "github.com/weaveworks/weave/net"
@@ -70,21 +69,6 @@ func attach(args []string) error {
 	return err
 }
 
-func containerPid(containerID string) (int, error) {
-	c, err := docker.NewVersionedClientFromEnv("1.18")
-	if err != nil {
-		return 0, fmt.Errorf("unable to connect to docker: %s", err)
-	}
-	container, err := c.InspectContainer(containerID)
-	if err != nil {
-		return 0, fmt.Errorf("unable to inspect container %s: %s", containerID, err)
-	}
-	if container.State.Pid == 0 {
-		return 0, fmt.Errorf("container %s not running", containerID)
-	}
-	return container.State.Pid, nil
-}
-
 func processExists(pid int) bool {
 	err := syscall.Kill(pid, syscall.Signal(0))
 	return err == nil || err == syscall.EPERM
@@ -119,21 +103,31 @@ func detach(args []string) error {
 }
 
 func rewriteEtcHosts(args []string) error {
-	if len(args) < 4 {
-		cmdUsage("rewrite-etc-hosts", "<hosts-path> <fqdn> <image> <cidr>... [name:addr...]")
+	if len(args) < 3 {
+		cmdUsage("rewrite-etc-hosts", "<container-id> <image> <cidr> [name:addr...]")
 	}
-	hostsPath := args[0]
-	fqdn := args[1]
-	image := args[2]
+	containerID := args[0]
+	image := args[1]
+	cidrs := args[2]
+	extraHosts := args[3:]
+
+	container, err := inspectContainer(containerID)
+	if err != nil {
+		return err
+	}
+
+	hostsPath := container.HostsPath
+	fqdn := container.Config.Hostname + "." + container.Config.Domainname
+
 	var ips []*net.IPNet
-	for _, cidr := range strings.Fields(args[3]) {
+	for _, cidr := range strings.Fields(cidrs) {
 		_, ipnet, err := net.ParseCIDR(cidr)
 		if err != nil {
 			return err
 		}
 		ips = append(ips, ipnet)
 	}
-	extraHosts := args[4:]
+
 	docker := os.Getenv("DOCKER_HOST")
 	if docker == "" {
 		docker = "unix:///var/run/docker.sock"

prog/weaveutil/bridge.go

@@ -4,6 +4,7 @@ import (
 	"fmt"
 	"strconv"
 
+	"github.com/weaveworks/weave/common"
 	weavenet "github.com/weaveworks/weave/net"
 )
 
@@ -47,7 +48,7 @@ func createBridge(args []string) error {
 		NPC:              args[9] == "--expect-npc",
 	}
 	procPath := args[8]
-	bridgeType, err := weavenet.EnsureBridge(procPath, &config)
+	bridgeType, err := weavenet.EnsureBridge(procPath, &config, common.Log)
 	if err != nil {
 		return err
 	}

prog/weaveutil/container.go

@@ -0,0 +1,285 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	docker "github.com/fsouza/go-dockerclient"
+)
+
+func inspectContainer(containerNameOrID string) (*docker.Container, error) {
+	c, err := docker.NewVersionedClientFromEnv("1.18")
+	if err != nil {
+		return nil, fmt.Errorf("unable to connect to docker: %s", err)
+	}
+
+	container, err := c.InspectContainer(containerNameOrID)
+	if err != nil {
+		return nil, fmt.Errorf("unable to inspect container %s: %s", containerNameOrID, err)
+	}
+	return container, nil
+}
+
+func containerPid(containerID string) (int, error) {
+	container, err := inspectContainer(containerID)
+	if err != nil {
+		return 0, err
+
+	}
+
+	if container.State.Pid == 0 {
+		return 0, fmt.Errorf("container %s not running", containerID)
+	}
+	return container.State.Pid, nil
+}
+
+func containerID(args []string) error {
+	if len(args) < 1 {
+		cmdUsage("container-id", "<container-name-or-short-id>")
+	}
+
+	container, err := inspectContainer(args[0])
+	if err != nil {
+		return err
+	}
+
+	fmt.Print(container.ID)
+	return nil
+}
+
+func containerState(args []string) error {
+	if len(args) < 1 {
+		cmdUsage("container-state", "<container-id> [<image-prefix>]")
+	}
+
+	container, err := inspectContainer(args[0])
+	if err != nil {
+		return err
+	}
+
+	if len(args) == 1 {
+		fmt.Print(container.State.StateString())
+	} else {
+		imagePrefix := args[1]
+		if strings.HasPrefix(container.Config.Image, imagePrefix) {
+			fmt.Print(container.State.StateString())
+		} else {
+			if container.State.Running {
+				fmt.Print("running image mismatch: ", container.Config.Image)
+			} else {
+				fmt.Print("image mismatch: ", container.Config.Image)
+			}
+		}
+	}
+	return nil
+}
+
+func containerFQDN(args []string) error {
+	if len(args) < 1 {
+		cmdUsage("container-fqdn", "<container-id>")
+	}
+
+	container, err := inspectContainer(args[0])
+	if err != nil {
+		return err
+	}
+
+	fmt.Print(container.Config.Hostname, ".", container.Config.Domainname)
+	return nil
+}
+
+func runContainer(args []string) error {
+	env := []string{}
+	name := ""
+	net := ""
+	pid := ""
+	privileged := false
+	restart := docker.NeverRestart()
+	volumes := []string{}
+	volumesFrom := []string{}
+
+	done := false
+	for i := 0; i < len(args) && !done; {
+		switch args[i] {
+		case "-e", "--env":
+			if v := os.Getenv(args[i+1]); v != "" {
+				env = append(env, args[i+1]+"="+v)
+			} else {
+				env = append(env, args[i+1])
+			}
+			args = append(args[:i], args[i+2:]...)
+		case "--name":
+			name = args[i+1]
+			args = append(args[:i], args[i+2:]...)
+		case "--net":
+			net = args[i+1]
+			args = append(args[:i], args[i+2:]...)
+		case "--pid":
+			pid = args[i+1]
+			args = append(args[:i], args[i+2:]...)
+		case "--privileged":
+			privileged = true
+			args = append(args[:i], args[i+1:]...)
+		case "--restart":
+			restart = docker.RestartPolicy{Name: args[i+1]}
+			args = append(args[:i], args[i+2:]...)
+		case "-v", "--volume":
+			// Must dedup binds, otherwise, container fails creation
+			skip := false
+			for _, v := range volumes {
+				if v == args[i+1] {
+					skip = true
+				}
+			}
+			if !skip {
+				volumes = append(volumes, args[i+1])
+			}
+			args = append(args[:i], args[i+2:]...)
+		case "--volumes-from":
+			volumesFrom = append(volumesFrom, args[i+1])
+			args = append(args[:i], args[i+2:]...)
+		default:
+			done = true
+		}
+	}
+
+	if len(args) < 2 {
+		cmdUsage("run-container", `[options] <image> <cmd> [[<cmd-options>] <cmd-arg1> [<cmd-arg2> ...]]
+
+  -e, --env list                       Set environment variables
+      --name string                    Assign a name to the container
+      --net string                     Network Mode
+      --pid string                     PID namespace to use
+      --privileged                     Give extended privileges to this container
+      --restart string                 Restart policy to apply when a container exits (default "no")
+  -v, --volume list                    Bind mount a volume
+      --volumes-from list              Mount volumes from the specified container(s)
+`)
+	}
+
+	image := args[0]
+	cmds := args[1:]
+
+	c, err := docker.NewVersionedClientFromEnv("1.18")
+	if err != nil {
+		return fmt.Errorf("unable to connect to docker: %s", err)
+	}
+
+	config := docker.Config{Image: image, Env: env, Cmd: cmds}
+	hostConfig := docker.HostConfig{NetworkMode: net, PidMode: pid, Privileged: privileged, RestartPolicy: restart, Binds: volumes, VolumesFrom: volumesFrom}
+	container, err := c.CreateContainer(docker.CreateContainerOptions{Name: name, Config: &config, HostConfig: &hostConfig})
+	if err != nil {
+		return fmt.Errorf("unable to create container: %s", err)
+	}
+
+	err = c.StartContainer(container.ID, &hostConfig)
+	if err != nil {
+		return fmt.Errorf("unable to start container: %s", err)
+	}
+
+	fmt.Print(container.ID)
+	return nil
+}
+
+func listContainers(args []string) error {
+	if len(args) > 1 {
+		cmdUsage("list-containers", "[<label>]")
+	}
+
+	c, err := docker.NewVersionedClientFromEnv("1.18")
+	if err != nil {
+		return fmt.Errorf("unable to connect to docker: %s", err)
+	}
+
+	opts := docker.ListContainersOptions{All: true}
+	if len(args) == 1 {
+		label := args[0]
+		opts.Filters = map[string][]string{"label": {label}}
+	}
+
+	containers, err := c.ListContainers(opts)
+	if err != nil {
+		return fmt.Errorf("unable to list containers: %s", err)
+	}
+
+	for _, container := range containers {
+		fmt.Println(container.ID)
+	}
+	return nil
+}
+
+func stopContainer(args []string) error {
+	if len(args) < 1 {
+		cmdUsage("stop-container", "<container-id> [<container-id2> ...]")
+	}
+
+	c, err := docker.NewVersionedClientFromEnv("1.18")
+	if err != nil {
+		return fmt.Errorf("unable to connect to docker: %s", err)
+	}
+
+	for _, containerID := range args {
+		err = c.StopContainer(containerID, 10)
+		if err != nil {
+			return fmt.Errorf("unable to stop container %s: %s", containerID, err)
+		}
+	}
+	return nil
+}
+
+func killContainer(args []string) error {
+	if len(args) < 1 {
+		cmdUsage("kill-container", "<container-id> [<container-id2> ...]")
+	}
+
+	c, err := docker.NewVersionedClientFromEnv("1.18")
+	if err != nil {
+		return fmt.Errorf("unable to connect to docker: %s", err)
+	}
+
+	for _, containerID := range args {
+		err = c.KillContainer(docker.KillContainerOptions{ID: containerID, Signal: docker.SIGKILL})
+		if err != nil {
+			return fmt.Errorf("unable to stop container %s: %s", containerID, err)
+		}
+	}
+	return nil
+}
+
+func removeContainer(args []string) error {
+	if len(args) < 1 {
+		cmdUsage("remove-container", "[-f | --force]  [-v | --volumes] <container-id> [<container-id2> ...]")
+	}
+
+	force := false
+	volumes := false
+	for i := 0; i < len(args); {
+		switch args[i] {
+		case "--force":
+		case "-f":
+			force = true
+			args = append(args[:i], args[i+1:]...)
+		case "--volumes":
+		case "-v":
+			volumes = true
+			args = append(args[:i], args[i+1:]...)
+		default:
+			i++
+		}
+	}
+
+	c, err := docker.NewVersionedClientFromEnv("1.18")
+	if err != nil {
+		return fmt.Errorf("unable to connect to docker: %s", err)
+	}
+
+	for _, containerID := range args {
+		err = c.RemoveContainer(docker.RemoveContainerOptions{
+			ID: containerID, Force: force, RemoveVolumes: volumes})
+		if err != nil {
+			return fmt.Errorf("unable to stop container %s: %s", containerID, err)
+		}
+	}
+	return nil
+}

prog/weaveutil/db_flag.go

@@ -0,0 +1,54 @@
+// Get/set arbitrary flags in the Weave Net persistence DB
+package main
+
+import (
+	"fmt"
+
+	"github.com/weaveworks/weave/db"
+)
+
+const dbFlagPrefix = "flag:"
+
+func getDBFlag(args []string) error {
+	if len(args) != 2 {
+		cmdUsage("check-db-flag", "<db-prefix> <flag-name>")
+	}
+	dbPrefix := args[0]
+	flagName := args[1]
+
+	d, err := db.NewBoltDBReadOnly(dbPrefix)
+	if err != nil {
+		return err
+	}
+	defer d.Close()
+	var value string
+	nameFound, err := d.Load(dbFlagPrefix+flagName, &value)
+	if err != nil {
+		return err
+	}
+	if !nameFound {
+		return fmt.Errorf("")
+	}
+	fmt.Print(value)
+	return nil
+}
+
+func setDBFlag(args []string) error {
+	if len(args) != 3 {
+		cmdUsage("set-db-flag", "<db-prefix> <flag-name> <flag-value>")
+	}
+	dbPrefix := args[0]
+	flagName := args[1]
+	value := args[2]
+
+	d, err := db.NewBoltDB(dbPrefix)
+	if err != nil {
+		return err
+	}
+	defer d.Close()
+	err = d.Save(dbFlagPrefix+flagName, value)
+	if err != nil {
+		return err
+	}
+	return nil
+}

prog/weaveutil/expose.go

@@ -1,23 +0,0 @@
-package main
-
-import (
-	weavenet "github.com/weaveworks/weave/net"
-)
-
-func exposeNAT(args []string) error {
-	if len(args) < 1 {
-		cmdUsage("expose-nat", "<cidr>...")
-	}
-
-	cidrs, err := parseCIDRs(args)
-	if err != nil {
-		return err
-	}
-
-	for _, cidr := range cidrs {
-		if err := weavenet.ExposeNAT(*cidr); err != nil {
-			return err
-		}
-	}
-	return nil
-}

prog/weaveutil/main.go

@@ -26,6 +26,15 @@ func init() {
 		"remove-plugin-network":    removePluginNetwork,
 		"container-addrs":          containerAddrs,
 		"process-addrs":            processAddrs,
+		"container-id":             containerID,
+		"container-state":          containerState,
+		"container-fqdn":           containerFQDN,
+		"run-container":            runContainer,
+		"list-containers":          listContainers,
+		"stop-container":           stopContainer,
+		"kill-container":           killContainer,
+		"remove-container":         removeContainer,
+		"create-volume-container":  createVolumeContainer,
 		"attach-container":         attach,
 		"detach-container":         detach,
 		"configure-arp":            configureARP,
@@ -36,12 +45,13 @@ func init() {
 		"list-netdevs":             listNetDevs,
 		"cni-net":                  cniNet,
 		"cni-ipam":                 cniIPAM,
-		"expose-nat":               exposeNAT,
 		"bridge-ip":                bridgeIP,
 		"unique-id":                uniqueID,
 		"swarm-manager-peers":      swarmManagerPeers,
 		"is-docker-plugin-enabled": isDockerPluginEnabled,
 		"rewrite-etc-hosts":        rewriteEtcHosts,
+		"get-db-flag":              getDBFlag,
+		"set-db-flag":              setDBFlag,
 	}
 }
 
@@ -75,7 +85,9 @@ func main() {
 		os.Exit(1)
 	}
 	if err := cmd(os.Args[2:]); err != nil {
-		fmt.Fprintln(os.Stderr, err)
+		if err.Error() != "" {
+			fmt.Fprintln(os.Stderr, err)
+		}
 		os.Exit(1)
 	}
 }

prog/weaveutil/volume.go

@@ -0,0 +1,42 @@
+package main
+
+import (
+	"fmt"
+
+	docker "github.com/fsouza/go-dockerclient"
+)
+
+func createVolumeContainer(args []string) error {
+	if len(args) < 3 {
+		cmdUsage("create-volume-container", "<name> <image-name-or-id> <label> [<bind-mount1> <bind-mount2> ...]")
+	}
+	containerName := args[0]
+	image := args[1]
+	label := args[2]
+	bindMounts := args[3:]
+
+	c, err := docker.NewVersionedClientFromEnv("1.18")
+	if err != nil {
+		return fmt.Errorf("unable to connect to docker: %s", err)
+	}
+
+	container, err := c.InspectContainer(containerName)
+	if err == nil {
+		fmt.Println("volume container already exists " + containerName + ":" + container.ID)
+		return nil
+	}
+
+	volumes := make(map[string]struct{})
+	for _, m := range bindMounts {
+		volumes[m] = struct{}{}
+	}
+	labels := map[string]string{label: ""}
+	config := docker.Config{Image: image, Volumes: volumes, Labels: labels, Entrypoint: []string{"data-only"}}
+	hostConfig := docker.HostConfig{}
+	_, err = c.CreateContainer(docker.CreateContainerOptions{Name: containerName, Config: &config, HostConfig: &hostConfig})
+	if err != nil {
+		return fmt.Errorf("unable to create container: %s", err)
+	}
+
+	return nil
+}

proxy/hosts.go

@@ -20,8 +20,7 @@ func (proxy *Proxy) RewriteEtcHosts(hostsPath, fqdn string, ips []*net.IPNet, ex
 	contents := buf.String()
 	cmdLine := fmt.Sprintf("echo '%s' > %s && rm -f %s && echo '%s' > %s", contents, mntHosts, mntHosts, contents, mntHosts)
 	mounts := []string{hostsPathDir + ":" + mnt}
-	proxy.runTransientContainer([]string{"sh"}, []string{"-c", cmdLine}, mounts)
-	return nil
+	return proxy.runTransientContainer([]string{"sh"}, []string{"-c", cmdLine}, mounts)
 }
 
 // we assume (for compatibility with the weave script) that fqdn has a dot

proxy/proxy.go

@@ -60,7 +60,6 @@ type Config struct {
 	NoDefaultIPAM       bool
 	NoRewriteHosts      bool
 	TLSConfig           TLSConfig
-	Version             string
 	WithoutDNS          bool
 	NoMulticastRoute    bool
 	KeepTXOn            bool
@@ -330,11 +329,7 @@ func copyOwnerAndPermissions(from, to string) error {
 		return nil
 	}
 
-	if err = os.Chown(to, int(moreStat.Uid), int(moreStat.Gid)); err != nil {
-		return err
-	}
-
-	return nil
+	return os.Chown(to, int(moreStat.Uid), int(moreStat.Gid))
 }
 
 func (proxy *Proxy) listen(protoAndAddr string) (net.Listener, string, error) {
@@ -510,7 +505,10 @@ func (proxy *Proxy) attach(containerID string) error {
 		if container.HostConfig != nil {
 			extraHosts = container.HostConfig.ExtraHosts
 		}
-		proxy.RewriteEtcHosts(container.HostsPath, fqdn, ips, extraHosts)
+		if err := proxy.RewriteEtcHosts(container.HostsPath, fqdn, ips, extraHosts); err != nil {
+			Log.Errorf("Error from rewriting container's /etc/hosts: %s", err)
+			// carry on since the original /etc/hosts is good enough in most cases
+		}
 	}
 
 	pid := container.State.Pid
@@ -684,6 +682,7 @@ func (proxy *Proxy) runTransientContainer(entrypoint, cmd, binds []string) (err
 		env = append(env, fmt.Sprintf("%s=%s", "WEAVE_DEBUG", val))
 	}
 
+	Log.Debugf("Running image %q; entrypoint=%q; cmd=%q; binds=%q", proxy.Image, entrypoint, fmt.Sprintf("%.72s", cmd), binds)
 	var container *docker.Container
 	container, err = proxy.client.CreateContainer(docker.CreateContainerOptions{
 		Config: &docker.Config{

router/http.go

@@ -5,7 +5,10 @@ import (
 	"net/http"
 
 	"github.com/gorilla/mux"
+
 	"github.com/weaveworks/weave/common"
+	"github.com/weaveworks/weave/net"
+	"github.com/weaveworks/weave/net/address"
 )
 
 func (router *NetworkRouter) HandleHTTP(muxRouter *mux.Router) {
@@ -26,4 +29,20 @@ func (router *NetworkRouter) HandleHTTP(muxRouter *mux.Router) {
 		router.ForgetConnections(r.Form["peer"])
 	})
 
+	muxRouter.Methods("POST").Path("/expose/{ip}/{prefixlen}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		vars := mux.Vars(r)
+		cidr, err := address.ParseCIDR(vars["ip"] + "/" + vars["prefixlen"])
+		if err != nil {
+			http.Error(w, fmt.Sprint("unable to parse ip addr: ", err.Error()), http.StatusBadRequest)
+			return
+		}
+
+		if err = net.Expose(router.BridgeConfig.WeaveBridgeName, cidr.IPNet(), router.BridgeConfig.AWSVPC, router.BridgeConfig.NPC); err != nil {
+			http.Error(w, fmt.Sprint("unable to expose: ", err.Error()), http.StatusInternalServerError)
+			return
+		}
+
+		w.WriteHeader(204)
+	})
+
 }

router/network_router.go

@@ -11,6 +11,7 @@ import (
 
 	"github.com/weaveworks/weave/common"
 	"github.com/weaveworks/weave/db"
+	weavenet "github.com/weaveworks/weave/net"
 )
 
 const (
@@ -47,11 +48,12 @@ type PacketLogging interface {
 type NetworkRouter struct {
 	*mesh.Router
 	NetworkConfig
+	weavenet.BridgeConfig
 	Macs *MacCache
 	db   db.DB
 }
 
-func NewNetworkRouter(config mesh.Config, networkConfig NetworkConfig, name mesh.PeerName, nickName string, overlay NetworkOverlay, db db.DB) (*NetworkRouter, error) {
+func NewNetworkRouter(config mesh.Config, networkConfig NetworkConfig, bridgeConfig weavenet.BridgeConfig, name mesh.PeerName, nickName string, overlay NetworkOverlay, db db.DB) (*NetworkRouter, error) {
 	if overlay == nil {
 		overlay = NullNetworkOverlay{}
 	}
@@ -63,7 +65,7 @@ func NewNetworkRouter(config mesh.Config, networkConfig NetworkConfig, name mesh
 	if err != nil {
 		return nil, err
 	}
-	router := &NetworkRouter{Router: meshRouter, NetworkConfig: networkConfig, db: db}
+	router := &NetworkRouter{Router: meshRouter, NetworkConfig: networkConfig, BridgeConfig: bridgeConfig, db: db}
 	router.Peers.OnInvalidateShortIDs(overlay.InvalidateShortIDs)
 	router.Routes.OnChange(overlay.InvalidateRoutes)
 	router.Macs = NewMacCache(macMaxAge,

router/sleeve.go

@@ -493,7 +493,7 @@ func (f curriedForward) Process(frame []byte, dec *EthernetDecoder, broadcast bo
 		// non-broadcast frames can be broadcast, if the
 		// destination MAC was not in our MAC cache.
 		if broadcast {
-			log.Print(fwd.logPrefix(), "dropping too big DF broadcast frame (", dec.IP.SrcIP, " -> ", dec.IP.DstIP, "): MTU=", mtu)
+			log.Print(fwd.logPrefix(), "dropping too big DF broadcast frame: len ", len(frame), " (", dec.IP.SrcIP, " -> ", dec.IP.DstIP, "): MTU=", mtu)
 			return
 		}
 

site/building.md

@@ -1,6 +1,6 @@
 ---
 title: Building Weave Net
-menu_order: 120
+menu_order: 220
 search_type: Documentation
 ---
 You only need to build Weave Net if you want to work on the Weave Net codebase

site/concepts.md

@@ -0,0 +1,23 @@
+---
+title: Concepts in Weave Net
+menu_order: 20
+search_type: Documentation
+---
+
+The section contains topics about how Weave Net works as well as providing a technical deep dive on how Weave Net implements fast datapath, and encryption.
+
+* [Understanding Weave Net](/site/concepts/how-it-works.md)
+* [Weave Net Router Sleeve Encapsulation](/site/concepts/router-encapsulation.md)
+* [How Weave Net Interprets Network Topology](/site/concepts/network-topology.md)
+     * [Communicating Topology Among Peers](https://weave.works/docs/net/latest/concepts/network-topology/#topology)
+     * [How Messages are Formed](https://weave.works/docs/net/latest/concepts/network-topology/#messages)
+     * [Removing Peers](https://weave.works/docs/net/latest/concepts/network-topology/#removing-peers)
+     * [What Happens When The Topology is Out of Date?](https://weave.works/docs/net/latest/concepts/network-topology/#out-of-date-topology)
+* [Fast Datapath & Weave Net](/site/concepts/fastdp-how-it-works.md)
+* [IP Addresses, Routes and Networks](/site/concepts/ip-addresses.md)
+* [Encryption and Weave Net](/site/concepts/encryption.md)
+     * [How Weave Net Implements Encryption](/site/concepts/encryption-implementation.md)
+          * [Establishing the Emphemeral Session Key](https://weave.works/docs/net/latest/concepts/network-topology/#ephemeral-key)
+          * [Key Generation and the Linux CSPRIN](https://weave.works/docs/net/latest/concepts/network-topology/#cspring)
+          * [Encrypting and Decrypting TCP Messages](https://weave.works/docs/net/latest/concepts/network-topology/#tcp)
+          * [Encrypting and Decrypting UDP Messages](https://weave.works/docs/net/latest/concepts/network-topology/#udp)

site/concepts/encryption-implementation.md

@@ -1,11 +1,11 @@
 ---
 title: How Weave Net Implements Encryption
-menu_order: 70
+menu_order: 60
 search_type: Documentation
 ---
 
 This section describes some details of Weave Net's built-in
-[encryption](/site/how-it-works/encryption.md):
+[encryption](/site/concepts/encryption.md):
 
  * [Establishing the Ephemeral Session Key](#ephemeral-key)
  * [Key Generation](#csprng)
@@ -148,7 +148,7 @@ follows:
     | Frame N: Captured payload         |  |
     +-----------------------------------+ -+
 
-This is very similar to the [non-crypto encapsulation](/site/how-it-works/router-encapsulation.md).
+This is very similar to the [non-crypto encapsulation](/site/concepts/router-encapsulation.md).
 
 All of the frames on a connection are encrypted with the same
 ephemeral session key, and a nonce constructed from a message sequence
@@ -211,4 +211,4 @@ Integrity Check Value of AES-GCM.
 
  * [architecture documentation](https://github.com/weaveworks/weave/blob/master/docs/architecture.txt)
  * [fastdp encryption](https://github.com/weaveworks/weave/blob/master/docs/fastdp-crypto.md)
- * [Securing Containers Across Untrusted Networks](/site/using-weave/security-untrusted-networks.md)
+ * [Securing Containers Across Untrusted Networks](/site/tasks/manage/security-untrusted-networks.md)

site/concepts/encryption.md

@@ -1,14 +1,14 @@
 ---
 title: Encryption and Weave Net
-menu_order: 60
+menu_order: 50
 search_type: Documentation
 ---
 
 
 Weave Net peers
-[communicate with each other over both TCP and UDP](/site/how-it-works.md). This
+[communicate with each other over both TCP and UDP](/site/concepts/how-it-works.md). This
 communication
-[can be encrypted](/site/using-weave/security-untrusted-networks.md).
+[can be encrypted](/site/tasks/manage/security-untrusted-networks.md).
 
 Encryption of control plane traffic (TCP) and data plane traffic (UDP) of sleeve
 overlay is accomplished using the [NaCl](http://nacl.cr.yp.to/)
@@ -41,5 +41,5 @@ the IP transformation framework (XFRM).
 
 **See Also**
 
- * [How Weave Implements Encryption](/site/how-it-works/encryption-implementation.md)
- * [Securing Containers Across Untrusted Networks](/site/using-weave/security-untrusted-networks.md)
+ * [How Weave Implements Encryption](/site/concepts/encryption-implementation.md)
+ * [Securing Containers Across Untrusted Networks](/site/tasks/manage/security-untrusted-networks.md)

site/concepts/fastdp-how-it-works.md

@@ -1,5 +1,5 @@
 ---
-title: How Fast Datapath Works
+title: Fast Datapath & Weave Net
 menu_order: 30
 search_type: Documentation
 ---
@@ -25,7 +25,7 @@ Prior to version 1.2, Weave Net used a custom encapsulation format. Fast datapat
 
 **See Also**
 
- * [Using Weave Net](/site/using-weave.md)
- * [Using Fast Datapath](/site/using-weave/fastdp.md)
+ * [Launching Weave Net](/site/install/using-weave.md)
+ * [Using Fast Datapath](/site/tasks/manage/fastdp.md)
 
 

site/concepts/how-it-works.md

@@ -1,11 +1,9 @@
 ---
-title: Understanding how Weave Net Works
-menu_order: 90
+title: Understanding Weave Net
+menu_order: 10
 search_type: Documentation
 ---
 
-
-
 A Weave network consists of a number of 'peers' - Weave Net routers
 residing on different hosts. Each peer has a name, which tends to
 remain the same over restarts, a human friendly nickname for use in
@@ -15,7 +13,7 @@ the router is concerned, although the name defaults to a MAC address.
 
 Weave Net routers establish TCP connections with each other, over which they
 perform a protocol handshake and subsequently exchange
-[topology](/site/router-topology/network-topology.md) information. 
+[topology](/site/concepts/network-topology.md) information. 
 These connections are encrypted if
 so configured. Peers also establish UDP "connections", possibly
 encrypted, which carry encapsulated network packets. These
@@ -27,7 +25,7 @@ is given an IP address and netmask supplied either by the user or
 by Weave Net's IP address allocator.
 
 Weave Net routes packets between containers on different hosts via two
-methods: a [fast data path](/site/using-weave/fastdp.md) method, which operates
+methods: a [fast data path](/site/tasks/manage/fastdp.md) method, which operates
 entirely in kernel space, and a fallback `sleeve` method, in which
 packets destined for non-local containers are captured by the kernel
 and processed by the Weave Net router in user space, forwarded over
@@ -47,5 +45,5 @@ first send it to peer 3:
 
 **See Also**
 
- * [Weave Router Sleeve Encapsulation](/site/how-it-works/router-encapsulation.md)
- * [How Weave Interprets Network Topology](/site/how-it-works/network-topology.md)
+ * [Weave Router Sleeve Encapsulation](/site/concepts/router-encapsulation.md)
+ * [How Weave Interprets Network Topology](/site/concepts/network-topology.md)

site/concepts/ip-addresses.md

@@ -1,6 +1,6 @@
 ---
 title: IP Addresses, Routes and Networks
-menu_order: 50
+menu_order: 40
 search_type: Documentation
 ---
 
@@ -8,7 +8,7 @@ search_type: Documentation
 Weave Net runs containers on a private network, which means that IP addresses are isolated from the rest of the
 Internet, and that you don't have to worry about addresses clashing. 
 
-You can of course also manually change the IP of any given container or subnet on a Weave network.  See, [How to Manually Specify IP Addresses and Subnets](/site/using-weave/manual-ip-address.md)
+You can of course also manually change the IP of any given container or subnet on a Weave network.  See, [How to Manually Specify IP Addresses and Subnets](/site/tasks/ipam/manual-ip-address.md)
 
 ### Some Definitions
 
@@ -32,7 +32,7 @@ You can of course also manually change the IP of any given container or subnet o
 ### IP Address Notation in Weave
 
 In the IP address `10.4.2.6/8`, the network prefix is the first 8 bits
-- `10`. Written out in full, that network is `10.0.0.0/8`.
+\- `10`. Written out in full, that network is `10.0.0.0/8`.
 
 The most common prefix lengths are 8, 16 and 24, but there is nothing
 stopping you using a /9 network or a /26. For example, `6.250.3.1/9` is on the
@@ -64,4 +64,4 @@ find a match, so uses the default rule.
 
 **See Also**
 
- * [Allocating IPs in a Specific Range](/site/using-weave/configuring-weave.md)
+ * [Allocating IPs in a Specific Range](/site/tasks/ipam/configuring-weave.md)

site/concepts/network-topology.md

@@ -145,7 +145,7 @@ which time the topology should have updated.
 
 **See Also**
 
- * [Weave Net Router Encapsulation](/site/how-it-works/router-encapsulation.md)
+ * [Weave Net Router Encapsulation](/site/concepts/router-encapsulation.md)
  
  
  

site/concepts/router-encapsulation.md

@@ -1,11 +1,11 @@
 ---
 title: Weave Net Router Sleeve Encapsulation
-menu_order: 10
+menu_order: 15
 search_type: Documentation
 ---
 
 When the Weave Net router forwards packets to peers in `sleeve` mode
-(rather than via the [fast data path](/site/using-weave/fastdp.md)), the
+(rather than via the [fast data path](/site/tasks/manage/fastdp.md)), the
 encapsulation looks something like this:
 
     +-----------------------------------+
@@ -62,4 +62,4 @@ MAC discovery.
 
 **See Also**
 
- * [How Weave Net Interprets Network Topology](/site/how-it-works/network-topology.md)
+ * [How Weave Net Interprets Network Topology](/site/concepts/network-topology.md)

site/faq.md

@@ -19,13 +19,13 @@ See [Troubleshooting Weave - List attached containers](/site/troubleshooting.md#
 
 You can manually change the IP of a container using [Classless Inter-Domain Routing or CIDR notation](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing). 
 
-For more information, refer to [Manually Specifying the IP Address of a Container](/site/using-weave/manual-ip-address.md). 
+For more information, refer to [Manually Specifying the IP Address of a Container](/site/tasks/ipam/manual-ip-address.md). 
 
 
 <a name="expose-container"></a>
 **Q: How do I expose one of my containers to the outside world?**
 
-Exposing a container to the outside world is described in [Exporting Services](/site/using-weave/service-management.md#exporting).
+Exposing a container to the outside world is described in [Exporting Services](/site/tasks/manage/service-management.md#exporting).
 
 
 <a name="legacy-network"></a>
@@ -35,9 +35,9 @@ Yes you can.
 
 For example, you have a Weave network that runs on hosts A, B, C. and you have an additional host, that we'll call P, where neither Weave nor Docker are running.  However, you need to connect from a process running on host P to a specific container running on host B on the Weave network.  Since the Weave network is completely separate from any network that P is connected to, you cannot connect the container using the container's IP address. 
 
-A simple way to accomplish this would be to run Weave on the host and then run, `weave expose` to expose the network to any running containers. Or you set up a route from P to one of A, B or C. See [Integrating a Network Host](/site/using-weave/host-network-integration.md).
+A simple way to accomplish this would be to run Weave on the host and then run, `weave expose` to expose the network to any running containers. Or you set up a route from P to one of A, B or C. See [Integrating a Network Host](/site/tasks/manage/host-network-integration.md).
 
-Yet another option is to expose a port from the container on host B and then connect to it. You can read about exposing ports in [Exporting Services](/site/using-weave/service-management.md#exporting).
+Yet another option is to expose a port from the container on host B and then connect to it. You can read about exposing ports in [Exporting Services](/site/tasks/manage/service-management.md#exporting).
 
 
 <a name="duplicate-ip"></a>
@@ -45,7 +45,7 @@ Yet another option is to expose a port from the container on host B and then con
 
 Under normal circumstances, this should never happen, but it can occur if `weave rmpeer` was run on more than one host. 
 
-For more information see [Starting, Stopping and Removing Peers](/site/ipam/stop-remove-peers-ipam.md).
+For more information see [Starting, Stopping and Removing Peers](/site/tasks/ipam/stop-remove-peers-ipam.md).
 
 
 <a name="dead-node"></a>
@@ -53,7 +53,7 @@ For more information see [Starting, Stopping and Removing Peers](/site/ipam/stop
 
 When a node goes out of service, the best option is to call `weave rmpeer` on one host and then `weave forget` on all the other hosts.
 
-See [Starting, Stopping and Removing Peers](/site/ipam/stop-remove-peers-ipam.md) for an in-depth discussion.
+See [Starting, Stopping and Removing Peers](/site/tasks/ipam/stop-remove-peers-ipam.md) for an in-depth discussion.
 
 
 <a name="performance"></a>
@@ -63,7 +63,7 @@ All virtualization techniques have some overhead, and Weave's overhead is typica
 
 Weave Net also automatically uses the fastest datapath between two hosts. When Weave Net can't use the fast datapath between two hosts, it falls back to the slower packet forwarding approach. Selecting the fastest forwarding approach is automatic, and is determined on a connection-by-connection basis. For example, a Weave network spanning two data centers might use fast datapath within the data centers, but not for the more constrained network link between them.
 
-For more information about fast datapath see [How Fast Datapath Works](/site/how-it-works/fastdp-how-it-works.md).
+For more information about fast datapath see [How Fast Datapath Works](/site/concepts/fastdp-how-it-works.md).
 
 
 <a name="query-fastdp"></a>
@@ -71,7 +71,7 @@ For more information about fast datapath see [How Fast Datapath Works](/site/how
 
 To view whether Weave is using fastdp or not, you can run, `weave status connections`
 
-For more information on this command, see [Using Fast Datapath](/site/using-weave/fastdp.md).
+For more information on this command, see [Using Fast Datapath](/site/tasks/manage/fastdp.md).
 
 
 <a name="encrypted-fastdp"></a>
@@ -79,12 +79,12 @@ For more information on this command, see [Using Fast Datapath](/site/using-weav
 
 Yes, 1.9 version of Weave Net added the encryption feature to fastdp.
 
-See [Using Fast Datapath](/site/using-weave/fastdp.md) for more information.
+See [Using Fast Datapath](/site/tasks/manage/fastdp.md) for more information.
 
 <a name="app-isolation"></a>
 **Q: Can I create multiple networks where containers can communicate on one network, but are isolated from containers on other networks?**
 
-Yes, of course!  Weave allows you to run isolated networks and still allow open communications between individual containers from those isolated networks. You can find information on how to do this in [Application Isolation](/site/using-weave/application-isolation.md).
+Yes, of course!  Weave allows you to run isolated networks and still allow open communications between individual containers from those isolated networks. You can find information on how to do this in [Application Isolation](/site/tasks/manage/application-isolation.md).
 
 
 **<a name="ports"></a>Q: Which ports does Weave Net use (e.g. if I am configuring a firewall) ?**
@@ -92,7 +92,7 @@ Yes, of course!  Weave allows you to run isolated networks and still allow open
 You must permit traffic to flow through TCP 6783 and UDP 6783/6784,
 which are Weave’s control and data ports.
 
-The daemon also uses TCP port 6782 for [metrics](/site/metrics.md), but
+The daemon also uses TCP port 6782 for [metrics](/site/tasks/manage/metrics.md), but
 you would only need to open up this port if you wish to collect metrics
 from another host.
 
@@ -116,6 +116,6 @@ Ubuntu image with those two commands added.
 **See Also**
 
  * [Troubleshooting Weave](/site/troubleshooting.md)
- * [Troubleshooting IPAM](/site/ipam.md)
- * [Troubleshooting the Proxy](/site/weave-docker-api/using-proxy.md)
+ * [Troubleshooting IPAM](/site/tasks/ipam/ipam.md)
+ * [Troubleshooting the Proxy](/site/tasks/weave-docker-api/using-proxy.md)
  

site/install.md

@@ -0,0 +1,42 @@
+---
+title: Installing Weave Net
+menu_order: 30
+search_type: Documentation
+---
+
+Install Weave Net from the command line on its own or if you are using Docker, Kubernetes or Mesosphere as a Docker or a CNI plugin. Weave Net can also be launched as a cloud agent within Weave Cloud to visualize and troubleshoot peer connections.
+
+
+## Installing Weave Net Stand-alone and in Weave Cloud
+
+* [Installing Weave Net](https://www.weave.works/docs/net/latest/install/installing-weave/)
+* [Launching Weave Net](https://www.weave.works/docs/net/latest/install/using-weave/)
+* [Visualizing Weave Net in Weave Cloud](https://www.weave.works/docs/net/latest/install/weave-cloud/)
+* [Using Weave with Systemd](https://www.weave.works/docs/net/latest/install/systemd/)
+
+
+## Weave Net Docker Plugin
+
+* [How the Weave Net Docker Network Plugins Work](https://www.weave.works/docs/net/latestplugin/plugin-how-it-works/)
+* [Integrating Docker via the Network Plugin (V2)](https://www.weave.works/docs/net/latest/install/plugin/plugin-v2/)
+     * [Installation](https://www.weave.works/docs/net/latest/install/plugin/plugin-v2/#installation)
+     * [Configuring the Plugin](https://www.weave.works/docs/net/latest/install/plugin/plugin-v2/#configuring)
+     * [Running Services or Containers Using the Plugin](https://www.weave.works/docs/net/latest/install/plugin/plugin-v2/#usage)
+* [Integrating Docker via the Network Plugin (Legacy)](https://www.weave.works/docs/net/latest/install/plugin/plugin/)
+     * [Launching Weave Net and Running Containers Using the Plugin](https://www.weave.works/docs/net/latest/install/plugin/plugin/#launching)
+     * [Restarting the Plugin](https://www.weave.works/docs/net/latest/install/plugin/plugin/#restarting)
+
+## Weave Net CNI plugin for Kubernetes and Mesosphere
+
+* [Integrating Kubernetes and Mesos via the CNI Plugin](https://www.weave.works/docs/net/latest/kubernetes/)
+* [Integrating Kubernetes via the Addon](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/)
+     * [Installation](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#install)
+     * [Upgrading Kubernetes to version 1.6](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#kube-1.6-upgrade)
+     * [Upgrading the Daemon Sets](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#daemon-sets)
+     * [CPU and Memory Requirements](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#resources)
+     * [Pod Eviction](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#eviction)
+     * [Network Policy Controller](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#npc)
+     * [Troubleshooting Blocked Connections](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#blocked-connections)
+     * [Changing Configuration Options](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#configuration-options)
+
+

site/install/installing-weave.md

@@ -1,6 +1,6 @@
 ---
 title: Installing Weave Net
-menu_order: 30
+menu_order: 10
 search_type: Documentation
 ---
 
@@ -17,9 +17,9 @@ If you are on OSX and you are using Docker Machine ensure that a VM is running a
 before downloading Weave Net. To set up a VM see [the Docker Machine
 documentation](https://docs.docker.com/installation/mac/#from-your-shell) or refer to ["Part 1: Launching Weave Net with Docker Machine"](/guides/part-1-launching-weave-net-with-docker-machine/).
 
-After your VM is setup with Docker Machine, Weave Net can be launched directly from the OSX host. Weave Net respects the environment variable `DOCKER_HOST`, so that you can run and control a Weave Network locally on a remote host. See [Using The Weave Docker API Proxy](/site/weave-docker-api/using-proxy.md).
+After your VM is setup with Docker Machine, Weave Net can be launched directly from the OSX host. Weave Net respects the environment variable `DOCKER_HOST`, so that you can run and control a Weave Network locally on a remote host. See [Using The Weave Docker API Proxy](/site/tasks/weave-docker-api/using-proxy.md).
 
-With Weave Net downloaded onto your VMs or hosts, you are ready to launch a Weave network and deploy apps onto it. See [Using Weave Net](/site/using-weave.md).
+With Weave Net downloaded onto your VMs or hosts, you are ready to launch a Weave network and deploy apps onto it. See [Launching Weave Net](/site/install/using-weave.md).
 
 ### Quick Start Screencast
 
@@ -32,7 +32,7 @@ With Weave Net downloaded onto your VMs or hosts, you are ready to launch a Weav
 Weave Net [periodically contacts Weaveworks servers for available
 versions](https://github.com/weaveworks/go-checkpoint).  New versions
 are announced in the log and in [the status
-summary](/site/troubleshooting.md#weave-status).
+summary](/site/troubleshooting.md/#weave-status).
 
 The information sent in this check is:
 
@@ -55,19 +55,19 @@ for the latest Weave AMIs and [here](https://www.weave.works/docs/tutorials/old-
 
 If you're on Amazon EC2, the standard installation instructions at the
 top of this page, provide the simplest setup and the most flexibility.
-A [special no-overlay mode for EC2](/site/using-weave/awsvpc.md) can
+A [special no-overlay mode for EC2](/site/tasks/manage/awsvpc.md) can
 optionally be enabled, which allows containers to communicate at the
 full speed of the underlying network.
 
 To make encryption in fast datapath work on Google Cloud Platform, see
-[here](/site/faq.md#ports).
+[here](/site/faq.md/#ports).
 
 **See Also** 
 
- * [Using Weave Net](/site/using-weave.md)
- * [Getting Started Guides](http://www.weave.works/guides/)
- * [Features](/site/features.md)
+ * [Launching Weave Net](/site/install/using-weave.md)
+ * [Tutorials](https://www.weave.works/docs/tutorials/)
+ * [Features](/site/overview/features.md)
  * [Troubleshooting](/site/troubleshooting.md)
  * [Building](/site/building.md)
- * [Using Weave with Systemd](/site/installing-weave/systemd.md)
+ * [Using Weave with Systemd](/site/install/systemd.md)
  

site/install/plugin.md

@@ -0,0 +1,6 @@
+---
+title: Weave Net Docker Plugin
+menu_order: 50
+search_type: Documentation
+---
+

site/install/plugin/plugin-how-it-works.md

@@ -1,6 +1,6 @@
 ---
 title: How the Weave Net Docker Network Plugins Work
-menu_order: 20
+menu_order: 10
 search_type: Documentation
 ---
 
@@ -8,10 +8,10 @@ search_type: Documentation
 The Weave Net legacy plugin actually provides *two* network drivers to Docker -
 one named `weavemesh` that can operate without a cluster store and another one
 named `weave` that can only work with one (like Docker's overlay driver), while
-the V2 plugin provides *one* - `store/weaveworks/net-plugin:latest_release`
+the V2 plugin provides *one* - `weaveworks/net-plugin:latest_release`
 operating only in swarm mode.
 
-### `weavemesh` driver
+### weavemesh driver
 
 * Weave Net handles all co-ordination between hosts (referred to by Docker as a "local scope" driver)
 * Supports a single network only. A network named `weave` is automatically created for you.
@@ -19,7 +19,7 @@ operating only in swarm mode.
 
 If you do create additional networks using the `weavemesh` driver, containers attached to them will be able to communicate with containers attached to `weave`. There is no isolation between those networks.
 
-### `weave` driver
+### weave driver
 
 * This runs in what Docker calls "global scope", which requires an external cluster store
 * Supports multiple networks that must be created using `docker network create --driver weave ...`
@@ -41,7 +41,7 @@ The plugin accepts the following options via `docker network create ... --opt`:
    network created when the plugin is first launched has the multicast
    option turned on, but for any networks you create it defaults to off.
 
-### `store/weaveworks/net-plugin:latest_release` driver
+### weaveworks/net-plugin:latest_release driver
 
 * The driver runs within the plugin V2.
 * Requires Docker to run in [swarm mode](https://docs.docker.com/engine/swarm/swarm-mode/).
@@ -51,6 +51,6 @@ The plugin accepts the following options via `docker network create ... --opt`:
 
 **See Also**
 
- * [Integrating Docker via the Network Plugin (Legacy)](/site/plugin.md)
- * [Integrating Docker via the Network Plugin (V2)](/site/plugin-v2.md)
+ * [Integrating Docker via the Network Plugin (Legacy)](/site/install/plugin/plugin.md)
+ * [Integrating Docker via the Network Plugin (V2)](/site/install/plugin/plugin-v2.md)
 

site/install/plugin/plugin-v2.md

@@ -1,6 +1,6 @@
 ---
 title: Integrating Docker via the Network Plugin (V2)
-menu_order: 65
+menu_order: 20
 search_type: Documentation
 ---
 
@@ -9,7 +9,7 @@ search_type: Documentation
  * [Running Services or Containers Using the Plugin](#usage)
 
 Docker Engine version 1.12 introduced [a new plugin system (V2)](https://docs.docker.com/engine/extend/).
-This document describes how to use [the Network Plugin V2 of Weave Net](https://store.docker.com/plugins/weave-net-plugin).
+This document describes how to use the Network Plugin V2 of Weave Net.
 
 Before using the plugin, please keep in mind the plugin works only in Swarm
 mode and requires Docker version 1.13 or later.
@@ -19,7 +19,7 @@ mode and requires Docker version 1.13 or later.
 To install the plugin run the following command on _each_ host already
 participating in a Swarm cluster, i.e. on all master and worker nodes:
 
-    $ docker plugin install store/weaveworks/net-plugin:latest_release
+    $ docker plugin install weaveworks/net-plugin:latest_release
 
 Docker will pull the plugin from Docker Store, and it will ask to grant
 privileges before installing the plugin. Afterwards, it will start `weaver`
@@ -29,42 +29,42 @@ process which will try to connect to Swarm masters running Weave Net.
 
 There are several configuration parameters which can be set with:
 
-    $ docker plugin set store/weaveworks/net-plugin:latest_release PARAM=VALUE
+    $ docker plugin set weaveworks/net-plugin:latest_release PARAM=VALUE
 
 The parameters include:
 
 * `WEAVE_PASSWORD` - if non empty, it will instruct Weave Net to encrypt
-   traffic - see [here](/site/using-weave/security-untrusted-networks.md) for
+   traffic - see [here](/site/tasks/manage/security-untrusted-networks.md) for
    more details.
 * `WEAVE_MULTICAST` - set to 1 on each host running the plugin to enable
   multicast traffic on any Weave Net network.
 * `WEAVE_MTU` - Weave Net defaults to 1376 bytes, but you can set a
   smaller size if your underlying network has a tighter limit, or set
   a larger size for better performance if your network supports jumbo
-  frames - see [here](/site/using-weave/fastdp.md#mtu) for more
+  frames - see [here](/site/tasks/manage/fastdp.md#mtu) for more
   details.
 * `IPALLOC_RANGE` - the range of IP addresses used by Weave Net and the subnet
   they are placed in (CIDR format; default 10.32.0.0/12).
-  See [here](/site/using-weave/configuring-weave.md) for more details.
+  See [here](/site/tasks/ipam/configuring-weave.md) for more details.
 
 Before setting any parameter, the plugin has to be disabled with:
 
-    $ docker plugin disable store/weaveworks/net-plugin:latest_release
+    $ docker plugin disable weaveworks/net-plugin:latest_release
 
 To re-enable the plugin run the following command:
 
-    $ docker plugin enable store/weaveworks/net-plugin:latest_release
+    $ docker plugin enable weaveworks/net-plugin:latest_release
 
 ### <a name="usage"></a>Running Services or Containers Using the Plugin
 
 After you have launched the plugin, you can create a network for Docker Swarm
 services by running the following command on any Docker Swarm master node:
 
-    $ docker network create --driver=store/weaveworks/net-plugin:latest_release mynetwork
+    $ docker network create --driver=weaveworks/net-plugin:latest_release mynetwork
 
 Or you can create a network for any Docker container with:
 
-    $ docker network create --driver=store/weaveworks/net-plugin:latest_release --attachable mynetwork
+    $ docker network create --driver=weaveworks/net-plugin:latest_release --attachable mynetwork
 
 To start a service attached to the network run, for example:
 
@@ -76,5 +76,6 @@ Or to start a container:
 
 **See Also**
 
- * [Integrating Docker via the Network Plugin (Legacy)](/site/plugin.md)
- * [How the Weave Network Plugins Work](/site/plugin/plugin-how-it-works.md)
+ * [Integrating Docker via the Network Plugin (Legacy)](/site/install/plugin/plugin.md)
+ * [How the Weave Network Plugins Work](/site/install/plugin/plugin-how-it-works.md)
+ * [Troubleshooting the V2 plugin](/site/troubleshooting.md#v2plugin)

site/install/plugin/plugin.md

@@ -1,6 +1,6 @@
 ---
 title: Integrating Docker via the Network Plugin (Legacy)
-menu_order: 60
+menu_order: 30
 search_type: Documentation
 ---
 
@@ -14,9 +14,9 @@ Networking Plugin is fast and easy to use, and, unlike other
 networking plugins, does not require an external cluster store.
 
 To create a network which can span multiple Docker hosts, Weave Net peers must be connected to each other, by specifying the other hosts during `weave launch` or via
-[`weave connect`](/site/using-weave/finding-adding-hosts-dynamically.md).
+[`weave connect`](/site/tasks/manage/finding-adding-hosts-dynamically.md).
 
-See [Using Weave Net](/site/using-weave.md#peer-connections) for a discussion on peer connections. 
+See [Launching Weave Net](/site/install/using-weave.md#peer-connections) for a discussion on peer connections. 
 
 After you've launched Weave Net and peered your hosts,  you can start containers using the following, for example:
 
@@ -25,7 +25,7 @@ After you've launched Weave Net and peered your hosts,  you can start containers
 on any of the hosts, and they can all communicate with each other
 using any protocol, even multicast.
 
-In order to use Weave Net's [Service Discovery](/site/weavedns.md) you
+In order to use Weave Net's [Service Discovery](/site/tasks/weavedns/weavedns.md) you
 must pass the additional arguments `--dns` and `-dns-search`, for
 which a helper is provided in the Weave script:
 
@@ -63,7 +63,7 @@ this between hosts requires that you run Docker in ["swarm mode"](https://docs.d
 ["key-value store"](https://docs.docker.com/engine/userguide/networking/get-started-overlay/#/set-up-a-key-value-store).
 
 To operate in swarm mode, you are required to use the plugin v2 of Weave Net.
-See [Integrating Docker via the Network Plugin (V2)](/site/plugin-v2.md) for
+See [Integrating Docker via the Network Plugin (V2)](/site/install/plugin/plugin-v2.md) for
 more details.
 
 If your Docker installation has a key-value store, create a network
@@ -96,4 +96,4 @@ If you are using `systemd` with Docker 1.9, it is advised that you modify the Do
 
 **See Also**
 
- * [How the Weave Network Plugins Work](/site/plugin/plugin-how-it-works.md)
+ * [How the Weave Network Plugins Work](/site/install/plugin/plugin-how-it-works.md)

site/install/systemd.md

@@ -1,11 +1,11 @@
 ---
 title: Using Weave with Systemd
-menu_order: 130
+menu_order: 40
 search_type: Documentation
 ---
 
 
-Having installed Weave Net as per [Installing Weave](/site/installing-weave.md), you might find it convenient to configure the
+Having installed Weave Net as per [Installing Weave](/site/install/installing-weave.md), you might find it convenient to configure the
 init daemon to start Weave on boot. Most recent Linux distribution releases ship with [systemd](http://www.freedesktop.org/wiki/Software/systemd/). The information below provides you with some initial guidance on getting a Weave Net service configured on a systemd-based OS.
 
 ## Weave Net Service Unit and Configuration
@@ -32,11 +32,11 @@ create the `/etc/sysconfig/weave` environment file using the following format:
 
     PEERS="HOST1 HOST2 .. HOSTn"
 
-You can also use the [`weave connect`](/site/using-weave/finding-adding-hosts-dynamically.md) command to add participating hosts dynamically.
+You can also use the [`weave connect`](/site/tasks/manage/finding-adding-hosts-dynamically.md) command to add participating hosts dynamically.
 
-Additionally, if you want to enable [encryption](/site/using-weave/security-untrusted-networks.md) specify a
+Additionally, if you want to enable [encryption](/site/tasks/manage/security-untrusted-networks.md) specify a
 password using `WEAVE_PASSWORD="wfvAwt7sj"` in the `/etc/sysconfig/weave` environment file, and it will get picked up by
-Weave Net on launch. Recommendations for choosing a suitably strong password can be found [here](/site/using-weave/security-untrusted-networks.md).
+Weave Net on launch. Recommendations for choosing a suitably strong password can be found [here](/site/tasks/manage/security-untrusted-networks.md).
 
 You can now launch Weave Net using
 
@@ -64,7 +64,7 @@ context with the commands shown below. You will need to have the
 
 **See Also**
 
- * [Using Weave Net](/site/using-weave.md)
- * [Getting Started Guides](http://www.weave.works/guides/)
- * [Features](/site/features.md)
+ * [Launching Weave Net](/site/install/using-weave.md)
+ * [Tutorials](https://www.weave.works/docs/tutorials/)
+ * [Features](/site/overview/features.md)
  * [Troubleshooting](/site/troubleshooting.md)

site/install/using-weave.md

@@ -1,6 +1,6 @@
 ---
-title: Using Weave Net
-menu_order: 40
+title: Launching Weave Net
+menu_order: 20
 search_type: Documentation
 ---
 
@@ -13,7 +13,6 @@ This section contains the following topics:
  * [Testing Container Communications](#testing)
  * [Starting the Netcat Service](#start-netcat)
 
-
 ### <a name="launching"></a>Launching Weave Net
 
 Before launching Weave Net and deploying your apps, ensure that Docker is [installed](https://docs.docker.com/engine/installation/) on both hosts. 
@@ -82,7 +81,7 @@ For example:
 
     host2$ weave launch <ip address> <ip address> 
 
-Peers can also be dynamically added. See [Adding Hosts Dynamically](/site/using-weave/finding-adding-hosts-dynamically.md) for more information.
+Peers can also be dynamically added. See [Adding Hosts Dynamically](/site/tasks/manage/finding-adding-hosts-dynamically.md) for more information.
 
 #### Restricting Access
 
@@ -94,7 +93,7 @@ Standard firewall rules can be deployed to restrict access to the
 Weave Net control and data ports.
 
 For communication across untrusted networks, connections can be
-[encrypted](/site/using-weave/security-untrusted-networks.md).
+[encrypted](/site/tasks/manage/security-untrusted-networks.md).
 
 ### <a name="testing"></a>Testing Container Communications
 
@@ -135,8 +134,8 @@ Weave Net supports *any* protocol, and it doesn't have to be over TCP/IP. For ex
 
 **See Also** 
 
- * [Installing Weave Net](/site/installing-weave.md)
- * [Using Weave Cloud to get started with Weave Net](/site/using-weave/weave-cloud.md)
- * [Using Fastdp With Weave Net](/site/using-weave/fastdp.md)
- * [Integrating Docker via the Network Plugin (Legacy)](/site/plugin.md)
- * [Integrating Docker via the Network Plugin (V2)](/site/plugin-v2.md)
+ * [Installing Weave Net](/site/install/installing-weave.md)
+ * [Using Weave Cloud to get started with Weave Net](/site/install/weave-cloud.md)
+ * [Using Fastdp With Weave Net](/site/tasks/manage/fastdp.md)
+ * [Integrating Docker via the Network Plugin (Legacy)](/site/install/plugin/plugin.md)
+ * [Integrating Docker via the Network Plugin (V2)](/site/install/plugin/plugin-v2.md)

site/install/weave-cloud.md

@@ -1,6 +1,6 @@
 ---
-title: Using Weave Cloud to get started with Weave Net
-menu_order: 10
+title: Visualizing Weave Net in Weave Cloud
+menu_order: 30
 search_type: Documentation
 ---
 
@@ -86,7 +86,7 @@ first and automatically create a connection to it.
 
 >**Note:** If there is a firewall between `$HOST1` and `$HOST2`,  you must permit traffic to flow through TCP 6783 and UDP 6783/6784, which are Weave’s control and data ports.
 
-Now you can proceed to [test container communications](/site/using-weave#testing)
+Now you can proceed to [test container communications](/site/install/using-weave#testing)
 
 ### <a name="how-it-works"></a>How Peer Discovery Works
 
@@ -110,19 +110,19 @@ error message in its log and exit. To check the logs run:
     host1$ docker logs weave
 
 When a Weave Net instance is formed for the first time, the peers must
-initialize the [IP Allocation Manager](/site/ipam.md). This process,
+initialize the [IP Allocation Manager](/site/tasks/ipam/ipam.md). This process,
 and the configuration options available, are unchanged when using
 Weave Cloud.
 
 When you shut down a peer for good, use [the `weave reset`
-command](/site/ipam/stop-remove-peers-ipam.md) and it will be removed
+command](/site/tasks/ipam/stop-remove-peers-ipam.md) and it will be removed
 from the Weave Cloud database so future peers will not attempt to
 connect to it.
 
 **See Also**
 
- * [Installing Weave Net](/site/installing-weave.md)
- * [Using Weave Net](/site/using-weave.md)
- * [Using Fastdp With Weave Net](/site/using-weave/fastdp.md)
- * [Integrating Docker via the Network Plugin (Legacy)](/site/plugin.md)
- * [Integrating Docker via the Network Plugin (V2)](/site/plugin-v2.md)
+ * [Installing Weave Net](/site/install/installing-weave.md)
+ * [Launching Weave Net](/site/install/using-weave.md)
+ * [Using Fastdp With Weave Net](/site/tasks/manage/fastdp.md)
+ * [Integrating Docker via the Network Plugin (Legacy)](/site/install/plugin/plugin.md)
+ * [Integrating Docker via the Network Plugin (V2)](/site/install/plugin/plugin-v2.md)

site/kubernetes.md

@@ -1,11 +1,11 @@
 ---
 title: Integrating Kubernetes and Mesos via the CNI Plugin
-menu_order: 65
+menu_order: 40
 search_type: Documentation
 ---
 
 > The recommended way of using Weave with Kubernetes is via the new
-> [Kubernetes Addon](/site/kube-addon.md). The instructions below
+> [Kubernetes Addon](/site/kubernetes/kube-addon.md). The instructions below
 > remain valid however, and are still the recommended method for
 > integrating with Mesos.
 
@@ -34,9 +34,9 @@ Then run:
 
 To create a network that spans multiple hosts, the Weave peers must be connected to each other.  
 This is accomplished by specifying the other hosts during `weave launch` or via
-[`weave connect`](/site/using-weave/finding-adding-hosts-dynamically.md).
+[`weave connect`](/site/tasks/manage/finding-adding-hosts-dynamically.md).
 
-See [Creating Peer Connections Between Hosts](/site/using-weave.md#peer-connections) 
+See [Creating Peer Connections Between Hosts](/site/install/using-weave.md#peer-connections) 
 for a discussion on peer connections. 
 
     weave launch <peer hosts>

site/kubernetes/kube-addon.md

@@ -1,6 +1,6 @@
 ---
 title: Integrating Kubernetes via the Addon
-menu_order: 63
+menu_order: 20
 search_type: Documentation
 ---
 
@@ -12,8 +12,10 @@ The following topics are discussed:
  * [CPU and Memory Requirements](#resources)
  * [Pod Eviction](#eviction)
 * [Network Policy Controller](#npc)
+* [Troubleshooting](#troubleshooting)
  * [Troubleshooting Blocked Connections](#blocked-connections)
- * [Changing Configuration Options](#configuration-options)
+ * [Things to watch out for](#key-points)
+* [Changing Configuration Options](#configuration-options)
 
 
 ## <a name="install"></a> Installation
@@ -22,7 +24,7 @@ Weave Net can be installed onto your CNI-enabled Kubernetes cluster
 with a single command:
 
 ```
-$ kubectl apply -n kube-system -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
+$ kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
 ```
 
 After a few seconds, a Weave Net pod should be running on each
@@ -43,7 +45,7 @@ recommend your master node has at least two CPU cores.
 > Alternatively, you can [configure CNI yourself](http://kubernetes.io/docs/admin/network-plugins/#cni)
 
 **Note:** If using the [Weave CNI
-Plugin](/site/cni-plugin.md) from a prior full install of Weave Net with your
+Plugin](/site/kubernetes.md) from a prior full install of Weave Net with your
 cluster, you must first uninstall it before applying the Weave-kube addon.
 Shut down Kubernetes, and _on all nodes_ perform the following:
 
@@ -55,21 +57,6 @@ Shut down Kubernetes, and _on all nodes_ perform the following:
 Then relaunch Kubernetes and install the addon as described
 above.
 
-**Note:** The URLs:
-
-- [https://git.io/weave-kube](https://git.io/weave-kube), and
-- [https://git.io/weave-kube-1.6](https://git.io/weave-kube-1.6)
-
-point to:
-
-- [https://cloud.weave.works/k8s/v1.5/net](https://cloud.weave.works/k8s/v1.5/net), and
-- [https://cloud.weave.works/k8s/v1.6/net](https://cloud.weave.works/k8s/v1.6/net).
-
-In the past, these URLs pointed to the static YAML files for the [latest release](https://github.com/weaveworks/weave/releases/tag/latest_release) of the Weave Net addon, respectively:
-
-- [`latest_release/weave-daemonset.yaml`](https://github.com/weaveworks/weave/releases/download/latest_release/weave-daemonset.yaml) and
-- [`latest_release/weave-daemonset-k8s-1.6.yaml`](https://github.com/weaveworks/weave/releases/download/latest_release/weave-daemonset-k8s-1.6.yaml).
-
 ## <a name="kube-1.6-upgrade"></a> Upgrading Kubernetes to version 1.6
 
 In version 1.6, Kubernetes has increased security, so we need to
@@ -95,11 +82,11 @@ the Weave Net pods one by one.
 Kubernetes v1.5 and below does not support rolling upgrades of daemon sets,
 and so you will need to perform the procedure manually:
 
-* Apply the updated addon manifest `kubectl apply -f https://git.io/weave-kube`
+* Apply the updated addon manifest `kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"`
 * Kill each Weave Net pod with `kubectl delete` and then wait for it to reboot before moving on to the next pod.
 
-**Note:** If you delete all Weave Net pods at the same time they will
-  lose track of IP address range ownership, possibly leading to
+**Note:** In versions prior to Weave Net 2.0, deleting all Weave Net pods at the same time
+  will result in them losing track of IP address range ownership, possibly leading to
   duplicate IP addresses if you then start a new copy of Weave Net.
 
 ## <a name="resources"></a>CPU and Memory Requirements
@@ -169,14 +156,88 @@ labels. For more information on configuring network policies in
 Kubernetes see the
 [walkthrough](http://kubernetes.io/docs/getting-started-guides/network-policy/walkthrough/)
 and the [NetworkPolicy API object
-definition](http://kubernetes.io/docs/api-reference/extensions/v1beta1/definitions/#_v1beta1_networkpolicy).
+definition](https://v1-7.docs.kubernetes.io/docs/api-reference/v1.7/#networkpolicy-v1-networking)
 
 **Note:** as of version 1.9 of Weave Net, the Network Policy
   Controller allows all multicast traffic. Since a single multicast
   address may be used by multiple pods, we cannot implement rules to
   isolate them individually.  You can turn this behaviour off (block
-  all multicast traffic) by adding `--allow-mcast` as an argument to
-  `weave-npc` in the YAML configuration.
+  all multicast traffic) by adding `--allow-mcast=false` as an
+  argument to `weave-npc` in the YAML configuration.
+
+## <a name="troubleshooting"></a> Troubleshooting
+
+The status of Weave Net can be checked by running its CLI commands. This can be done in various ways:
+
+1\. [Install the `weave` script](/site/installing-weave.md) and run:
+
+```
+$ weave status
+        Version: 2.0.1 (up to date; next check at 2017/07/10 13:49:29)
+
+        Service: router
+       Protocol: weave 1..2
+           Name: 42:8e:e8:c4:52:1b(host-0)
+     Encryption: disabled
+  PeerDiscovery: enabled
+        Targets: 3
+    Connections: 3 (2 established, 1 failed)
+          Peers: 3 (with 6 established connections)
+ TrustedSubnets: none
+
+        Service: ipam
+         Status: ready
+          Range: 10.32.0.0/12
+  DefaultSubnet: 10.32.0.0/12
+```
+
+2\. If you don't want to install additional software onto your hosts, run via `kubectl` commands, which produce the exact same outcome as the previous example:
+
+```
+### Identify the Weave Net pods:
+
+$ kubectl get pods -n kube-system -l name=weave-net -o wide
+NAME              READY  STATUS   RESTARTS  AGE  IP           NODE
+weave-net-1jkl6   2/2    Running  0         1d   10.128.0.4   host-0
+weave-net-bskbv   2/2    Running  0         1d   10.128.0.5   host-1
+weave-net-m4x1b   2/2    Running  0         1d   10.128.0.6   host-2
+```
+
+The above shows all Weave Net pods available in your cluster.
+You can see Kubernetes has deployed one Weave Net pod per host, in order to interconnect all hosts.
+
+You then need to:
+ - choose which pod you want to run your command from (in most cases it doesn't matter
+   which one you pick so just pick the first one, e.g. pod `weave-net-1jkl6` here)
+ - use `kubectl exec` to run the `weave status` command
+ - specify the absolute path `/home/weave/weave` and add `--local` because it's running inside a container
+
+```
+$ kubectl exec -n kube-system weave-net-1jkl6 -c weave -- /home/weave/weave --local status
+
+        Version: 2.0.1 (up to date; next check at 2017/07/10 13:49:29)
+
+        Service: router
+       Protocol: weave 1..2
+           Name: 42:8e:e8:c4:52:1b(host-0)
+     Encryption: disabled
+  PeerDiscovery: enabled
+        Targets: 3
+    Connections: 3 (2 established, 1 failed)
+          Peers: 3 (with 6 established connections)
+ TrustedSubnets: none
+
+        Service: ipam
+         Status: ready
+          Range: 10.32.0.0/12
+  DefaultSubnet: 10.32.0.0/12
+```
+
+3\. Finally you could also use [Weave Cloud](https://cloud.weave.works/) and monitor all your pods, including Weave Net's ones, from there.
+
+![Weave Net status screen in Weave Cloud](weave-cloud-net-status.png)
+
+For more information see [What is Weave Cloud?](https://www.weave.works/docs/cloud/latest/overview/)
 
 ### <a name="blocked-connections"></a> Troubleshooting Blocked Connections
 
@@ -210,11 +271,20 @@ TCP connection from 10.32.0.7:56648 to 10.32.0.11:80 blocked by Weave NPC.
 UDP connection from 10.32.0.7:56648 to 10.32.0.11:80 blocked by Weave NPC.
 ```
 
-### <a name="configuration-options"></a> Changing Configuration Options
+### <a name="key-points"></a> Things to watch out for
+
+- Don't turn on `--masquerade-all` on kube-proxy: this will change the
+  source address of every pod-to-pod conversation which will make it
+  impossible to correctly enforce network policies that restrict which
+  pods can talk.
+- If you do set the `--cluster-cidr` option on kube-proxy, make sure
+  it matches the `IPALLOC_RANGE` given to Weave Net (see below)
+
+## <a name="configuration-options"></a> Changing Configuration Options
 
 #### Using `cloud.weave.works`
 
-If the YAML file is from `cloud.weave.works` as described above, then you can customise it by passing some of Weave Net's options, arguments and environment variables as query parameters:
+You can customise the YAML you get from `cloud.weave.works` by passing some of Weave Net's options, arguments and environment variables as query parameters:
 
   - `version`: Weave Net's version. Default: `latest`, i.e. latest release. *N.B.*: This only changes the specified version inside the generated YAML file, it does not ensure that the rest of the YAML is compatible with that version. To freeze the YAML version save a copy of the YAML file from the [release page](https://github.com/weaveworks/weave/releases) and use that copy instead of downloading it each time from `cloud.weave.works`.
   - `password-secret`: name of the Kubernetes secret containing your password.
@@ -222,13 +292,14 @@ If the YAML file is from `cloud.weave.works` as described above, then you can cu
 
         $ echo "s3cr3tp4ssw0rd" > /var/lib/weave/weave-passwd.txt
         $ kubectl create secret -n kube-system generic weave-passwd --from-file=/var/lib/weave/weave-passwd.txt
-        $ kubectl apply -n kube-system -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')&password-secret=weave-passwd"
+        $ kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')&password-secret=weave-passwd"
 
   - `known-peers`: comma-separated list of hosts. Default: empty.
   - `trusted-subnets`: comma-separated list of CIDRs. Default: empty.
   - `disable-npc`: boolean (`true|false`). Default: `false`.
   - `env.NAME=VALUE`: add environment variable `NAME` and set it to `VALUE`.
   - `seLinuxOptions.NAME=VALUE`: add SELinux option `NAME` and set it to `VALUE`, e.g. `seLinuxOptions.type=spc_t`
+  - `use-legacy-netpol`: use [legacy NetworkPolicy semantics](https://v1-7.docs.kubernetes.io/docs/api-reference/v1.7/#networkpolicy-v1-networking), boolean (`true|false`). Default: `true` for Kubernetes version <= 1.6, `false` for > 1.6.
 
 The list of variables you can set is:
 
@@ -252,12 +323,12 @@ The list of variables you can set is:
 * `WEAVE_MTU` - Weave Net defaults to 1376 bytes, but you can set a
   smaller size if your underlying network has a tighter limit, or set
   a larger size for better performance if your network supports jumbo
-  frames - see [here](/site/using-weave/fastdp.md#mtu) for more
+  frames - see [here](/site/tasks/manage/fastdp.md#mtu) for more
   details.
 
 Example:
 ```
-$ kubectl apply -n kube-system -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')&env.WEAVE_MTU=1337"
+$ kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')&env.WEAVE_MTU=1337"
 ```
 This command -- notice `&env.WEAVE_MTU=1337` at the end of the URL -- generates a YAML file containing, among others:
 

site/operational-guide/autoscaling.md

@@ -25,7 +25,7 @@ and partitions.
 ### Scaling Out
 
 On the additional dynamic peer, at boot, via
-[systemd](/site/installing-weave/systemd.md) or equivalent:
+[systemd](/site/install/systemd.md) or equivalent:
 
     weave launch --no-restart --ipalloc-init=observer $PEERS
 
@@ -39,7 +39,7 @@ You do not have to keep track of and specify the addresses
 of other dynamic peers in `$PEERS` - they will discover and connect to
 each other via the fixed cluster.
 
->>**Note:** The use of `--ipalloc-init=observer` prevents dynamic peers from
+>**Note:** The use of `--ipalloc-init=observer` prevents dynamic peers from
 coming to a consensus on their own - this is important to stop a
 clique forming amongst a group of dynamically added peers if they
 become partitioned from the fixed cluster after having learned about

site/operational-guide/concepts.md

@@ -72,7 +72,7 @@ detail below.
 Peer discovery is a mechanism that allows peers to learn about new
 Weave Net hosts from existing peers without being explicitly told. Peer
 discovery is
-[enabled by default](/site/using-weave/finding-adding-hosts-dynamically.md).
+[enabled by default](/site/tasks/manage/finding-adding-hosts-dynamically.md).
 
 ## <a name="network-partition"></a>Network Partition
 
@@ -85,7 +85,7 @@ partition, healing automatically once the partition is over.
 
 ## <a name="ip-address-manager"></a>IP Address Manager (IPAM)
 
-[IPAM](/site/ipam.md) is the subsystem responsible for dividing up a
+[IPAM](/site/tasks/ipam/ipam.md) is the subsystem responsible for dividing up a
 large contiguous block of IP addresses (known as the IP allocation
 range) amongst peers so that individual addresses may be uniquely
 assigned to containers anywhere on the overlay network.

site/operational-guide/tasks.md

@@ -21,7 +21,7 @@ once and your system is configured to start Docker on boot, then Weave Net
 will start automatically on system restarts.
 
 If you are aiming for a non-interactive installation, use
-[systemd](/site/installing-weave/systemd.md) or a similar init system to
+[systemd](/site/install/systemd.md) or a similar init system to
 launch Weave using the `--no-restart` flag after Docker has been started.
 
 ## <a name="detect-reclaim-ipam"></a>Detecting and Reclaiming Lost IP Address Space
@@ -66,7 +66,7 @@ after which one of the remaining peers could decide unilaterally that
 it is safe to appropriate the space for itself, and so an
 administrative action is required to reclaim it.
 
-The [`weave rmpeer`](/site/ipam/stop-remove-peers-ipam.md)
+The [`weave rmpeer`](/site/tasks/ipam/stop-remove-peers-ipam.md)
 command is provided to perform this task, and must
 be executed on _one_ of the remaining peers. That peer will then take
 ownership of the freed address space.
@@ -99,7 +99,7 @@ To minimize downtime while the new script is pulling the new container images:
 * Start the new Weave Net with `weave launch <existing peer list>` (or
   `systemctl start weave` if you're using a systemd unit file)
 
->>**Note:** Always check the Release Notes for specific versions in case
+>**Note:** Always check the Release Notes for specific versions in case
 there are any special caveats or deviations from the standard
 procedure.
 
@@ -115,6 +115,6 @@ you can issue the following command:
 
 **See Also**
 
- * [Allocating IP Addresses](/site/ipam.md)
- * [Troubleshooting the IP Allocator](/site/ipam/troubleshooting-ipam.md)
+ * [Allocating IP Addresses](/site/tasks/ipam/ipam.md)
+ * [Troubleshooting the IP Allocator](/site/tasks/ipam/troubleshooting-ipam.md)
 

site/operational-guide/uniform-dynamic-cluster.md

@@ -18,7 +18,7 @@ A uniform dynamic cluster has the following characteristics:
 ## Bootstrapping
 
 On each initial peer N, at boot, via
-[systemd](/site/installing-weave/systemd.md):
+[systemd](/site/install/systemd.md):
 
     hostN$ weave launch --no-restart --name ::N --ipalloc-init seed=$SEED $PEERS
 
@@ -45,7 +45,7 @@ Where
 ## Adding a Peer
 
 On each new peer, at boot, via
-[systemd](/site/installing-weave/systemd.md):
+[systemd](/site/install/systemd.md):
 
     hostN$ weave launch --no-restart --name ::N --ipalloc-init seed=$SEED $PEERS
 

site/operational-guide/uniform-fixed-cluster.md

@@ -24,7 +24,7 @@ A uniform fixed cluster has the following characteristics:
 ### Bootstrapping
 
 On each initial peer, at boot, via
-[systemd](/site/installing-weave/systemd.md):
+[systemd](/site/install/systemd.md):
 
     weave launch --no-restart $PEERS
 
@@ -42,7 +42,7 @@ Then on _any_ peer run the following to force consensus:
 
     weave prime
 
->>**Note:** You can run this safely on more than one or
+>**Note:** You can run this safely on more than one or
 even all peers, but it's only strictly necessary to run it on one of
 them. 
 
@@ -56,7 +56,7 @@ that](/site/troubleshooting.md) before moving on.
 ### Adding a Peer
 
 On the new peer, at boot, via
-[systemd](/site/installing-weave/systemd.md) run:
+[systemd](/site/install/systemd.md) run:
 
     weave launch --no-restart $PEERS
 

site/overview.md

@@ -27,7 +27,7 @@ Weave Net implements service discovery by providing a fast "micro DNS" server at
 All other Docker networking plugins, including Docker's own "Overlay" driver, require that you set up Docker with a cluster store – a central database like Consul or Zookeeper – before you can even use them. Besides being difficult to set up, maintain and manage, every Docker host must also be in constant contact with the cluster store: if you lose the connection, even temporarily, then you cannot start or stop any containers.
 
 Weave Net is bundled with a Docker Network plugin that doesn't require an external cluster store. You can get started right away and you can start and stop containers even when there are network connectivity problems.  
-For information about the Weave Docker Plugin, see [How The Weave Network Plugin Works](/site/plugin/plugin-how-it-works.md).
+For information about the Weave Docker Plugin, see [How The Weave Network Plugin Works](/site/install/plugin/plugin-how-it-works.md).
 
 ### Operates in Partially Connected Networks
 
@@ -37,7 +37,7 @@ Weave Net can forward traffic between nodes, and it works even if the mesh netwo
 
 Weave Net automatically chooses the fastest path between two hosts, offering near native throughput and latency, all without your intervention.  
 
-See [How Fast Datapath Works](/site/using-weave/fastdp.md) for more information.
+See [How Fast Datapath Works](/site/tasks/manage/fastdp.md) for more information.
 
 ### Network Operations Friendly
 
@@ -60,7 +60,7 @@ With Weave Net, deploy your applications - whether peer-to-peer file sharing, vo
 Weave Net is a good choice if you want one tool for everything.  For example: In addition to Docker plugins, you can also use Weave as a Kubernetes plugin.  You can also use Weave with Amazon ECS or with Mesos and Marathon.  
 Refer to our Getting Started and Integration Guides for more information.
 
-For a complete list and description of Weave Net’s current feature set see, [Weave Net Features](/site/features.md).
+For a complete list and description of Weave Net’s current feature set see, [Weave Net Features](/site/overview/features.md).
 
 
 

site/overview/features.md

@@ -26,7 +26,7 @@ search_type: Documentation
  * [Container Mobility](#container-mobility)
  * [Fault Tolerance](#fault-tolerance) 
 
-For step-by-step instructions on how to use Weave Net, see [Using Weave Net](/site/using-weave.md).
+For step-by-step instructions on how to use Weave Net, see [Launching Weave Net](/site/install/using-weave.md).
 
 ### <a name="virtual-ethernet-switch"></a>Virtual Ethernet Switch
 
@@ -44,8 +44,8 @@ a container network.
 Broadcast and Multicast protocols can also be used
 over Weave Net.
 
-To start using Weave Net, see [Installing Weave Net](/site/installing-weave.md) 
-and [Using Weave Net](/site/using-weave.md).
+To start using Weave Net, see [Installing Weave Net](/site/install/installing-weave.md) 
+and [Launching Weave Net](/site/install/using-weave.md).
 
 ### <a name="fast-data-path"></a>Fast Datapath
 
@@ -53,12 +53,12 @@ Weave Net automatically chooses the fastest available method to
 transport data between peers. The best performing of these 
 (the 'fast datapath') offers near-native throughput and latency.
 
-See [Using Fast Datapath](/site/using-weave/fastdp.md) and
-[How Fast Datapath Works](/site/how-it-works/fastdp-how-it-works.md).
+See [Using Fast Datapath](/site/tasks/manage/fastdp.md) and
+[How Fast Datapath Works](/site/concepts/fastdp-how-it-works.md).
 
 ### <a name="docker"></a>Seamless Docker Integration (Weave Docker API Proxy)
 
-Weave Net includes a [Docker API Proxy](/site/weave-docker-api.md), which can be 
+Weave Net includes a [Docker API Proxy](/site/tasks/weave-docker-api/weave-docker-api.md), which can be 
 used to start containers using the Docker [command-line interface](https://docs.docker.com/reference/commandline/cli/) or the [remote API](https://docs.docker.com/reference/api/docker_remote_api/), and attach them to the Weave network before they begin execution.
 
 To use the proxy, run:
@@ -71,7 +71,7 @@ Containers started in this way that subsequently restart, either
 by an explicit `docker restart` command or by Docker restart 
 policy, are re-attached to the Weave network by the `Weave Docker API Proxy`.
 
-See [Integrating Docker via the API Proxy](/site/weave-docker-api.md).
+See [Integrating Docker via the API Proxy](/site/tasks/weave-docker-api/weave-docker-api.md).
 
 
 ### <a name="plugin"></a>Weave Network Docker Plugin
@@ -84,15 +84,15 @@ named `weave` is created by `weave launch`, which is used as follows:
 Using the Weave plugin enables you to take advantage of [Docker's network functionality](https://docs.docker.com/engine/extend/plugins_network/).
 
 There are two plugin implementations for Weave Net: the
-["Legacy"](/site/plugin.md) plugin which doesn't require an external
-cluster store, and the ["V2"](/site/plugin-v2.md) plugin which
+["Legacy"](/site/install/plugin/plugin.md) plugin which doesn't require an external
+cluster store, and the ["V2"](/site/install/plugin/plugin-v2.md) plugin which
 supports Docker swarm mode.
 
 ### <a name="cniplugin"></a>Weave Network CNI Plugin
 
 Weave can be used as a plugin to systems that support the [Container Network Interface](https://github.com/appc/cni), such as Kubernetes and Mesosphere.
 
-See [Integrating Kubernetes and Mesos via the CNI Plugin](/site/cni-plugin.md) for more details.
+See [Integrating Kubernetes and Mesos via the CNI Plugin](/site/kubernetes.md) for more details.
 
 
 ### <a name="addressing"></a>IP Address Management (IPAM)
@@ -100,24 +100,24 @@ See [Integrating Kubernetes and Mesos via the CNI Plugin](/site/cni-plugin.md) f
 Containers are automatically allocated a unique IP address. To view the addresses allocated by Weave, run `weave ps`.
 
 Instead of allowing Weave to automatically allocate addresses, an IP address and a network can be explicitly 
-specified. See [How to Manually Specify IP Addresses and Subnets](/site/using-weave/manual-ip-address.md) for instructions. 
+specified. See [How to Manually Specify IP Addresses and Subnets](/site/tasks/ipam/manual-ip-address.md) for instructions. 
 
-For a discussion on how Weave Net uses IPAM, see [Automatic IP Address Management](/site/ipam.md). And also review the 
-[the basics of IP addressing](/site/how-it-works/ip-addresses.md) for an explanation of addressing and private networks. 
+For a discussion on how Weave Net uses IPAM, see [Automatic IP Address Management](/site/tasks/ipam/ipam.md). And also review the 
+[the basics of IP addressing](/site/concepts/ip-addresses.md) for an explanation of addressing and private networks. 
 
 
 ### <a name="naming-and-discovery"></a>Naming and Discovery
  
-Named containers are automatically registered in [weaveDNS](/site/weavedns.md), 
+Named containers are automatically registered in [weaveDNS](/site/tasks/weavedns/weavedns.md), 
 and are discoverable by using standard, simple name lookups:
 
     host1$ docker run -dti --name=service weaveworks/ubuntu
     host1$ docker run -ti weaveworks/ubuntu
     root@7b21498fb103:/# ping service
 
-WeaveDNS also supports [load balancing](/site/weavedns/load-balance-fault-weavedns.md), [fault resilience](/site/weavedns/load-balance-fault-weavedns.md) and [hot swapping](/site/weavedns/managing-entries-weavedns.md). 
+WeaveDNS also supports [load balancing](/site/tasks/weavedns/load-balance-fault-weavedns.md), [fault resilience](/site/tasks/weavedns/load-balance-fault-weavedns.md) and [hot swapping](/site/tasks/weavedns/managing-entries-weavedns.md). 
 
-See [Discovering Containers with WeaveDNS](/site/weavedns.md).
+See [Discovering Containers with WeaveDNS](/site/tasks/weavedns/weavedns.md).
  
 ### <a name="application-isolation"></a>Application Isolation
 
@@ -132,13 +132,13 @@ _isolation-through-subnets_ technique. This common strategy
  techniques can be used to deploy your applications to 
  containers.
 
-See [Isolating Applications](/site/using-weave/application-isolation.md) 
+See [Isolating Applications](/site/tasks/manage/application-isolation.md) 
 for information on how to use the isolation-through-subnets 
 technique with Weave Net.
 
 ### <a name="network-policy"></a>Network Policy
 
-The Weave [Kubernetes Addon](/site/kube-addon.md) includes a network
+The Weave [Kubernetes Addon](/site/kubernetes/kube-addon.md) includes a network
 policy controller that implements [Kubernetes Network
 Policies](http://kubernetes.io/docs/user-guide/networkpolicies/).
 
@@ -149,7 +149,7 @@ given container in advance. In these cases, you can take
 advantage of Weave's ability to attach and detach running 
 containers to and from any network. 
 
-See [Dynamically Attaching and Detaching Containers](/site/using-weave/dynamically-attach-containers.md) 
+See [Dynamically Attaching and Detaching Containers](/site/tasks/manage/dynamically-attach-containers.md) 
 for details. 
 
 
@@ -176,7 +176,7 @@ running alongside Weave.
 Weave Net implements encryption and security using the Go version of [Daniel J.  Bernstein's NaCl library](http://nacl.cr.yp.to/index.html),
 and, additionally in the case of encrypted fast datapath using [the cryptography framework of the Linux kernel](https://en.wikipedia.org/wiki/Crypto_API_(Linux)).
 
-For information on how to secure your Docker network connections, see [Securing Connections Across Untrusted Networks](/site/using-weave/security-untrusted-networks.md) and for a more technical discussion on how Weave implements encryption see, [Weave Encryption](/site/how-it-works/encryption.md) and [How Weave Implements Encryption](/site/how-it-works/encryption-implementation.md).
+For information on how to secure your Docker network connections, see [Securing Connections Across Untrusted Networks](/site/tasks/manage/security-untrusted-networks.md) and for a more technical discussion on how Weave implements encryption see, [Weave Encryption](/site/concepts/encryption.md) and [How Weave Implements Encryption](/site/concepts/encryption-implementation.md).
 
 
 ### <a name="host-network-integration"></a>Host Network Integration
@@ -184,7 +184,7 @@ For information on how to secure your Docker network connections, see [Securing
 Weave Net application networks can be integrated with a host's network, and establish connectivity between the host and 
 application containers anywhere.
 
-See [Integrating with the Host Network](/site/using-weave/host-network-integration.md).
+See [Integrating with the Host Network](/site/tasks/manage/host-network-integration.md).
 
 ### <a name="services"></a>Managing Services: Exporting, Importing, Binding and Routing
  
@@ -193,7 +193,7 @@ See [Integrating with the Host Network](/site/using-weave/host-network-integrati
  * **Binding Services** - A container can be bound to a particular IP and port without having to change your application code, while at the same time will maintain its original endpoint. 
  * **Routing Services** - By combining the importing and exporting features, you can connect to disjointed networks, even when separated by firewalls and where there may be overlapping IP addresses.  
 
-See [Managing Services - Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md) for instructions on how to manage services on a Weave container network. 
+See [Managing Services - Exporting, Importing, Binding and Routing](/site/tasks/manage/service-management.md) for instructions on how to manage services on a Weave container network. 
 
 ### <a name="multi-cloud-networking"></a>Multi-Cloud Networking
 
@@ -203,7 +203,7 @@ of containers that run on [Google Compute Engine](https://cloud.google.com/compu
 (GCE), [Amazon Elastic Compute Cloud](https://aws.amazon.com/ec2/) 
 (EC2) and in a local data centre all at the same time.
 
-See [Enabling Multi-Cloud networking and Muti-hop Routing](/site/using-weave/multi-cloud-multi-hop.md).
+See [Enabling Multi-Cloud networking and Muti-hop Routing](/site/tasks/manage/multi-cloud-multi-hop.md).
 
 
 ### <a name="multi-hop-routing"></a>Multi-Hop Routing
@@ -213,14 +213,14 @@ even when there is only partial connectivity between the hosts. Weave Net
 routes traffic between containers as long as there is at least one *path* 
 of connected hosts between them.
 
-See [Enabling Multi-Cloud networking and Multi-hop Routing](/site/using-weave/multi-cloud-multi-hop.md).
+See [Enabling Multi-Cloud networking and Multi-hop Routing](/site/tasks/manage/multi-cloud-multi-hop.md).
 
 
 ### <a name="dynamic-topologies"></a>Dynamic Topologies
 
 Hosts can be added to or removed from a Weave network without stopping
 or reconfiguring the remaining hosts. See [Adding and Removing Hosts
-Dynamically.](/site/using-weave/finding-adding-hosts-dynamically.md)
+Dynamically.](/site/tasks/manage/finding-adding-hosts-dynamically.md)
 
 
 ### <a name="container-mobility"></a>Container Mobility
@@ -230,7 +230,7 @@ reconfiguration or, in many cases, restarts of other containers.
 All that is required is for the migrated container to be started 
 with the same IP address as it was given originally.
 
-See [Managing Services - Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md), in particular, Routing Services for more information on container mobility. 
+See [Managing Services - Exporting, Importing, Binding and Routing](/site/tasks/manage/service-management.md), in particular, Routing Services for more information on container mobility. 
 
 
 ### <a name="fault-tolerance"></a>Fault Tolerance

site/tasks.md

@@ -0,0 +1,71 @@
+---
+title: Weave Net Tasks
+menu_order: 50
+search_type: Documentation
+---
+
+The section describes the configuration options available in Weave Net. It is divided into: Managing Containerized Applications, Administering IP Addresses, Managing WeaveDNS, and finally attaching Docker Containers via the Weave API Proxy. 
+
+
+## Managing Containerized Applications
+
+
+* [Isolating Applications on a Weave Network](https://weave.works/docs/net/latest/tasks/manage/application-isolation/)
+* [Dynamically Attaching and Detaching Applications](https://weave.works/docs/net/latest/tasks/dynamically-attach-containers/)
+* [Integrating with the Host Network](https://weave.works/docs/net/latest/tasks/host-network-integration/)
+* [Managing Services - Exporting, Importing, Binding and Routing](https://weave.works/docs/net/latest/tasks/manage/service-management/)
+     * [Exporting Services](https://weave.works/docs/net/latest/tasks/manage/service-management/#exporting)
+     * [Importing Services](https://weave.works/docs/net/latest/tasks/manage/service-management/#importing)
+     * [Binding Services](https://weave.works/docs/net/latest/tasks/manage/service-management/#binding)
+     * [Routing Services](https://weave.works/docs/net/latest/tasks/manage/service-management/#routing)
+     * [Dynamically Changing Service Locations](https://weave.works/docs/net/latest/tasks/manage/service-management/#change-location)
+* [Securing Connections Across Untrusted Networks](https://weave.works/docs/net/latest/tasks//manage/security-untrusted-networks/)
+* [Adding and Removing Hosts Dynamically](https://weave.works/docs/net/latest/tasks/manage/finding-adding-hosts-dynamically/)
+* [Using Fast Datapath](https://weave.works/docs/net/latest/tasks/manage/fastdp/)
+* [Enabling Multi-Cloud, Multi-Hop Networking and Routing](https://weave.works/docs/net/latest/tasks/manage/multi-cloud-multi-hop/)
+* [Configuring IP Routing on an Amazon Web Services Virtual Private Cloud](https://weave.works/docs/net/latest/tasks/manage/awsvpc/)
+* [Monitoring with Prometheus](https://weave.works/docs/net/latest/tasks/manage/metrics/)
+
+
+## Administering IP Addresses
+
+
+* [Allocating IP Addresses](https://weave.works/docs/net/latest/tasks/ipam/ipam/)
+     * [Initializing Peers on a Weave Network](https://weave.works/docs/net/latest/tasks/ipam/ipam/#initialization)
+     * [--ipalloc-init:consensus and How Quorum is Achieved](https://weave.works/docs/net/latest/tasks/ipam/ipam/#quorum)
+     * [Priming a Peer](https://weave.works/docs/net/latest/tasks/ipam/ipam/#priming-a-peer)
+     * [Choosing an Allocation Range](https://weave.works/docs/net/latest/tasks/ipam/ipam/#range)
+* [Allocating IPs in a Specific Range](https://weave.works/docs/net/latest/tasks/ipam/configuring-weave/)
+* [Manually Specifying the IP Address of a Container](https://weave.works/docs/net/latest/tasks/ipam/manual-ip-address/)
+* [Automatic Allocation Across Multiple Subnets](https://weave.works/docs/net/latest/tasks/ipam/allocation-multi-ipam/)
+* [Starting, Stopping and Removing Peers](https://weave.works/docs/net/latest/tasks/ipam/stop-remove-peers-ipam/)
+* [Troubleshooting the IP Allocator](https://weave.works/docs/net/latest/tasks/ipam/troubleshooting-ipam/)
+
+
+## Managing WeaveDNS
+
+
+* [Discovering Containers with WeaveDNS](https://weave.works/docs/net/latest/tasks/weavedns/weavedns/)
+* [How Weave Finds Containers](https://weave.works/docs/net/latest/tasks/weavedns/how-works-weavedns/)
+* [Load Balancing and Fault Resilience with weaveDNS](https://weave.works/docs/net/latest/tasks/weavedns/load-balance-fault-weavedns/)
+* [Managing Domains](https://weave.works/docs/net/latest/tasks/weavedns/managing-domains-weavedns/)
+     * [Configuring the domain search path](https://weave.works/docs/net/latest/tasks/weavedns/managing-domains-weavedns/#domain-search-path)
+     * [Using a different local domain](https://weave.works/docs/net/latest/tasks/weavedns/managing-domains-weavedns/#local-domain)
+* [Managing Domain Entries](https://weave.works/docs/net/latest/tasks/weavedns/managing-entries-weavedns/)
+     * [Adding and removing extra DNS entries](https://weave.works/docs/net/latest/tasks/weavedns/managing-entries-weavedns/#add-remove)
+     * [Resolving WeaveDNS entries from the Host](https://weave.works/docs/net/latest/tasks/weavedns/managing-entries-weavedns/#resolve-weavedns-entries-from-host)
+     * [Hot-swapping Service Containers](https://weave.works/docs/net/latest/tasks/weavedns/managing-entries-weavedns/#hot-swapping)
+     * [Configuring a Custom TTL](https://weave.works/docs/net/latest/tasks/weavedns/managing-entries-weavedns/#ttl)
+* [Troubleshooting and Present Limitations](https://weave.works/docs/net/latest/tasks/weavedns/troubleshooting-weavedns/)
+
+
+## Attaching Docker Containers via the Weave API Proxy
+
+
+* [Integrating Docker via the API Proxy](https://weave.works/docs/net/latest/tasks/weave-docker-api/weave-docker-api/)
+* [Using The Weave Docker API Proxy](https://weave.works/docs/net/latest/tasks/weave-docker-api/using-proxy/)
+* [Securing the Docker Communications With TLS](https://weave.works/docs/net/latest/tasks/weave-docker-api/securing-proxy/)
+* [Automatic IP Allocation and the Weave Proxy](https://weave.works/docs/net/latest/tasks/weave-docker-api/ipam-proxy/)
+* [Using Automatic Discovery With the Weave Net Proxy](https://weave.works/docs/net/latest/tasks/weave-docker-api/automatic-discovery-proxy/)
+* [Name resolution via `/etc/hosts`](https://weave.works/docs/net/latest/tasks/weave-docker-api/name-resolution-proxy/)
+* [Launching Containers With Weave Run (without the Proxy)](https://weave.works/docs/net/latest/tasks/weave-docker-api/launching-without-proxy/)

site/tasks/ipam.md

@@ -0,0 +1,5 @@
+---
+title: Administering IP Addresses
+menu_order: 20
+search_type: Documentation
+---

site/tasks/ipam/allocation-multi-ipam.md

@@ -1,6 +1,6 @@
 ---
 title: Automatic Allocation Across Multiple Subnets
-menu_order: 10
+menu_order: 30
 search_type: Documentation
 ---
 
@@ -10,7 +10,7 @@ puts all containers into a subnet that spans the entire allocation
 range, so that every Weave-attached container can communicate with every other
 Weave-attached container.
 
-If you want some [isolation](/site/using-weave/application-isolation.md), you
+If you want some [isolation](/site/tasks/manage/application-isolation.md), you
 can choose to run containers on different subnets.  To request the
 allocation of an address from a particular subnet, set the
 `WEAVE_CIDR` environment variable to `net:<subnet>` when creating the
@@ -63,6 +63,6 @@ for manual allocation.
 
 **See Also**
 
- * [Address Allocation with IP Address Management (IPAM)](/site/ipam.md)
- * [Isolating Applications on a Weave Network](/site/using-weave/application-isolation.md)
- * [Starting, Stopping and Removing Peers](/site/ipam/stop-remove-peers-ipam.md)
+ * [Address Allocation with IP Address Management (IPAM)](/site/tasks/ipam/ipam.md)
+ * [Isolating Applications on a Weave Network](/site/tasks/manage/application-isolation.md)
+ * [Starting, Stopping and Removing Peers](/site/tasks/ipam/stop-remove-peers-ipam.md)

site/tasks/ipam/configuring-weave.md

@@ -1,6 +1,6 @@
 ---
 title: Allocating IPs in a Specific Range
-menu_order: 15
+menu_order: 10
 search_type: Documentation
 ---
 
@@ -40,4 +40,4 @@ Networks](https://en.wikipedia.org/wiki/Private_network).  For example
 
 **See Also**
 
- * [IP Addresses, Routes and Networks](/site/how-it-works/ip-addresses.md)
+ * [IP Addresses, Routes and Networks](/site/concepts/ip-addresses.md)

site/tasks/ipam/ipam.md

@@ -1,6 +1,6 @@
 ---
 title: Allocating IP Addresses
-menu_order: 70
+menu_order: 5
 search_type: Documentation
 ---
 
@@ -180,7 +180,7 @@ The range parameter is written in
 in this example "/16" means the first 16 bits of the address form the
 network address and the allocator is to allocate container addresses
 that all start 10.2. See [IP
-addresses and routes](/site/how-it-works/ip-addresses.md) for more information.
+addresses and routes](/site/concepts/ip-addresses.md) for more information.
 
 Weave shares the IP address range across all peers, dynamically
 according to their needs.  If a group of peers becomes isolated from
@@ -202,5 +202,5 @@ named `weavedb` is used to store this data.
 
  **See Also**
 
- * [Automatic Allocation Across Multiple Subnets](/site/ipam/allocation-multi-ipam.md)
- * [Integrating Docker via the Network Plugin (Legacy)](/site/plugin.md)
+ * [Automatic Allocation Across Multiple Subnets](/site/tasks/ipam/allocation-multi-ipam.md)
+ * [Integrating Docker via the Network Plugin (Legacy)](/site/install/plugin/plugin.md)

site/tasks/ipam/manual-ip-address.md

@@ -11,7 +11,7 @@ Containers are automatically allocated an IP address that is unique across the W
 
 Weave Net detects when a container has exited and releases its allocated addresses so they can be re-used by the network.
 
-See the [Automatic IP Address Management](/site/ipam.md) and also an explanation of [the basics of IP addressing](/site/how-it-works/ip-addresses.md) for further details.
+See the [Automatic IP Address Management](/site/tasks/ipam/ipam.md) and also an explanation of [the basics of IP addressing](/site/concepts/ip-addresses.md) for further details.
 
 Instead of allowing Weave Net to allocate IP addresses automatically (using IPAM), there may be instances where you need to control a particular container or a cluster by setting an IP address for it.  
 
@@ -52,7 +52,7 @@ Individual IP addresses given to containers must, of course, be unique. If you p
 
 **See Also**
 
- * [Managing Services - Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md)
- * [Allocating IPs in a Specific Range](/site/using-weave/configuring-weave.md) 
- * [Automatic IP Address Management](/site/ipam.md)   
+ * [Managing Services - Exporting, Importing, Binding and Routing](/site/tasks/manage/service-management.md)
+ * [Allocating IPs in a Specific Range](/site/tasks/ipam/configuring-weave.md) 
+ * [Automatic IP Address Management](/site/tasks/ipam/ipam.md)