Add release v1.9.6

This means we only filter connections at the destination end,
which is a better match for the ingress-only rules.

.gitignore

@@ -68,3 +68,8 @@ test/tls/*.pem
 test/coverage
 test/coverage.*
 *qemu-*-static
+terraform.tfstate
+terraform.tfstate.backup
+
+# Website
+site-build

.gitmodules

@@ -3,7 +3,7 @@
 	url = https://github.com/weaveworks/build-tools
 [submodule "vendor/github.com/vishvananda/netlink"]
 	path = vendor/github.com/vishvananda/netlink
-	url = https://github.com/weaveworks/netlink.git
+	url = https://github.com/vishvananda/netlink.git
 [submodule "vendor/github.com/gorilla/mux"]
 	path = vendor/github.com/gorilla/mux
 	url = https://github.com/gorilla/mux.git
@@ -61,9 +61,6 @@
 [submodule "vendor/github.com/hashicorp/go-cleanhttp"]
 	path = vendor/github.com/hashicorp/go-cleanhttp
 	url = http://github.com/hashicorp/go-cleanhttp
-[submodule "vendor/github.com/appc/cni"]
-	path = vendor/github.com/appc/cni
-	url = https://github.com/appc/cni
 [submodule "vendor/github.com/coreos/go-iptables"]
 	path = vendor/github.com/coreos/go-iptables
 	url = https://github.com/coreos/go-iptables
@@ -124,3 +121,6 @@
 [submodule "vendor/github.com/weaveworks/common"]
 	path = vendor/github.com/weaveworks/common
 	url = https://github.com/weaveworks/common
+[submodule "vendor/github.com/containernetworking/cni"]
+	path = vendor/github.com/containernetworking/cni
+	url = https://github.com/containernetworking/cni

.lintignore

@@ -1,8 +1,27 @@
 # IMPORTANT: Please only use GLOBs to ignore files and directories.
 
-# Do not lint any bash script in Weave Net, in order to 
+# Do not lint the below bash scripts in Weave Net, in order to 
 # avoid breaking the build on make lint:
-*.sh
-bin/*
+bin/circle-deploy-issues
+bin/circle-deploy-master
+bin/circle-teardown-pre
+bin/circle-test-teardown
+bin/circle-test-unit
+bin/install-wordepress
+bin/multiweave
+bin/publish-site
+bin/release
+build/build.sh
+prog/weave-kube/launch.sh
 prog/weaveexec/symlink
+test/*_test.sh
+test/600_proxy_docker_py.sh
+test/assert.sh
+test/config.sh
+test/gce.sh
+test/gen_coverage_reports.sh
+test/run_all.sh
+test/sanity_check.sh
+test/setup.sh
+vendor/*.sh
 weave

.ruby-version

@@ -0,0 +1 @@
+2.3.0

CHANGELOG.md

@@ -1,3 +1,117 @@
+## Release 1.9.6
+
+Bug fixes and minor improvements
+
+* Ensure that Kubernetes pods can contact a service implemented within
+  the same pod, by turning on "hairpin mode". This is required because
+  of a quiet change between Kubernetes 1.5 and 1.6. #2993
+* Network Policy Controller (`weave-npc`) now checks local addresses
+  only, so it doesn't interfere with cross-cluster traffic. It should
+  be more efficient too #2622,#2973,#2979
+* Stop reporting back to Kubernetes any issues encountered when
+  deleting a pod's network interface. This is required because
+  of a quiet change between Kubernetes 1.5 and 1.6. #2921,#2928
+* Fixed an issue whereby `weave-npc` couldn't start because one
+  `ipset` was referring to another one and could not be destroyed #2915,#2949
+* Improved the code which checks whether the kernel supports `ipset` #2934,#2935
+* `weave-npc` now creates ipsets with only valid xml characters in the
+  name #2958,#2959
+
+Build and Testing
+
+* In build container use cross-compilers from debian package
+  repository, so they match other components #2940
+* Pin the version of the linting tool `shfmt` so the set of things it
+  checks is stable #2987
+* Fix lint error in script that runs smoke-tests #2962
+* Moved website publishing from Wordpress to Netlify #2986
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/55?closed=1).
+
+## Release 1.9.5
+
+Bug fixes and minor improvements
+
+* Improve log messages generated if "hairpin" conditions are detected,
+  to make clear which kind is likely to cause problems #2808/#2926
+* Filter out IPv6 peer addresses from Kubernetes; Weave Net currently
+  only supports IPv4 #2904/#2912
+* Fix rare crash during initialization of weave-kube #2893/#2892
+* Include overlay and encryption modes in checkpoint reports, in case
+  this is relevant to a version upgrade #2771/#2907
+
+Build and Testing
+
+* Ensure CI build can run gcloud tools  #2887
+* Prevent kubeadm from upgrading Kubernetes if we are trying to test an older version #2886
+* Upgrade build scripts to support Kubernetes 1.6 #2880
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/54?closed=1).
+
+## Release 1.9.4
+
+Bug fixes and minor improvements
+
+* Support Kubernetes 1.6 by creating a new DaemonSet #2777,#2801
+* Support Kubernetes 1.6 by allowing CNI callers to send a
+  network-delete request for a container that is not running or has
+  never been attached to the network #2850
+* Leave non-weave ipsets alone in Network Policy Controller (e.g. when
+  running Weave Net alonside keepalived-vip) #2751,#2846
+* Fix various small issues revealed by 'staticcheck' tool #2843,#2857
+* Avoid leaving 'defunct' processes when weave-kube container restarts #2836,#2845
+* When using the CNI plugin with a non-standard network configuration
+  file, the weave bridge could get the same IP as a container, if
+  'weave expose' hadn't run at that point #2839,#2856
+
+Build and Testing
+
+* Check that no defunct processes remain after each test #2852
+* Update build and test scripts to work with Kubernetes 1.6 beta #2851
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/53?closed=1).
+
+## Release 1.9.3
+
+Bug fixes and minor improvements
+
+* Fixed a race condition in Fast Datapath encrypted connections which could lead Weave Net to crash: #2824, #2825.
+
+## Release 1.9.2
+
+Bug fixes and minor improvements
+
+* Fix a weave-kube bug when `br_netfilter` or `xt_set` module is compiled into
+  kernel #2820/#2821
+* Detect the absence of the required `xt_set` kernel module #2821
+
+## Release 1.9.1
+
+Bug fixes and minor improvements
+
+* Fix a race condition when the Weave Net container is restarted
+  which could allow a new container to be allocated the same IP
+  address as an existing one #2784,#2787
+* Handle the message type received when a pod has been deleted during
+  Kubernetes api-server fail-over #2772,#2773
+* Make weave-kube work with `dockerd --iptables=false` #2726
+* Ensure we have the right kernel modules loaded for Network Policy in weave-kube #2819
+* Reference-count addresses in Network Policy Controller, to avoid
+  errors when updates come in an unexpected order #2792,#2795
+* Allow the soft connection limit to be raised in weave-kube, so
+  larger clusters can be created #2781
+* WeaveDNS was incorrectly case-sensitive for reverse DNS lookups #2817,#2818
+
+Build and Testing
+
+* Scripts to create VMs to run automated tests were rewritten to use
+  Terraform and Ansible, to make it much easier to test with different
+  versions of components such as Docker and Kubernetes #2647,#2694,#2775,#2796
+* Upgrade to latest Weaveworks common build-tools #2780
+* Improve encryption tests #2793
+* Update vishvananda/netlink library to bring in changes we had previously forked #2790
+* Slight change to the build container to avoid permission errors and slow builds #2761,#2802
+
 ## Release 1.9.0
 
 Highlights:

DEPENDENCIES

@@ -0,0 +1,3 @@
+DOCKER_VERSION=1.11.2
+KUBERNETES_VERSION=1.5.2
+KUBERNETES_CNI_VERSION=0.3.0.1

Makefile

@@ -173,6 +173,8 @@ ifeq ($(BUILD_IN_CONTAINER),true)
 # It bind-mounts the source into the container and passes all important variables
 exes $(EXES) tests lint: $(BUILD_UPTODATE)
 	git submodule update --init
+# Containernetworking has another copy of vishvananda/netlink which leads to duplicate definitions
+	-@rm -r vendor/github.com/containernetworking/cni/vendor
 	@mkdir -p $(shell pwd)/.pkg
 	$(SUDO) docker run $(RM) $(RUN_FLAGS) \
 	    -v $(shell pwd):/go/src/github.com/weaveworks/weave \
@@ -214,7 +216,7 @@ tests:
 	./tools/test -no-go-get -netgo -timeout 8m
 
 lint:
-	./tools/lint -nocomment -notestpackage .
+	./tools/lint -nocomment -notestpackage
 
 endif
 
@@ -359,3 +361,31 @@ build:
 
 run-smoketests: all testrunner
 	cd test && ./setup.sh && ./run_all.sh
+
+integration-tests: all testrunner
+# Usage:
+#   $ make \
+#     NAME="<prefix used to name VMs and other resources>" \
+#     PROVIDER="<provider among {vagrant|gcp|aws|do}>" \
+#     NUM_HOSTS="<# test machines>" \
+#     PLAYBOOK="<filename>" \
+#     RUNNER_ARGS="<...>" \
+#     TESTS="<...>" \  # Can be set to only run one or a few tests instead of the full test suite.
+#     DOCKER_VERSION=<...> \
+#     KUBERNETES_VERSION=<...> \
+#     KUBERNETES_CNI_VERSION=<...> \
+#     <...> # See also run-integration-test.sh for all variables and individual functions.
+#     integration-tests
+#
+	RUNNER_ARGS="-parallel" ./test/run-integration-tests.sh
+
+# This target is only intended for use in Netlify CI environment for generating preview pages on feature branches and pull requests.
+# We need to obtain website repository (checked out under `site-build`) and place `site` directory into the context (`site-build/_weave_net_docs`).
+# We then run make in `site-build` and Netlify will publish the output (`site-build/_site`).
+netlify-site-preview:
+	@mkdir -p site-build
+	@curl --user $(WEBSITE_GITHUB_USER) --silent 'https://codeload.github.com/weaveworks/website-next/tar.gz/$(WEBSITE_BRANCH)' \
+	  | tar --extract --gunzip --directory site-build --strip 1
+	@cp -r site site-build/_weave_net_docs
+	@$(MAKE) -C site-build netlify_ensure_install
+	@$(MAKE) -C site-build BUILD_ENV=netlify

README.md

@@ -23,5 +23,20 @@ Weave Net creates a virtual network that connects Docker containers across multi
 
 Services provided by application containers on the Weave network can be exposed to the outside world, regardless of where they are running. Similarly, existing internal systems can be opened to accept connections from application containers irrespective of their location.
 
-* [Install Weave Net](https://www.weave.works/docs/net/latest/installing-weave/)
-* [Find out more about Weave Net](https://www.weave.works/products/weave-net/)
+## <a name="help"></a>Getting help
+
+If you have any questions about, feedback for or a problem with Weave Net:
+
+- Read [the Weave Net docs](https://www.weave.works/docs/net/latest/introducing-weave/).
+- Invite yourself to the <a href="https://weaveworks.github.io/community-slack/" target="_blank"> #weave-community </a> slack channel.
+- Ask a question on the <a href="https://weave-community.slack.com/messages/general/"> #weave-community</a> slack channel.
+- Join the <a href="https://www.meetup.com/pro/Weave/"> Weave User Group </a> and get invited to online talks, hands-on training and meetups in your area.
+- Send an email to <a href="mailto:weave-users@weave.works">weave-users@weave.works</a>
+- <a href="https://github.com/weaveworks/weave/issues/new">File an issue.</a>
+
+Your feedback is always welcome!
+
+##Further Information
+
+* [Installing Weave Net](https://www.weave.works/docs/net/latest/installing-weave/)
+* [Weave Net Product Overview](https://www.weave.works/products/weave-net/)

Vagrantfile

@@ -1,45 +1,36 @@
-VAGRANTFILE_API_VERSION = "2"
-
-require './vagrant-common.rb'
+require File.expand_path(File.join(File.dirname(__FILE__), 'vagrant-common.rb'))
 
 vm_ip = "172.16.0.3" # arbitrary private IP
 
-pkgs = %w(
-  docker-engine=1.10.2-0~wily
-  aufs-tools
-  build-essential
-  ethtool
-  iputils-arping
-  libpcap-dev
-  git
-  mercurial
-  bc
-  jq
-)
-
 Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
 
-  config.vm.box = "ubuntu/wily64"
+  config.vm.box = VAGRANT_IMAGE
 
   config.vm.network "private_network", ip: vm_ip
   config.vm.provider :virtualbox do |vb|
     vb.memory = 2048
-    vb.customize ["modifyvm", :id, "--natdnshostresolver1", "off"]
-    vb.customize ["modifyvm", :id, "--natdnsproxy1", "off"]
+    configure_nat_dns(vb)
   end
 
+  # Disable default Vagrant shared folder, which we don't need:
   config.vm.synced_folder ".", "/vagrant", disabled: true
+  # Keep Weave Net sources' in sync:
   config.vm.synced_folder ".", "/home/vagrant/src/github.com/weaveworks/weave"
+  # Create a convenience symlink to $HOME/src/github.com/weaveworks/weave
+  config.vm.provision :shell, :inline => 'ln -sf ~vagrant/src/github.com/weaveworks/weave ~vagrant/'
 
   # Set SSH keys up to be able to run smoke tests straightaway:
   config.vm.provision "file", source: "~/.vagrant.d/insecure_private_key", destination: "/home/vagrant/src/github.com/weaveworks/weave/test/insecure_private_key"
-
-  install_build_deps config.vm, pkgs
-  install_go_toochain config.vm
-  tweak_user_env config.vm
-  tweak_docker_daemon config.vm
+  # Grant permissions on sources:
+  config.vm.provision :shell, :inline => 'sudo chown -R vagrant:vagrant ~vagrant/src', :privileged => false
   cleanup config.vm
 
+  config.vm.provision 'ansible' do |ansible|
+    ansible.playbook = 'tools/config_management/setup_weave-net_dev.yml'
+    ansible.extra_vars = {
+      go_version: GO_VERSION
+    }.merge(ansibleize(get_dependencies_version_from_file_and_env()))
+  end
 end
 
 begin

bin/circle-dependencies-post

@@ -2,45 +2,16 @@
 
 set -e
 
-echo "TEST_AND_PUBLISH=1" > "$STATE"
-
-# Avoid doing a full build if we can.  Note if the PR doesn't exist
-# when the build is kicked off, we can't run this check so we do a
-# full build.  Subsequent pushes to the same branch will have the desired behaviour
-if [ -n "$CI_PULL_REQUEST" -a "$CI_PULL_REQUEST" = "$CI_PULL_REQUESTS" ] ; then
-    if PR=$(echo "$CI_PULL_REQUEST" | grep -oP '(?<=^https://github.com/weaveworks/weave/pull/)[0-9]+$') ; then
-        echo "GH_PR=$PR" >> "$STATE"
-        if FILES=$(curl -s https://api.github.com/repos/weaveworks/weave/pulls/$PR/files | jq -r '.[]|.filename') ; then
-            echo "PR touches [$FILES]"
-            if ! echo "$FILES" | grep -v ^site/ ; then
-                echo "PR contains site changes only - skipping tests"
-                echo "TEST_AND_PUBLISH=" >> "$STATE"
-            else
-                echo "PR contains non-site changes - running tests"
-            fi
-        else
-            echo "Unable to get files for PR $PR" >&2
-        fi
-    else
-        echo "Unable to extract PR number from $CI_PULL_REQUEST" >&2
-    fi
-fi
-
 source "$STATE"
 
-mkdir -p $(dirname $SRCDIR) && cp -r $(pwd)/ $SRCDIR
-
-if [ -n "$TEST_AND_PUBLISH" ] ; then
-    [ -z "$SECRET_PASSWORD" ] || bin/setup-circleci-secrets "$SECRET_PASSWORD"
-    cd $SRCDIR; git submodule update --init
-    if [ "$CIRCLE_NODE_INDEX" = "0" -a -n "$SECRET_PASSWORD" ] ; then
-        cd $SRCDIR/test; ./gce.sh make_template
-    fi
-    cd $SRCDIR/build
+if [ -n "$TEST_AND_PUBLISH" ]; then
+    cd "$SRCDIR"
+    git submodule update --init
+    cd "$SRCDIR/build"
     ../tools/rebuild-image weaveworks/weavebuild . Dockerfile build.sh
-    touch $SRCDIR/.build.uptodate
+    touch "$SRCDIR/.build.uptodate"
     sudo chown -R ubuntu /usr/local/go
-    cd $SRCDIR
+    cd "$SRCDIR"
     make testrunner
     make COVERAGE=true RM= exes all
 fi

bin/circle-dependencies-post-sources

@@ -0,0 +1,36 @@
+#!/bin/bash
+
+set -e
+
+echo "TEST_AND_PUBLISH=1" >"$STATE"
+
+# Avoid doing a full build if we can.  Note if the PR doesn't exist
+# when the build is kicked off, we can't run this check so we do a
+# full build.  Subsequent pushes to the same branch will have the desired behaviour
+if [ -n "$CI_PULL_REQUEST" -a "$CI_PULL_REQUEST" = "$CI_PULL_REQUESTS" ]; then
+    if PR=$(echo "$CI_PULL_REQUEST" | grep -oP '(?<=^https://github.com/weaveworks/weave/pull/)[0-9]+$'); then
+        echo "GH_PR=$PR" >>"$STATE"
+        if FILES=$(curl -s "https://api.github.com/repos/weaveworks/weave/pulls/$PR/files" | jq -r '.[]|.filename'); then
+            echo "PR touches [$FILES]"
+            if ! echo "$FILES" | grep -v ^site/; then
+                echo "PR contains site changes only - skipping tests"
+                echo "TEST_AND_PUBLISH=" >>"$STATE"
+            else
+                echo "PR contains non-site changes - running tests"
+            fi
+        else
+            echo "Unable to get files for PR $PR" >&2
+        fi
+    else
+        echo "Unable to extract PR number from $CI_PULL_REQUEST" >&2
+    fi
+fi
+
+source "$STATE"
+
+mkdir -p "$(dirname "$SRCDIR")" && cp -r "$(pwd)/" "$SRCDIR"
+
+if [ -n "$TEST_AND_PUBLISH" ]; then
+    cd "$SRCDIR"
+    git submodule update --init
+fi

bin/circle-deploy-issues

@@ -1,43 +0,0 @@
-#!/bin/bash
-
-set -e
-set -o pipefail
-
-source "$STATE"
-
-site_changes() {
-    curl -s https://api.github.com/repos/weaveworks/weave/pulls/$1/files |
-        jq -r '.[]|.status + " " + .filename' |
-        grep -oP '^(added|modified|removed) site/.*(?=\.md$)'
-}
-
-format_comment() {
-    if [ ! -d .git ] || ! BRANCH=$(git symbolic-ref --short HEAD) || [ -z "$BRANCH" ] ; then
-        echo "Could not determine branch" >&2
-        return 1
-    fi
-
-    echo "#### Documentation changes:"
-    while read ACTION PAGE; do
-        echo "$ACTION [${PAGE#site/}]($WP_DEV_URL/docs/net/${BRANCH#issues/}/${PAGE#site/}/)"
-    done | sort
-}
-
-post_comment() {
-    jq -s -R '. | tostring | { body: . }' |
-        curl -s -d @- \
-            -H "Authorization: Bearer $WP_GH_COMMENT_TOKEN" \
-            -H 'Content-Type: application/json' \
-            https://api.github.com/repos/weaveworks/weave/issues/$GH_PR/comments > /dev/null
-}
-
-cd $SRCDIR
-bin/install-wordepress v1.0.0
-bin/publish-site "$WP_DEV_URL" "$WP_DEV_USER" "$WP_DEV_PASSWORD"
-
-if [ -n "$WP_GH_COMMENT_TOKEN" -a -n "$GH_PR" ] ; then
-    echo ">>> Adding doc changelog to GitHub PR $GH_PR"
-    if CHANGES=$(site_changes $GH_PR) && [ -n "$CHANGES" ] ; then
-        echo "$CHANGES" | format_comment | post_comment
-    fi
-fi

bin/circle-deploy-master

@@ -11,7 +11,3 @@ if [ -n "$TEST_AND_PUBLISH" ] ; then
     # CircleCI doesn't let us install the qemu module to do ARM docker builds
     make publish ALL_ARCHITECTURES=amd64 UPDATE_LATEST=latest-only
 fi
-
-cd $SRCDIR
-bin/install-wordepress v1.0.0
-bin/publish-site "$WP_DEV_URL" "$WP_DEV_USER" "$WP_DEV_PASSWORD"

bin/circle-deploy-release

@@ -1,7 +0,0 @@
-#!/bin/bash
-
-set -e
-
-cd $SRCDIR
-bin/install-wordepress v1.0.0
-bin/publish-site "$WP_LIVE_URL" "$WP_LIVE_USER" "$WP_LIVE_PASSWORD"

bin/circle-test-pre

@@ -2,9 +2,29 @@
 
 set -e
 
-source "$STATE"
+# Signal failures in lock file, in order to fail fast:
+function signal_failure() {
+    echo "KO" >"$TEST_VMS_READY_LOCK_FILE"
+    exit 1
+}
+trap signal_failure ERR
 
-if [ -n "$TEST_AND_PUBLISH" ] ; then
-    cd $SRCDIR/test
-    [ -z "$SECRET_PASSWORD" ] || (./gce.sh setup && eval $(./gce.sh hosts) && ./setup.sh)
+source "$STATE"
+source "$(dirname "$0")/wait_for.sh"
+
+if [ -n "$TEST_AND_PUBLISH" ]; then
+    [ -n "$SECRET_KEY" ] || {
+        echo "Cannot run smoke tests: no secret key"
+        exit 1
+    }
+
+    # Provisioning from image typically take 90 seconds, 
+    # but if we are creating a GCP image, it can take more than 10 minutes:
+    wait_for 900 "$TEST_VMS_PROV_AND_CONF_LOCK_FILE"
+
+    # Upload built binaries to testing VMs:
+    cd "$SRCDIR/test" # Ensures we generate Terraform state files in the right folder, for later use by integration tests.
+    ./run-integration-tests.sh setup >>"$TEST_VMS_SETUP_OUTPUT_FILE" 2>&1
+    echo "OK" >"$TEST_VMS_READY_LOCK_FILE"
+    echo "Test VMs now ready. $(date)." >>"$TEST_VMS_SETUP_OUTPUT_FILE"
 fi

bin/circle-test-smoke

@@ -3,12 +3,21 @@
 set -e
 
 source "$STATE"
+source "$(dirname "$0")/wait_for.sh"
 
-if [ -n "$TEST_AND_PUBLISH" ] ; then
-    [ -n "$SECRET_PASSWORD" ] || { echo "Cannot run smoke tests: no secret key"; exit 1; }
-    cd $SRCDIR/test
-    eval $(./gce.sh hosts)
+if [ -n "$TEST_AND_PUBLISH" ]; then
+    [ -n "$SECRET_KEY" ] || {
+        echo "Cannot run smoke tests: no secret key"
+        exit 1
+    }
+
+    # Provisioning from image typically take 90 seconds, 
+    # but if we are creating a GCP image, it can take more than 10 minutes:
+    wait_for 900 "$TEST_VMS_READY_LOCK_FILE"
+
+    # Run integration tests:
     export COVERAGE=true
     export WEAVE_NET_SANITY_CHECKS_FILES="$CIRCLE_ARTIFACTS/weave_net_sanity_check_*.log"
-    ./run_all.sh
+    cd "$SRCDIR/test" # Ensures we generate code coverage files in the right folder, and use the Terraform state of previously provisionned VMs.
+    ./run-integration-tests.sh test
 fi

bin/circle-test-teardown

@@ -6,6 +6,5 @@ source "$STATE"
 
 if [ -n "$TEST_AND_PUBLISH" ] ; then
     cd $SRCDIR/test
-    [ -z "$SECRET_PASSWORD" ] || ./gce.sh destroy
     test "$CIRCLE_NODE_INDEX" != "0" || (cd $SRCDIR/test; ./gen_coverage_reports.sh)
 fi

bin/install-wordepress

@@ -1,12 +0,0 @@
-#!/bin/bash
-
-if [ $# -ne 1 ] ; then
-    echo USAGE: install-wordepress [COMMIT-ISH] >&2
-    exit 1
-fi
-
-go get github.com/weaveworks/wordepress
-cd /home/ubuntu/src/github.com/weaveworks/wordepress
-git checkout $1
-cd cmd/wordepress
-go get

bin/provision_test_vms.sh

@@ -0,0 +1,41 @@
+#!/bin/bash
+
+set -e
+
+# Signal failures in lock file, in order to fail fast:
+function signal_failure() {
+    echo "KO" >"$TEST_VMS_PROV_AND_CONF_LOCK_FILE"
+    exit 1
+}
+trap signal_failure ERR
+
+source "$STATE"
+
+function install_terraform() {
+    curl -fsS https://releases.hashicorp.com/terraform/0.8.5/terraform_0.8.5_linux_amd64.zip | gunzip >terraform && chmod +x terraform && sudo mv terraform /usr/bin
+}
+
+function install_ansible() {
+    sudo apt-get update || true
+    sudo apt-get install -qq -y python-pip python-dev libffi-dev libssl-dev && pip install --user -U cffi && pip install --user ansible
+    export PATH="$PATH:$HOME/.local/bin"
+}
+
+if [ -n "$TEST_AND_PUBLISH" ]; then
+    [ -n "$SECRET_KEY" ] || {
+        echo "Cannot run smoke tests: no secret key"
+        exit 1
+    }
+
+    install_terraform >>"$TEST_VMS_SETUP_OUTPUT_FILE" 2>&1
+    install_ansible >>"$TEST_VMS_SETUP_OUTPUT_FILE" 2>&1
+
+    # Only attempt to create GCP image in first container, wait for it to be created otherwise:
+    [ "$CIRCLE_NODE_INDEX" != "0" ] && export CREATE_IMAGE=0
+
+    # Provision and configure testing VMs:
+    cd "$SRCDIR/test" # Ensures we generate Terraform state files in the right folder, for later use by integration tests.
+    ./run-integration-tests.sh configure >>"$TEST_VMS_SETUP_OUTPUT_FILE" 2>&1
+    echo "OK" >"$TEST_VMS_PROV_AND_CONF_LOCK_FILE"
+    echo "Test VMs now provisioned and configured. $(date)." >>"$TEST_VMS_SETUP_OUTPUT_FILE"
+fi

bin/publish-site

@@ -1,46 +0,0 @@
-#!/bin/bash
-
-set -e
-set -o pipefail
-
-PRODUCT=net
-
-fatal() {
-    echo "$@" >&2
-    exit 1
-}
-
-if [ ! -d .git ] ; then
-    fatal "Current directory is not a git clone"
-fi
-
-if ! BRANCH=$(git symbolic-ref --short HEAD) || [ -z "$BRANCH" ] ; then
-    fatal "Could not determine branch"
-fi
-
-case "$BRANCH" in
-    issues/*)
-        VERSION="${BRANCH#issues/}"
-        TAGS="$VERSION"
-        ;;
-    *)
-        if echo "$BRANCH" | grep -qE '^[0-9]+\.[0-9]+' ; then
-            DESCRIBE=$(git describe --match 'v*')
-            if ! VERSION=$(echo "$DESCRIBE" | grep -oP '(?<=^v)[0-9]+\.[0-9]+\.[0-9]+') ; then
-                fatal "Could not infer latest $BRANCH version from $DESCRIBE"
-            fi
-            TAGS="latest"
-        else
-            VERSION="$BRANCH"
-            TAGS="$VERSION"
-        fi
-        ;;
-esac
-
-for TAG in $TAGS ; do
-    echo ">>> Publishing $PRODUCT $VERSION to $1/docs/$PRODUCT/$TAG"
-    wordepress \
-        --url "$1" --user "$2" --password "$3" \
-        --product "$PRODUCT" --version "$VERSION" --tag "$TAG" \
-        publish site
-done

bin/release

@@ -89,6 +89,8 @@ build() {
     ## Inject the version numbers and build the distributables
     ## (library versions?)
     sed -i "/SCRIPT_VERSION=/ c\SCRIPT_VERSION=\"$VERSION\"" ./weave
+    sed -i -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset.yaml
+    sed -i -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset-k8s-1.6.yaml
     make SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER
 
     if make tests; then
@@ -157,17 +159,9 @@ draft() {
         --name "weave" \
         --file "./weave"
 
-    github-release release $RELEASE_ARGS \
-        --user $GITHUB_USER \
-        --repo weave-kube \
-        --tag $LATEST_TAG \
-        --name "$RELEASE_NAME $VERSION" \
-        --description "$RELEASE_DESCRIPTION"
-
-    sed -i -e s/:latest/:$VERSION/ -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset.yaml
     github-release upload \
         --user $GITHUB_USER \
-        --repo weave-kube \
+        --repo weave \
         --tag $LATEST_TAG \
         --name "weave-daemonset.yaml" \
         --file "./prog/weave-kube/weave-daemonset.yaml"
@@ -176,8 +170,8 @@ draft() {
         --user $GITHUB_USER \
         --repo weave \
         --tag $LATEST_TAG \
-        --name "weave-daemonset.yaml" \
-        --file "./prog/weave-kube/weave-daemonset.yaml"
+        --name "weave-daemonset-k8s-1.6.yaml" \
+        --file "./prog/weave-kube/weave-daemonset-k8s-1.6.yaml"
 
     echo "** Draft $TYPE $RELEASE_NAME $VERSION created at"
     echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/$LATEST_TAG"
@@ -258,29 +252,9 @@ publish() {
             --name "weave" \
             --file "./weave"
 
-        github-release publish \
-            --user $GITHUB_USER \
-            --repo weave-kube \
-            --tag $LATEST_TAG
-
-        if github-release info --user $GITHUB_USER --repo weave-kube \
-            --tag latest_release >/dev/null 2>&1; then
-            github-release delete \
-                --user $GITHUB_USER \
-                --repo weave-kube \
-                --tag latest_release
-        fi
-
-        github-release release \
-            --user $GITHUB_USER \
-            --repo weave-kube \
-            --tag latest_release \
-            --name "$RELEASE_NAME latest ($VERSION)" \
-            --description "[Release Notes](https://github.com/$GITHUB_USER/weave/releases/$LATEST_TAG)"
-
         github-release upload \
             --user $GITHUB_USER \
-            --repo weave-kube \
+            --repo weave \
             --tag latest_release \
             --name "weave-daemonset.yaml" \
             --file "./prog/weave-kube/weave-daemonset.yaml"
@@ -289,8 +263,8 @@ publish() {
             --user $GITHUB_USER \
             --repo weave \
             --tag latest_release \
-            --name "weave-daemonset.yaml" \
-            --file "./prog/weave-kube/weave-daemonset.yaml"
+            --name "weave-daemonset-k8s-1.6.yaml" \
+            --file "./prog/weave-kube/weave-daemonset-k8s-1.6.yaml"
 
         echo "** Release $RELEASE_NAME $VERSION published at"
         echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/$LATEST_TAG"

bin/setup-circleci-secrets

@@ -1,115 +0,0 @@
-#!/bin/bash
-
-set -eu
-
-# openssl enc -in do-setup-circleci-secrets.orig -out setup-circleci-secrets.orig -e -aes256 -pass stdin
-# openssl base64 < setup-circleci-secrets.orig
-
-openssl base64 -d << EOF \
-  | openssl enc \
-    -out bin/do-setup-circleci-secrets \
-		-d -aes256 -pass pass:$1
-U2FsdGVkX1/3QHnDgpZn0srp+GiEQBTlag0CBdsw6Bp6iaIKy6Jy8gFshrsmvl3+
-c+7hn1nqsBVpzVH0S5QA3Ex5nx9rl7/dcGoogsySFcwMmBFcauT0UTMK1bvX0Hqx
-QrshYSndUkww72KA7EsSnjR1BBJWm3AhCDOTIcnO6nQhig1Fc+SBEj5FGS5xTQ5Z
-/8hDsKf54Ms49vOO6QeE5w5ndZ+k7IvVJfq9Evhb7htD20RntDhR86foAwMnLV7R
-Wh9qAqI3gnRq0GmbeRKaHmB7pcLq9f0OdScqM/Tn1D+q0556fqJjBziXpZiGbxVs
-DxEp8zDJ8yNDAi0Wb2NXjoA79SXJ8Q7iTieaST4y2OmXQpY+Vw8RVDXxi6ERyVnH
-41OeMzyLm0qvCoMl32pe1sSG5DRCUiAm7wOtDTVvNDRqmGJYUgwwexyayecJI1wQ
-LlhYLJavUvw3UGDZOld8BaMQhfOKBVCdFrjmA4HuVxmOf3029qkgzOCSZiQVNqs+
-fvM6J05zBuk+GPy8YInvRkzfxum/x00vmUyGjMs/lHDFfrmYDFK3ux41EoVqvWLG
-S5uYxdZJFdR3Xsrs+sMr51XNBQGMmz6KoEEzFYyHs5fZgtdpRdPAuQS1m6FqgZyi
-wSz6+EKsALtmGorEp+CF8JN5YzKJCrKzNkg0fILBqsqmJZGP64lmX69950lRLNGM
-/l3KVR2zH31b4AnnzTcvldVSCKDXjaMw3JA2Ce/MK6XhOWOcpEv4oDt/62lbhUKP
-QUQTPbfpQRvVfBcoIrfAm3ghi/CZc2TDVEuD7DhasLTZD3r7LoK261BWC9O5Wubi
-vlXuFIF+uyrpTDf6EJKE/+XywtArppffMP6RkHu1Iqnd8JtsqPaIZsmMmZbjgb6j
-BX4ZLb5U9R7jYoSgIke4+u6geFcrMQ+Lny4r7MxXaS0Y1UJ6Jo1bwpYw1TBIobZc
-eQ30+xhbSeRC9vEw1NXI6qM2Ai2gIHtAZvEiBkBBXTG5Ld7N/mBAmDIJEzVRxh6F
-jRkB3dzESF6eeTBm61OQepwV3aW6lslMaf26drPGv2W2q+K/WOpQOejfggtHUgVs
-9SWqtn7hHYSpIASe1Ihi14HAWv3kPhQien0zY1DLKJWf4ilbRaICn1QSJjNuKwtl
-A2ZwEw7FFddFcsF4okHPkxEoCre3shq0DEw9UA1gnwEaEF1Q+i9cvLtQg56zK3+R
-MfTzFUv2K1z3j1s5tC6Z6LzL8j682EFOll7MkihqiIWh+po1zcNs6Ln0SkjB3v9T
-OFRYVJis9GqXYO9PUW1mI69udR4nXbKlBl5tWNyAxn9hufJHLSAmbGMwActKLRM5
-kROapKrN1Ra/sHUmhoSSDUK9AXeYKkqPXVMbhly9xC1Kya4hFaFokjRQIMLPmXdQ
-lP46RL3HOq0v1BkXg05poSwDm+BtitZkcIhOzcmeucUqn1whsz39y3VoOhJqQu2j
-0dUcj19Fue1CDZfwzACb9mLZr643h0yCIHQaVT7is/yiN2vEt0S0zynvOA1D+Xgy
-4bFdjqed9DgBShLwF/lh4/hur5iZPpVr9A8rt3Vq0k5A1tdjsZpXP3qv5jLc7lJv
-cAQxdVLEuv8WPwwTV/EQsoH7lpoEdjktw/gu0v5PkUgR/WBxEqJq4liTekViQ+8X
-BAmdm/FGWeIxSSEm0a8I+muiBpIuKjuQoJit1f0k+lLUohoVDqpDDfeYHh1TxXHE
-xSKAKNmFSc2XOr5Zg2mOInxeH7WoxH6iylXKADBgEgAQ4C55DoAPRjFNtGGz2iJl
-WkzhQOfgsDss1VxpPZ4SWDQO+xxuuEnYcCYdJ6Tyi5hAwZ0+RZmXx0g9BCK+5qzc
-Ffei066kCItBcayekwCs0i86r8JvDtsMUUkzEKDQTSWHpP8ypIZoz3P+Skdqcs6p
-IcZHy2fODIsClctRAsHOjZ9BnWSY2rqf1lysqvRfOkNTViTOUVx2hf+LuL+8rT6g
-K4GvS39Q+c9FYSMjCNxv1m37NfHJv2W7AlqVV95gMBEFB97M3388T7PfvmSLsyPv
-qAKDE7genTOycQvEX9oeEACxPo08ai2Mgs/YXCig0g0btZoScoKPelQKtD0HLpZ+
-j7ruaKFy4/vySDMostB3AKSkCE/SkGu/W9oI37n1guSYC/9yJjtTcwZULXAClfwt
-LyHvLAw4+7BdGJqDe9Tf8z4zqVLax+BcBHkW7pbP+pvxC6fhIA4PO8k5HwwAVlRw
-68lG6e3z9+Vst4zrFLewHJvkDRSOTSecs9h9SPOjReTK0ovlDzMHVakfUvqpk9mv
-nCj8SWaMPF70z138AuRuKN7X4exNXPBFnOcseKp/NpT1DyvgoHm/oIO65mq9vvo+
-42uSbyyegYiTdVK4SaXBc2hixkhaTteqdEy6GxfWqKCvHtYuRD2LyUSjZn/0Y6CG
-3wZapylKL1sxfH+b1gogv2wAA8KtZAfiTlMISuymwQr5RbOFma7UVMVecCa7XrNG
-fZwO8R+6++ru0efDzeC/gtLOItjn+yVfsbk2gfu0FikHIV1sogOjnsv3dl6IpH/s
-w6WbuK/yZhyrrmhIIueHF30ES512Peogf/Gxp8U/ZAmqBa7p4qsv93XWDR52IPYz
-yU1MtboLD02J39daBNrYZuO4Yfo/xAvU7Ijg2KIR9HS0bSWNuSFAZaZXlJ2bSag3
-8l0dZMx2fZa0FALxWvY2+nd1tWcH+Wo99RSxX+4TH3guCpe1x/58yKOKL4d1xRpu
-02nGaqerV9bHU6WjCUYTrrZurTp6yqivpSFCALDMhAU5Ihd/Qjz69zg4uZ7iqbNd
-WqIYJ3SAOei2tSfoOJbdh8mohotnMFFbupck3Fv+lXAXYDp0V/sSuK32MSeJxan1
-q3t0eenAh6zoBP0Tj53N8B9YmrgZ1i1AljQqYsAdb8N5Z3vecuzQa946U8H4axEc
-BL5fgmqSfsbj1EjJVUrw3cE9hgqeqAYBBvZsyW5zSAsLq/RJKYtBxDGADqr0TOOu
-TjVqfPObb1u9EAQj15ALBhhPORa2gBAK5gnhJWW5deNgA0UOYoWARFYfATuQvr1T
-9xTVbqzZ0VJR4n7X5JyxJo2tZ44QYVzA0fqX0lt+uRkarrkbZMgplKp4MJTFCp2N
-tLciWS0YSHPbKTm7svw5mROazUvEoj1N3SYrJT+Ai9UgSL1cA4FmQ5Jsi+GOeQbH
-7E4pzjeST4b1aATY9ZZSy4TzK+UNYBcCVqY1dN5Xnk0e5yPMnrkkcYI41s/6oOaW
-lQkp/yrHkvpzQWZa7vF2yWg54j3eX8YWx/TsLvmuJmSQqiIHfu9TXcAIPDispCpF
-gp4ZbYPZ8RDQLw3ROc+bZ9E0J8e+FHwgXEELsMLW9n4d3PTj9pLFej1vmBKa03o3
-g+JViNxb1gUPNmC6ADk4o7TQzBrjV2pmmuCza16kOmq6oAbJPOeCLtVWA5HJTX8d
-AuOzBLO37tWP457dNqoNDaCMik3LCy8cslZjAz02D6VG/QGOAXP9x8bJ3zg9N7vW
-EOn1BMtIeMI4yIxpC6HgotNWDNbUBWremAFWYQRgYQqo9L1SdW4hz6X8FqdlBFvK
-ml7sdUwjEy2tNW+aiZI4cgAlI/VvMWYr5kPfhvaUu1wGYeS1TD1B+DPcjcXpOGWo
-vB1f/uvGc61zZWD0ph9f3S9OnvEko5qV3VhxOnxYUZTrqlM7LVgy/sHrSaPdR7Eo
-nb4QGdi5wEGMxVgQlDGwOmEmS5nCaA0RcfHbkQV4SebRleocLB9qcSwj5JzPICAB
-sn1qeSv45LnGdTzUnCQErw1EzgrK1FXOknujjZ3GX5dM7a6Gag0E6cAxoQeC8Mh0
-aUWolyBo9aSXltn7Vnywoe4X3v96eVH6tUQtN3wt51LBlfSDTWOQDeXLFyW57oYL
-/ErbmJLaSHpANBcC9InI+rxNYePTTba7K8rdPDppmMEmdOXjaL+5aPay2Bay9K8T
-FUyZdoyrbzxFZYT9/YyVJV/ukVcT9ru3HpzLTg9Vgc/CyRfIvCZY/Gzh+pPUbRsR
-pHYDT09BFS+mjszTkcdbceweYXJvknXpajMiISrPUUkb9rwCTmLS4QR4y7I9zAgJ
-9U72mCZB9yhhzS17kyKAANihXvOFrMXCO7QTWZbtSkobMBS2SybDxYP3rM5b6824
-HQaY4MZW5qCndydfz4fa908XbPTDka9Nu3CZfI3fhGiiGDFWViotGEYE4i9BuGZJ
-vmtzII7zeIoHwM4t5fb5mYDd9H+Rkct1yATGEIFEx7cu1fj4Ht2177EzhdkF3Q6I
-Fccn43yGDW06nFvEjnjb+Lz4EvtGp6dDefUhNBohdzd4Rm2o+etKobm0lZ1xOX74
-2gOGBLfXS6PF+msNkWP76mqKE6hG0+R1iPrN06Arti+ZRBhNGJDp1pnOERl7IyB0
-hiLUqae7xVSMVRDeq4O04EUpMXh5ixs3RJXjthYILt7vX+6qskJC3rMuyThrsvMi
-Kk0wErDpzR5U7KFnM2fk4jY5by/0b/jHZ/PWFGnaFrXNOiwK/mV1RIJtg5FCrvMv
-xJN5Qo7o2ZYzUL+IBf5mP5O49FCnz3e++9r4ySY/K57ybhQft1l+cQPWS7xuvVAg
-wZGM9DPl9AhBus0I3V3XzfkcWWHkMynMRcfTnkBHrmKjMopIOggsq05JPalM9b0V
-NfwZngVvTFbQejbGjgQQQ+G7spav/icCIA8Iu8JPHaKZt2IX77L1xMcQOouqMrx8
-JCXnJ7zsuU+8W3GQGm70H2kcen4bf4beA8TSkFmbVsaqDNtGaG+ZopWO8ctZX3CP
-jFgIYgEGi137IWvs1KD5QNFHZI6l0JMzglmvcfnViRPDHLi+EQXAoL/98j/YzA9t
-a9L4xMO0oucovKkqYOS2cdFaQKVIUM9L5ibcbW7Tol4ugMo5ibIsTgjZoyFohG6t
-acg+h2wTTyxtrIapvOOBeAo9V/iQowTAPQf38LKQZ5tApksiCG1qvJerKALTMp1J
-yUHjTCPCPsaiXVVO0mfa9OPtzVvi/icLmlz7umk82mew13djedgMS3/ghq5aEHbg
-e7rtI5snFlm/GQnV6IaCDytb+jYxqQ0DZ9Mb0CHcMpnLyiBO6nzHfadpZYlrxrpx
-dPx9MVqfLavj1UVjKuVj4Mgec9J38IBBRGMJZ4MGhde2hrw/s2jVzG1BzaIzcvFa
-bWiJOkKKRC4BY9DBDYfae5c/obwzkxLmvkbWAVzoFXc+46K1IK6bbC2a4EerOAxc
-C7B3X8BucD8OtiPORwXgjctd9AJ7cXq0EEVGoZZ8jkILnklsK6RXsxrT4gSOGFpx
-cBU7TF4kHpH+o4KGQmjlFA2VYP1tXirjEep57LOHfL+zy+HiSXdsbexf6odkqLYA
-1RR0TzSkmqJYvP0GfHPMvp2xredx1kP934fhTIUcWvErrG3NPd/Dod4BzQye1nNK
-8Y8aFYF6fACcD4SFZ11ecK9DJmB+WXGlsxtO3w1LeEysny2lVkkH6HuMRIzPyXAM
-/sEpwh3hwPJosAh6vGM6JxuL7CXP8hC1oUvK91VR0hxeDcO/LgdcHtDnlNMqt1wC
-jvYt9Hg+3aY5GehBnQPRppnIMaGhiBrRC6HjBg8Umt6UTUr5PXNY2aia5wRcx4M6
-Tj0O0dAt8WWZ9ySgzOoq4SRoaAp6m4E7CpRNMgXH031OWEeHqtt8rsYqAcvcQLuW
-vIwts5hiXZOfgXyR6SBY3ANE1SjDY46na2G4oiG8FI7eAqYgehjHrXqCnGrzjefE
-V1eVcIr7DBYuM8wq7C9x+qdMir6RNV4MrVD+7rO/drivnd3MaqDHlG0LfMZo6GIK
-QZGIy0z1W5lOXm9x3RAz7/Ea11ynkZyPDDxBE5NRcV6JfukdjDrvsaQA/Qk2Wo7h
-PdUZjv17H4syMdap6tUDTc8Cohut6n1UxnDnhaAYEDGR3TgPqI5vEaJ+yoaM9itv
-b2I/K+0uK6GfWwca7PeTtYn92/5etO4czG99xEBFzw7tiEmBTA8KfseZIUe/i3fr
-9LO+rKEGalFB4mVTYyCKPYYx4hOIvjdCox/jEBTFyYUEhgGnZvArgw9XabvII7eM
-c1FmwnN2CM/MHUzM0Kw7cS/oaoRFcG/PqioO/krL2Bd+xThwSH4Zn+Y1UVO7aXNy
-90MOehHSc2nhN984Y2re3ffVsjPhC3KdWxr8jbho/Az65rUkYg64roiO2ZqdqRvf
-aYtX9fCe8N8SzgLTlWZt+l8sjY8HP2k9NoajphV1nG4iC/pX0ZJiVQ+VBRNaGhSl
-6IhM9gM1ZsI86cULJpOTAhNu1fZBv+Mq8IXYD8SqlNVwVtjvshtlzBN4InSGEKiO
-HM7coo7aDuu+Go5AEiIVMbR09jAa0VINsCVScfSbTFhlGRTrZr2DSO/s8WxbV+Mb
-EQAfZmyrLF71Z3MfIukg+ps5iuqFhZ+gJSXXSFwumpg=
-EOF
-
-exec sh bin/do-setup-circleci-secrets

bin/wait_for.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+
+# shellcheck disable=SC2155
+wait_for() {
+    local timeout=$1
+    local lock_file=$2
+    for i in $(seq 1 "$timeout"); do
+        if [ -f "$lock_file" ]; then
+            local status="$(cat "$lock_file")"
+            if [ "$status" == "OK" ]; then
+                echo "[$i seconds]: $lock_file found and status: $status." && return
+            else
+                echo "[$i seconds]: $lock_file found and status: $status." && return 1
+            fi
+        fi
+        if ! ((i % 10)); then echo "[$i seconds]: Waiting for $lock_file to be created..."; fi
+        sleep 1
+    done
+    echo "Timed out waiting for test VMs to be ready. See details in: $TEST_VMS_SETUP_OUTPUT_FILE" >&2
+    return 1
+}

build/Dockerfile

@@ -20,13 +20,12 @@ ENV GCC_CROSSCOMPILERS \
 
 # Temporarily add the Ubuntu repositories, because we're going to install gcc cross-compilers from there
 # Install the build-essential and crossbuild-essential-ARCH packages
-RUN echo "deb http://archive.ubuntu.com/ubuntu xenial main universe" > /etc/apt/sources.list.d/cgocrosscompiling.list \
-  && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 40976EAF437D05B5 3B4FE6ACC0B21F32 \
+RUN echo "deb http://emdebian.org/tools/debian/ jessie main" > /etc/apt/sources.list.d/cgocrosscompiling.list \
+  && curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add - \
+  && for platform in ${DEB_CROSSPLATFORMS}; do dpkg --add-architecture $platform; done \
   && apt-get update \
   && apt-get install -y build-essential \
   && for platform in ${DEB_CROSSPLATFORMS}; do apt-get install -y crossbuild-essential-${platform}; done \
-  && rm /etc/apt/sources.list.d/cgocrosscompiling.list \
-  && apt-get update \
   && apt-get clean && rm -rf /var/lib/apt/lists/*
 
 # Install some required packages
@@ -41,10 +40,16 @@ RUN apt-get update \
 		flex \
 		bison
 
+RUN curl -fsSLo shfmt https://github.com/mvdan/sh/releases/download/v1.3.0/shfmt_v1.3.0_linux_amd64 && \
+	echo "b1925c2c405458811f0c227266402cf1868b4de529f114722c2e3a5af4ac7bb2  shfmt" | sha256sum -c && \
+	chmod +x shfmt && \
+	mv shfmt /usr/bin
+
 # Install common Go tools
 RUN go get \
 	github.com/golang/lint/golint \
 	github.com/fzipp/gocyclo \
+	github.com/fatih/hclfmt \
 	github.com/client9/misspell/cmd/misspell
 
 # First clean, because we're gonna rebuild the std both with -race enabled and disabled
@@ -60,6 +65,9 @@ RUN for platform in ${GO_CROSSPLATFORMS}; do \
 		GOOS=${platform%/*} GOARCH=${platform##*/} \
 			go install -tags netgo std; done
 
+# Allow full write access to the Go folders for anyone
+RUN chmod -R a+w /usr/local/go
+
 # The libpcap version from Debian packages is 1.6.2, matching that version here although newer versions of libpcap have been released
 # We have to cross-compile the libpcap library for the new architectures, that's what we're doing here
 ENV LIBPCAP_CROSS_VERSION=1.6.2

build/build.sh

@@ -25,7 +25,4 @@ echo "weave:x:$uid:$gid::$WEAVE_SRC:/bin/sh" >>/etc/passwd
 echo "weave:*:::::::" >>/etc/shadow
 echo "weave	ALL=(ALL)	NOPASSWD: ALL" >>/etc/sudoers
 
-# Allow full write access to the Go folders for the user Weave
-chown -R $uid:$gid /usr/local/go
-
 su weave -c "PATH=$PATH make -C $WEAVE_SRC BUILD_IN_CONTAINER=false $*"

circle.yml

@@ -11,20 +11,34 @@ machine:
   environment:
     GOPATH: /home/ubuntu:$GOPATH
     SRCDIR: /home/ubuntu/src/github.com/weaveworks/weave
-    PATH: $PATH:$HOME/.local/bin
+    PATH: $PATH:$HOME/.local/bin:/opt/google-cloud-sdk/bin
     CLOUDSDK_CORE_DISABLE_PROMPTS: 1
     STATE: /home/ubuntu/state.env
+    # Base name of VMs for integration tests:
+    NAME: test-$CIRCLE_BUILD_NUM-$CIRCLE_NODE_INDEX
+    TEST_VMS_SETUP_OUTPUT_FILE: $CIRCLE_ARTIFACTS/test_vms_setup_output.txt
+    TEST_VMS_PROV_AND_CONF_LOCK_FILE: $CIRCLE_ARTIFACTS/test_vms_provisioned_and_configured.txt
+    TEST_VMS_READY_LOCK_FILE: $CIRCLE_ARTIFACTS/test_vms_ready.txt
+    # Google Cloud Platorm images for integration tests:
+    USE_IMAGE: 1
+    DISK_NAME_PREFIX: test-$CIRCLE_BUILD_NUM-0
 
 dependencies:
   cache_directories:
     - "~/docker"
   post:
+    - bin/circle-dependencies-post-sources
+    # Provision & configure testing VMs:
+    - "while [ ! -d $SRCDIR ]; do sleep 1; done; $SRCDIR/bin/provision_test_vms.sh":
+        parallel: true
+        background: true
     - bin/circle-dependencies-post
 
 test:
   pre:
     - "$SRCDIR/bin/circle-test-pre":
         parallel: true
+        background: true
   override:
     - "$SRCDIR/bin/circle-test-unit":
         parallel: true
@@ -37,22 +51,16 @@ test:
 
 teardown:
   pre:
+    # Destroy testing VMs:
+    - "cd $SRCDIR/test && ./run-integration-tests.sh destroy":
+        parallel: true
+        background: true
     - "$SRCDIR/bin/circle-teardown-pre":
         parallel: true
 
 deployment:
-  release:
-    branch: /[0-9]+\.[0-9]+/
-    owner: weaveworks
-    commands:
-      - "$SRCDIR/bin/circle-deploy-release"
   master:
     branch: master
     owner: weaveworks
     commands:
       - "$SRCDIR/bin/circle-deploy-master"
-  issues:
-    branch: /.*/
-    owner: weaveworks
-    commands:
-      - "$SRCDIR/bin/circle-deploy-issues"

docs/fastdp-crypto.md

@@ -4,14 +4,15 @@ This document describes implementation details of the fast datapath encryption.
 
 At the high level, we use the ESP protocol ([RFC 2406][esp]) in the Transport
 mode. Each packet is encrypted with AES in GCM mode ([RFC 4106][aesgcm]), with
-32byte key and 4byte salt. This combo provides the following security
+32 byte key and 4 byte salt. This combo provides the following security
 properties:
 
 * Data confidentiality.
 * Data origin authentication.
 * Integrity.
-* Anti-replay service.
-* Limited traffic flow confidentiality.
+* Anti-replay.
+* Limited traffic flow confidentiality as fast datapath VXLAN packets are fully
+  encrypted.
 
 ## Notation
 

docs/fastdp.md

@@ -52,9 +52,9 @@ overlay, which has a more sophisticated dynamic mechanism for coping
 with low path MTUs.
 
 To avoid triggering this fallback in typical deployments, the datapath
-interface is statically configured with an MTU of 1410 bytes allowing
+interface is statically configured with an MTU of 1376 bytes allowing
 it to work with most underlay network provider MTUs, including GCE at
-1460 bytes (the fifty byte difference accommodates the vxlan overhead).
+1460 bytes (the eighty four byte difference accommodates the encrypted vxlan overhead).
 This value can be overridden by setting `WEAVE_MTU` at launch if
 necessary.
 

docs/ipam.md

@@ -65,14 +65,30 @@ peer for space:
   - it will continue to ask for space until it receives some, or its
     copy of the ring tells it all peers are full in that subnet.
 
+### Data persistence
+
+Key IPAM data is saved to disk, in a [BoltDB](https://github.com/boltdb/bolt)
+file, stored within a [data volume container](https://docs.docker.com/engine/userguide/containers/dockervolumes/#creating-and-mounting-a-data-volume-container)
+named `weavedb`.
+
+This file is used to persist data for various Weave Net components; for IPAM it contains:
+
+* the division of the IP allocation range amongst peers, and
+* allocation of addresses to containers on the local peer,
+
+so that it is immediately available when the peer restarts.
+
 ### Claiming an address
 
-If a Weave process is restarted, in most cases it will hear from
-another peer which ranges it used to own, but it needs to know which
-individual IP addresses are assigned to containers in order to avoid
-giving the same address out in subsequent allocation requests. The
-weave script invokes the `claim` command in `populate_ipam` to do
-this.
+If a Weave process is restarted,
+
+- if `weavedb` is present, then it loads persisted IPAM data from there, as
+described, in the previous section;
+- else, it learns from other peers which ranges it used to own.
+
+Weave Net then consults the allocator to know which individual IP addresses
+are assigned to containers, and therefore avoid giving the same address out in
+subsequent allocation requests.
 
 When the Allocator is told to claim an address, there are four
 possibilities:

docs/release-process.md

@@ -138,6 +138,8 @@ Finally, for **Mainline** releases only:
   published to https://www.weave.works
 * Add the new version of `weave-net` to the checkpoint system at
   https://checkpoint-api.weave.works/admin
+* File a PR to update the version of the daemonset at
+  https://github.com/kubernetes/kops/tree/master/upup/models/cloudup/resources/addons/networking.weave
 
 ## Troubleshooting
 

docs/weavenpc-design.md

@@ -19,6 +19,9 @@ NetworkPolicy object updates from the k8s API server:
   network policy, containing the IP addresses of all pods in the
   namespace whose labels match that selector
 
+IPsets are implemented by the kernel module `xt_set`, without which
+weave-npc will not work.
+
 ipset names are generated deterministically from a string
 representation of the corresponding label selector. Because ipset
 names are limited to 31 characters in length, this is done by taking a
@@ -74,8 +77,10 @@ iptables -A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-INGRESS
 
 To direct traffic into the policy engine:
 
+```
 iptables -A FORWARD -o weave -m physdev ! --physdev-out vethwe-bridge -j WEAVE-NPC
 iptables -A FORWARD -o weave -m physdev ! --physdev-out vethwe-bridge -j DROP
+```
 
 Note this only affects traffic which egresses the bridge on a physical
 port which is not the Weave Net router - in other words, it is
@@ -94,6 +99,9 @@ The following traffic is NOT affected:
   (e.g. kubelet health checks)
 * Traffic routed from an application container to the internet
 
+The above mechanism relies on the kernel module `br_netfilter` being
+loaded and enabled via `/proc/sys/net/bridge/bridge-nf-call-iptables`.
+
 See these resources for helpful context:
 
 * http://ebtables.netfilter.org/br_fw_ia/br_fw_ia.html

ipam/allocate.go

@@ -14,9 +14,9 @@ type allocateResult struct {
 
 type allocate struct {
 	resultChan       chan<- allocateResult
-	ident            string
+	ident            string       // a container ID, something like "weave:expose", or api.NoContainerID
 	r                address.CIDR // Subnet we are trying to allocate within
-	isContainer      bool
+	isContainer      bool         // true if ident is a container ID
 	hasBeenCancelled func() bool
 }
 

ipam/allocator.go

@@ -76,6 +76,13 @@ type Allocator struct {
 	now               func() time.Time
 }
 
+// PreClaims are IP addresses discovered before we could initialize IPAM
+type PreClaim struct {
+	Ident       string // a container ID, something like "weave:expose", or api.NoContainerID
+	IsContainer bool   // true if Ident is a container ID
+	Cidr        address.CIDR
+}
+
 type Config struct {
 	OurName     mesh.PeerName
 	OurUID      mesh.PeerUID
@@ -83,6 +90,7 @@ type Config struct {
 	Seed        []mesh.PeerName
 	Universe    address.CIDR
 	IsObserver  bool
+	PreClaims   []PreClaim
 	Quorum      func() uint
 	Db          db.DB
 	IsKnownPeer func(name mesh.PeerName) bool
@@ -123,6 +131,12 @@ func NewAllocator(config Config) *Allocator {
 		dead:        make(map[string]time.Time),
 		now:         time.Now,
 	}
+
+	alloc.pendingClaims = make([]operation, len(config.PreClaims))
+	for i, c := range config.PreClaims {
+		alloc.pendingClaims[i] = &claim{ident: c.Ident, cidr: c.Cidr}
+	}
+
 	return alloc
 }
 
@@ -156,6 +170,9 @@ func (alloc *Allocator) Start() {
 	default:
 		alloc.infof("Initialising as observer - awaiting IPAM data from another peer")
 	}
+	if loadedPersistedData { // do any pre-claims right away
+		alloc.tryOps(&alloc.pendingClaims)
+	}
 	actionChan := make(chan func(), mesh.ChannelSize)
 	stopChan := make(chan struct{})
 	alloc.actionChan = actionChan
@@ -193,9 +210,9 @@ func (alloc *Allocator) doOperation(op operation, ops *[]operation) {
 //  already succeeded.  If it is on the queue, we call
 //  cancel on it, allowing callers waiting for the resultChans
 //  to unblock.
-func (alloc *Allocator) cancelOp(op operation, ops *[]operation) {
+func (alloc *Allocator) cancelOp(opToCancel operation, ops *[]operation) {
 	for i, op := range *ops {
-		if op == op {
+		if op == opToCancel {
 			*ops = append((*ops)[:i], (*ops)[i+1:]...)
 			op.Cancel()
 			break
@@ -248,6 +265,10 @@ func (alloc *Allocator) tryPendingOps() {
 	alloc.tryOps(&alloc.pendingAllocates)
 }
 
+func (alloc *Allocator) havePendingOps() bool {
+	return len(alloc.pendingPrimes)+len(alloc.pendingClaims)+len(alloc.pendingAllocates) > 0
+}
+
 func (alloc *Allocator) spaceRequestDenied(sender mesh.PeerName, r address.Range) {
 	for i := 0; i < len(alloc.pendingClaims); {
 		claim := alloc.pendingClaims[i].(*claim)
@@ -679,8 +700,12 @@ func (alloc *Allocator) actorLoop(actionChan <-chan func(), stopChan <-chan stru
 			// Retry things in case messages got lost between here and recipients
 			if alloc.awaitingConsensus {
 				alloc.propose()
-			} else {
-				alloc.tryPendingOps()
+			} else if alloc.havePendingOps() {
+				if alloc.ring.Empty() {
+					alloc.establishRing()
+				} else {
+					alloc.tryPendingOps()
+				}
 			}
 			alloc.removeDeadContainers()
 		}

ipam/allocator_test.go

@@ -168,11 +168,13 @@ func TestAllocatorClaim(t *testing.T) {
 		container1 = "abcdef"
 		container3 = "b01df00d"
 		universe   = "10.0.3.0/24"
-		testAddr1  = "10.0.3.2/24"
+		testAddr1  = "10.0.3.5/24"
 		testAddr2  = "10.0.4.2/24"
+		testPre    = "10.0.3.1/24"
 	)
 
-	allocs, router, subnet := makeNetworkOfAllocators(2, universe)
+	preAddr, _ := address.ParseCIDR(testPre)
+	allocs, router, subnet := makeNetworkOfAllocators(2, universe, []PreClaim{{container1, true, preAddr}})
 	defer stopNetworkOfAllocators(allocs, router)
 	alloc := allocs[1]
 	addr1, _ := address.ParseCIDR(testAddr1)
@@ -184,6 +186,9 @@ func TestAllocatorClaim(t *testing.T) {
 	alloc.Prime()
 	// Do an allocate on the other peer, which we will try to claim later
 	addrx, err := allocs[0].Allocate(container1, subnet, true, returnFalse)
+	require.NoError(t, err)
+	// Should not get the address we pre-claimed
+	require.NotEqual(t, addrx, preAddr)
 	router.Flush()
 
 	// Now try the claim again
@@ -201,6 +206,11 @@ func TestAllocatorClaim(t *testing.T) {
 	// claiming the address allocated on the other peer should fail
 	err = alloc.SimplyClaim(container1, address.MakeCIDR(subnet, addrx))
 	require.Error(t, err, "claiming address allocated on other peer should fail")
+	// claiming the pre-claimed address should fail on both peers
+	err = alloc.SimplyClaim(container3, preAddr)
+	require.Error(t, err, "claiming address allocated on other peer should fail")
+	err = allocs[0].SimplyClaim(container3, preAddr)
+	require.Error(t, err, "claiming address allocated on other peer should fail")
 	// Check an address outside of our universe
 	addr2, _ := address.ParseCIDR(testAddr2)
 	err = alloc.SimplyClaim(container1, addr2)

ipam/claim.go

@@ -12,10 +12,10 @@ import (
 
 type claim struct {
 	resultChan       chan<- error
-	ident            string
-	cidr             address.CIDR
-	isContainer      bool
-	noErrorOnUnknown bool
+	ident            string       // a container ID, something like "weave:expose", or api.NoContainerID
+	cidr             address.CIDR // single address being claimed
+	isContainer      bool         // true if ident is a container ID
+	noErrorOnUnknown bool         // if false, error or block if we don't know; if true return ok but keep trying
 	hasBeenCancelled func() bool
 }
 
@@ -35,7 +35,7 @@ func (c *claim) sendResult(result error) {
 
 // Try returns true for success (or failure), false if we need to try again later
 func (c *claim) Try(alloc *Allocator) bool {
-	if c.hasBeenCancelled() {
+	if c.hasBeenCancelled != nil && c.hasBeenCancelled() {
 		c.Cancel()
 		return true
 	}

ipam/space/space_test.go

@@ -52,12 +52,12 @@ func TestLowlevel(t *testing.T) {
 
 	s := New()
 	require.Equal(t, address.Count(0), s.NumFreeAddresses())
-	ok, got := s.Allocate(address.NewRange(0, 1000))
+	ok, _ := s.Allocate(address.NewRange(0, 1000))
 	require.False(t, ok, "allocate in empty space should fail")
 
 	s.Add(100, 100)
 	require.Equal(t, address.Count(100), s.NumFreeAddresses())
-	ok, got = s.Allocate(address.NewRange(0, 1000))
+	ok, got := s.Allocate(address.NewRange(0, 1000))
 	require.True(t, ok && got == 100, "allocate")
 	require.Equal(t, address.Count(99), s.NumFreeAddresses())
 	require.NoError(t, s.Claim(150))
@@ -68,6 +68,7 @@ func TestLowlevel(t *testing.T) {
 	wt.AssertErrorInterface(t, (*error)(nil), s.Free(100), "double free")
 
 	r, ok := s.Donate(address.NewRange(0, 1000))
+	require.True(t, ok, "donate")
 	require.Equal(t, address.NewRange(0xa0, 0x20), r, "donate")
 
 	// test Donate when addresses are scarce
@@ -78,6 +79,7 @@ func TestLowlevel(t *testing.T) {
 	require.NoError(t, s.Claim(0))
 	require.NoError(t, s.Claim(2))
 	r, ok = s.Donate(address.NewRange(0, 1000))
+	require.True(t, ok, "donate")
 	require.Equal(t, address.NewRange(1, 1), r, "donate")
 	r, ok = s.Donate(address.NewRange(0, 1000))
 	require.True(t, !ok, "donate should fail")

ipam/testutils_test.go

@@ -128,7 +128,7 @@ type mockDB struct{}
 func (d *mockDB) Load(_ string, _ interface{}) (bool, error) { return false, nil }
 func (d *mockDB) Save(_ string, _ interface{}) error         { return nil }
 
-func makeAllocator(name string, cidrStr string, quorum uint) (*Allocator, address.CIDR) {
+func makeAllocator(name string, cidrStr string, quorum uint, preClaims ...PreClaim) (*Allocator, address.CIDR) {
 	peername, err := mesh.PeerNameFromString(name)
 	if err != nil {
 		panic(err)
@@ -145,6 +145,7 @@ func makeAllocator(name string, cidrStr string, quorum uint) (*Allocator, addres
 		OurNickname: "nick-" + name,
 		Universe:    cidr,
 		IsObserver:  quorum == 0,
+		PreClaims:   preClaims,
 		Quorum:      func() uint { return quorum },
 		Db:          new(mockDB),
 		IsKnownPeer: func(mesh.PeerName) bool { return true },
@@ -213,15 +214,19 @@ func AssertNothingSentErr(t *testing.T, ch <-chan error) {
 	}
 }
 
-func makeNetworkOfAllocators(size int, cidr string) ([]*Allocator, *gossip.TestRouter, address.CIDR) {
+func makeNetworkOfAllocators(size int, cidr string, preClaims ...[]PreClaim) ([]*Allocator, *gossip.TestRouter, address.CIDR) {
 	gossipRouter := gossip.NewTestRouter(0.0)
 	allocs := make([]*Allocator, size)
 	var subnet address.CIDR
 
 	for i := 0; i < size; i++ {
 		var alloc *Allocator
+		preClaim := []PreClaim{}
+		if i < len(preClaims) {
+			preClaim = preClaims[i]
+		}
 		alloc, subnet = makeAllocator(fmt.Sprintf("%02d:00:00:02:00:00", i),
-			cidr, uint(size/2+1))
+			cidr, uint(size/2+1), preClaim...)
 		alloc.SetInterfaces(gossipRouter.Connect(alloc.ourName, alloc))
 		alloc.Start()
 		allocs[i] = alloc

nameserver/dns.go

@@ -222,7 +222,7 @@ func (h *handler) handleReverse(w dns.ResponseWriter, req *dns.Msg) {
 		return
 	}
 
-	ipStr := strings.TrimSuffix(req.Question[0].Name, "."+reverseDNSdomain)
+	ipStr := strings.TrimSuffix(strings.ToLower(req.Question[0].Name), "."+reverseDNSdomain)
 	ip, err := address.ParseIP(ipStr)
 	if err != nil {
 		h.nameError(w, req)

net/veth.go

@@ -113,7 +113,7 @@ func interfaceExistsInNamespace(netNSPath string, ifName string) bool {
 
 // NB: This function can be used only by a process that terminates immediately
 //     after calling the function as it changes netns via WithNetNSLinkUnsafe.
-func AttachContainer(netNSPath, id, ifName, bridgeName string, mtu int, withMulticastRoute bool, cidrs []*net.IPNet, keepTXOn bool) error {
+func AttachContainer(netNSPath, id, ifName, bridgeName string, mtu int, withMulticastRoute bool, cidrs []*net.IPNet, keepTXOn bool, hairpinMode bool) error {
 	ns, err := netns.GetFromPath(netNSPath)
 	if err != nil {
 		return err
@@ -131,7 +131,7 @@ func AttachContainer(netNSPath, id, ifName, bridgeName string, mtu int, withMult
 			id = id[:maxIDLen] // trim passed ID if too long
 		}
 		name, peerName := vethPrefix+"pl"+id, vethPrefix+"pg"+id
-		_, err := CreateAndAttachVeth(name, peerName, bridgeName, mtu, keepTXOn, func(veth netlink.Link) error {
+		veth, err := CreateAndAttachVeth(name, peerName, bridgeName, mtu, keepTXOn, func(veth netlink.Link) error {
 			if err := netlink.LinkSetNsFd(veth, int(ns)); err != nil {
 				return fmt.Errorf("failed to move veth to container netns: %s", err)
 			}
@@ -143,6 +143,10 @@ func AttachContainer(netNSPath, id, ifName, bridgeName string, mtu int, withMult
 		if err != nil {
 			return err
 		}
+		if err = netlink.LinkSetHairpin(veth, hairpinMode); err != nil {
+			return fmt.Errorf("unable to set hairpin mode to %t for bridge side of veth %s: %s", hairpinMode, name, err)
+		}
+
 	}
 
 	if err := WithNetNSLinkUnsafe(ns, ifName, func(veth netlink.Link) error {

npc/constants.go

@@ -6,4 +6,8 @@ const (
 	MainChain    = "WEAVE-NPC"
 	DefaultChain = "WEAVE-NPC-DEFAULT"
 	IngressChain = "WEAVE-NPC-INGRESS"
+
+	IpsetNamePrefix = "weave-"
+
+	LocalIpset = IpsetNamePrefix + "local-pods"
 )

npc/controller.go

@@ -29,6 +29,8 @@ type NetworkPolicyController interface {
 type controller struct {
 	sync.Mutex
 
+	nodeName string // my node name
+
 	ipt *iptables.IPTables
 	ips ipset.Interface
 
@@ -36,11 +38,12 @@ type controller struct {
 	nsSelectors *selectorSet   // selector string -> nsSelector
 }
 
-func New(ipt *iptables.IPTables, ips ipset.Interface) NetworkPolicyController {
+func New(nodeName string, ipt *iptables.IPTables, ips ipset.Interface) NetworkPolicyController {
 	c := &controller{
-		ipt: ipt,
-		ips: ips,
-		nss: make(map[string]*ns)}
+		nodeName: nodeName,
+		ipt:      ipt,
+		ips:      ips,
+		nss:      make(map[string]*ns)}
 
 	c.nsSelectors = newSelectorSet(ips, c.onNewNsSelector)
 
@@ -82,10 +85,21 @@ func (npc *controller) withNS(name string, f func(ns *ns) error) error {
 	return nil
 }
 
+func (npc *controller) checkLocalPod(obj *coreapi.Pod, event string) bool {
+	if obj.Spec.NodeName != npc.nodeName {
+		common.Log.Debugf("%s ignored for pod %s/%s on node %s", event, obj.ObjectMeta.Namespace, obj.ObjectMeta.Name, obj.Spec.NodeName)
+		return false
+	}
+	return true
+}
+
 func (npc *controller) AddPod(obj *coreapi.Pod) error {
 	npc.Lock()
 	defer npc.Unlock()
 
+	if !npc.checkLocalPod(obj, "AddPod") {
+		return nil
+	}
 	common.Log.Debugf("EVENT AddPod %s", js(obj))
 	return npc.withNS(obj.ObjectMeta.Namespace, func(ns *ns) error {
 		return errors.Wrap(ns.addPod(obj), "add pod")
@@ -96,6 +110,10 @@ func (npc *controller) UpdatePod(oldObj, newObj *coreapi.Pod) error {
 	npc.Lock()
 	defer npc.Unlock()
 
+	// Pods don't move across nodes so we only check the new version
+	if !npc.checkLocalPod(newObj, "UpdatePod") {
+		return nil
+	}
 	common.Log.Debugf("EVENT UpdatePod %s %s", js(oldObj), js(newObj))
 	return npc.withNS(oldObj.ObjectMeta.Namespace, func(ns *ns) error {
 		return errors.Wrap(ns.updatePod(oldObj, newObj), "update pod")
@@ -106,6 +124,9 @@ func (npc *controller) DeletePod(obj *coreapi.Pod) error {
 	npc.Lock()
 	defer npc.Unlock()
 
+	if !npc.checkLocalPod(obj, "DeletePod") {
+		return nil
+	}
 	common.Log.Debugf("EVENT DeletePod %s", js(obj))
 	return npc.withNS(obj.ObjectMeta.Namespace, func(ns *ns) error {
 		return errors.Wrap(ns.deletePod(obj), "delete pod")

npc/ipset/ipset.go

@@ -1,7 +1,9 @@
 package ipset
 
 import (
+	"log"
 	"os/exec"
+	"strings"
 
 	"github.com/pkg/errors"
 )
@@ -22,15 +24,19 @@ type Interface interface {
 	Flush(ipsetName Name) error
 	Destroy(ipsetName Name) error
 
+	List(prefix string) ([]Name, error)
+
 	FlushAll() error
 	DestroyAll() error
 }
 
 type ipset struct {
+	refCount
+	*log.Logger
 }
 
-func New() Interface {
-	return &ipset{}
+func New(logger *log.Logger) Interface {
+	return &ipset{refCount: newRefCount(), Logger: logger}
 }
 
 func (i *ipset) Create(ipsetName Name, ipsetType Type) error {
@@ -38,32 +44,97 @@ func (i *ipset) Create(ipsetName Name, ipsetType Type) error {
 }
 
 func (i *ipset) AddEntry(ipsetName Name, entry string) error {
+	i.Logger.Printf("adding entry %s to %s", entry, ipsetName)
+	if i.inc(ipsetName, entry) > 1 { // already in the set
+		return nil
+	}
 	return doExec("add", string(ipsetName), entry)
 }
 
 func (i *ipset) DelEntry(ipsetName Name, entry string) error {
+	i.Logger.Printf("deleting entry %s from %s", entry, ipsetName)
+	if i.dec(ipsetName, entry) > 0 { // still needed
+		return nil
+	}
 	return doExec("del", string(ipsetName), entry)
 }
 
 func (i *ipset) Flush(ipsetName Name) error {
+	i.removeSet(ipsetName)
 	return doExec("flush", string(ipsetName))
 }
 
 func (i *ipset) FlushAll() error {
+	i.refCount = newRefCount()
 	return doExec("flush")
 }
 
 func (i *ipset) Destroy(ipsetName Name) error {
+	i.removeSet(ipsetName)
 	return doExec("destroy", string(ipsetName))
 }
 
 func (i *ipset) DestroyAll() error {
+	i.refCount = newRefCount()
 	return doExec("destroy")
 }
 
+// Fetch a list of all existing sets with a given prefix
+func (i *ipset) List(prefix string) ([]Name, error) {
+	output, err := exec.Command("ipset", "list", "-name", "-output", "plain").Output()
+	if err != nil {
+		return nil, err
+	}
+
+	var selected []Name
+	sets := strings.Split(string(output), "\n")
+	for _, v := range sets {
+		if strings.HasPrefix(v, prefix) {
+			selected = append(selected, Name(v))
+		}
+	}
+
+	return selected, err
+}
+
 func doExec(args ...string) error {
 	if output, err := exec.Command("ipset", args...).CombinedOutput(); err != nil {
 		return errors.Wrapf(err, "ipset %v failed: %s", args, output)
 	}
 	return nil
 }
+
+// Reference-counting
+type key struct {
+	ipsetName Name
+	entry     string
+}
+
+// note no locking is required as all operations are serialised in the controller
+type refCount struct {
+	ref map[key]int
+}
+
+func newRefCount() refCount {
+	return refCount{ref: make(map[key]int)}
+}
+
+func (rc *refCount) inc(ipsetName Name, entry string) int {
+	k := key{ipsetName, entry}
+	rc.ref[k]++
+	return rc.ref[k]
+}
+
+func (rc *refCount) dec(ipsetName Name, entry string) int {
+	k := key{ipsetName, entry}
+	rc.ref[k]--
+	return rc.ref[k]
+}
+
+func (rc *refCount) removeSet(ipsetName Name) {
+	for k := range rc.ref {
+		if k.ipsetName == ipsetName {
+			delete(rc.ref, k)
+		}
+	}
+}

npc/namespace.go

@@ -88,6 +88,7 @@ func (ns *ns) addPod(obj *coreapi.Pod) error {
 		return nil
 	}
 
+	ns.ips.AddEntry(LocalIpset, obj.Status.PodIP)
 	return ns.podSelectors.addToMatching(obj.ObjectMeta.Labels, obj.Status.PodIP)
 }
 
@@ -100,10 +101,12 @@ func (ns *ns) updatePod(oldObj, newObj *coreapi.Pod) error {
 	}
 
 	if hasIP(oldObj) && !hasIP(newObj) {
+		ns.ips.DelEntry(LocalIpset, oldObj.Status.PodIP)
 		return ns.podSelectors.delFromMatching(oldObj.ObjectMeta.Labels, oldObj.Status.PodIP)
 	}
 
 	if !hasIP(oldObj) && hasIP(newObj) {
+		ns.ips.AddEntry(LocalIpset, newObj.Status.PodIP)
 		return ns.podSelectors.addToMatching(newObj.ObjectMeta.Labels, newObj.Status.PodIP)
 	}
 
@@ -139,6 +142,7 @@ func (ns *ns) deletePod(obj *coreapi.Pod) error {
 		return nil
 	}
 
+	ns.ips.DelEntry(LocalIpset, obj.Status.PodIP)
 	return ns.podSelectors.delFromMatching(obj.ObjectMeta.Labels, obj.Status.PodIP)
 }
 

npc/selector.go

@@ -29,7 +29,7 @@ func newSelectorSpec(json *unversioned.LabelSelector, nsName string, ipsetType i
 		// We prefix the selector string with the namespace name when generating
 		// the shortname because you can specify the same selector in multiple
 		// namespaces - we need those to map to distinct ipsets
-		ipsetName: ipset.Name("weave-" + shortName(nsName+":"+key)),
+		ipsetName: ipset.Name(IpsetNamePrefix + shortName(nsName+":"+key)),
 		ipsetType: ipsetType}, nil
 }
 
@@ -43,12 +43,10 @@ func (s *selector) matches(labelMap map[string]string) bool {
 }
 
 func (s *selector) addEntry(entry string) error {
-	common.Log.Infof("adding entry %s to %s", entry, s.spec.ipsetName)
 	return s.ips.AddEntry(s.spec.ipsetName, entry)
 }
 
 func (s *selector) delEntry(entry string) error {
-	common.Log.Infof("deleting entry %s from %s", entry, s.spec.ipsetName)
 	return s.ips.DelEntry(s.spec.ipsetName, entry)
 }
 

npc/shortname.go

@@ -7,12 +7,13 @@ import (
 
 const (
 	// This array:
-	// * Must include only printable UTF8 characters that are represented with a single byte
+	// * Must include only printable UTF8 characters that are represented with a single byte (http://www.utf8-chartable.de/)
 	// * Must be at least of length 85 (`len("weave-") + l(2^160)/l(85)` equals 31, the maximum ipset name length)
 	// * Must not include commas as those are treated specially by `ipset add` when adding a named set to a list:set
 	// * Should not include space for readability
+	// * Should not include invalid chars for XML `<>&"'`
 	// * Should not include single quote or backslash to be nice to shell users
-	ShortNameSymbols = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789(){}[]<>_$%^&*|/?.;:@#~"
+	ShortNameSymbols = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789(){}[]+=_$%^!*|/?.;:@#~"
 )
 
 // sha1 hash an arbitrary string and represent it using the full range of

plugin/ipam/cni.go

@@ -5,19 +5,24 @@ import (
 	"fmt"
 	"net"
 
-	"github.com/appc/cni/pkg/skel"
-	"github.com/appc/cni/pkg/types"
+	"github.com/containernetworking/cni/pkg/skel"
+	"github.com/containernetworking/cni/pkg/types"
+	"github.com/containernetworking/cni/pkg/types/current"
 )
 
 func (i *Ipam) CmdAdd(args *skel.CmdArgs) error {
+	var conf types.NetConf
+	if err := json.Unmarshal(args.StdinData, &conf); err != nil {
+		return fmt.Errorf("failed to load netconf: %v", err)
+	}
 	result, err := i.Allocate(args)
 	if err != nil {
 		return err
 	}
-	return result.Print()
+	return types.PrintResult(result, conf.CNIVersion)
 }
 
-func (i *Ipam) Allocate(args *skel.CmdArgs) (*types.Result, error) {
+func (i *Ipam) Allocate(args *skel.CmdArgs) (types.Result, error) {
 	// extract the things we care about
 	conf, err := loadIPAMConf(args.StdinData)
 	if err != nil {
@@ -46,12 +51,13 @@ func (i *Ipam) Allocate(args *skel.CmdArgs) (*types.Result, error) {
 	if err != nil {
 		return nil, err
 	}
-	result := &types.Result{
-		IP4: &types.IPConfig{
-			IP:      *ipnet,
+	result := &current.Result{
+		IPs: []*current.IPConfig{{
+			Version: "4",
+			Address: *ipnet,
 			Gateway: conf.Gateway,
-			Routes:  conf.Routes,
-		},
+		}},
+		Routes: conf.Routes,
 	}
 	return result, nil
 }
@@ -65,9 +71,9 @@ func (i *Ipam) Release(args *skel.CmdArgs) error {
 }
 
 type ipamConf struct {
-	Subnet  string        `json:"subnet,omitempty"`
-	Gateway net.IP        `json:"gateway,omitempty"`
-	Routes  []types.Route `json:"routes"`
+	Subnet  string         `json:"subnet,omitempty"`
+	Gateway net.IP         `json:"gateway,omitempty"`
+	Routes  []*types.Route `json:"routes"`
 }
 
 type netConf struct {

plugin/net/cni.go

@@ -5,11 +5,13 @@ import (
 	"encoding/json"
 	"fmt"
 	"net"
+	"os"
 	"syscall"
 
-	"github.com/appc/cni/pkg/ipam"
-	"github.com/appc/cni/pkg/skel"
-	"github.com/appc/cni/pkg/types"
+	"github.com/containernetworking/cni/pkg/ipam"
+	"github.com/containernetworking/cni/pkg/skel"
+	"github.com/containernetworking/cni/pkg/types"
+	"github.com/containernetworking/cni/pkg/types/current"
 	"github.com/vishvananda/netlink"
 	"github.com/vishvananda/netns"
 	weaveapi "github.com/weaveworks/weave/api"
@@ -32,7 +34,8 @@ func NewCNIPlugin(weave *weaveapi.Client) *CNIPlugin {
 
 func loadNetConf(bytes []byte) (*NetConf, error) {
 	n := &NetConf{
-		BrName: weavenet.WeaveBridgeName,
+		BrName:      weavenet.WeaveBridgeName,
+		HairpinMode: true,
 	}
 	if err := json.Unmarshal(bytes, n); err != nil {
 		return nil, fmt.Errorf("failed to load netconf: %v", err)
@@ -40,17 +43,26 @@ func loadNetConf(bytes []byte) (*NetConf, error) {
 	return n, nil
 }
 
-func (c *CNIPlugin) getIP(ipamType string, args *skel.CmdArgs) (result *types.Result, err error) {
+func (c *CNIPlugin) getIP(ipamType string, args *skel.CmdArgs) (newResult *current.Result, err error) {
+	var result types.Result
 	// Default IPAM is Weave's own
 	if ipamType == "" {
 		result, err = ipamplugin.NewIpam(c.weave).Allocate(args)
 	} else {
 		result, err = ipam.ExecAdd(ipamType, args.StdinData)
 	}
-	if err == nil && result.IP4 == nil {
+	if err != nil {
+		return nil, err
+	}
+	if result == nil {
+		return nil, fmt.Errorf("Received no usable result from IPAM plugin")
+	}
+	newResult, err = current.NewResultFromResult(result)
+	// Check if ipam returned no results without error
+	if err == nil && len(newResult.IPs) == 0 {
 		return nil, fmt.Errorf("IPAM plugin failed to allocate IP address")
 	}
-	return result, err
+	return newResult, err
 }
 
 func (c *CNIPlugin) CmdAdd(args *skel.CmdArgs) error {
@@ -70,28 +82,35 @@ func (c *CNIPlugin) CmdAdd(args *skel.CmdArgs) error {
 	if err != nil {
 		return fmt.Errorf("unable to allocate IP address: %s", err)
 	}
+	// Only expecting one address
+	ip := result.IPs[0]
 
 	// If config says nothing about routes or gateway, default one will be via the bridge
-	if result.IP4.Routes == nil && result.IP4.Gateway == nil {
-		bridgeIP, err := weavenet.FindBridgeIP(conf.BrName, &result.IP4.IP)
+	if len(result.Routes) == 0 && ip.Gateway == nil {
+		bridgeIP, err := weavenet.FindBridgeIP(conf.BrName, &ip.Address)
 		if err == weavenet.ErrBridgeNoIP {
 			bridgeArgs := *args
 			bridgeArgs.ContainerID = "weave:expose"
+			// It would be better if libcni let us send just the desired parameters,
+			// but there is a bug: https://github.com/containernetworking/cni/issues/410
+			// so just blank out the one we want to change
+			os.Setenv("CNI_CONTAINERID", bridgeArgs.ContainerID)
 			bridgeIPResult, err := c.getIP(conf.IPAM.Type, &bridgeArgs)
 			if err != nil {
 				return fmt.Errorf("unable to allocate IP address for bridge: %s", err)
 			}
-			if err := assignBridgeIP(conf.BrName, bridgeIPResult.IP4.IP); err != nil {
+			bridgeCIDR := bridgeIPResult.IPs[0].Address
+			if err := assignBridgeIP(conf.BrName, bridgeCIDR); err != nil {
 				return fmt.Errorf("unable to assign IP address to bridge: %s", err)
 			}
-			if err := weavenet.ExposeNAT(bridgeIPResult.IP4.IP); err != nil {
+			if err := weavenet.ExposeNAT(bridgeCIDR); err != nil {
 				return fmt.Errorf("unable to create NAT rules: %s", err)
 			}
-			bridgeIP = bridgeIPResult.IP4.IP.IP
+			bridgeIP = bridgeCIDR.IP
 		} else if err != nil {
 			return err
 		}
-		result.IP4.Gateway = bridgeIP
+		result.IPs[0].Gateway = bridgeIP
 	}
 
 	ns, err := netns.GetFromPath(args.Netns)
@@ -110,20 +129,20 @@ func (c *CNIPlugin) CmdAdd(args *skel.CmdArgs) error {
 		id = fmt.Sprintf("%x", data)
 	}
 
-	if err := weavenet.AttachContainer(args.Netns, id, args.IfName, conf.BrName, conf.MTU, false, []*net.IPNet{&result.IP4.IP}, false); err != nil {
+	if err := weavenet.AttachContainer(args.Netns, id, args.IfName, conf.BrName, conf.MTU, false, []*net.IPNet{&ip.Address}, false, conf.HairpinMode); err != nil {
 		return err
 	}
 	if err := weavenet.WithNetNSLinkUnsafe(ns, args.IfName, func(link netlink.Link) error {
-		return setupRoutes(link, args.IfName, result.IP4.IP, result.IP4.Gateway, result.IP4.Routes)
+		return setupRoutes(link, args.IfName, ip.Address, ip.Gateway, result.Routes)
 	}); err != nil {
 		return fmt.Errorf("error setting up routes: %s", err)
 	}
 
 	result.DNS = conf.DNS
-	return result.Print()
+	return types.PrintResult(result, conf.CNIVersion)
 }
 
-func setupRoutes(link netlink.Link, name string, ipnet net.IPNet, gw net.IP, routes []types.Route) error {
+func setupRoutes(link netlink.Link, name string, ipnet net.IPNet, gw net.IP, routes []*types.Route) error {
 	var err error
 	if routes == nil { // If config says nothing about routes, add a default one
 		if !ipnet.Contains(gw) {
@@ -133,7 +152,7 @@ func setupRoutes(link netlink.Link, name string, ipnet net.IPNet, gw net.IP, rou
 				return err
 			}
 		}
-		routes = []types.Route{{Dst: zeroNetwork}}
+		routes = []*types.Route{{Dst: zeroNetwork}}
 	}
 	for _, r := range routes {
 		if r.GW != nil {
@@ -164,14 +183,22 @@ func assignBridgeIP(bridgeName string, ipnet net.IPNet) error {
 	return nil
 }
 
+// As of CNI 0.5 spec:
+//   "Plugins should generally complete a DEL action without error even if some resources are missing"
+// this method should therefore return nil in most, if not all, cases.
 func (c *CNIPlugin) CmdDel(args *skel.CmdArgs) error {
 	conf, err := loadNetConf(args.StdinData)
 	if err != nil {
-		return err
+		logOnStderr(err)
+		return nil
 	}
 
-	if _, err = weavenet.WithNetNS(args.Netns, "del-iface", args.IfName); err != nil {
-		return fmt.Errorf("error removing interface %q: %s", args.IfName, err)
+	// As of CNI 0.3 spec, runtimes can send blank if they just want the address deallocated
+	if args.Netns != "" {
+		if _, err = weavenet.WithNetNS(args.Netns, "del-iface", args.IfName); err != nil {
+			// We log the error and carry on instead of returning nil, as there may still be resources to free in IPAM.
+			logOnStderr(fmt.Errorf("error removing interface %q: %s", args.IfName, err))
+		}
 	}
 
 	// Default IPAM is Weave's own
@@ -180,16 +207,21 @@ func (c *CNIPlugin) CmdDel(args *skel.CmdArgs) error {
 	} else {
 		err = ipam.ExecDel(conf.IPAM.Type, args.StdinData)
 	}
-	if err != nil {
-		return fmt.Errorf("unable to release IP address: %s", err)
-	}
+	logOnStderr(fmt.Errorf("unable to release IP address: %s", err))
 	return nil
 }
 
+// Standard error is inherited from the caller (e.g. Kubelet, when running Weave Net's CNI plugin
+// under Kubernetes) hence provided errors should eventually appear in the caller's log files.
+func logOnStderr(err error) {
+	fmt.Fprintln(os.Stderr, "weave-cni:", err)
+}
+
 type NetConf struct {
 	types.NetConf
-	BrName string `json:"bridge"`
-	IsGW   bool   `json:"isGateway"`
-	IPMasq bool   `json:"ipMasq"`
-	MTU    int    `json:"mtu"`
+	BrName      string `json:"bridge"`
+	IsGW        bool   `json:"isGateway"`
+	IPMasq      bool   `json:"ipMasq"`
+	MTU         int    `json:"mtu"`
+	HairpinMode bool   `json:"hairpinMode"`
 }

prog/kube-peers/main.go

@@ -3,6 +3,7 @@ package main
 import (
 	"fmt"
 	"log"
+	"net"
 
 	"k8s.io/client-go/kubernetes"
 	api "k8s.io/client-go/pkg/api/v1"
@@ -35,10 +36,25 @@ func getKubePeers() ([]string, error) {
 	}
 	addresses := make([]string, 0, len(nodeList.Items))
 	for _, peer := range nodeList.Items {
+		var internalIP, externalIP string
 		for _, addr := range peer.Status.Addresses {
-			if addr.Type == "InternalIP" {
-				addresses = append(addresses, addr.Address)
+			// Check it's a valid ipv4 address
+			ip := net.ParseIP(addr.Address)
+			if ip == nil || ip.To4() == nil {
+				continue
 			}
+			if addr.Type == "InternalIP" {
+				internalIP = ip.To4().String()
+			} else if addr.Type == "ExternalIP" {
+				externalIP = ip.To4().String()
+			}
+		}
+
+		// Fallback for cases where a Node has an ExternalIP but no InternalIP
+		if internalIP != "" {
+			addresses = append(addresses, internalIP)
+		} else if externalIP != "" {
+			addresses = append(addresses, externalIP)
 		}
 	}
 	return addresses, nil

prog/plugin/main.go

@@ -94,7 +94,7 @@ func run(dockerClient *docker.Client, weave *weaveapi.Client, address, meshAddre
 	}
 	go serveStatus(statusListener)
 	sigChan := make(chan os.Signal, 1)
-	signal.Notify(sigChan, os.Interrupt, os.Kill, syscall.SIGTERM)
+	signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
 
 	select {
 	case sig := <-sigChan:

prog/weave-kube/launch.sh

@@ -2,11 +2,34 @@
 
 set -e
 
+modprobe_safe() {
+    modprobe $1 || echo "Ignore the error if \"$1\" is built-in in the kernel" >&2
+}
+
+# Check whether xt_set actually exists
+xt_set_exists() {
+    # Clean everything up in advance, in case there's leftovers
+    iptables -F WEAVE-KUBE-TEST 2>/dev/null || true
+    iptables -X WEAVE-KUBE-TEST 2>/dev/null || true
+    ipset destroy weave-kube-test 2>/dev/null || true
+
+    ipset create weave-kube-test hash:ip
+    iptables -t filter -N WEAVE-KUBE-TEST
+    if ! iptables -A WEAVE-KUBE-TEST -m set --match-set weave-kube-test src -j DROP; then
+        NOT_EXIST=1
+    fi
+    iptables -F WEAVE-KUBE-TEST
+    iptables -X WEAVE-KUBE-TEST
+    ipset destroy weave-kube-test
+    [ -z "$NOT_EXIST" ] || (echo "\"xt_set\" does not exist" >&2 && return 1)
+}
+
 # Default if not supplied - same as weave net default
 IPALLOC_RANGE=${IPALLOC_RANGE:-10.32.0.0/12}
 HTTP_ADDR=${WEAVE_HTTP_ADDR:-127.0.0.1:6784}
 STATUS_ADDR=${WEAVE_STATUS_ADDR:-0.0.0.0:6782}
 HOST_ROOT=${HOST_ROOT:-/host}
+CONN_LIMIT=${CONN_LIMIT:-30}
 
 # Check if the IP range overlaps anything existing on the host
 /usr/bin/weaveutil netcheck $IPALLOC_RANGE weave
@@ -14,6 +37,13 @@ HOST_ROOT=${HOST_ROOT:-/host}
 # Default for network policy
 EXPECT_NPC=${EXPECT_NPC:-1}
 
+# Ensure we have the required modules for NPC
+if [ "${EXPECT_NPC}" != "0" ]; then
+    modprobe_safe br_netfilter
+    modprobe_safe xt_set
+    xt_set_exists
+fi
+
 # kube-proxy requires that bridged traffic passes through netfilter
 if ! BRIDGE_NF_ENABLED=$(cat /proc/sys/net/bridge/bridge-nf-call-iptables); then
     echo "Cannot detect bridge-nf support - network policy and iptables mode kubeproxy may not work reliably" >&2
@@ -66,14 +96,6 @@ if [ -z "$IPALLOC_INIT" ]; then
     IPALLOC_INIT="consensus=$(peer_count $KUBE_PEERS)"
 fi
 
-reclaim_ips() {
-    ID=$1
-    shift
-    for CIDR in "$@" ; do
-        curl -s -S -X PUT "$HTTP_ADDR/ip/$ID/$CIDR" || true
-    done
-}
-
 post_start_actions() {
     # Wait for weave process to become responsive
     while true ; do
@@ -81,15 +103,6 @@ post_start_actions() {
         sleep 1
     done
 
-    # Tell the newly-started weave about existing weave bridge IPs
-    /usr/bin/weaveutil container-addrs weave weave:expose | while read ID IFACE MAC IPS; do
-        reclaim_ips "weave:expose" $IPS
-    done
-    # Tell weave about existing weave process IPs
-    /usr/bin/weaveutil process-addrs weave | while read ID IFACE MAC IPS; do
-        reclaim_ips "_" $IPS
-    done
-
     # Install CNI plugin binary to typical CNI bin location
     # with fall-back to CNI directory used by kube-up on GCI OS
     if ! mkdir -p $HOST_ROOT/opt/cni/bin ; then
@@ -109,9 +122,10 @@ post_start_actions() {
 
 post_start_actions &
 
-exec /home/weave/weaver $EXTRA_ARGS --port=6783 $BRIDGE_OPTIONS \
+/home/weave/weaver $EXTRA_ARGS --port=6783 $BRIDGE_OPTIONS \
      --http-addr=$HTTP_ADDR --status-addr=$STATUS_ADDR --docker-api='' --no-dns \
      --ipalloc-range=$IPALLOC_RANGE $NICKNAME_ARG \
      --ipalloc-init $IPALLOC_INIT \
+     --conn-limit=$CONN_LIMIT \
      "$@" \
      $KUBE_PEERS

prog/weave-kube/weave-daemonset-k8s-1.6.yaml

@@ -0,0 +1,120 @@
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: weave-net
+rules:
+- apiGroups:
+  - ""
+  resources:
+  - pods
+  - namespaces
+  - nodes
+  verbs:
+  - get
+  - list
+  - watch
+- apiGroups:
+  - extensions
+  resources:
+  - networkpolicies
+  verbs:
+  - get
+  - list
+  - watch
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: weave-net
+  namespace: kube-system
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: weave-net
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: weave-net
+subjects:
+- kind: ServiceAccount
+  name: weave-net
+  namespace: kube-system
+---
+apiVersion: extensions/v1beta1
+kind: DaemonSet
+metadata:
+  name: weave-net
+  namespace: kube-system
+spec:
+  template:
+    metadata:
+      labels:
+        name: weave-net
+    spec:
+      hostNetwork: true
+      hostPID: true
+      containers:
+        - name: weave
+          image: weaveworks/weave-kube:latest
+          imagePullPolicy: Always
+          command:
+            - /home/weave/launch.sh
+          livenessProbe:
+            initialDelaySeconds: 30
+            httpGet:
+              host: 127.0.0.1
+              path: /status
+              port: 6784
+          securityContext:
+            privileged: true
+          volumeMounts:
+            - name: weavedb
+              mountPath: /weavedb
+            - name: cni-bin
+              mountPath: /host/opt
+            - name: cni-bin2
+              mountPath: /host/home
+            - name: cni-conf
+              mountPath: /host/etc
+            - name: dbus
+              mountPath: /host/var/lib/dbus
+            - name: lib-modules
+              mountPath: /lib/modules
+          resources:
+            requests:
+              cpu: 10m
+        - name: weave-npc
+          image: weaveworks/weave-npc:latest
+          imagePullPolicy: Always
+          resources:
+            requests:
+              cpu: 10m
+          securityContext:
+            privileged: true
+      restartPolicy: Always
+      tolerations:
+      - key: node-role.kubernetes.io/master
+        effect: NoSchedule
+      serviceAccountName: weave-net
+      securityContext:
+        seLinuxOptions:
+          type: spc_t
+      volumes:
+        - name: weavedb
+          emptyDir: {}
+        - name: cni-bin
+          hostPath:
+            path: /opt
+        - name: cni-bin2
+          hostPath:
+            path: /home
+        - name: cni-conf
+          hostPath:
+            path: /etc
+        - name: dbus
+          hostPath:
+            path: /var/lib/dbus
+        - name: lib-modules
+          hostPath:
+            path: /lib/modules

prog/weave-kube/weave-daemonset.yaml

@@ -46,6 +46,8 @@ spec:
               mountPath: /host/etc
             - name: dbus
               mountPath: /host/var/lib/dbus
+            - name: lib-modules
+              mountPath: /lib/modules
           resources:
             requests:
               cpu: 10m
@@ -73,3 +75,6 @@ spec:
         - name: dbus
           hostPath:
             path: /var/lib/dbus
+        - name: lib-modules
+          hostPath:
+            path: /lib/modules

prog/weave-npc/main.go

@@ -28,6 +28,7 @@ var (
 	metricsAddr string
 	logLevel    string
 	allowMcast  bool
+	nodeName    string
 )
 
 func handleError(err error) { common.CheckFatal(err) }
@@ -53,6 +54,41 @@ func resetIPTables(ipt *iptables.IPTables) error {
 		return err
 	}
 
+	return nil
+}
+
+func resetIPSets(ips ipset.Interface) error {
+	// Remove ipsets prefixed `weave-` only
+
+	sets, err := ips.List(npc.IpsetNamePrefix)
+	if err != nil {
+		common.Log.Errorf("Failed to retrieve list of ipsets")
+		return err
+	}
+
+	common.Log.Debugf("Got list of ipsets: %v", sets)
+
+	// Must remove references to ipsets by other ipsets before they're destroyed
+	for _, s := range sets {
+		common.Log.Debugf("Flushing ipset '%s'", string(s))
+		if err := ips.Flush(s); err != nil {
+			common.Log.Errorf("Failed to flush ipset '%s'", string(s))
+			return err
+		}
+	}
+
+	for _, s := range sets {
+		common.Log.Debugf("Destroying ipset '%s'", string(s))
+		if err := ips.Destroy(s); err != nil {
+			common.Log.Errorf("Failed to destroy ipset '%s'", string(s))
+			return err
+		}
+	}
+
+	return nil
+}
+
+func createBaseRules(ipt *iptables.IPTables, ips ipset.Interface) error {
 	// Configure main chain static rules
 	if err := ipt.Append(npc.TableFilter, npc.MainChain,
 		"-m", "state", "--state", "RELATED,ESTABLISHED", "-j", "ACCEPT"); err != nil {
@@ -76,17 +112,13 @@ func resetIPTables(ipt *iptables.IPTables) error {
 		return err
 	}
 
-	return nil
-}
-
-func resetIPSets(ips ipset.Interface) error {
-	// TODO should restrict ipset operations to the `weave-` prefix:
-
-	if err := ips.FlushAll(); err != nil {
+	// If the destination address is not any of the local pods, let it through
+	if err := ips.Create(npc.LocalIpset, ipset.HashIP); err != nil {
 		return err
 	}
-
-	if err := ips.DestroyAll(); err != nil {
+	if err := ipt.Append(npc.TableFilter, npc.MainChain,
+		"-m", "set", "--match-set", npc.LocalIpset, "src",
+		"-m", "set", "!", "--match-set", npc.LocalIpset, "dst", "-j", "ACCEPT"); err != nil {
 		return err
 	}
 
@@ -95,7 +127,14 @@ func resetIPSets(ips ipset.Interface) error {
 
 func root(cmd *cobra.Command, args []string) {
 	common.SetLogLevel(logLevel)
-	common.Log.Infof("Starting Weaveworks NPC %s", version)
+	if nodeName == "" {
+		// HOSTNAME is set by Kubernetes for pods in the host network namespace
+		nodeName = os.Getenv("HOSTNAME")
+	}
+	if nodeName == "" {
+		common.Log.Fatalf("Must set node name via --node-name or $HOSTNAME")
+	}
+	common.Log.Infof("Starting Weaveworks NPC %s; node name %q", version, nodeName)
 
 	if err := metrics.Start(metricsAddr); err != nil {
 		common.Log.Fatalf("Failed to start metrics: %v", err)
@@ -114,12 +153,13 @@ func root(cmd *cobra.Command, args []string) {
 	ipt, err := iptables.New()
 	handleError(err)
 
-	ips := ipset.New()
+	ips := ipset.New(common.LogLogger())
 
 	handleError(resetIPTables(ipt))
 	handleError(resetIPSets(ips))
+	handleError(createBaseRules(ipt, ips))
 
-	npc := npc.New(ipt, ips)
+	npc := npc.New(nodeName, ipt, ips)
 
 	nsController := makeController(client.Core().RESTClient(), "namespaces", &coreapi.Namespace{},
 		cache.ResourceEventHandlerFuncs{
@@ -127,7 +167,15 @@ func root(cmd *cobra.Command, args []string) {
 				handleError(npc.AddNamespace(obj.(*coreapi.Namespace)))
 			},
 			DeleteFunc: func(obj interface{}) {
-				handleError(npc.DeleteNamespace(obj.(*coreapi.Namespace)))
+				switch obj := obj.(type) {
+				case *coreapi.Namespace:
+					handleError(npc.DeleteNamespace(obj))
+				case cache.DeletedFinalStateUnknown:
+					// We know this object has gone away, but its final state is no longer
+					// available from the API server. Instead we use the last copy of it
+					// that we have, which is good enough for our cleanup.
+					handleError(npc.DeleteNamespace(obj.Obj.(*coreapi.Namespace)))
+				}
 			},
 			UpdateFunc: func(old, new interface{}) {
 				handleError(npc.UpdateNamespace(old.(*coreapi.Namespace), new.(*coreapi.Namespace)))
@@ -139,7 +187,15 @@ func root(cmd *cobra.Command, args []string) {
 				handleError(npc.AddPod(obj.(*coreapi.Pod)))
 			},
 			DeleteFunc: func(obj interface{}) {
-				handleError(npc.DeletePod(obj.(*coreapi.Pod)))
+				switch obj := obj.(type) {
+				case *coreapi.Pod:
+					handleError(npc.DeletePod(obj))
+				case cache.DeletedFinalStateUnknown:
+					// We know this object has gone away, but its final state is no longer
+					// available from the API server. Instead we use the last copy of it
+					// that we have, which is good enough for our cleanup.
+					handleError(npc.DeletePod(obj.Obj.(*coreapi.Pod)))
+				}
 			},
 			UpdateFunc: func(old, new interface{}) {
 				handleError(npc.UpdatePod(old.(*coreapi.Pod), new.(*coreapi.Pod)))
@@ -151,7 +207,15 @@ func root(cmd *cobra.Command, args []string) {
 				handleError(npc.AddNetworkPolicy(obj.(*extnapi.NetworkPolicy)))
 			},
 			DeleteFunc: func(obj interface{}) {
-				handleError(npc.DeleteNetworkPolicy(obj.(*extnapi.NetworkPolicy)))
+				switch obj := obj.(type) {
+				case *extnapi.NetworkPolicy:
+					handleError(npc.DeleteNetworkPolicy(obj))
+				case cache.DeletedFinalStateUnknown:
+					// We know this object has gone away, but its final state is no longer
+					// available from the API server. Instead we use the last copy of it
+					// that we have, which is good enough for our cleanup.
+					handleError(npc.DeleteNetworkPolicy(obj.Obj.(*extnapi.NetworkPolicy)))
+				}
 			},
 			UpdateFunc: func(old, new interface{}) {
 				handleError(npc.UpdateNetworkPolicy(old.(*extnapi.NetworkPolicy), new.(*extnapi.NetworkPolicy)))
@@ -175,6 +239,7 @@ func main() {
 	rootCmd.PersistentFlags().StringVar(&metricsAddr, "metrics-addr", ":6781", "metrics server bind address")
 	rootCmd.PersistentFlags().StringVar(&logLevel, "log-level", "debug", "logging level (debug, info, warning, error)")
 	rootCmd.PersistentFlags().BoolVar(&allowMcast, "allow-mcast", true, "allow all multicast traffic")
+	rootCmd.PersistentFlags().StringVar(&nodeName, "node-name", "", "only generate rules that apply to this node")
 
 	handleError(rootCmd.Execute())
 }

prog/weaveexec/Dockerfile.template

@@ -19,6 +19,7 @@ RUN apk add --update \
     curl \
     ethtool \
     iptables \
+    ipset \
     iproute2 \
     util-linux \
     conntrack-tools \

prog/weaver/checkpoint.go

@@ -6,6 +6,7 @@ import (
 	"time"
 
 	"github.com/weaveworks/go-checkpoint"
+	weave "github.com/weaveworks/weave/router"
 )
 
 var checker *checkpoint.Checker
@@ -16,7 +17,7 @@ const (
 	updateCheckPeriod = 6 * time.Hour
 )
 
-func checkForUpdates(dockerVersion string, network string) {
+func checkForUpdates(dockerVersion string, router *weave.NetworkRouter) {
 	newVersion.Store("")
 	success.Store(true)
 
@@ -51,9 +52,6 @@ func checkForUpdates(dockerVersion string, network string) {
 		"docker-version": dockerVersion,
 		"kernel-version": kernelVersion,
 	}
-	if network != "" {
-		flags["network"] = network
-	}
 
 	// Start background version checking
 	params := checkpoint.CheckParams{
@@ -61,6 +59,21 @@ func checkForUpdates(dockerVersion string, network string) {
 		Version:       version,
 		SignatureFile: "",
 		Flags:         flags,
+		ExtraFlags:    func() []checkpoint.Flag { return checkpointFlags(router) },
 	}
 	checker = checkpoint.CheckInterval(&params, updateCheckPeriod, handleResponse)
 }
+
+func checkpointFlags(router *weave.NetworkRouter) []checkpoint.Flag {
+	flags := []checkpoint.Flag{}
+	status := weave.NewNetworkRouterStatus(router)
+	for _, conn := range status.Connections {
+		if connectionName, ok := conn.Attrs["name"].(string); ok {
+			if _, encrypted := conn.Attrs["encrypted"]; encrypted {
+				connectionName = connectionName + " encrypted"
+			}
+			flags = append(flags, checkpoint.Flag{Key: "network", Value: connectionName})
+		}
+	}
+	return flags
+}

prog/weaver/main.go

@@ -231,7 +231,8 @@ func main() {
 
 	if bridge != nil {
 		if err := weavenet.DetectHairpin("vethwe-bridge", Log); err != nil {
-			Log.Errorf("DetectHairpin failed: %s", err)
+			Log.Errorf("Setting may cause connectivity issues : %s", err)
+			Log.Infof("Hairpin mode may have been enabled by other software on this machine")
 		}
 	}
 
@@ -254,7 +255,8 @@ func main() {
 	checkFatal(err)
 	defer db.Close()
 
-	router := weave.NewNetworkRouter(config, networkConfig, name, nickName, overlay, db)
+	router, err := weave.NewNetworkRouter(config, networkConfig, name, nickName, overlay, db)
+	checkFatal(err)
 	Log.Println("Our name is", router.Ourself)
 
 	if peers, err = router.InitialPeers(resume, peers); err != nil {
@@ -274,11 +276,7 @@ func main() {
 		dockerVersion = dockerCli.DockerVersion()
 	}
 
-	network := ""
-	if isAWSVPC {
-		network = "awsvpc"
-	}
-	checkForUpdates(dockerVersion, network)
+	checkForUpdates(dockerVersion, router)
 
 	observeContainers := func(o docker.ContainerObserver) {
 		if dockerCli != nil {
@@ -306,13 +304,16 @@ func main() {
 			}
 			trackerName = "awsvpc"
 		}
-		allocator, defaultSubnet = createAllocator(router, ipamConfig, db, t, isKnownPeer)
-		observeContainers(allocator)
+		var allContainerIDs []string
 		if dockerCli != nil {
-			ids, err := dockerCli.AllContainerIDs()
+			allContainerIDs, err = dockerCli.AllContainerIDs()
 			checkFatal(err)
-			allocator.PruneOwned(ids)
 		}
+		preClaims, err := findExistingAddresses(dockerCli, allContainerIDs, weavenet.WeaveBridgeName)
+		checkFatal(err)
+		allocator, defaultSubnet = createAllocator(router, ipamConfig, preClaims, db, t, isKnownPeer)
+		observeContainers(allocator)
+		allocator.PruneOwned(allContainerIDs)
 	}
 
 	var (
@@ -445,7 +446,7 @@ func createOverlay(datapathName string, ifaceName string, isAWSVPC bool, host st
 	return overlay, bridge
 }
 
-func createAllocator(router *weave.NetworkRouter, config ipamConfig, db db.DB, track tracker.LocalRangeTracker, isKnownPeer func(mesh.PeerName) bool) (*ipam.Allocator, address.CIDR) {
+func createAllocator(router *weave.NetworkRouter, config ipamConfig, preClaims []ipam.PreClaim, db db.DB, track tracker.LocalRangeTracker, isKnownPeer func(mesh.PeerName) bool) (*ipam.Allocator, address.CIDR) {
 	ipRange, err := ipam.ParseCIDRSubnet(config.IPRangeCIDR)
 	checkFatal(err)
 	defaultSubnet := ipRange
@@ -464,6 +465,7 @@ func createAllocator(router *weave.NetworkRouter, config ipamConfig, db db.DB, t
 		Seed:        config.SeedPeerNames,
 		Universe:    ipRange,
 		IsObserver:  config.Observer,
+		PreClaims:   preClaims,
 		Quorum:      func() uint { return determineQuorum(config.PeerCount, router) },
 		Db:          db,
 		IsKnownPeer: isKnownPeer,
@@ -472,7 +474,9 @@ func createAllocator(router *weave.NetworkRouter, config ipamConfig, db db.DB, t
 
 	allocator := ipam.NewAllocator(c)
 
-	allocator.SetInterfaces(router.NewGossip("IPallocation", allocator))
+	gossip, err := router.NewGossip("IPallocation", allocator)
+	checkFatal(err)
+	allocator.SetInterfaces(gossip)
 	allocator.Start()
 	router.Peers.OnGC(func(peer *mesh.Peer) { allocator.PeerGone(peer.Name) })
 
@@ -482,7 +486,9 @@ func createAllocator(router *weave.NetworkRouter, config ipamConfig, db db.DB, t
 func createDNSServer(config dnsConfig, router *mesh.Router, isKnownPeer func(mesh.PeerName) bool) (*nameserver.Nameserver, *nameserver.DNSServer) {
 	ns := nameserver.New(router.Ourself.Peer.Name, config.Domain, isKnownPeer)
 	router.Peers.OnGC(func(peer *mesh.Peer) { ns.PeerGone(peer.Name) })
-	ns.SetGossip(router.NewGossip("nameserver", ns))
+	gossip, err := router.NewGossip("nameserver", ns)
+	checkFatal(err)
+	ns.SetGossip(gossip)
 	upstream := nameserver.NewUpstream(config.ResolvConf, config.EffectiveListenAddress)
 	dnsserver, err := nameserver.NewDNSServer(ns, config.Domain, config.ListenAddress,
 		upstream, uint32(config.TTL), config.ClientTimeout)

prog/weaver/metrics.go

@@ -82,6 +82,18 @@ var metrics = []metric{
 				ch <- intGauge(desc, metrics.Flows)
 			}
 		}},
+	{desc("weave_ipam_pending_allocates", "Number of pending allocates."),
+		func(s WeaveStatus, desc *prometheus.Desc, ch chan<- prometheus.Metric) {
+			if s.IPAM != nil {
+				ch <- intGauge(desc, len(s.IPAM.PendingAllocates))
+			}
+		}},
+	{desc("weave_ipam_pending_claims", "Number of pending claims."),
+		func(s WeaveStatus, desc *prometheus.Desc, ch chan<- prometheus.Metric) {
+			if s.IPAM != nil {
+				ch <- intGauge(desc, len(s.IPAM.PendingClaims))
+			}
+		}},
 }
 
 func fastDPMetrics(s WeaveStatus) *weave.FastDPMetrics {

prog/weaver/reclaim.go

@@ -0,0 +1,84 @@
+package main
+
+import (
+	"net"
+
+	docker "github.com/fsouza/go-dockerclient"
+
+	"github.com/weaveworks/weave/api"
+	"github.com/weaveworks/weave/common"
+	weavedocker "github.com/weaveworks/weave/common/docker"
+	"github.com/weaveworks/weave/ipam"
+	weavenet "github.com/weaveworks/weave/net"
+	"github.com/weaveworks/weave/net/address"
+)
+
+func a(cidr *net.IPNet) address.CIDR {
+	prefixLength, _ := cidr.Mask.Size()
+	return address.CIDR{Addr: address.FromIP4(cidr.IP), PrefixLen: prefixLength}
+}
+
+// Get all the existing Weave IPs at startup, so we can stop IPAM
+// giving out any as duplicates
+func findExistingAddresses(dockerCli *weavedocker.Client, containerIDs []string, bridgeName string) (addrs []ipam.PreClaim, err error) {
+	Log.Infof("Checking for pre-existing addresses on %s bridge", bridgeName)
+	// First get the address for the bridge
+	bridgeNetDev, err := weavenet.GetBridgeNetDev(bridgeName)
+	if err != nil {
+		return nil, err
+	}
+	for _, cidr := range bridgeNetDev.CIDRs {
+		Log.Infof("%s bridge has address %v", bridgeName, cidr)
+		addrs = append(addrs, ipam.PreClaim{Ident: "weave:expose", Cidr: a(cidr)})
+	}
+
+	add := func(cid string, isContainer bool, netDevs []weavenet.Dev) {
+		for _, netDev := range netDevs {
+			for _, cidr := range netDev.CIDRs {
+				Log.Infof("Found address %v for ID %s", cidr, cid)
+				addrs = append(addrs, ipam.PreClaim{Ident: cid, IsContainer: isContainer, Cidr: a(cidr)})
+			}
+		}
+	}
+
+	// Then find all veths connected to the bridge
+	peerIDs, err := weavenet.ConnectedToBridgeVethPeerIds(bridgeName)
+	if err != nil {
+		return nil, err
+	}
+
+	// Now iterate over all containers to see if they have a network
+	// namespace with an attached interface
+	if dockerCli != nil {
+		for _, cid := range containerIDs {
+			container, err := dockerCli.InspectContainer(cid)
+			if err != nil {
+				if _, ok := err.(*docker.NoSuchContainer); ok {
+					continue
+				}
+				return nil, err
+			}
+			if container.State.Pid != 0 {
+				netDevs, err := weavenet.GetNetDevsByVethPeerIds(container.State.Pid, peerIDs)
+				if err != nil {
+					return nil, err
+				}
+				add(cid, true, netDevs)
+			}
+		}
+	} else {
+		// If we don't have a Docker connection, iterate over all processes
+		pids, err := common.AllPids("/proc")
+		if err != nil {
+			return nil, err
+		}
+		for _, pid := range pids {
+			netDevs, err := weavenet.GetNetDevsByVethPeerIds(pid, peerIDs)
+			if err != nil {
+				return nil, err
+			}
+			add(api.NoContainerID, false, netDevs)
+		}
+	}
+	return addrs, nil
+}

prog/weaveutil/attach.go

@@ -14,11 +14,12 @@ import (
 
 func attach(args []string) error {
 	if len(args) < 4 {
-		cmdUsage("attach-container", "[--no-multicast-route] [--keep-tx-on] <container-id> <bridge-name> <mtu> <cidr>...")
+		cmdUsage("attach-container", "[--no-multicast-route] [--keep-tx-on] [--hairpin-mode=true|false] <container-id> <bridge-name> <mtu> <cidr>...")
 	}
 
 	keepTXOn := false
 	withMulticastRoute := true
+	hairpinMode := true
 	for i := 0; i < len(args); {
 		switch args[i] {
 		case "--no-multicast-route":
@@ -27,6 +28,9 @@ func attach(args []string) error {
 		case "--keep-tx-on":
 			keepTXOn = true
 			args = append(args[:i], args[i+1:]...)
+		case "--hairpin-mode=false":
+			hairpinMode = false
+			args = append(args[:i], args[i+1:]...)
 		default:
 			i++
 		}
@@ -55,7 +59,7 @@ func attach(args []string) error {
 		return err
 	}
 
-	err = weavenet.AttachContainer(weavenet.NSPathByPid(pid), fmt.Sprint(pid), weavenet.VethName, args[1], mtu, withMulticastRoute, cidrs, keepTXOn)
+	err = weavenet.AttachContainer(weavenet.NSPathByPid(pid), fmt.Sprint(pid), weavenet.VethName, args[1], mtu, withMulticastRoute, cidrs, keepTXOn, hairpinMode)
 	// If we detected an error but the container has died, tell the user that instead.
 	if err != nil && !processExists(pid) {
 		err = fmt.Errorf("Container %s died", args[0])

prog/weaveutil/cni.go

@@ -3,7 +3,8 @@ package main
 import (
 	"os"
 
-	cni "github.com/appc/cni/pkg/skel"
+	cni "github.com/containernetworking/cni/pkg/skel"
+	"github.com/containernetworking/cni/pkg/version"
 
 	weaveapi "github.com/weaveworks/weave/api"
 	"github.com/weaveworks/weave/common"
@@ -14,13 +15,13 @@ import (
 func cniNet(args []string) error {
 	weave := weaveapi.NewClient(os.Getenv("WEAVE_HTTP_ADDR"), common.Log)
 	n := netplugin.NewCNIPlugin(weave)
-	cni.PluginMain(n.CmdAdd, n.CmdDel)
+	cni.PluginMain(n.CmdAdd, n.CmdDel, version.PluginSupports("0.1.0", "0.2.0", "0.3.0"))
 	return nil
 }
 
 func cniIPAM(args []string) error {
 	weave := weaveapi.NewClient(os.Getenv("WEAVE_HTTP_ADDR"), common.Log)
 	i := ipamplugin.NewIpam(weave)
-	cni.PluginMain(i.CmdAdd, i.CmdDel)
+	cni.PluginMain(i.CmdAdd, i.CmdDel, version.PluginSupports("0.1.0", "0.2.0", "0.3.0"))
 	return nil
 }

prog/weaveutil/docker_tls_args.go

@@ -45,7 +45,6 @@ func dockerTLSArgs(args []string) error {
 			switch {
 			case arg == "-d" || arg == "daemon":
 				isDaemon = true
-				break
 			case arg == "--tls", arg == "--tlsverify":
 				tlsArgs = append(tlsArgs, arg)
 			case strings.HasPrefix(arg, "--tls"):

prog/weaveutil/unique_id.go

@@ -16,6 +16,6 @@ func uniqueID(args []string) error {
 	if err != nil {
 		return err
 	}
-	fmt.Printf(uid)
+	fmt.Print(uid)
 	return nil
 }

router/fastdp.go

@@ -577,8 +577,8 @@ type fastDatapathForwarder struct {
 	confirmed         bool
 	remoteAddr        *net.UDPAddr
 	heartbeatInterval time.Duration
-	heartbeatTimer    *time.Timer
-	heartbeatTimeout  *time.Timer
+	heartbeatTimer    *time.Timer // for sending
+	heartbeatTimeout  *time.Timer // for receiving
 	ackedHeartbeat    bool
 	stopChan          chan struct{}
 	stopped           bool
@@ -856,7 +856,9 @@ func (fwd *fastDatapathForwarder) handleCryptoInitSARemote(msg []byte) {
 		return
 	}
 
-	if !fwd.isOutboundIPSecEstablished {
+	// FastDatapathCryptoInitSARemote can be received before Confirm'ing
+	// connection, thus before InitSALocal.
+	if fwd.confirmed && !fwd.isOutboundIPSecEstablished {
 		fwd.isOutboundIPSecEstablished = true
 		fwd.heartbeatTimer.Reset(0)
 	}
@@ -1222,7 +1224,7 @@ func (fastdp *FastDatapath) send(fops FlowOp, frame []byte, lock *fastDatapathLo
 	}
 
 	if fastdp.isHairpinFlow(&flow) {
-		log.Error("Vetoed installation of hairpin flow ", flow)
+		log.Warning("Vetoed installation of hairpin flow ", flow)
 		return
 	}
 

router/network_router.go

@@ -51,7 +51,7 @@ type NetworkRouter struct {
 	db   db.DB
 }
 
-func NewNetworkRouter(config mesh.Config, networkConfig NetworkConfig, name mesh.PeerName, nickName string, overlay NetworkOverlay, db db.DB) *NetworkRouter {
+func NewNetworkRouter(config mesh.Config, networkConfig NetworkConfig, name mesh.PeerName, nickName string, overlay NetworkOverlay, db db.DB) (*NetworkRouter, error) {
 	if overlay == nil {
 		overlay = NullNetworkOverlay{}
 	}
@@ -59,7 +59,11 @@ func NewNetworkRouter(config mesh.Config, networkConfig NetworkConfig, name mesh
 		networkConfig.Bridge = NullBridge{}
 	}
 
-	router := &NetworkRouter{Router: mesh.NewRouter(config, name, nickName, overlay, common.LogLogger()), NetworkConfig: networkConfig, db: db}
+	meshRouter, err := mesh.NewRouter(config, name, nickName, overlay, common.LogLogger())
+	if err != nil {
+		return nil, err
+	}
+	router := &NetworkRouter{Router: meshRouter, NetworkConfig: networkConfig, db: db}
 	router.Peers.OnInvalidateShortIDs(overlay.InvalidateShortIDs)
 	router.Routes.OnChange(overlay.InvalidateRoutes)
 	router.Macs = NewMacCache(macMaxAge,
@@ -67,7 +71,7 @@ func NewNetworkRouter(config mesh.Config, networkConfig NetworkConfig, name mesh
 			log.Println("Expired MAC", mac, "at", peer)
 		})
 	router.Peers.OnGC(func(peer *mesh.Peer) { router.Macs.Delete(peer) })
-	return router
+	return router, nil
 }
 
 // Start listening for TCP connections, locally captured packets, and

router/sleeve.go

@@ -6,7 +6,6 @@
 package router
 
 import (
-	"bytes"
 	"encoding/binary"
 	"fmt"
 	"io"
@@ -1039,6 +1038,9 @@ func (sender *udpSenderDF) dial() error {
 	laddr := &net.IPAddr{IP: sender.localIP}
 	raddr := &net.IPAddr{IP: sender.remoteIP}
 	s, err := net.DialIP("ip4:UDP", laddr, raddr)
+	if err != nil {
+		return err
+	}
 
 	f, err := s.File()
 	if err != nil {
@@ -1059,7 +1061,7 @@ func (sender *udpSenderDF) dial() error {
 
 func (sender *udpSenderDF) send(msg []byte, raddr *net.UDPAddr) error {
 	// Ensure we have a socket sending to the right IP address
-	if sender.socket == nil || !bytes.Equal(sender.remoteIP, raddr.IP) {
+	if sender.socket == nil || !sender.remoteIP.Equal(raddr.IP) {
 		sender.remoteIP = raddr.IP
 		if err := sender.dial(); err != nil {
 			return err
@@ -1111,7 +1113,7 @@ func (sender *udpSenderDF) close() error {
 }
 
 func udpAddrsEqual(a *net.UDPAddr, b *net.UDPAddr) bool {
-	return bytes.Equal(a.IP, b.IP) && a.Port == b.Port && a.Zone == b.Zone
+	return a.IP.Equal(b.IP) && a.Port == b.Port && a.Zone == b.Zone
 }
 
 func allZeros(s []byte) bool {

site/building.md

@@ -1,6 +1,7 @@
 ---
 title: Building Weave Net
 menu_order: 120
+search_type: Documentation
 ---
 You only need to build Weave Net if you want to work on the Weave Net codebase
 (or you just enjoy building software).

site/cni-plugin.md

@@ -1,6 +1,7 @@
 ---
 title: Integrating Kubernetes and Mesos via the CNI Plugin
 menu_order: 65
+search_type: Documentation
 ---
 
 > The recommended way of using Weave with Kubernetes is via the new
@@ -11,11 +12,11 @@ menu_order: 65
 CNI, the [_Container Network Interface_](https://github.com/containernetworking/cni),
 is a proposed standard for configuring network interfaces for Linux
 application containers.  CNI is supported by
-[Kubernetes](http://kubernetes.io/), [Apache Mesos](mesos.apache.org)
+[Kubernetes](http://kubernetes.io/), [Apache Mesos](http://mesos.apache.org)
 and 
 [others](https://github.com/containernetworking/cni#who-is-using-cni).
 
-###Installing the Weave Net CNI plugin
+### Installing the Weave Net CNI plugin
 
 If your machine has the directories normally used to host CNI plugins, 
 then the Weave Net CNI plugin is installed when you run `weave setup`.
@@ -29,7 +30,7 @@ Then run:
 
     weave setup
 
-####Launching Weave Net
+#### Launching Weave Net
 
 To create a network that spans multiple hosts, the Weave peers must be connected to each other.  
 This is accomplished by specifying the other hosts during `weave launch` or via
@@ -40,7 +41,7 @@ for a discussion on peer connections.
 
     weave launch <peer hosts>
 
-####Using the CNI network configuration file
+#### Using the CNI network configuration file
 
 All CNI plugins are configured by a JSON file in the directory
 `/etc/cni/net.d/`.  `weave setup` installs a minimal configuration
@@ -64,9 +65,9 @@ The following other fields in the spec are supported:
 - `ipam / subnet` - default is to use Weave's IPAM default subnet
 - `ipam / gateway` - default is to use the Weave bridge IP address (allocated by `weave expose`)
 
-###Using the Weave Net CNI plugin
+### Using the Weave Net CNI plugin
 
-####Configuring Kubernetes to use the CNI Plugin
+#### Configuring Kubernetes to use the CNI Plugin
 
 After you've launched Weave and peered your hosts, you can configure
 Kubernetes to use Weave, by adding the following options to the
@@ -79,7 +80,7 @@ for more details.
 
 Now, whenever Kubernetes starts a pod, it will be attached to the Weave network.
 
-####Configuring Mesos to use the CNI plugin
+#### Configuring Mesos to use the CNI plugin
 
 To use the CNI plugin, the Mesos Agent must be started with reference
 to the CNI configuration and binary directories:
@@ -105,7 +106,7 @@ connected to the weave network)"
 For more information, see the 
 [Mesos documentation](http://mesos.apache.org/documentation/cni/).
 
-###Caveats
+### Caveats
 
 - The Weave Net router container must be running for CNI to allocate addresses
 - The CNI plugin does not add entries to weaveDNS.

site/faq.md

@@ -1,10 +1,11 @@
 ---
 title: FAQ
-menu_order: 100
+menu_order: 200
+search_type: Documentation
 ---
 
 
-
+<a name="container-ip"></a>
 **Q: How do I obtain the IP of a specific container when I'm using Weave?**
 
 You can use `weave ps <container>` to see the allocated address of a container on a Weave network.  
@@ -12,6 +13,7 @@ You can use `weave ps <container>` to see the allocated address of a container o
 See [Troubleshooting Weave - List attached containers](/site/troubleshooting.md#list-attached-containers).
 
 
+<a name="specific-ip"></a>
 **Q: My dockerized app needs to check the request of an application that uses a static IP. Is it possible to manually change the IP of a container?**
 
 
@@ -20,11 +22,13 @@ You can manually change the IP of a container using [Classless Inter-Domain Rout
 For more information, refer to [Manually Specifying the IP Address of a Container](/site/using-weave/manual-ip-address.md). 
 
 
+<a name="expose-container"></a>
 **Q: How do I expose one of my containers to the outside world?**
 
 Exposing a container to the outside world is described in [Exporting Services](/site/using-weave/service-management.md#exporting).
 
 
+<a name="legacy-network"></a>
 **Q: Can I connect my existing 'legacy' network with a Weave container network?**
 
 Yes you can. 
@@ -36,19 +40,15 @@ A simple way to accomplish this would be to run Weave on the host and then run,
 Yet another option is to expose a port from the container on host B and then connect to it. You can read about exposing ports in [Exporting Services](/site/using-weave/service-management.md#exporting).
 
 
+<a name="duplicate-ip"></a>
 **Q: Why am I seeing the same IP address assigned to two different containers on different hosts?**
 
-Under normal circumstances, this should never happen, but it can occur if  `weave forget` and `weave rmpeer` was run on more than one host. 
+Under normal circumstances, this should never happen, but it can occur if `weave rmpeer` was run on more than one host. 
 
-You cannot call `weave rmpeer` on more than one host. The address space, which was owned by the stale peer cannot be left dangling, and as a result it gets reassigned. In this instance, the address is reassigned to the peer on which `weave rmpeer` was run. Therefore, if you run `weave forget` and then `weave rmpeer` on more than one host at a time, it results in duplicate IPs on more than one host.
-
-Once the peers detect the inconsistency, they log the error and drop the connection that supplied the inconsistent data. The rest of the peers will carry on with their view of the world, but the network will not function correctly.
-
-Some peers may be able to communicate their claim to the others before they run `rmpeer` (i.e. it's a race), so what you can expect is a few cliques of peers that are still talking to each other, but repeatedly dropping attempted connections with peers in other cliques.
-
-For more information on see [Allocating IP Addresses](/site/ipam.md) and also, [Starting, Stopping and Removing Peers](/site/ipam/stop-remove-peers-ipam.md).
+For more information see [Starting, Stopping and Removing Peers](/site/ipam/stop-remove-peers-ipam.md).
 
 
+<a name="dead-node"></a>
 **Q: What is the best practice for resetting a node that goes out of service?**
 
 When a node goes out of service, the best option is to call `weave rmpeer` on one host and then `weave forget` on all the other hosts.
@@ -56,6 +56,7 @@ When a node goes out of service, the best option is to call `weave rmpeer` on on
 See [Starting, Stopping and Removing Peers](/site/ipam/stop-remove-peers-ipam.md) for an in-depth discussion.
 
 
+<a name="performance"></a>
 **Q: What about Weave's performance? Are software defined network overlays just as fast as native networking?**
 
 All virtualization techniques have some overhead, and Weave's overhead is typically around 2-3%. Unless your system is completely bottlenecked on the network, you won't notice this during normal operation. 
@@ -65,6 +66,7 @@ Weave Net also automatically uses the fastest datapath between two hosts. When W
 For more information about fast datapath see [How Fast Datapath Works](/site/how-it-works/fastdp-how-it-works.md).
 
 
+<a name="query-fastdp"></a>
 **Q: How can I tell if Weave is using fast datapath (fastdp) or not?**
 
 To view whether Weave is using fastdp or not, you can run, `weave status connections`
@@ -72,24 +74,36 @@ To view whether Weave is using fastdp or not, you can run, `weave status connect
 For more information on this command, see [Using Fast Datapath](/site/using-weave/fastdp.md).
 
 
+<a name="encrypted-fastdp"></a>
 **Q: Does encryption work with fastdp?**
 
-Encryption does not work with fast datapath. If you enable encryption using the `--password` option to launch Weave (or you use the `WEAVE_PASSWORD` environment variable), fast datapath will by default be disabled. 
-
-You can however have a mixture of fast datapath connections over trusted links, as well as, encrypted connections over untrusted links.
+Yes, 1.9 version of Weave Net added the encryption feature to fastdp.
 
 See [Using Fast Datapath](/site/using-weave/fastdp.md) for more information.
 
+<a name="app-isolation"></a>
 **Q: Can I create multiple networks where containers can communicate on one network, but are isolated from containers on other networks?**
 
 Yes, of course!  Weave allows you to run isolated networks and still allow open communications between individual containers from those isolated networks. You can find information on how to do this in [Application Isolation](/site/using-weave/application-isolation.md).
 
 
-**Q: Which ports does Weave Net use (e.g. if I am configuring a firewall) ?**
+**<a name=ports></a>Q: Which ports does Weave Net use (e.g. if I am configuring a firewall) ?**
 
 You must permit traffic to flow through TCP 6783 and UDP 6783/6784,
 which are Weave’s control and data ports.
 
+The daemon also uses TCP port 6782 for [metrics](/site/metrics.md), but
+you would only need to open up this port if you wish to collect metrics
+from another host.
+
+The Weave Net daemon listens on localhost (127.0.0.1) TCP port 6784
+for commands from other Weave Net components. This port should not be
+opened to other hosts.
+
+When using encrypted fast datapath, make sure that underlying
+network does not block ESP traffic (IP protocol 50). For instance
+on Google Cloud Platform a firewall rule for allowing ESP traffic has
+to be installed.
 
 **<a name=own-image></a>Q: Why do you use your own Docker image `weaveworks/ubuntu`?**
 

site/features.md

@@ -1,6 +1,7 @@
 ---
 title: Feature Overview
 menu_order: 20
+search_type: Documentation
 ---
 
  * [Virtual Ethernet Switch](#virtual-ethernet-switch)
@@ -27,7 +28,7 @@ menu_order: 20
 
 For step-by-step instructions on how to use Weave Net, see [Using Weave Net](/site/using-weave.md).
 
-###<a name="virtual-ethernet-switch"></a>Virtual Ethernet Switch
+### <a name="virtual-ethernet-switch"></a>Virtual Ethernet Switch
 
 Weave Net creates a virtual network that connects Docker containers
 deployed across multiple hosts.
@@ -46,20 +47,16 @@ over Weave Net.
 To start using Weave Net, see [Installing Weave Net](/site/installing-weave.md) 
 and [Using Weave Net](/site/using-weave.md).
 
-###<a name="fast-data-path"></a>Fast Datapath
+### <a name="fast-data-path"></a>Fast Datapath
 
 Weave Net automatically chooses the fastest available method to 
 transport data between peers. The best performing of these 
 (the 'fast datapath') offers near-native throughput and latency.
 
-Fast datapath does not support encryption. For full details on configuring
-Weave when you have connections that traverse untrusted networks,
-see [Securing Connections Across Untrusted Networks](/site/using-weave/security-untrusted-networks.md) for more details.
-
 See [Using Fast Datapath](/site/using-weave/fastdp.md) and
 [How Fast Datapath Works](/site/how-it-works/fastdp-how-it-works.md).
 
-###<a name="docker"></a>Seamless Docker Integration (Weave Docker API Proxy)
+### <a name="docker"></a>Seamless Docker Integration (Weave Docker API Proxy)
 
 Weave Net includes a [Docker API Proxy](/site/weave-docker-api.md), which can be 
 used to start containers using the Docker [command-line interface](https://docs.docker.com/reference/commandline/cli/) or the [remote API](https://docs.docker.com/reference/api/docker_remote_api/), and attach them to the Weave network before they begin execution.
@@ -77,7 +74,7 @@ policy, are re-attached to the Weave network by the `Weave Docker API Proxy`.
 See [Integrating Docker via the API Proxy](/site/weave-docker-api.md).
 
 
-###<a name="plugin"></a>Weave Network Docker Plugin
+### <a name="plugin"></a>Weave Network Docker Plugin
 
 Weave Net can also be used as a [Docker plugin](https://docs.docker.com/engine/extend/plugins_network/).  A Docker network 
 named `weave` is created by `weave launch`, which is used as follows:
@@ -92,14 +89,14 @@ when there are network connectivity problems.
 See [Integrating Docker via the Network Plugin](/site/plugin.md) for more details.
 
 
-###<a name="cniplugin"></a>Weave Network CNI Plugin
+### <a name="cniplugin"></a>Weave Network CNI Plugin
 
 Weave can be used as a plugin to systems that support the [Container Network Interface](https://github.com/appc/cni), such as Kubernetes and Mesosphere.
 
 See [Integrating Kubernetes and Mesos via the CNI Plugin](/site/cni-plugin.md) for more details.
 
 
-###<a name="addressing"></a>IP Address Management (IPAM)
+### <a name="addressing"></a>IP Address Management (IPAM)
  
 Containers are automatically allocated a unique IP address. To view the addresses allocated by Weave, run `weave ps`.
 
@@ -110,7 +107,7 @@ For a discussion on how Weave Net uses IPAM, see [Automatic IP Address Managemen
 [the basics of IP addressing](/site/how-it-works/ip-addresses.md) for an explanation of addressing and private networks. 
 
 
-###<a name="naming-and-discovery"></a>Naming and Discovery
+### <a name="naming-and-discovery"></a>Naming and Discovery
  
 Named containers are automatically registered in [weaveDNS](/site/weavedns.md), 
 and are discoverable by using standard, simple name lookups:
@@ -123,7 +120,7 @@ WeaveDNS also supports [load balancing](/site/weavedns/load-balance-fault-weaved
 
 See [Discovering Containers with WeaveDNS](/site/weavedns.md).
  
-###<a name="application-isolation"></a>Application Isolation
+### <a name="application-isolation"></a>Application Isolation
 
 A single Weave network can host multiple, isolated 
 applications, with each application's containers being able 
@@ -140,13 +137,13 @@ See [Isolating Applications](/site/using-weave/application-isolation.md)
 for information on how to use the isolation-through-subnets 
 technique with Weave Net.
 
-###<a name="network-policy"></a>Network Policy
+### <a name="network-policy"></a>Network Policy
 
 The Weave [Kubernetes Addon](/site/kube-addon.md) includes a network
 policy controller that implements [Kubernetes Network
 Policies](http://kubernetes.io/docs/user-guide/networkpolicies/).
 
-###<a name="dynamic-network-attachment"></a>Dynamic Network Attachment
+### <a name="dynamic-network-attachment"></a>Dynamic Network Attachment
 
 At times, you may not know the application network for a 
 given container in advance. In these cases, you can take 
@@ -157,7 +154,7 @@ See [Dynamically Attaching and Detaching Containers](/site/using-weave/dynamical
 for details. 
 
 
-###<a name="security"></a>Security
+### <a name="security"></a>Security
 
 In keeping with our ease-of-use philosophy, the cryptography 
 in Weave Net is intended to satisfy a particular user requirement: 
@@ -177,19 +174,20 @@ mechanism which you can use in conjunction with or as an
 alternative to any other security technologies you have 
 running alongside Weave. 
 
-Weave Net implements encryption and security using the Go version of [Daniel J. Bernstein's NaCl library](http://nacl.cr.yp.to/index.html).
+Weave Net implements encryption and security using the Go version of [Daniel J.  Bernstein's NaCl library](http://nacl.cr.yp.to/index.html),
+and, additionally in the case of encrypted fast datapath using [the cryptography framework of the Linux kernel](https://en.wikipedia.org/wiki/Crypto_API_(Linux)).
 
 For information on how to secure your Docker network connections, see [Securing Connections Across Untrusted Networks](/site/using-weave/security-untrusted-networks.md) and for a more technical discussion on how Weave implements encryption see, [Weave Encryption](/site/how-it-works/encryption.md) and [How Weave Implements Encryption](/site/how-it-works/encryption-implementation.md).
 
 
-###<a name="host-network-integration"></a>Host Network Integration
+### <a name="host-network-integration"></a>Host Network Integration
 
 Weave Net application networks can be integrated with a host's network, and establish connectivity between the host and 
 application containers anywhere.
 
 See [Integrating with the Host Network](/site/using-weave/host-network-integration.md).
 
-###<a name="services"></a>Managing Services: Exporting, Importing, Binding and Routing
+### <a name="services"></a>Managing Services: Exporting, Importing, Binding and Routing
  
  * **Exporting Services** - Services running in containers on a Weave network can be made accessible to the outside world or to other networks.
  * **Importing Services** - Applications can run anywhere, and yet still be made accessible by specific application containers or services.
@@ -198,7 +196,7 @@ See [Integrating with the Host Network](/site/using-weave/host-network-integrati
 
 See [Managing Services - Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md) for instructions on how to manage services on a Weave container network. 
 
-###<a name="multi-cloud-networking"></a>Multi-Cloud Networking
+### <a name="multi-cloud-networking"></a>Multi-Cloud Networking
 
 Weave can network containers hosted in different cloud providers 
 or data centers. For example, you can run an application consisting 
@@ -209,7 +207,7 @@ of containers that run on [Google Compute Engine](https://cloud.google.com/compu
 See [Enabling Multi-Cloud networking and Muti-hop Routing](/site/using-weave/multi-cloud-multi-hop.md).
 
 
-###<a name="multi-hop-routing"></a>Multi-Hop Routing
+### <a name="multi-hop-routing"></a>Multi-Hop Routing
 
 A network of containers across more than two hosts can be established 
 even when there is only partial connectivity between the hosts. Weave Net
@@ -219,14 +217,14 @@ of connected hosts between them.
 See [Enabling Multi-Cloud networking and Multi-hop Routing](/site/using-weave/multi-cloud-multi-hop.md).
 
 
-###<a name="dynamic-topologies"></a>Dynamic Topologies
+### <a name="dynamic-topologies"></a>Dynamic Topologies
 
 Hosts can be added to or removed from a Weave network without stopping
 or reconfiguring the remaining hosts. See [Adding and Removing Hosts
 Dynamically.](/site/using-weave/finding-adding-hosts-dynamically.md)
 
 
-###<a name="container-mobility"></a>Container Mobility
+### <a name="container-mobility"></a>Container Mobility
 
 Containers can be moved between hosts without requiring any 
 reconfiguration or, in many cases, restarts of other containers. 
@@ -236,7 +234,7 @@ with the same IP address as it was given originally.
 See [Managing Services - Exporting, Importing, Binding and Routing](/site/using-weave/service-management.md), in particular, Routing Services for more information on container mobility. 
 
 
-###<a name="fault-tolerance"></a>Fault Tolerance
+### <a name="fault-tolerance"></a>Fault Tolerance
 
 Weave Net peers continually exchange topology information, and 
 monitor and (re)establish network connections to other peers. 

site/how-it-works.md

@@ -1,6 +1,7 @@
 ---
 title: Understanding how Weave Net Works
 menu_order: 90
+search_type: Documentation
 ---
 
 

site/how-it-works/encryption-implementation.md

@@ -1,6 +1,7 @@
 ---
 title: How Weave Net Implements Encryption
 menu_order: 70
+search_type: Documentation
 ---
 
 This section describes some details of Weave Net's built-in
@@ -13,7 +14,7 @@ This section describes some details of Weave Net's built-in
 
 
 
-####<a name="ephemeral-key"></a>Establishing the Ephemeral Session Key
+#### <a name="ephemeral-key"></a>Establishing the Ephemeral Session Key
 
 For every connection between peers, a fresh public/private key pair is
 created at both ends, using NaCl's `GenerateKey` function. The public
@@ -48,7 +49,7 @@ not be able to form valid ephemeral session keys.
 The same ephemeral session key is used for both TCP and UDP traffic
 between two peers.
 
-###<a name="csprng"></a> Key Generation and The Linux CSPRNG
+### <a name="csprng"></a> Key Generation and The Linux CSPRNG
 
 Generating fresh keys for every connection
 provides forward secrecy at the cost of placing a demand on the Linux
@@ -84,7 +85,7 @@ Weave Net's demand on `/dev/urandom` is causing you problems with blocking
 `/dev/random` reads, please get in touch with us - we'd love to hear
 about your use case.
 
-####<a name="tcp"></a>Encypting and Decrypting TCP Messages
+#### <a name="tcp"></a>Encypting and Decrypting TCP Messages
 
 TCP connection are only used to exchange topology information between
 peers, via a message-based protocol. Encryption of each message is
@@ -105,7 +106,9 @@ polarity. As a result the receiver will only be able to decrypt a
 message if it has the expected sequence number. This prevents replay
 attacks.
 
-####<a name="udp"></a>Encrypting and Decrypting UDP Packets
+#### <a name="udp"></a>Encrypting and Decrypting UDP Packets
+
+##### Sleeve
 
 UDP connections carry captured traffic between peers. For a UDP packet
 sent between peers that are using crypto, the encapsulation looks as
@@ -179,7 +182,33 @@ contained in the set. The window spans at least 2^20 message sequence
 numbers, and hence any re-ordering between the most recent ~1 million
 messages is handled without dropping messages.
 
+##### Fast Datapath
+
+Encryption in fastdp uses [the ESP protocol of IPsec](https://tools.ietf.org/html/rfc2406)
+in the transport mode. Each VXLAN packet is encrypted with
+[AES in GCM mode](https://tools.ietf.org/html/rfc4106aesgcm), with 32 byte key and
+4 byte salt. This combo provides the following security properties:
+
+* Data confidentiality.
+* Data origin authentication.
+* Integrity.
+* Anti-replay.
+* Limited traffic flow confidentiality as VXLAN packets are fully encrypted.
+
+For each connection direction, a different AES-GCM key and salt is used.
+The pairs are derived with [HKDF](https://tools.ietf.org/html/rfc5869)
+to which we pass a randomly generated 32 byte salt transferred over the encrypted
+control plane channel between peers.
+
+To prevent from replay attacks, which are possible because of the size of
+sequence number field in ESP (4 bytes), we use extended sequence numbers
+implemented by [ESN](https://tools.ietf.org/html/rfc4304).
+
+Authentication of ESP packet integrity and origin is ensured by 16 byte
+Integrity Check Value of AES-GCM.
+
 **See Also**
 
  * [architecture documentation](https://github.com/weaveworks/weave/blob/master/docs/architecture.txt)
+ * [fastdp encryption](https://github.com/weaveworks/weave/blob/master/docs/fastdp-crypto.md)
  * [Securing Containers Across Untrusted Networks](/site/using-weave/security-untrusted-networks.md)

site/how-it-works/encryption.md

@@ -1,6 +1,7 @@
 ---
 title: Encryption and Weave Net
 menu_order: 60
+search_type: Documentation
 ---
 
 
@@ -9,7 +10,8 @@ Weave Net peers
 communication
 [can be encrypted](/site/using-weave/security-untrusted-networks.md).
 
-Encryption is accomplished using the [NaCl](http://nacl.cr.yp.to/)
+Encryption of control plane traffic (TCP) and data plane traffic (UDP) of sleeve
+overlay is accomplished using the [NaCl](http://nacl.cr.yp.to/)
 crypto libraries, employing Curve25519, XSalsa20 and Poly1305 to
 encrypt and authenticate messages. Weave Net protects against
 injection and replay attacks for traffic forwarded between peers.
@@ -32,6 +34,11 @@ extensions to TLS such as [DTLS](https://tools.ietf.org/html/rfc4347)
 which can operate over UDP, these are not widely implemented and
 deployed.
 
+In the case of fast datapath, data plane traffic is encrypted by using
+[ESP of IPsec](https://tools.ietf.org/html/rfc2406).
+The process of encryption is handled by the Linux kernel and is controlled via
+the IP transformation framework (XFRM).
+
 **See Also**
 
  * [How Weave Implements Encryption](/site/how-it-works/encryption-implementation.md)

site/how-it-works/fastdp-how-it-works.md

@@ -1,6 +1,7 @@
 ---
 title: How Fast Datapath Works
 menu_order: 30
+search_type: Documentation
 ---
 
 

site/how-it-works/ip-addresses.md

@@ -1,6 +1,7 @@
 ---
 title: IP Addresses, Routes and Networks
 menu_order: 50
+search_type: Documentation
 ---
 
 

site/how-it-works/network-topology.md

@@ -1,6 +1,7 @@
 ---
 title: How Weave Net Interprets Network Topology
 menu_order: 20
+search_type: Documentation
 ---
 
 This section contains the following topics: 
@@ -11,7 +12,7 @@ This section contains the following topics:
  * [What Happens When The Topology is Out of Date?](#out-of-date-topology)
 
 
-###<a name="topology"></a>Communicating Topology Among Peers
+### <a name="topology"></a>Communicating Topology Among Peers
 
 Topology messages capture which peers are connected to other peers. 
 Weave peers communicate their knowledge of the topology
@@ -49,7 +50,7 @@ improved update containing them is gossiped.
 If the update mentions a peer that the receiver does not know, then
 the entire update is ignored.
 
-####<a name="messages"></a>How Messages Are Formed
+#### <a name="messages"></a>How Messages Are Formed
 
 Every gossip message is structured as follows:
 
@@ -120,14 +121,14 @@ which the structure is:
     | Connection N: Established         |
     +-----------------------------------+
 
-####<a name="removing-peers"></a>Removing Peers
+#### <a name="removing-peers"></a>Removing Peers
 
 If a peer, after receiving a topology update, sees that another peer
 no longer has any connections within the network, it drops all
 knowledge of that second peer.
 
 
-####<a name="out-of-date-topology"></a>What Happens When The Topology is Out of Date?
+#### <a name="out-of-date-topology"></a>What Happens When The Topology is Out of Date?
 
 The propagation of topology changes to all peers is not instantaneous.
 Therefore, it is very possible for a node elsewhere in the network to have an

site/how-it-works/router-encapsulation.md

@@ -1,6 +1,7 @@
 ---
 title: Weave Net Router Sleeve Encapsulation
 menu_order: 10
+search_type: Documentation
 ---
 
 When the Weave Net router forwards packets to peers in `sleeve` mode

site/installing-weave.md

@@ -1,11 +1,12 @@
 ---
 title: Installing Weave Net
 menu_order: 30
+search_type: Documentation
 ---
 
 
 Ensure you are running Linux (kernel 3.8 or later) and have Docker
-(version 1.6.0 or later) installed. 
+(version 1.10.0 or later) installed.
 
 Install Weave Net by running the following:
 
@@ -20,13 +21,13 @@ After your VM is setup with Docker Machine, Weave Net can be launched directly f
 
 With Weave Net downloaded onto your VMs or hosts, you are ready to launch a Weave network and deploy apps onto it. See [Using Weave Net](/site/using-weave.md).
 
-###Quick Start Screencast
+### Quick Start Screencast
 
 <a href="https://youtu.be/kihQCCT1ykE" target="_blank">
   <img src="hello-screencast.png" alt="Click to watch the screencast" />
 </a>
 
-###Checkpoint
+### Checkpoint
 
 Weave Net [periodically contacts Weaveworks servers for available
 versions](https://github.com/weaveworks/go-checkpoint).  New versions
@@ -45,12 +46,12 @@ To disable this check, run the following before launching Weave Net:
 
     export CHECKPOINT_DISABLE=1
 
-###Guides for Specific Platforms
+### Guides for Specific Platforms
 
 CoreOS users see [here](/guides/networking-docker-containers-with-weave-on-coreos/) for an example of installing Weave using cloud-config.
 
 Amazon ECS users see [here](https://github.com/weaveworks/integrations/blob/master/aws/ecs/README.md)
-for the latest Weave AMIs and [here](http://weave.works/guides/service-discovery-with-weave-aws-ecs.html) to get started with Weave Net on ECS.
+for the latest Weave AMIs and [here](https://www.weave.works/docs/tutorials/old-guides/ecs/) to get started with Weave Net on ECS.
 
 If you're on Amazon EC2, the standard installation instructions at the
 top of this page, provide the simplest setup and the most flexibility.
@@ -58,6 +59,9 @@ A [special no-overlay mode for EC2](/site/using-weave/awsvpc.md) can
 optionally be enabled, which allows containers to communicate at the
 full speed of the underlying network.
 
+To make encryption in fast datapath work on Google Cloud Platform, see
+[here](/site/faq.md#ports).
+
 **See Also** 
 
  * [Using Weave Net](/site/using-weave.md)

site/installing-weave/systemd.md

@@ -1,6 +1,7 @@
 ---
 title: Using Weave with Systemd
 menu_order: 130
+search_type: Documentation
 ---
 
 

site/introducing-weave.md

@@ -1,6 +1,7 @@
 ---
 title: Introducing Weave Net
 menu_order: 10
+search_type: Documentation
 ---
 
 
@@ -9,52 +10,52 @@ Weave Net creates a virtual network that connects Docker containers across multi
 Services provided by application containers on the weave network can be exposed to the outside world, regardless of where they are running. Similarly, existing internal systems can be opened to accept connections from application containers irrespective of their location.
 
 
-##Why Weave?
+## Why Weave?
 
-###Hassle Free Configuration
+### Hassle Free Configuration
 
 Weave Net simplifies setting up a container network. Because containers on a Weave network use standard port numbers, (for example MySQL’s default is port 3306), managing microservices is straightforward. Every container can find the IP of any other container using a simple DNS query on the container's name, and it can also communicate directly without NAT, without using port mappings or complicated ambassador linking.  And best of all deploying a Weave container network requires zero changes to your application’s code. 
 
 ![Weave Net Encapsulation](weave-net-overview.png)
 
-###Service Discovery
+### Service Discovery
 
 Weave Net implements service discovery by providing a fast "micro DNS" server at each node. You simply name containers and everything 'just works', including load balancing across multiple containers with the same name.  
 
-###No External Cluster Store Required
+### No External Cluster Store Required
 
 All other Docker networking plugins, including Docker's own "Overlay" driver, require that you set up Docker with a cluster store – a central database like Consul or Zookeeper – before you can even use them. Besides being difficult to set up, maintain and manage, every Docker host must also be in constant contact with the cluster store: if you lose the connection, even temporarily, then you cannot start or stop any containers.
 
 Weave Net is bundled with a Docker Network plugin that doesn't require an external cluster store. You can get started right away and you can start and stop containers even when there are network connectivity problems.  
 For information about the Weave Docker Plugin, see [How The Weave Network Plugin Works](/site/plugin/plugin-how-it-works.md).
 
-###Operates in Partially Connected Networks
+### Operates in Partially Connected Networks
 
 Weave Net can forward traffic between nodes, and it works even if the mesh network is only partially connected.  This means that you can have a mix of legacy systems and containerized apps and still use Weave Net to keep everything in communication. 
 
-###Weave Net is Fast
+### Weave Net is Fast
 
 Weave Net automatically chooses the fastest path between two hosts, offering near native throughput and latency, all without your intervention.  
 
 See [How Fast Datapath Works](/site/using-weave/fastdp.md) for more information.
 
-###Network Operations Friendly
+### Network Operations Friendly
 
 Weave uses industry-standard VXLAN encapsulation between hosts. This means you can continue using your favorite packet analyzing tools, such as ‘Wireshark’ to inspect and troubleshoot protocols.
 
-###Weave Net is Secure With Built-in Encryption
+### Weave Net is Secure
 
 Weave Net traverses firewalls without requiring a TCP add-on. You can encrypt your traffic, which allows you to connect to apps on hosts even across an untrusted network.  
 
-###Multicast Support
+### Multicast Support
 
 Multicast addressing and routing is fully supported in Weave Net. Data can be sent to one multicast address and it will be automatically broadcast to all of its recipients. 
 
-###NAT Traversal
+### NAT Traversal
 
 With Weave Net, deploy your applications - whether peer-to-peer file sharing, voice over IP or anything else - and take advantage of built-in NAT traversal. With Weave your app is portable, containerized and with its standardized approach to networking it gives you one less thing to worry about.
 
-###Run with Anything: Kubernetes, Mesos, Amazon ECS, ...
+### Run with Anything: Kubernetes, Mesos, Amazon ECS, ...
 
 Weave Net is a good choice if you want one tool for everything.  For example: In addition to Docker plugins, you can also use Weave as a Kubernetes plugin.  You can also use Weave with Amazon ECS or with Mesos and Marathon.  
 Refer to our Getting Started and Integration Guides for more information.

site/ipam.md

@@ -1,6 +1,7 @@
 ---
 title: Allocating IP Addresses
 menu_order: 70
+search_type: Documentation
 ---
 
 
@@ -112,7 +113,7 @@ peers. This can be useful to add peers to an existing fixed cluster
 (for example in response to a scale-out event) without worrying about
 adjusting initial peer counts accordingly.
 
-####<a name="quorum"></a> `--ipalloc-init consensus=` and How Quorum is Achieved
+#### <a name="quorum"></a> `--ipalloc-init consensus=` and How Quorum is Achieved
 
 Normally it isn't a problem to over-estimate the value supplied to
 `--ipalloc-init consensus=`, but if you supply a number that is too

site/ipam/allocation-multi-ipam.md

@@ -1,6 +1,7 @@
 ---
 title: Automatic Allocation Across Multiple Subnets
 menu_order: 10
+search_type: Documentation
 ---
 
 

site/ipam/stop-remove-peers-ipam.md

@@ -1,6 +1,7 @@
 ---
 title: Starting, Stopping and Removing Peers
 menu_order: 20
+search_type: Documentation
 ---
 
 
@@ -32,8 +33,26 @@ caught fire, you can go to one of the other hosts and run:
 Weave Net takes all the IP address ranges owned by host3 and transfers
 them to be owned by host1. The name "host3" is resolved via the
 'nickname' feature of Weave Net, which defaults to the local host
-name. Alternatively, you can supply a peer name as shown in `weave
-status`.
+name. Alternatively, you can supply a peer name as shown in `weave status`.
+
+### <a name="caution-rmpeer"></a>Caution###
+
+You cannot call `weave rmpeer` on more than one host. The address
+space, which was owned by the stale peer cannot be left dangling, and
+as a result it gets reassigned. In this instance, the address is
+reassigned to the peer on which `weave rmpeer` was run. Therefore, if
+you run `weave forget` and then `weave rmpeer` on more than one host
+at a time, it results in duplicate IPs on more than one host.
+
+Once the peers detect the inconsistency, they log the error and drop
+the connection that supplied the inconsistent data. The rest of the
+peers will carry on with their view of the world, but the network will
+not function correctly.
+
+Some peers may be able to communicate their claim to the others before
+they run `rmpeer` (i.e. it's a race), so what you can expect is a few
+cliques of peers that are still talking to each other, but repeatedly
+dropping attempted connections with peers in other cliques.
 
 **See Also**
 

site/ipam/troubleshooting-ipam.md

@@ -1,6 +1,7 @@
 ---
 title: Troubleshooting the IP Allocator
 menu_order: 30
+search_type: Documentation
 ---
 
 

site/kube-addon.md

@@ -1,32 +1,44 @@
 ---
 title: Integrating Kubernetes via the Addon
 menu_order: 63
+search_type: Documentation
 ---
 
 The following topics are discussed:
 
 * [Installation](#install)
-* [Network Policy Controller](#npc)
+ * [Upgrading Kubernetes to version 1.6](#kube-1.6-upgrade)
  * [Upgrading the Daemon Sets](#daemon-sets)
+ * [CPU and Memory Requirements](#resources)
+ * [Pod Eviction](#eviction)
 * [Network Policy Controller](#npc)
  * [Troubleshooting Blocked Connections](#blocked-connections)
  * [Changing Configuration Options](#configuration-options)
 
 
-##<a name="install"></a> Installation
+## <a name="install"></a> Installation
 
 Weave Net can be installed onto your CNI-enabled Kubernetes cluster
 with a single command:
 
+* Kubernetes versions `1.6` and above:
+
 ```
-kubectl apply -f https://git.io/weave-kube
+$ kubectl apply -f https://git.io/weave-kube-1.6
+```
+
+* Kubernetes versions up to `1.5`:
+
+```
+$ kubectl apply -f https://git.io/weave-kube
 ```
 
 After a few seconds, a Weave Net pod should be running on each
 Node and any further pods you create will be automatically attached to the Weave
 network.
 
-**Note:** This command requires Kubernetes 1.4 or later.
+**Note:** This command requires Kubernetes 1.4 or later, and we
+recommend your master node has at least two CPU cores.
 
 > CNI, the [_Container Network Interface_](https://github.com/containernetworking/cni),
 > is a proposed standard for configuring network interfaces for Linux
@@ -51,12 +63,27 @@ Shut down Kubernetes, and _on all nodes_ perform the following:
 Then relaunch Kubernetes and install the addon as described
 above.
 
-The URL [https://git.io/weave-kube](https://git.io/weave-kube) points
-to the YAML file for the latest release of the Weave Net addon.
+The URLs [https://git.io/weave-kube](https://git.io/weave-kube) and [https://git.io/weave-kube-1.6](https://git.io/weave-kube-1.6) point
+to the YAML file for the [latest release](https://github.com/weaveworks/weave/releases/tag/latest_release) of the Weave Net addon.
 Historic versions are archived on our [GitHub release
 page](https://github.com/weaveworks/weave/releases).
 
-###<a name="daemon-sets"></a> Upgrading the Daemon Sets
+## <a name="kube-1.6-upgrade"></a> Upgrading Kubernetes to version 1.6
+
+In version 1.6, Kubernetes has increased security, so we need to
+create a special service account to run Weave Net. This is done in
+the file `weave-daemonset-k8s-1.6.yaml` attached to the [Weave Net
+release](https://github.com/weaveworks/weave/releases/latest).
+
+Also, the
+[toleration](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/taint-toleration-dedicated.md)
+required to let Weave Net run on master nodes has moved from an
+annotation to a field on the DaemonSet spec object.
+
+If you have edited the Weave Net DaemonSet from a previous release,
+you will need to re-make your changes against the new version.
+
+### <a name="daemon-sets"></a> Upgrading the Daemon Sets
 
 Kubernetes does not currently support rolling upgrades of daemon sets,
 and so you will need to perform the procedure manually:
@@ -68,7 +95,65 @@ and so you will need to perform the procedure manually:
   lose track of IP address range ownership, possibly leading to
   duplicate IP addresses if you then start a new copy of Weave Net.
 
-##<a name="npc"></a>Network Policy Controller
+## <a name="resources"></a>CPU and Memory Requirements
+
+Kubernetes manages
+[resources](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)
+on each node, and only schedules pods to run on nodes that have enough
+free resources.
+
+The components of a typical Kubernetes installation (with the master
+node running etcd, scheduler, api-server, etc.) take up about 95% of a
+CPU, which leaves little room to run anything else. For all of Weave
+Net's features to work, it must run on every node, including the
+master.
+
+The best way to resolve this issue is to use machines with at least
+two CPU cores. However if you are installing Kubernetes and Weave Net
+for the first time, you may not be aware of this requirement. For this
+reason, Weave Net launches as a DaemonSet with a specification that
+reserves at least 1% CPU for each container. This enables Weave Net to
+start up seamlessly on a single-CPU node.
+
+Depending on the workload, Weave Net may need more than 1% of the
+CPU. The percentage set in the DaemonSet is the minimum and not a
+limit. This minimum setting allows Weave Net to take advantage of
+available CPU and "burst" above that limit if it needs to.
+
+## <a name="eviction"></a>Pod Eviction
+
+If a node runs out of CPU, memory or disk, Kubernetes may [decide to
+evict](https://kubernetes.io/docs/concepts/cluster-administration/out-of-resource/)
+one or more pods. It may choose to evict the Weave Net pod, which will
+disrupt pod network operations.
+
+You can reduce the chance of eviction by changing the DaemonSet to
+have a much bigger request, and a limit of the same value.
+
+This causes Kubernetes to apply a ["guaranteed" rather than a
+"burstable" policy](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/resource-qos.md).
+However a similar request for disk space can not
+be made, and so please be aware of this issue and monitor your
+resources to ensure that they stay below 100%.
+
+You can see when pods have been evicted via the `kubectl get events` command
+
+```
+LASTSEEN   COUNT     NAME          KIND    TYPE      REASON     SOURCE            MESSAGE
+1m         1         mypod-09vkd   Pod     Warning   Evicted    kubelet, node-1   The node was low on resource: memory.
+```
+
+or `kubectl get pods`
+
+```
+NAME                READY     STATUS    RESTARTS   AGE       IP          NODE
+mypod-09vkd         0/1       Evicted   0          1h        <none>      node-1
+```
+
+If you see this in your cluster, consider some of the above steps to
+reduce disruption.
+
+## <a name="npc"></a>Network Policy Controller
 
 The addon also supports the [Kubernetes policy
 API](http://kubernetes.io/docs/user-guide/networkpolicies/) so that
@@ -86,7 +171,7 @@ definition](http://kubernetes.io/docs/api-reference/extensions/v1beta1/definitio
   all multicast traffic) by adding `--allow-mcast` as an argument to
   `weave-npc` in the YAML configuration.
 
-###<a name="blocked-connections"></a> Troubleshooting Blocked Connections
+### <a name="blocked-connections"></a> Troubleshooting Blocked Connections
 
 If you suspect that legitimate traffic is being blocked by the Weave Network Policy Controller, the first thing to do is check the `weave-npc` container's logs.
 
@@ -118,7 +203,7 @@ TCP connection from 10.32.0.7:56648 to 10.32.0.11:80 blocked by Weave NPC.
 UDP connection from 10.32.0.7:56648 to 10.32.0.11:80 blocked by Weave NPC.
 ```
 
-###<a name="configuration-options"></a> Changing Configuration Options
+### <a name="configuration-options"></a> Changing Configuration Options
 
 The default configuration settings can be changed by saving and editing the
 addon YAML before running `kubectl apply`. Additional arguments may be
@@ -138,21 +223,25 @@ inserted into the YAML file like this:
 
 The list of variables you can set is:
 
-* CHECKPOINT\_DISABLE - if set to 1, disable checking for new Weave Net
+* `CHECKPOINT_DISABLE` - if set to 1, disable checking for new Weave Net
   versions (default is blank, i.e. check is enabled)
-* IPALLOC\_RANGE - the range of IP addresses used by Weave Net
-  and the subnet they are placed in (CIDR format; default 10.32.0.0/12)
-* EXPECT\_NPC - set to 0 to disable Network Policy Controller (default is on)
-* KUBE\_PEERS - list of addresses of peers in the Kubernetes cluster
+* `IPALLOC_RANGE` - the range of IP addresses used by Weave Net
+  and the subnet they are placed in (CIDR format; default `10.32.0.0/12`)
+* `EXPECT_NPC` - set to 0 to disable Network Policy Controller (default is on)
+* `KUBE_PEERS` - list of addresses of peers in the Kubernetes cluster
   (default is to fetch the list from the api-server)
-* IPALLOC\_INIT - set the initialization mode of the [IP Address
+* `IPALLOC_INIT` - set the initialization mode of the [IP Address
   Manager](/site/operational-guide/concepts.md#ip-address-manager)
-  (defaults to consensus amongst the KUBE\_PEERS)
-* WEAVE\_EXPOSE\_IP - set the IP address used as a gateway from the
+  (defaults to consensus amongst the `KUBE_PEERS`)
+* `WEAVE_EXPOSE_IP` - set the IP address used as a gateway from the
   Weave network to the host network - this is useful if you are
   configuring the addon as a static pod.
-* WEAVE\_MTU - Weave Net defaults to 1410 bytes, but you can set a
+* `WEAVE_MTU` - Weave Net defaults to 1376 bytes, but you can set a
   smaller size if your underlying network has a tighter limit, or set
   a larger size for better performance if your network supports jumbo
   frames - see [here](/site/using-weave/fastdp.md#mtu) for more
   details.
+* `HAIRPIN_MODE` - Weave Net defaults to enabling hairpin on the bridge side of
+  the `veth` pair for containers attached. If you need to disable hairpin, e.g. your
+  kernel is one of those that can panic if hairpin is enabled, then you can disable it
+  by setting `HAIRPIN_MODE=false`.

site/metrics.md

@@ -1,6 +1,7 @@
 ---
 title: Monitoring with Prometheus
 menu_order: 105
+search_type: Documentation
 ---
 
 Two endpoints are exposed: one for the Weave Net router, and, when deployed as
@@ -19,6 +20,8 @@ exposed:
 * `weave_max_ips` - Size of IP address space used by allocator.
 * `weave_dns_entries` - Number of DNS entries.
 * `weave_flows` - Number of FastDP flows.
+* `weave_ipam_pending_allocates` - Number of pending allocates.
+* `weave_ipam_pending_claims` - Number of pending claims.
 
 #### Publish Router Metrics Endpoint
 

site/operational-guide.md

@@ -1,6 +1,7 @@
 ---
 title: Operational Guide
 menu_order: 45
+search_type: Documentation
 ---
 This operational guide is intended to give you an overview of how to
 operate and manage a Weave Network in production. It consists of three

site/operational-guide/autoscaling.md

@@ -1,6 +1,7 @@
 ---
 title: Autoscaling
 menu_order: 40
+search_type: Documentation
 ---
 
 ### Bootstrapping

site/operational-guide/concepts.md

@@ -1,6 +1,7 @@
 ---
 title: Concepts
 menu_order: 10
+search_type: Documentation
 ---
 This section describes some of the essential concepts with which you will
 need to be familiar before continuing to the example deployment
@@ -18,18 +19,18 @@ The following concepts are described:
     * [Observers](#observers)
  * [Persistence](#persistence)
 
-##<a name="host"></a>Host
+## <a name="host"></a>Host
 
 For the purposes of this documentation a host is an
 installation of the Linux operating system that is running an
 instance of the Docker Engine. The host may be executing directly on bare
 hardware or inside a virtual machine.
 
-##<a name="peer"></a>Peer
+## <a name="peer"></a>Peer
 
 A peer is a running instance of Weave Net, typically one per host.
 
-##<a name="peer-name"></a>Peer Name
+## <a name="peer-name"></a>Peer Name
 
 Weave Net peers are identified by a 48-bit value formatted like an
 ethernet MAC address, for example, `01:23:45:67:89:ab`. The 'peer
@@ -66,14 +67,14 @@ The appropriate strategy for assigning peer names depends on the type
 and method of your particular deployment and is discussed in more
 detail below.
 
-##<a name="peer-discovery"></a>Peer Discovery
+## <a name="peer-discovery"></a>Peer Discovery
 
 Peer discovery is a mechanism that allows peers to learn about new
 Weave Net hosts from existing peers without being explicitly told. Peer
 discovery is
 [enabled by default](/site/using-weave/finding-adding-hosts-dynamically.md).
 
-##<a name="network-partition"></a>Network Partition
+## <a name="network-partition"></a>Network Partition
 
 A network partition is a transient condition whereby some arbitrary
 subsets of peers are unable to communicate with each other for the
@@ -82,7 +83,7 @@ optic line severed. Weave Net is designed to allow peers and their
 containers to make maximum safe progress under conditions of
 partition, healing automatically once the partition is over.
 
-##<a name="ip-address-manager"></a>IP Address Manager (IPAM)
+## <a name="ip-address-manager"></a>IP Address Manager (IPAM)
 
 [IPAM](/site/ipam.md) is the subsystem responsible for dividing up a
 large contiguous block of IP addresses (known as the IP allocation
@@ -116,7 +117,7 @@ scenarios:
 
 * [Uniform Dynamic Cluster](/site/operational-guide/uniform-dynamic-cluster.md)
 
-###<a name="consensus"></a>Consensus
+### <a name="consensus"></a>Consensus
 
 Alternatively, when a new network is formed for the first time, peers
 can be configured to co-ordinate amongst themselves to automatically
@@ -132,7 +133,7 @@ scenarios:
 * [Interactive Deployment](/site/operational-guide/interactive.md)
 * [Uniform Fixed Cluster](/site/operational-guide/uniform-fixed-cluster.md)
 
-###<a name="observers"></a>Observers
+### <a name="observers"></a>Observers
 
 Finally, an option is provided to start a peer as an _observer_. Such
 peers do not require a seed peer name list or an initial peer
@@ -146,7 +147,7 @@ scenarios:
 
 * [Autoscaling](/site/operational-guide/autoscaling.md)
 
-##<a name="persistence"></a>Persistence
+## <a name="persistence"></a>Persistence
 
 Certain information is remembered between launches of Weave Net (for
 example across reboots):

site/operational-guide/interactive.md

@@ -1,6 +1,7 @@
 ---
 title: Interactive Deployment
 menu_order: 20
+search_type: Documentation
 ---
 Weave Net can be launched interactively on the command line, and as 
 long as Docker is configured to start on boot, the network will survive 

site/operational-guide/tasks.md

@@ -1,18 +1,19 @@
 ---
 title: Administrative Tasks
 menu_order: 60
+search_type: Documentation
 ---
 
 The following administrative tasks are discussed: 
 
 * [Configuring Weave Net to Start Automatically on Boot](#start-on-boot)
 * [Detecting and Reclaiming Lost IP Address Space](#detect-reclaim-ipam)
-    *[Manually Reclaiming Lost Address Space](#reclaim-address-space)
+* [Manually Reclaiming Lost Address Space](#manually-reclaim-address-space)
 * [Upgrading a Cluster](#cluster-upgrade)
 * [Resetting Persisted Data](#reset)
 
 
-##<a name="start-on-boot"></a>Configuring Weave Net to Start Automatically on Boot
+## <a name="start-on-boot"></a>Configuring Weave Net to Start Automatically on Boot
 
 `weave launch` runs all of Weave Net's containers with a Docker restart
 policy set to `always`.  If you have launched Weave Net manually at least
@@ -23,7 +24,7 @@ If you are aiming for a non-interactive installation, use
 [systemd](/site/installing-weave/systemd.md) or a similar init system to
 launch Weave using the `--no-restart` flag after Docker has been started.
 
-##<a name="detect-reclaim-ipam"></a>Detecting and Reclaiming Lost IP Address Space
+## <a name="detect-reclaim-ipam"></a>Detecting and Reclaiming Lost IP Address Space
 
 The recommended method of removing a peer is to run `weave reset` on that
 peer before the underlying host is decommissioned or repurposed. This
@@ -43,7 +44,7 @@ or possible:
 In some cases you may already be aware of the problem, as you were
 unable to execute `weave reset` successfully or because you know
 through other channels that the host has died. In these cases you can
-proceed straight to [Manually Reclaiming Lost Address Space](#reclaim-address-space).
+proceed straight to [Manually Reclaiming Lost Address Space](#manually-reclaim-address-space).
 
 However in some scenarios it may not be obvious that space has been
 lost, in which case you can check for it periodically with the
@@ -56,7 +57,7 @@ range managed by peer and also identifies the names of unreachable peers. If you
 that the peer is truly gone, rather than temporarily unreachable due to a
 partition, you can reclaim their space manually.
 
-###<a name="manually-reclaim-address-space"></a>Manually Reclaiming Address Space
+### <a name="manually-reclaim-address-space"></a>Manually Reclaiming Address Space
 
 When a peer dies unexpectedly the remaining peers will consider its
 address space to be unavailable even after it has remained unreachable
@@ -70,7 +71,7 @@ command is provided to perform this task, and must
 be executed on _one_ of the remaining peers. That peer will then take
 ownership of the freed address space.
 
-##<a name="cluster-upgrade"></a>Upgrading a Cluster
+## <a name="cluster-upgrade"></a>Upgrading a Cluster
 
 Protocol versioning and feature negotiation are employed in Weave Net
 to enable incremental rolling upgrades. Each major maintains
@@ -102,7 +103,7 @@ To minimize downtime while the new script is pulling the new container images:
 there are any special caveats or deviations from the standard
 procedure.
 
-##<a name="reset"></a>Resetting Persisted Data
+## <a name="reset"></a>Resetting Persisted Data
 
 Weave Net persists information in a data volume container named
 `weavedb`. If you wish to start from a completely clean slate (for

site/operational-guide/uniform-dynamic-cluster.md

@@ -1,6 +1,7 @@
 ---
 title: Uniform Dynamic Clusters
 menu_order: 50
+search_type: Documentation
 ---
 
 A uniform dynamic cluster has the following characteristics:

site/operational-guide/uniform-fixed-cluster.md

@@ -1,6 +1,7 @@
 ---
 title: Uniform Fixed Clusters
 menu_order: 30
+search_type: Documentation
 ---
 
 This scenario describes a production deployment of a fixed number of N

site/plugin.md

@@ -1,11 +1,11 @@
 ---
 title: Integrating Docker via the Network Plugin
 menu_order: 60
+search_type: Documentation
 ---
 
  * [Launching Weave Net and Running Containers Using the Plugin](#launching)
  * [Restarting the Plugin](#restarting)
- * [Configuring the Plugin](#configuring)
 
 Docker versions 1.9 and later have a plugin mechanism for adding
 different network providers. Weave Net installs itself as a network
@@ -34,7 +34,7 @@ which a helper is provided in the Weave script:
     # ping foo
 
 
-###<a name="launching"></a>Launching Weave Net and Running Containers Using the Plugin
+### <a name="launching"></a>Launching Weave Net and Running Containers Using the Plugin
 
 Just launch the Weave Net router onto each host and make a peer connection with the other hosts:
 
@@ -55,7 +55,7 @@ then run your containers using the Docker command-line:
     64 bytes from 10.32.0.2: icmp_seq=2 ttl=64 time=0.052 ms
 
 
-###<a name="multi"></a>Creating multiple Docker Networks
+### <a name="multi"></a>Creating multiple Docker Networks
 
 Docker enables you to create multiple independent networks and attach
 different sets of containers to each network. However, coordinating
@@ -79,7 +79,7 @@ Containers attached to different Docker Networks are
 [isolated through subnets](https://www.weave.works/docs/net/latest/using-weave/application-isolation/).
 
 
-###<a name="restarting"></a>Restarting the Plugin
+### <a name="restarting"></a>Restarting the Plugin
 
 The plugin, like all Weave Net components, is started with a policy of `--restart=always`, so that it is always there after a restart or reboot. If you remove this container (for example, when using `weave reset`) before removing all endpoints created using `--net=weave`, Docker may hang for a long time when it subsequently tries to re-establish communications to the plugin.
 
@@ -90,7 +90,7 @@ If you are using `systemd` with Docker 1.9, it is advised that you modify the Do
     TimeoutStartSec=0
 
 
-###<a name="configuring"></a>Configuring the Plugin
+### <a name="configuring"></a>Configuring the Plugin
 
 The plugin accepts a number of configuration parameters. To supply
 these, instead of running `weave launch`, start the router and plugin

site/plugin/plugin-how-it-works.md

@@ -1,6 +1,7 @@
 ---
 title: How the Weave Net Docker Network Plugin Works
 menu_order: 20
+search_type: Documentation
 ---
 
 

site/troubleshooting.md

@@ -1,6 +1,7 @@
 ---
 title: Troubleshooting Weave Net
 menu_order: 110
+search_type: Documentation
 ---
 
 

site/using-weave.md

@@ -1,9 +1,10 @@
 ---
 title: Using Weave Net
 menu_order: 40
+search_type: Documentation
 ---
 
-Weave Net provides a simple to deploy networking solution for containerized apps. Here, we describe how to manage a Weave container network using a sample application which consists of two simple `netcat` services deployed to containers on two separate hosts, and sometimes extended across three hosts.  
+Weave Net provides a simple to deploy networking solution for containerized apps. Here, we describe how to manage a Weave container network using a sample application which consists of two simple `netcat` services deployed to containers on two separate hosts.
 
 This section contains the following topics: 
 
@@ -13,7 +14,7 @@ This section contains the following topics:
  * [Starting the Netcat Service](#start-netcat)
 
 
-###<a name="launching"></a>Launching Weave Net
+### <a name="launching"></a>Launching Weave Net
 
 Before launching Weave Net and deploying your apps, ensure that Docker is [installed](https://docs.docker.com/engine/installation/) on both hosts. 
 
@@ -57,7 +58,7 @@ You can also preload the images by running `weave setup`. Preloaded images are u
 If you are deploying an application that consists of more than one container to the same host, launch them one after another using `docker run`, as appropriate.  
 
 
-###<a name="peer-connections"></a>Creating Peer Connections Between Hosts
+### <a name="peer-connections"></a>Creating Peer Connections Between Hosts
 
 To launch Weave Net on an additional host and create a peer connection, run the following:
 
@@ -73,7 +74,7 @@ You can also peer with other hosts by specifying the IP address, and a `:port` b
 
 There are a number of different ways that you can specify peers on a Weave network. You can launch Weave Net on `$HOST1` and then peer with `$HOST2`, or you can launch on `$HOST2` and peer with `$HOST1` or you can tell both hosts about each other at launch. The order in which peers are specified is not important. Weave Net automatically (re)connects to peers when they become available. 
 
-####Specifying Multiple Peers at Launch
+#### Specifying Multiple Peers at Launch
 
 To specify multiple peers, supply a list of addresses to which you want to connect, all separated by spaces. 
 
@@ -83,7 +84,7 @@ For example:
 
 Peers can also be dynamically added. See [Adding Hosts Dynamically](/site/using-weave/finding-adding-hosts-dynamically.md) for more information.
 
-####Restricting Access
+#### Restricting Access
 
 By default Weave Net listens on all host IPs (i.e. 0.0.0.0). This
 can be altered with the `--host` parameter to `weave launch`, for example, 
@@ -95,7 +96,7 @@ Weave Net control and data ports.
 For communication across untrusted networks, connections can be
 [encrypted](/site/using-weave/security-untrusted-networks.md).
 
-###<a name="testing"></a>Testing Container Communications
+### <a name="testing"></a>Testing Container Communications
 
 With two containers running on separate hosts, test that both containers are able to find and communicate with one another using ping. 
 
@@ -116,7 +117,7 @@ Similarly, in the container started on `$HOST2`...
     1 packets transmitted, 1 received, 0% packet loss, time 0ms
     rtt min/avg/max/mdev = 0.366/0.366/0.366/0.000 ms
 
-###<a name="start-netcat"></a>Starting the Netcat Service
+### <a name="start-netcat"></a>Starting the Netcat Service
 
 The `netcat` service can be started using the following commands:  
 

site/using-weave/application-isolation.md

@@ -1,6 +1,7 @@
 ---
 title: Isolating Applications on a Weave Network
 menu_order: 30
+search_type: Documentation
 ---
 
 A single Weave network can host multiple, isolated applications where each application's containers are able 
@@ -32,7 +33,7 @@ Next, launch the two netcat containers onto the default subnet:
 And then to test the isolation, launch a few more containers onto a different subnet:
 
     host1$ docker run -e WEAVE_CIDR=net:10.2.2.0/24 --name b1 -ti weaveworks/ubuntu
-    host2$ docker run -e WEAVE_CIDR=net:10.2.2.0.24 --name b2 -ti weaveworks/ubuntu
+    host2$ docker run -e WEAVE_CIDR=net:10.2.2.0/24 --name b2 -ti weaveworks/ubuntu
 
 Ping each container to confirm that they can talk to each other, but not to the containers of our first subnet:
 

site/using-weave/awsvpc.md

@@ -1,6 +1,7 @@
 ---
 title: Using IP Routing on an Amazon Web Services Virtual Private Cloud
 menu_order: 110
+search_type: Documentation
 ---
 
 If your container infrastructure is running entirely within Amazon Web Services (AWS) 
@@ -16,7 +17,7 @@ on which they live.
 
 ![Weave Net AWS-VPC Mode](weave-net-awsvpc-1007x438.png)
 
-###Configuring EC2 Instances to use Weave AWS-VPC Mode
+### Configuring EC2 Instances to use Weave AWS-VPC Mode
 
 First, your AWS instances need to be given write access to the route
 table via its
@@ -59,7 +60,7 @@ addresses will flow over the AWS network unmodified.
 Finally, since Weave will be operating with IP addresses outside of the 
 range allocated by Amazon, you must disable "Source/Destination check" on each machine.
 
-###Using AWS-VPC Mode
+### Using AWS-VPC Mode
 
 Launch Weave Net with the `--awsvpc` flag:
 
@@ -68,7 +69,7 @@ Launch Weave Net with the `--awsvpc` flag:
  >>**Note:** You will still need to supply the names or IP addresses of other hosts in
 your cluster.
 
-###Present Limitations
+### Present Limitations
 
 - AWS-VPC mode does not inter-operate with other Weave Net modes; it
   is all or nothing.  In this mode, all hosts in a cluster must be AWS
@@ -85,11 +86,11 @@ your cluster.
 - All of your containers must be on the same network, with no subnet
   isolation. (We hope to ease this limitation in future.)
 
-###Packet size (MTU)
+### Packet size (MTU)
 
 The Maximum Transmission Unit, or MTU, is the technical term for the
 limit on how big a single packet can be on the network. Weave Net
-defaults to 1410 bytes. This default works across almost all networks, but for better 
+defaults to 1376 bytes. This default works across almost all networks, but for better 
 performance you can set it to a larger MTU size. 
 
 The AWS network supports packets of up to 9000 bytes.  

site/using-weave/configuring-weave.md

@@ -1,6 +1,7 @@
 ---
 title: Allocating IPs in a Specific Range
 menu_order: 15
+search_type: Documentation
 ---
 
 The default configurations for both Weave Net and Docker use [Private

site/using-weave/dynamically-attach-containers.md

@@ -1,6 +1,7 @@
 ---
 title: Dynamically Attaching and Detaching Applications
 menu_order: 40
+search_type: Documentation
 ---
 
 
@@ -20,7 +21,7 @@ where,
 
 >Note If you are using the Weave Docker API proxy, it will have modified `DOCKER_HOST` to point to the proxy and therefore you will have to pass `-e WEAVE_CIDR=none` to start a container that _doesn't_ get automatically attached to the weave network for the purposes of this example.
 
-###Dynamically Detaching Containers
+### Dynamically Detaching Containers
 
 A container can be detached from a subnet, by using the `weave detach` command: