lb: members' ping identifiers in their node's block too

The live run found ping from a member on one node failing right after a
member on another node pinged the same host: busybox ping uses its PID as
the ICMP echo identifier, containers' PIDs are small and alike, so both
nodes' echoes reached the holder as the same connection, and the holder
dropped the second as a collision (fail closed, as intended). NAT treats
the echo identifier as ICMP's port, and iptables accepts a range for it:
members' ICMP is now SNATed into the node's block like TCP, UDP and SCTP,
so every node's pings are distinct on the holder. Replies follow the
holder's tag as before.

The end-to-end test's ping and traceroute pods run as root (the curl
image's user cannot use raw sockets even with NET_RAW).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UHXzGrsxfkGk8LmwrBbHNZ

.dockerignore

@@ -0,0 +1,13 @@
+# Git status should be ignored
+.git
+
+# .dockerignore should be ignored
+.dockerignore
+
+# Almost all of the reweave directory 
+# should be ignored
+reweave/build/Dockerfile
+reweave/tools/
+reweave/scans/
+reweave/Makefile
+reweave/*.md

.gitea/workflows/images.yml

@@ -0,0 +1,67 @@
+# Build the images the cluster runs (weave-kube, weave-npc) and push them to
+# this Gitea's container registry. CI stops here: rolling them out is the
+# CD side's job, not this workflow's.
+#
+# Tags are the full commit SHA, so a tag never moves: rebuilding a commit
+# pushes the same name again instead of changing what an existing name means.
+# linux/amd64 only for now (every node is amd64).
+name: images
+
+on:
+  push:
+    paths-ignore:
+      - "website/**"
+      - "docs/**"
+      - "**.md"
+  workflow_dispatch:
+
+# The job token can't log in to the registry (no package scope there), so
+# the push uses the reweave-ci bot's token from the org secret REGISTRY_TOKEN.
+permissions:
+  contents: read
+
+jobs:
+  images:
+    runs-on: ubuntu-latest
+    env:
+      REGISTRY: git.levk.dev
+      IMAGE_PREFIX: git.levk.dev/reweave
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          fetch-depth: 0 # tags, for the version string
+          submodules: true # tools/ (weaveworks/build-tools): make all builds tools/runner
+
+      - name: Log in to the registry
+        run: echo "${{ secrets.REGISTRY_TOKEN }}" | docker login "$REGISTRY" -u reweave-ci --password-stdin
+
+      - name: Build and push (linux/amd64)
+        run: |
+          set -eu
+          sha=$(git rev-parse HEAD)
+          version=$(git describe --tags --always | sed 's/^v//')
+          for pair in weavekubeimage:weave-kube weavenpcimage:weave-npc; do
+            target=${pair%%:*}
+            image="$IMAGE_PREFIX/${pair#*:}:$sha"
+            docker buildx build --push --progress=plain \
+              --platform=linux/amd64 \
+              --target="$target" \
+              --build-arg=WEAVE_VERSION="$version" \
+              --build-arg=revision="$sha" \
+              --build-arg=imageversion="$version" \
+              -f reweave/build/Dockerfile \
+              -t "$image" \
+              .
+            echo "pushed $image"
+          done
+
+      # The images live in the registry now; the runner keeps only a bounded build cache. Without this the
+      # runner's 10Gi Docker volume fills up after a handful of builds (no space left on device mid-build).
+      - name: Free the runner's disk
+        if: always()
+        run: |
+          set -u
+          docker image ls --format '{{.Repository}}:{{.Tag}}' | grep "^$IMAGE_PREFIX/" | xargs -r docker image rm -f
+          docker image prune -f
+          docker builder prune -f --keep-storage 4GB
+          docker system df

.github/workflows/github-pages-publish.yml

@@ -0,0 +1,53 @@
+# Workflow for building and deploying a Jekyll site to GitHub Pages
+name: Build using Jekyll, deploy to GitHub Pages
+
+on:
+  # Runs on pushes targeting the master branch, if changes found in
+  # the website directory
+  push:
+    branches: ["master"]
+    paths: ["website/**"]
+
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+
+# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
+# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
+concurrency:
+  group: "pages"
+  cancel-in-progress: false
+
+jobs:
+  # Build job
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+      - name: Setup Pages
+        uses: actions/configure-pages@v4
+      - name: Build with Jekyll
+        uses: actions/jekyll-build-pages@v1
+        with:
+          source: ./website/
+          destination: ./_site
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v3
+
+  # Deployment job
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    needs: build
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

.gitignore

@@ -73,6 +73,3 @@ test/images/network-tester/webserver
 terraform.tfstate
 terraform.tfstate.backup
 prog/net-plugin/rootfs/
-
-# Website
-site-build

.ruby-version

@@ -1 +0,0 @@
-2.3.0

CHANGELOG.md

@@ -1,3 +1,228 @@
+## Release 2.9.0
+
+This release has upgraded dependencies based on CVE recommendations.
+
+Also, from this release onwards, all weave images will use the `iptables-nft` backend for `iptables` by default. To use the legacy mode, set the environment variable `IPTABLES_BACKEND` to `legacy`. To do this for the Docker plugin, export the variable before using the `weave` script. For the Kubernetes addon, set the variable in the manifest.
+
+## Release 2.8.10
+
+This release bumps the Go version to 1.23.1 and the alpine base image to version 3.20.3 as per CVE recommendations.
+
+## Release 2.8.9
+
+This release bumps the Go version to 1.22.2, the alpine base image to version 3.20.2, and upgrades a dependency as per CVE recommendations.
+
+## Release 2.8.8
+
+The alpine base image used by `weave-kube` and `weave-npc` has been upgraded to version 3.20.0 in this release. This removes several CVEs.
+
+## Release 2.8.7
+
+This release has upgraded dependencies based on CVE recommendations.
+
+Also, this release does not provides images for the `ppc64le` and `s390x` architectures, because a crucial upstream dependency (libpcap-0.8, from the debian buster respository) does not provide binaries for those architectures any more.
+
+## Release 2.8.6
+
+From this release onwards, the Docker V2 Plugin will be published for all supported architectures. It will be named using the pattern `rajchaudhuri/net-plugin:<version>-<arch>`. The latest version will also be published as `rajchaudhuri/net-plugin:latest_release-<arch>`. Supported architectures are `amd64`,`arm`,`arm64`,`ppc64le` and `s390x`. It can be installed on a Swarm-mode-enabled Docker host as follows:
+
+```bash
+ARCH=$(docker version -f '{{.Server.Arch}}')
+docker plugin install rajchaudhuri/net-plugin:latest_release-$ARCH
+```
+
+## Release 2.8.5
+
+This release has upgraded dependencies based on CVE recommendations.
+
+* Build with Go version 1.21.9
+* A dependency has been updated
+
+## Release 2.8.4
+
+The alpine base image used by `weave-kube` and `weave-npc` was upgraded to version 3.19.1 in the previous release. In this version, the default iptables backend is nftables, and the legacy backend is not included. Our scripts and programs assume legacy as the default backend, and change to nft if autodetected, or if we ask for it. So, the build Dockerfile (reweave/build/Dockerfile) was changed to also install the Alpine `iptables-legacy` package , and change the `iptables-{save,restore}` symbolic links to point to the legacy backend by default.
+
+The `weave-kube` and `weave-npc` images can now log traces if the environment WEAVE_DEBUG is set in the manifest.
+
+## Release 2.8.3
+
+The docker API client version, used by the proxy package and the weaveutil command, was bumped from 1.18 to 1.24. As of March 2024, Docker API versions below 1.24 are deprecated. This means that the minimum supported Docker version is now 1.12.0.
+
+Provision was made in weaveutil program and the weave script to override the API version used, via the environment variable `DOCKER_API_VERSION`. The same variable is used by standard docker clients.
+
+## Release 2.8.2
+
+This is the first release since forking from weaveworks/weave. While no new functionality has been added, there have been many changes.
+
+* Build with Go version 1.21.6
+* Dependencies, especially ones with associated CVEs, have been updated
+* libCNI has been upgraded to v1.1.2
+* A new, Docker-only build process has been created, which produces proper multi-arch images
+* A vulnerability scanning process has been created using [grype](https://github.com/anchore/grype)
+
+## Release 2.8.1
+
+Fixes a problem introduced in 2.8.0 for machines whose ID is in /etc/machine-id #3886
+Many thanks to contributor @avestuk for this fix.
+
+Also move Kubernetes API calls out of Weave Net daemon #3885
+- this reduces the size of the 'weaver' binary and stops it crashing when run on 32-bit ARM.
+
+## Release 2.8.0
+
+This release makes some important changes to trim the "attack surface" of the
+Kubernetes install, and improves a couple of reported issues.
+
+* Workaround to fix ipset conflict with iptables #3851, #3882
+* Kubernetes: move kernel and CNI setup to init container #3880
+* For K8s, stop running in host PID namespace #3876
+* NetworkPolicy: avoid logging dropped packets that were not actually dropped #3852
+* Build with Go version 1.15.6 #3883
+
+Many thanks to contributors @drigz, @KevDBG and @NeonSludge.
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/87?closed=1)
+
+## Release 2.7.0
+
+This release improves resiliency in a number of areas,
+and extends the Prometheus metrics exported by Weave Net.
+
+Change in behaviour: on Kubernetes, the client source IP is preserved 
+when calling from a pod to a service.
+This feature, introduced in version 2.4.0 and previously turned on by 
+setting `NO_MASQ_LOCAL=1` is now on by default. #3389, #3756
+
+### Features
+
+* Reload router iptables rules if they get cleared, e.g. when firewalld restarts. #3802
+  (weave-npc rules are not reloaded)
+* Add new `type` and `encryption` labels to `weave_connections` metric #3788, #3789
+* Weave Net now exports Go metrics for heap size, garbage collection, etc. #3838
+* Register container name and its network aliases with weaveDNS #3084, #3090
+* Make DNS listen address configurable #1770, #3231
+
+### Bug fixes
+
+* weave-npc could crash if you deleted a Kubernetes Namespace containing pods #3833, #3836
+* Ensure that weave-npc exits and restarts if it crashes #3764, #3792, #3841
+* Avoid weave-kube failing on startup due to iptables lock #3828, #3835
+
+### Build and test
+
+* Reduce size of containers (weaveworks/weave goes from 99MB to 83MB) #3624, #3726
+* Weave Net is now built with Go 1.14.4, which should improve performance #3838
+* CI tests are now run against Docker 19.03.1, Kubernetes 1.14.0 #3687
+
+Many thanks to contributors @berlic, @gobomb, @hairyhenderson, @naemono, @nesc58
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/81?closed=1)
+
+
+## Release 2.6.5
+
+Fixes a bug that would leak memory every time a fast-datapath connection
+was stopped. #3808
+Also avoid a crash when the machine has ipv6 disabled. #3815
+
+
+## Release 2.6.4
+
+Improves the iptables rule added in 2.6.3 to block just the Weave Net
+control port and avoid blocking other uses of 127.0.0.1.	#3811
+
+
+## Release 2.6.3
+
+This release has a couple of security improvements, and some other fixes.
+
+* Block non-local traffic to the Weave control port	#3805
+* Tell Linux not to accept router advisory messages	#3801
+* Network Policy Controller: add a metric to show errors while operating	#3804
+* Network Policy Controller: don't treat named port as a fatal error	#3790
+
+## Release 2.6.2
+
+This release fixes a regression found in 2.6.1 release and removes a possible case
+of CPU spinning when pcap is used in sleeve mode
+
+### Bug fixes
+
+* Weave Net can not be used in fastdp mode and always falls back to sleeve mode #3781, #3783
+* Restrict timeout value passed to pcap library to a value less than 2^31 microseconds to
+  prevent CPU spinning #3782
+
+## Release 2.6.1
+
+This release fixes a bug discovered since the release of Weave Net 2.6.0 and
+an improvement to support iptables 1.8
+
+### Bug fixes
+
+* Weave Net on node restart unable to form connections to peers and connections get stuck in
+  pending state and stops responding to API endpoint #3762 #3763
+
+### Other improvements
+
+* Weave Net Kubernetes images by default uses iptables 1.8 with legacy (netfilter) backend with
+  option to choose netfilter as backend #3465, #3747
+
+
+## Release 2.6.0
+
+This release reduces CPU and memory usage in larger clusters, by
+sending notifications to a smaller set of peers and coalescing updates
+to reduce topology recalculation. #3715, #3732
+
+The default soft limit on connections has been raised from 100 to 200.
+
+### Bug fixes
+
+* Fix a race condition in Kubernetes addon when reclaiming IP
+  addresses after node deletion #3724, #3716
+* Buffer events so Docker won't drop them, and Weave Net can clean up
+  after dead containers #3432, #3705
+* Weave reconnect occasionally fails after network interface
+  disconnect #3666, #3669, #3676
+* Ingress NetworkPolicy would accepts all traffic when specifying both
+  IPBlock and port #3653, #3654
+
+### Kubernetes improvements
+
+* Support both podSelector and namespaceSelector in NetworkPolicy #3312, #3647
+* Only add default-drop egress rule if network policies are in use #3639
+* Manifests use 'apps/v1' rather than deprecated 'apps/v1beta1' #3660
+* Avoid Weave Net pods being evicted by setting priorityClassName: system-node-critical #3697
+* Manifests use recommended DNS policy ClusterFirstWithHostNet #3692
+* Weave Net now tolerates 'NoExecute' taint #3655
+* Allow extra arguments to NetworkPolicy controller to be set in an environment variable #3683
+* Stop reporting a failure to connect to self #3454, #3585
+* Minor reduction in log noise when reclaiming IPs #3710
+
+### Other improvements
+
+* Avoid isolating nodes which have restarted by automatically
+  repairing inconsistencies in IP allocation data #1962, #3637, #3708
+* Build Weave Net for the s390x architecture #3685
+* When a IP address is requested that may be in use, make several
+  attempts to claim it before returning an error #3725
+* Improve logging for IP allocation updates #3627, #3630
+* Improve 'expecting PMTU update' log message on initial connection #3603
+
+### Build and test
+
+* Shut down Kubernetes on node when testing node deletion #3716
+* Update the base 'Alpine' container image to version 3.8 #3701
+* Update Go to version 1.13.3 #3712
+* Update gopacket library #3590
+* Pin busybox version to 1.28 to avoid CI failure in Python test #3689
+* Remove obsolete weave-daemonset.yaml file #3674
+
+Thanks to contributors @christian-2, @hpdvanwyk, @guirish, @kitt1987,
+@mmerrill3, @Pensu, @scritchley, @sidharthsurana, @tanishq-dubey
+
+[Full list of changes](https://github.com/weaveworks/weave/milestone/78?closed=1)
+
 
 ## Release 2.5.2
 

DEPENDENCIES

@@ -1,3 +1,3 @@
-DOCKER_VERSION=17.06
-KUBERNETES_VERSION=1.11.0
-KUBERNETES_CNI_VERSION=0.6.0
+DOCKER_VERSION=19.03.1
+KUBERNETES_VERSION=1.14.0
+KUBERNETES_CNI_VERSION=0.7.5

HISTORY.md

@@ -0,0 +1,44 @@
+# History of the ReWeave project
+
+## How it started
+
+In June 2022, Weave Net had not been updated for a year. Problems were starting to appear in the field. In particular, the last published images on the Docker Hub ([weaveworks/weave-kube](https://hub.docker.com/r/weaveworks/weave-kube) and [weaveworks/weave-npc](https://hub.docker.com/r/weaveworks/weave-npc), v2.8.1) had issues supporting multiple processor architectures, and security scanners showed multiple vulnerabilities. 
+
+A call went out from Weaveworks to get the community more involved in maintaining it. After some discussion on GitHub issues and e-mail, and even a few online meetings, things were not moving forward. 
+
+## ReWeave begins
+
+Finally, in March 2023, this fork was created, with the following goals in mind:
+
+* Update dependencies, especially ones with security vulnerabilities
+* Make minimal code changes _only_ when required by updating dependencies
+* Create true multi-arch images using modern tools
+* Create a new build process to automate all this
+* Do all this with _minimal_ changes to the existing codebase. Keep all new things in the `reweave` folder.
+
+These goals were achieved. Details can be found in the [reweave](reweave/README.md) directory. A [pull request](https://github.com/weaveworks/weave/pull/3996) was submitted on the weaveworks repo, with the aim of getting a new official release out.
+
+## Weaveworks ends
+
+On February 5th, 2024, Weaveworks CEO Alexis Richardson announced via [LinkedIn](https://www.linkedin.com/posts/richardsonalexis_hi-everyone-i-am-very-sad-to-announce-activity-7160295096825860096-ZS67/) and [Twitter](https://twitter.com/monadic/status/1754530336120140116) that Weaveworks is winding down. 
+
+So, a decision was taken to maintain this fork independently.
+
+## We're forked
+
+Two major changes were introduced at this point:
+
+* The module name was changed to `github.com/rajch/weave` (previously `github.com/weaveworks/weave`)
+* The default registry account for publishing images was changed to `docker.io/rajchaudhuri` (previously `docker.io/weaveworks`)
+
+In addition, the old repo structure and codebase is not longer sacrosanct. Things can be moved around, new code can be added outside the `reweave` directory, old code can be modified or deleted as necessary.
+
+The version numbers will continue from where Weaveworks left off. 
+
+## New Goals
+
+The old goals, listed above (except the last one), remain the priority. In addition, this project aims to:
+
+* Remove dependencies on Weaveworks infrastructure, starting with telemetry (what weaveworks called checkpoint)
+* Publish new releases regularly, duly security scanned
+* Provide supporting infrastructure, such as weave's famous one-line installation, where possible

Makefile

@@ -1,435 +0,0 @@
-.DEFAULT: all
-.PHONY: all exes testrunner update tests lint publish-one-arch $(PUBLISH) clean clean-bin clean-work-dir prerequisites build run-smoketests
-
-# If you can use docker without being root, you can do "make SUDO="
-SUDO=$(shell docker info >/dev/null 2>&1 || echo "sudo -E")
-BUILD_IN_CONTAINER=true
-RM=--rm
-RUN_FLAGS=-ti
-COVERAGE=
-
-# This specifies the architecture we're building for
-ARCH?=amd64
-
-# We're using QEMU to be able to build Docker images for other architectures
-# from amd64 machines.
-QEMU_VERSION=v2.7.0
-
-# A list of all supported architectures here. Should be named as Go is naming platforms
-# All supported architectures must have an "ifeq" block below that customizes the parameters
-ALL_ARCHITECTURES=amd64 arm arm64 ppc64le
-ML_PLATFORMS=linux/amd64,linux/arm,linux/arm64,linux/ppc64le
-
-ifeq ($(ARCH),amd64)
-# The architecture to use when downloading the docker binary
-	WEAVEEXEC_DOCKER_ARCH?=x86_64
-
-# The name of the alpine baseimage to use as the base for weave images
-	ALPINE_BASEIMAGE?=alpine:3.6
-
-# The extension for the made images
-# Specifying none means for example weaveworks/weave:latest
-	ARCH_EXT?=
-
-# The name of the gcc we're using for compiling C code
-	CC=gcc
-
-# Optional parameters that can be passed when linking C code into the Go binary
-	CGO_LDFLAGS=
-endif
-ifeq ($(ARCH),arm)
-# The architecture to use when downloading the docker binary
-	WEAVEEXEC_DOCKER_ARCH?=armel
-
-# Using the (semi-)official alpine image
-	ALPINE_BASEIMAGE?=arm32v6/alpine:3.6
-
-# arm images have the -arm suffix, for instance weaveworks/weave-arm:latest
-	ARCH_EXT?=-arm
-
-# The name of the gcc binary
-	CC=arm-linux-gnueabihf-gcc
-
-# The architecture name to use when downloading a prebuilt QEMU binary
-	QEMUARCH=arm
-
-# In the weaveworks/build image; libpcap libraries for arm are placed here
-# Tell the gcc linker to search for libpcap here
-	CGO_LDFLAGS="-L/usr/local/lib/$(CC)"
-endif
-ifeq ($(ARCH),arm64)
-# Use the arm 32-bit docker client variant
-	WEAVEEXEC_DOCKER_ARCH?=armel
-
-# Using the (semi-)official alpine image
-	ALPINE_BASEIMAGE?=arm64v8/alpine:3.6
-
-# arm64 images have the -arm64 suffix, for instance weaveworks/weave-arm64:latest
-	ARCH_EXT?=-arm64
-
-# The name of the gcc binary
-	CC=aarch64-linux-gnu-gcc
-
-# The architecture name to use when downloading a prebuilt QEMU binary
-	QEMUARCH=aarch64
-
-# In the weaveworks/build image; libpcap libraries for arm64 are placed here
-# Tell the gcc linker to search for libpcap here
-	CGO_LDFLAGS="-L/usr/local/lib/$(CC)"
-endif
-ifeq ($(ARCH),ppc64le)
-# The architecture to use when downloading the docker binary
-	WEAVEEXEC_DOCKER_ARCH?=ppc64le
-
-# Using the (semi-)official alpine image
-	ALPINE_BASEIMAGE?=ppc64le/alpine:3.6
-
-# ppc64le images have the -ppc64le suffix, for instance weaveworks/weave-ppc64le:latest
-	ARCH_EXT?=-ppc64le
-
-# The name of the gcc binary
-	CC=powerpc64le-linux-gnu-gcc
-
-# The architecture name to use when downloading a prebuilt QEMU binary
-	QEMUARCH=ppc64le
-
-# In the weaveworks/build image; libpcap libraries for ppc64le are placed here
-# Tell the gcc linker to search for libpcap here
-	CGO_LDFLAGS="-L/usr/local/lib/$(CC)"
-endif
-
-# The name of the user that this Makefile should produce image artifacts for. Can/should be overridden
-DOCKERHUB_USER?=weaveworks
-# The default version that's chosen when pushing the images. Can/should be overridden
-GIT_REVISION=$(shell git rev-parse HEAD)
-WEAVE_VERSION?=git-$(shell git rev-parse --short=12 HEAD)
-# Docker Store does not allow the "latest" tag.
-NET_PLUGIN_LATEST=latest_release
-
-# Paths to all relevant binaries that should be compiled
-WEAVER_EXE=prog/weaver/weaver
-SIGPROXY_EXE=prog/sigproxy/sigproxy
-KUBEUTILS_EXE=prog/kube-utils/kube-utils
-WEAVENPC_EXE=prog/weave-npc/weave-npc
-WEAVEWAIT_EXE=prog/weavewait/weavewait
-WEAVEWAIT_NOOP_EXE=prog/weavewait/weavewait_noop
-WEAVEWAIT_NOMCAST_EXE=prog/weavewait/weavewait_nomcast
-WEAVEUTIL_EXE=prog/weaveutil/weaveutil
-RUNNER_EXE=tools/runner/runner
-# Get this from https://github.com/estesp/manifest-tool/releases
-MANIFEST_TOOL_CMD=manifest-tool
-TEST_TLS_EXE=test/tls/tls
-NETWORKTESTER_EXE=test/images/network-tester/webserver
-
-# All binaries together in a list
-EXES=$(WEAVER_EXE) $(SIGPROXY_EXE) $(KUBEUTILS_EXE) $(WEAVENPC_EXE) $(WEAVEWAIT_EXE) $(WEAVEWAIT_NOOP_EXE) $(WEAVEWAIT_NOMCAST_EXE) $(WEAVEUTIL_EXE) $(RUNNER_EXE) $(TEST_TLS_EXE) $(NETWORKTESTER_EXE)
-
-# These stamp files are used to mark the current state of the build; whether an image has been built or not
-BUILD_UPTODATE=.build.uptodate
-WEAVER_UPTODATE=.weaver$(ARCH_EXT).uptodate
-WEAVEEXEC_UPTODATE=.weaveexec$(ARCH_EXT).uptodate
-PLUGIN_UPTODATE=.net-plugin$(ARCH_EXT).uptodate
-WEAVEKUBE_UPTODATE=.weavekube$(ARCH_EXT).uptodate
-WEAVENPC_UPTODATE=.weavenpc$(ARCH_EXT).uptodate
-WEAVEDB_UPTODATE=.weavedb.uptodate
-NETWORKTESTER_UPTODATE=.networktester$(ARCH_EXT).uptodate
-
-IMAGES_UPTODATE=$(WEAVER_UPTODATE) $(WEAVEEXEC_UPTODATE) $(WEAVEKUBE_UPTODATE) $(WEAVENPC_UPTODATE) $(NETWORKTESTER_UPTODATE)
-
-# The names of the images. Note that the images for other architectures than amd64 have a suffix in the image name.
-WEAVER_IMAGE=$(DOCKERHUB_USER)/weave$(ARCH_EXT)
-WEAVEEXEC_IMAGE=$(DOCKERHUB_USER)/weaveexec$(ARCH_EXT)
-WEAVEKUBE_IMAGE=$(DOCKERHUB_USER)/weave-kube$(ARCH_EXT)
-WEAVENPC_IMAGE=$(DOCKERHUB_USER)/weave-npc$(ARCH_EXT)
-BUILD_IMAGE=weaveworks/weavebuild
-WEAVEDB_IMAGE=$(DOCKERHUB_USER)/weavedb
-PLUGIN_IMAGE=$(DOCKERHUB_USER)/net-plugin
-NETWORKTESTER_IMAGE=$(DOCKERHUB_USER)/network-tester
-
-IMAGES=$(WEAVER_IMAGE) $(WEAVEEXEC_IMAGE) $(WEAVEKUBE_IMAGE) $(WEAVENPC_IMAGE) $(WEAVEDB_IMAGE) $(NETWORKTESTER_IMAGE)
-
-PLUGIN_WORK_DIR="prog/net-plugin/rootfs"
-PLUGIN_BUILD_IMG="plugin-builder"
-
-PUBLISH=publish_weave publish_weaveexec publish_weave-kube publish_weave-npc
-PUSH_ML=push_ml_weave push_ml_weaveexec push_ml_weave-kube push_ml_weave-npc
-
-WEAVE_EXPORT=weave$(ARCH_EXT).tar.gz
-
-NETGO_CHECK=@strings $@ | grep cgo_stub\\\.go >/dev/null || { \
-	rm $@; \
-	echo "\nYour go standard library was built without the 'netgo' build tag."; \
-	echo "To fix that, run"; \
-	echo "    sudo go clean -i net"; \
-	echo "    sudo go install -tags netgo std"; \
-	false; \
-}
-# The flags we are passing to go build. -extldflags -static for making a static binary, 
-# -linkmode external for linking external C libraries into the binary, -X main.version for telling the
-# Go binary which version it is, -tags netgo for enforcing the native Go DNS resolver
-BUILD_FLAGS=-i -ldflags "-linkmode external -extldflags -static -X main.version=$(WEAVE_VERSION)" -tags netgo
-
-PACKAGE_BASE=$(shell go list -e ./)
-
-all: $(WEAVE_EXPORT)
-testrunner: $(RUNNER_EXE) $(TEST_TLS_EXE)
-
-$(WEAVER_EXE) $(WEAVEUTIL_EXE): common/*.go common/*/*.go net/*.go net/*/*.go
-$(WEAVER_EXE): router/*.go ipam/*.go ipam/*/*.go db/*.go nameserver/*.go prog/weaver/*.go
-$(WEAVER_EXE): api/*.go plugin/*.go plugin/*/*
-$(WEAVER_EXE):  proxy/*.go
-$(WEAVEUTIL_EXE): prog/weaveutil/*.go net/*.go plugin/net/*.go plugin/ipam/*.go db/*.go
-$(SIGPROXY_EXE): prog/sigproxy/*.go
-$(KUBEUTILS_EXE): prog/kube-utils/*.go
-$(WEAVENPC_EXE): prog/weave-npc/*.go npc/*.go npc/*/*.go
-$(TEST_TLS_EXE): test/tls/*.go
-$(RUNNER_EXE): tools/runner/*.go
-$(WEAVEWAIT_NOOP_EXE): prog/weavewait/*.go
-$(WEAVEWAIT_EXE): prog/weavewait/*.go net/*.go
-$(WEAVEWAIT_NOMCAST_EXE): prog/weavewait/*.go net/*.go
-$(NETWORKTESTER_EXE): test/images/network-tester/*.go
-tests: tools/.git
-lint: tools/.git
-
-ifeq ($(BUILD_IN_CONTAINER),true)
-
-# This make target compiles all binaries inside of the weaveworks/build container
-# It bind-mounts the source into the container and passes all important variables
-exes $(EXES) tests lint: $(BUILD_UPTODATE)
-	@mkdir -p $(shell pwd)/.pkg
-	$(SUDO) docker run $(RM) $(RUN_FLAGS) \
-	    -v $(shell pwd):/go/src/github.com/weaveworks/weave \
-		-v $(shell pwd)/.pkg:/go/pkg \
-		-e GOARCH=$(ARCH) -e CGO_ENABLED=1 -e GOOS=linux -e CIRCLECI -e CIRCLE_BUILD_NUM -e CIRCLE_NODE_TOTAL -e CIRCLE_NODE_INDEX -e COVERDIR -e SLOW -e DEBUG \
-		$(BUILD_IMAGE) COVERAGE=$(COVERAGE) WEAVE_VERSION=$(WEAVE_VERSION) CC=$(CC) QEMUARCH=$(QEMUARCH) CGO_LDFLAGS=$(CGO_LDFLAGS) $@
-	touch $@
-
-else
-
-exes: $(EXES)
-
-$(WEAVER_EXE):
-ifeq ($(COVERAGE),true)
-	$(eval COVERAGE_MODULES := $(shell (go list ./$(@D); go list -f '{{join .Deps "\n"}}' ./$(@D) | grep "^$(PACKAGE_BASE)/") | grep -v "^$(PACKAGE_BASE)/vendor/" | paste -s -d,))
-	go test -c -o ./$@ $(BUILD_FLAGS) -v -covermode=atomic -coverpkg $(COVERAGE_MODULES) ./$(@D)/
-else
-	go build $(BUILD_FLAGS) -o $@ ./$(@D)
-endif
-	$(NETGO_CHECK)
-
-$(WEAVEUTIL_EXE) $(KUBEUTILS_EXE) $(WEAVENPC_EXE) $(NETWORKTESTER_EXE):
-	go build $(BUILD_FLAGS) -o $@ ./$(@D)
-	$(NETGO_CHECK)
-
-$(WEAVEWAIT_EXE):
-	go build $(BUILD_FLAGS) -tags "netgo iface mcast" -o $@ ./$(@D)
-	$(NETGO_CHECK)
-
-$(WEAVEWAIT_NOMCAST_EXE):
-	go build $(BUILD_FLAGS) -tags "netgo iface" -o $@ ./$(@D)
-	$(NETGO_CHECK)
-
-# These programs need a separate rule as they fail the netgo check in
-# the main build stanza due to not importing net package
-$(SIGPROXY_EXE) $(TEST_TLS_EXE) $(WEAVEWAIT_NOOP_EXE) $(RUNNER_EXE):
-	go build $(BUILD_FLAGS) -o $@ ./$(@D)
-
-tests:
-	./tools/test -no-go-get -netgo -timeout 8m
-
-lint:
-	./tools/lint -nocomment -notestpackage
-
-endif
-
-# This rule makes sure the build image is up-to-date.
-# It also makes sure the multiarch hooks are reqistered in the kernel so the QEMU emulation works
-$(BUILD_UPTODATE): build/* tools/.git
-	$(SUDO) docker build -t $(BUILD_IMAGE) build/
-	$(SUDO) docker tag $(BUILD_IMAGE) $(BUILD_IMAGE):$(shell tools/image-tag)
-	$(SUDO) docker run --rm --privileged multiarch/qemu-user-static:register --reset
-	touch $@
-
-# Creates the Dockerfile.your-user-here file from the template
-# Also replaces all placeholders with real values
-# If the architecture is amd64, it deletes all CROSS_BUILD lines
-# but otherwise, it only removes the "CROSS_BUILD_" placeholder and downloads QEMU
-%/Dockerfile.$(DOCKERHUB_USER): %/Dockerfile.template
-	echo "DOCKERHUB_USER|$(DOCKERHUB_USER)|g;s|ARCH_EXT|$(ARCH_EXT)|g;s|ALPINE_BASEIMAGE|$(ALPINE_BASEIMAGE)|g;s|QEMUARCH|$(QEMUARCH)"
-	sed -e "s|DOCKERHUB_USER|$(DOCKERHUB_USER)|g;s|ARCH_EXT|$(ARCH_EXT)|g;s|ALPINE_BASEIMAGE|$(ALPINE_BASEIMAGE)|g;s|QEMUARCH|$(QEMUARCH)|g" $^ > $@
-ifeq ($(ARCH),amd64)
-# When building "normally" for amd64, remove the whole line, it has no part in the amd64 image
-	sed -i.bak "/CROSS_BUILD_/d" $@
-else
-# When cross-building, only the placeholder "CROSS_BUILD_" should be removed
-# Register /usr/bin/qemu-ARCH-static as the handler for ARM and ppc64le binaries in the kernel
-	curl -sSL https://github.com/multiarch/qemu-user-static/releases/download/$(QEMU_VERSION)/x86_64_qemu-$(QEMUARCH)-static.tar.gz | tar -xz -C $(shell dirname $@)
-	cd $(shell dirname $@) && sha256sum -c $(shell pwd)/build/shasums/qemu-$(QEMUARCH)-static.sha256sum
-	sed -i.bak "s/CROSS_BUILD_//g" $@
-endif
-
-# The targets below builds the weave images
-$(WEAVER_UPTODATE): prog/weaver/Dockerfile.$(DOCKERHUB_USER) $(WEAVER_EXE) weave $(WEAVEUTIL_EXE)
-	cp $(WEAVEUTIL_EXE) prog/weaver/weaveutil
-	cp weave prog/weaver/weave
-	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker build --build-arg=revision=$(GIT_REVISION) -f prog/weaver/Dockerfile.$(DOCKERHUB_USER) -t $(WEAVER_IMAGE) prog/weaver
-	touch $@
-
-$(WEAVEEXEC_UPTODATE): prog/weaveexec/Dockerfile.$(DOCKERHUB_USER) prog/weaveexec/symlink $(SIGPROXY_EXE) $(WEAVEWAIT_EXE) $(WEAVEWAIT_NOOP_EXE) $(WEAVEWAIT_NOMCAST_EXE) $(WEAVER_UPTODATE)
-	cp $(SIGPROXY_EXE) prog/weaveexec/sigproxy
-	cp $(WEAVEWAIT_EXE) prog/weaveexec/weavewait
-	cp $(WEAVEWAIT_NOOP_EXE) prog/weaveexec/weavewait_noop
-	cp $(WEAVEWAIT_NOMCAST_EXE) prog/weaveexec/weavewait_nomcast
-	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker build --build-arg=revision=$(GIT_REVISION) -f prog/weaveexec/Dockerfile.$(DOCKERHUB_USER) -t $(WEAVEEXEC_IMAGE) prog/weaveexec
-	touch $@
-
-# Builds Docker plugin.
-$(PLUGIN_UPTODATE): prog/net-plugin/launch.sh prog/net-plugin/config.json $(WEAVER_UPTODATE)
-	-$(SUDO) docker rm -f $(PLUGIN_BUILD_IMG) 2>/dev/null
-	$(SUDO) docker create --name=$(PLUGIN_BUILD_IMG) $(WEAVER_IMAGE) true
-	rm -rf $(PLUGIN_WORK_DIR)
-	mkdir $(PLUGIN_WORK_DIR)
-	$(SUDO) docker export $(PLUGIN_BUILD_IMG) | tar -x -C $(PLUGIN_WORK_DIR)
-	$(SUDO) docker rm -f $(PLUGIN_BUILD_IMG)
-	cp prog/net-plugin/launch.sh $(PLUGIN_WORK_DIR)/home/weave/launch.sh
-	-$(SUDO) docker plugin disable $(PLUGIN_IMAGE):$(WEAVE_VERSION) 2>/dev/null
-	-$(SUDO) docker plugin rm $(PLUGIN_IMAGE):$(WEAVE_VERSION) 2>/dev/null
-	$(SUDO) docker plugin create $(PLUGIN_IMAGE):$(WEAVE_VERSION) prog/net-plugin
-	-$(SUDO) docker plugin disable $(PLUGIN_IMAGE):$(NET_PLUGIN_LATEST) 2>/dev/null
-	-$(SUDO) docker plugin rm $(PLUGIN_IMAGE):$(NET_PLUGIN_LATEST) 2>/dev/null
-	$(SUDO) docker plugin create $(PLUGIN_IMAGE):$(NET_PLUGIN_LATEST) prog/net-plugin
-	touch $@
-
-$(WEAVEKUBE_UPTODATE): prog/weave-kube/Dockerfile.$(DOCKERHUB_USER) prog/weave-kube/launch.sh $(KUBEUTILS_EXE) $(WEAVER_UPTODATE)
-	cp $(KUBEUTILS_EXE) prog/weave-kube/
-	$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) -f prog/weave-kube/Dockerfile.$(DOCKERHUB_USER) -t $(WEAVEKUBE_IMAGE) prog/weave-kube
-	touch $@
-
-$(WEAVENPC_UPTODATE): prog/weave-npc/Dockerfile.$(DOCKERHUB_USER) $(WEAVENPC_EXE) prog/weave-npc/ulogd.conf
-	$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) -f prog/weave-npc/Dockerfile.$(DOCKERHUB_USER) -t $(WEAVENPC_IMAGE) prog/weave-npc
-	touch $@
-
-$(WEAVEDB_UPTODATE): prog/weavedb/Dockerfile
-	$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) -t $(WEAVEDB_IMAGE) prog/weavedb
-	touch $@
-
-$(NETWORKTESTER_UPTODATE): test/images/network-tester/Dockerfile $(NETWORKTESTER_EXE)
-	$(SUDO) docker build -t $(NETWORKTESTER_IMAGE) test/images/network-tester
-	touch $@
-
-$(WEAVE_EXPORT): $(IMAGES_UPTODATE) $(WEAVEDB_UPTODATE)
-	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker save $(addsuffix :latest,$(IMAGES)) | gzip > $@
-
-tools/.git:
-	git submodule update --init
-
-# CODE FOR PUBLISHING THE IMAGES
-
-# Push plugin
-plugin_publish: $(PLUGIN_UPTODATE)
-	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker plugin push $(PLUGIN_IMAGE):$(WEAVE_VERSION)
-# "$(NET_PLUGIN_LATEST)" means "stable release" here, so only push that if explicitly told to
-ifeq ($(UPDATE_LATEST),true)
-	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker plugin push $(PLUGIN_IMAGE):$(NET_PLUGIN_LATEST)
-endif
-
-# This target first runs "make publish" for each architecture
-# Then it pushes the manifest lists
-publish: $(addprefix sub-publish-,$(ALL_ARCHITECTURES)) $(WEAVEDB_UPTODATE)
-	$(MAKE) DOCKER_HOST=$(DOCKER_HOST) DOCKERHUB_USER=$(DOCKERHUB_USER) WEAVE_VERSION=$(WEAVE_VERSION) UPDATE_LATEST=$(UPDATE_LATEST) $(PUSH_ML)
-ifeq ($(PUBLISH_WEAVEDB),true)
-	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker push $(DOCKERHUB_USER)/weavedb:latest
-endif
-
-sub-publish-%:
-	$(MAKE) ARCH=$* DOCKER_HOST=$(DOCKER_HOST) clean-bin
-	$(MAKE) ARCH=$* DOCKER_HOST=$(DOCKER_HOST) DOCKERHUB_USER=$(DOCKERHUB_USER) WEAVE_VERSION=$(WEAVE_VERSION) UPDATE_LATEST=$(UPDATE_LATEST) $(IMAGES_UPTODATE)
-	$(MAKE) ARCH=$* DOCKER_HOST=$(DOCKER_HOST) DOCKERHUB_USER=$(DOCKERHUB_USER) WEAVE_VERSION=$(WEAVE_VERSION) UPDATE_LATEST=$(UPDATE_LATEST) publish-one-arch
-
-publish-one-arch: $(PUBLISH)
-
-# This rule handles the pushing of the built images
-$(PUBLISH): publish_%: $(IMAGES_UPTODATE)
-# Tag :latest with the real version
-	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker tag  $(DOCKERHUB_USER)/$*$(ARCH_EXT) $(DOCKERHUB_USER)/$*$(ARCH_EXT):$(WEAVE_VERSION)
-# Push the image with the arch suffix for arm, arm64 and ppc64le, and without suffix for amd64
-	$(MAKE) DOCKER_HOST=$(DOCKER_HOST) DOCKERHUB_USER=$(DOCKERHUB_USER) WEAVE_VERSION=$(WEAVE_VERSION) UPDATE_LATEST=$(UPDATE_LATEST) push_$*$(ARCH_EXT)
-
-ifeq ($(ARCH),amd64)
-# If the architecture is amd64, add the -amd64 suffix.
-	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker tag  $(DOCKERHUB_USER)/$*:$(WEAVE_VERSION) $(DOCKERHUB_USER)/$*-amd64:$(WEAVE_VERSION)
-# If the version is latest, tag the -amd64 with latest as well
-ifneq ($(UPDATE_LATEST),false)
-	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker tag  $(DOCKERHUB_USER)/$*-amd64:$(WEAVE_VERSION) $(DOCKERHUB_USER)/$*-amd64:latest
-endif
-# Push the image with the -amd64-suffix so BINARY-ARCH-named images exist for all arches, so manifest lists may be made and ARCH is replaceable in scripts
-	$(MAKE) DOCKER_HOST=$(DOCKER_HOST) DOCKERHUB_USER=$(DOCKERHUB_USER) WEAVE_VERSION=$(WEAVE_VERSION) UPDATE_LATEST=$(UPDATE_LATEST) push_$*-amd64
-endif
-
-# This target pushes an image, and if UPDATE_LATEST is anything but false, also updates the latest tag
-# It takes one parameter, the image name
-push_%:
-	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker push $(DOCKERHUB_USER)/$*:$(WEAVE_VERSION)
-ifneq ($(UPDATE_LATEST),false)
-	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker push $(DOCKERHUB_USER)/$*:latest
-endif
-
-# This target pushes a manifest list; it takes one parameter, the image name.
-# The variable UPDATE_LATEST controls whether it updates the versioned tag and/or the latest tag
-$(PUSH_ML): push_ml_%:
-ifneq ($(UPDATE_LATEST),latest-only)
-	$(MANIFEST_TOOL_CMD) push from-args --platforms $(ML_PLATFORMS) --template $(DOCKERHUB_USER)/$*-ARCH:$(WEAVE_VERSION) --target $(DOCKERHUB_USER)/$*:$(WEAVE_VERSION)
-endif
-ifneq ($(UPDATE_LATEST),false)
-# Push the manifest list to :latest as well
-	$(MANIFEST_TOOL_CMD) push from-args --platforms $(ML_PLATFORMS) --template $(DOCKERHUB_USER)/$*-ARCH:latest --target $(DOCKERHUB_USER)/$*:latest
-endif
-
-clean-work-dir:
-	rm -rf $(PLUGIN_WORK_DIR)
-
-clean-bin:
-	-$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker rmi $(IMAGES)
-	find prog -type f -name "Dockerfile.*" -not -name "Dockerfile.template" -print | xargs rm -f
-	find prog -type f -name "*qemu-*" -print | xargs rm -f
-	rm -rf $(EXES) $(IMAGES_UPTODATE) $(WEAVEDB_UPTODATE) weave*.tar.gz .pkg
-
-clean: clean-bin clean-work-dir
-	-$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker rmi $(BUILD_IMAGE)
-	rm -rf test/tls/*.pem test/coverage.* test/coverage $(BUILD_UPTODATE)
-
-build:
-	$(SUDO) go clean -i net
-	$(SUDO) go install -tags netgo std
-	$(MAKE)
-
-run-smoketests: all testrunner
-	cd test && ./setup.sh && ./run_all.sh
-
-integration-tests: all testrunner
-# Usage:
-#   $ make \
-#     NAME="<prefix used to name VMs and other resources>" \
-#     PROVIDER="<provider among {vagrant|gcp|aws|do}>" \
-#     NUM_HOSTS="<# test machines>" \
-#     PLAYBOOK="<filename>" \
-#     RUNNER_ARGS="<...>" \
-#     TESTS="<...>" \  # Can be set to only run one or a few tests instead of the full test suite.
-#     DOCKER_VERSION=<...> \
-#     KUBERNETES_VERSION=<...> \
-#     KUBERNETES_CNI_VERSION=<...> \
-#     <...> # See also run-integration-test.sh for all variables and individual functions.
-#     integration-tests
-#
-	RUNNER_ARGS="-parallel" ./test/run-integration-tests.sh
-
-# This target is only intended for use in Netlify CI environment for generating preview pages on feature branches and pull requests.
-# We need to obtain website repository (checked out under `site-build`) and place `site` directory into the context (`site-build/_weave_net_docs`).
-# We then run make in `site-build` and Netlify will publish the output (`site-build/_site`).
-netlify-site-preview:
-	@mkdir -p site-build
-	@curl --user $(WEBSITE_GITHUB_USER) --silent 'https://codeload.github.com/weaveworks/website-next/tar.gz/$(WEBSITE_BRANCH)' \
-	  | tar --extract --gunzip --directory site-build --strip 1
-	@cp -r site site-build/_weave_net_docs
-	@$(MAKE) -C site-build netlify_ensure_install
-	@$(MAKE) -C site-build BUILD_ENV=netlify

README.md

@@ -1,43 +1,36 @@
-# Weave Net - Weaving Containers into Applications
-
-[![Integration Tests](https://circleci.com/gh/weaveworks/weave/tree/master.svg?style=shield)](https://circleci.com/gh/weaveworks/weave)
-[![Coverage Status](https://coveralls.io/repos/weaveworks/weave/badge.svg)](https://coveralls.io/r/weaveworks/weave)
-[![Go Report Card](https://goreportcard.com/badge/github.com/weaveworks/weave)](https://goreportcard.com/report/github.com/weaveworks/weave)
-[![Docker Pulls](https://img.shields.io/docker/pulls/weaveworks/weave.svg?maxAge=604800)](https://hub.docker.com/r/weaveworks/weave/)
-
-# About Weaveworks
-
-[Weaveworks](https://www.weave.works) is the company that delivers the most productive way for developers to connect, observe and control
-Docker containers.
-
-This repository contains [Weave Net](https://www.weave.works/products/weave-net/), the first product developed by Weaveworks, and with over 8 million downloads to date, enables you to get started with Docker clusters and portable apps in a fraction of the time compared with other solutions.
-
-## Weave Cloud: 
-
-[Weave Cloud](https://www.weave.works/product/cloud/) is software-as-a-service that simplifies deployment, monitoring and management for containers and microservices. It extends and complements popular orchestrators, and enables developers and DevOps to realize faster deployments, insightful monitoring, visualization and networking.
-
-Weave Cloud is built using these Open Source projects: [Weave Scope](https://www.weave.works/products/weave-scope/), a powerful container visualization tool that automatically maps Docker containers and their interactions, [Weave Cortex](https://github.com/weaveworks/cortex), a horizontally-scalable version of Prometheus, and [Weave Flux](https://www.weave.works/products/weave-flux/), a continuous deployment tool that works with Kubernetes.
-
 # Weave Net
 
-Weave Net creates a virtual network that connects Docker containers across multiple hosts and enables their automatic discovery. With Weave Net, portable microservices-based applications consisting of multiple containers can run anywhere: on one host, multiple hosts or even across cloud providers and data centers. Applications use the network just as if the containers were all plugged into the same network switch, without having to configure port mappings, ambassadors or links.
+This repository contains a fork of Weave Net, the first product developed by Weaveworks. Since Weaveworks has shut down, this repo aims to continue maintaining Weave Net, and to publish releases regularly.
 
-Services provided by application containers on the Weave network can be exposed to the outside world, regardless of where they are running. Similarly, existing internal systems can be opened to accept connections from application containers irrespective of their location.
+[![Go Report Card](https://goreportcard.com/badge/github.com/rajch/weave)](https://goreportcard.com/report/github.com/rajch/weave)
+[![Docker Pulls](https://img.shields.io/docker/pulls/rajchaudhuri/weave-kube "Number of times the weave-kube image was pulled from the Docker Hub")](https://hub.docker.com/r/rajchaudhuri/weave-kube)
+[![GitHub release (latest by date)](https://img.shields.io/github/v/release/rajch/weave?include_prereleases)](https://github.com/rajch/weave/releases)
+[![Unique CVE count in all images](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Frajch%2Fweave%2Fmaster%2Freweave%2Fscans%2Fbadge.json&label=CVE%20count "The number of unique CVEs reported by scanning all images")](reweave/scans/report.md)
 
-## <a name="help"></a>Getting help
+The history of the fork can be found in [HISTORY.md](HISTORY.md).
 
-If you have any questions about, feedback for or a problem with Weave Net:
+## Using Weave Net on Kubernetes
 
-- Read [the Weave Net docs](https://www.weave.works/docs/net/latest/overview).
-- Invite yourself to the <a href="https://weaveworks.github.io/community-slack/" target="_blank"> #weave-community </a> slack channel.
-- Ask a question on the <a href="https://weave-community.slack.com/messages/general/"> #weave-community</a> slack channel.
-- Join the <a href="https://www.meetup.com/pro/Weave/"> Weave User Group </a> and get invited to online talks, hands-on training and meetups in your area.
-- Send an email to <a href="mailto:weave-users@weave.works">weave-users@weave.works</a>
-- <a href="https://github.com/weaveworks/weave/issues/new">File an issue.</a>
+On a newly created Kubernetes cluster, the Weave Net CNI pluging can be installed by running the following command:
 
-Your feedback is always welcome!
+```
+kubectl apply -f https://reweave.azurewebsites.net/k8s/v1.29/net.yaml
+```
 
-## Further Information
+Replace `v1.29` with the version on Kubernetes on your cluster.
 
-* [Installing Weave Net](https://www.weave.works/docs/net/latest/install/installing-weave)
-* [Weave Net Product Overview](https://www.weave.works/products/weave-net)
+That endpoint is provided by the companion project [weave-endpoint](https://github.com/rajch/weave-endpoint).
+
+## Using Weave Net in other ways
+
+Please refer to the [documentation](https://rajch.github.io/weave).
+
+## Building Weave Net
+
+Details can be found [here](reweave/BUILDING.md). 
+
+## Documentation status
+
+The public documentation that used to exist in the `site` directory has been moved to the `original/site` directory. A new `website` directory has been created, and populated with the content of the `original/site` directory, rearranged and reformatted for being built with Jekyll and published to the GitHub pages site [https://rajch.github.io/weave](https://rajch.github.io/weave).
+
+The documentation will now be maintained and published from the `website` directory exclusively.

bin/provision_test_vms.sh

@@ -1,40 +0,0 @@
-#!/bin/bash
-
-set -e
-
-# Signal failures in lock file, in order to fail fast:
-function signal_failure() {
-    echo "KO" >"$TEST_VMS_PROV_AND_CONF_LOCK_FILE"
-    exit 1
-}
-trap signal_failure ERR
-
-function install_terraform() {
-    curl -fsS https://releases.hashicorp.com/terraform/0.8.5/terraform_0.8.5_linux_amd64.zip | gunzip >terraform && chmod +x terraform && sudo mv terraform /usr/bin
-}
-
-function install_ansible() {
-    sudo apt-get update || true
-    sudo apt-get install -qq -y python-pip python-dev libffi-dev libssl-dev \
-        && pip install --user -U setuptools cffi \
-        && pip install --user ansible
-}
-
-[ -n "$SECRET_KEY" ] || {
-    echo "Cannot run smoke tests: no secret key"
-    exit 1
-}
-
-source "$SRCDIR/bin/circle-env"
-
-install_terraform
-install_ansible
-
-# Only attempt to create GCP image in first container, wait for it to be created otherwise:
-[ "$CIRCLE_NODE_INDEX" != "0" ] && export CREATE_IMAGE=0
-
-# Provision and configure testing VMs:
-cd "$SRCDIR/test" # Ensures we generate Terraform state files in the right folder, for later use by integration tests.
-./run-integration-tests.sh configure
-echo "OK" >"$TEST_VMS_PROV_AND_CONF_LOCK_FILE"
-echo "Test VMs now provisioned and configured. $(date)."

build/Dockerfile

@@ -1,95 +0,0 @@
-FROM golang:1.10.1-stretch
-
-# Support Raspberry Pi 2 and newer
-ENV GOARM 7
-
-# The names of the architectures that we're fetching cross-compilers for from the Debian packages
-ENV DEB_CROSSPLATFORMS \
-	armhf \
-	arm64 \
-	ppc64el
-
-# The go platforms we are supporting to cross-compile
-ENV GO_CROSSPLATFORMS \
-	linux/arm \
-	linux/arm64 \
-	linux/ppc64le
-
-# The names of the gcc cross-compilers we have installed
-ENV GCC_CROSSCOMPILERS \
-	arm-linux-gnueabihf \
-	aarch64-linux-gnu \
-	powerpc64le-linux-gnu
-
-# Temporarily add the Debian repositories, because we're going to install gcc cross-compilers from there
-# Install the build-essential and crossbuild-essential-ARCH packages
-RUN echo "deb http://emdebian.org/tools/debian/ jessie main" > /etc/apt/sources.list.d/cgocrosscompiling.list \
-  && curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add - \
-  && for platform in ${DEB_CROSSPLATFORMS}; do dpkg --add-architecture $platform; done \
-  && apt-get update \
-  && apt-get install -y build-essential \
-  && for platform in ${DEB_CROSSPLATFORMS}; do apt-get install -y crossbuild-essential-${platform}; done \
-  && apt-get clean && rm -rf /var/lib/apt/lists/*
-
-# Install some required packages
-# libpcap is required because we're linking against its C libraries from the prog/weaver binary
-# flex and bison are required packages for compiling libpcap manually later
-RUN apt-get update \
-	&& apt-get install -y \
-		libpcap-dev \
-		python-requests \
-		time \
-		file \
-		flex \
-		bison
-
-RUN curl -fsSLo shfmt https://github.com/mvdan/sh/releases/download/v1.3.0/shfmt_v1.3.0_linux_amd64 && \
-	echo "b1925c2c405458811f0c227266402cf1868b4de529f114722c2e3a5af4ac7bb2  shfmt" | sha256sum -c && \
-	chmod +x shfmt && \
-	mv shfmt /usr/bin
-
-# Install common Go tools
-RUN go get \
-    github.com/weaveworks/build-tools/cover \
-    github.com/mattn/goveralls \
-	golang.org/x/lint/golint \
-	github.com/fzipp/gocyclo \
-	github.com/fatih/hclfmt \
-	github.com/client9/misspell/cmd/misspell
-
-# First clean, because we're gonna rebuild the std both with -race enabled and disabled
-# Running go install std twice is intentional and required, as content is written in two different directories
-# (/usr/local/go/pkg/linux_amd64 vs /usr/local/go/pkg/linux_amd64_race)
-# The other architectures do not have support for -race
-RUN go clean -i net \
-	&& go install -tags netgo std \
-	&& go install -race -tags netgo std
-
-# Prebuild std for the other architectures
-RUN for platform in ${GO_CROSSPLATFORMS}; do \
-		GOOS=${platform%/*} GOARCH=${platform##*/} \
-			go install -tags netgo std; done
-
-# Allow full write access to the Go folders for anyone
-RUN chmod -R a+w /usr/local/go
-
-# The libpcap version from Debian packages is 1.6.2, matching that version here although newer versions of libpcap have been released
-# We have to cross-compile the libpcap library for the new architectures, that's what we're doing here
-ENV LIBPCAP_CROSS_VERSION=1.6.2
-RUN curl -sSL https://www.tcpdump.org/release/libpcap-${LIBPCAP_CROSS_VERSION}.tar.gz | tar -xz \
-	&& cd libpcap-${LIBPCAP_CROSS_VERSION} \
-	&& for crosscompiler in ${GCC_CROSSCOMPILERS}; do \
-		CC=${crosscompiler}-gcc ac_cv_linux_vers=2 ./configure --host=${crosscompiler} --with-pcap=linux \
-		&& make \
-		&& export LIB_DIR="/usr/local/lib/${crosscompiler}-gcc" VER="$(cat ./VERSION)" MAJOR_VER="$(sed 's/\([0-9][0-9]*\)\..*/\1/' ./VERSION)" \
-		&& mkdir -p ${LIB_DIR} \
-		&& cp -f libpcap.a libpcap.so.${VER} ${LIB_DIR} \
-		&& ln -sf libpcap.so.${VER} /usr/local/lib/libpcap.so.${MAJOR_VER} \
-		&& ln -sf libpcap.so.${MAJOR_VER} /usr/local/lib/libpcap.so \
-		&& make clean; done
-
-# Install Docker
-RUN curl -fsSL https://get.docker.com | VERSION=18.06.0-ce /bin/sh
-
-COPY build.sh /
-ENTRYPOINT ["sh", "/build.sh"]

common/chains/npc.go

@@ -0,0 +1,12 @@
+package chains
+
+const (
+	MainChain    = "WEAVE-NPC"
+	DefaultChain = "WEAVE-NPC-DEFAULT"
+	IngressChain = "WEAVE-NPC-INGRESS"
+
+	EgressChain        = "WEAVE-NPC-EGRESS"
+	EgressDefaultChain = "WEAVE-NPC-EGRESS-DEFAULT"
+	EgressCustomChain  = "WEAVE-NPC-EGRESS-CUSTOM"
+	EgressMarkChain    = "WEAVE-NPC-EGRESS-ACCEPT"
+)

common/docker/client.go

@@ -8,7 +8,7 @@ import (
 
 	docker "github.com/fsouza/go-dockerclient"
 
-	"github.com/weaveworks/weave/common"
+	"github.com/rajch/weave/common"
 )
 
 const (
@@ -99,7 +99,9 @@ func (c *Client) AddObserver(ob ContainerObserver) error {
 		pending := make(pendingStarts)
 		retryInterval := InitialInterval
 		for {
-			events := make(chan *docker.APIEvents)
+			// Use a buffered chan so the client library can run ahead of the listener
+			// - Docker will drop an event if it is not collected quickly enough.
+			events := make(chan *docker.APIEvents, 1024)
 			if err := c.AddEventListener(events); err != nil {
 				c.errorf("Unable to add listener to Docker API: %s - retrying in %ds", err, retryInterval/time.Second)
 			} else {

common/logging.go

@@ -54,6 +54,12 @@ func SetLogLevel(levelname string) {
 	Log.Level = level
 }
 
+func CheckError(e error) {
+	if e != nil {
+		Log.Error(e)
+	}
+}
+
 func CheckFatal(e error) {
 	if e != nil {
 		Log.Fatal(e)

docs/entrypoints.md

@@ -64,7 +64,7 @@ To add new options to how weave should run with each invocation, you would do th
 5. Add a command-line option `--option` to `weaveutil` or `weaver` as appropriate.
 6. If the option can be configured for CNI:
     * have `prog/weave-kube/launch.sh` read it as an environment variable and set inside
-    * set a default for the environment variable in `prog/weave-kube/weave-daemonset-k8s-1.6.yaml` and `weave-daemonset.yaml`
+    * set a default for the environment variable in `prog/weave-kube/weave-daemonset-k8s-N.N.yaml`
     * if it should be set via `weaver` globally on its one-time initialization invocation, pass it on in `launch.sh`
     * if it needs to be set via `weaveutil` on each invocation of `weave-net`, have `launch.sh` save it as an option in the CNI config file `/etc/cni/net.d/10-weave.conflist` and then have the CNI code in `plugin/net/cni.go` in `weaveutil` read it and use where appropriate
 7. Document it!

docs/k8s-src-ip.md

@@ -4,7 +4,7 @@
 
 * Weave Net: v2.2.1.
 * Kubernetes: v1.10.0.
-* kube-proxy: iptables mode (default kubeadm options).
+* kube-proxy: iptables/ipvs mode with `--masquerade-all=false` and `--cluster-cidr` unspecified(default kubeadm options).
 * Hosts: `h1` and `h2`.
 
 ## Source IP
@@ -14,7 +14,7 @@
 1  | Pod_h1      | ip(Pod_h2)        | Pod_h2 | ip(Pod_h1)       | OK
 2  | h1          | ClusterIP         | Pod_h1 | ip(weave_h1)     | **NOK**
 3  | h1          | ClusterIP         | Pod_h2 | ip(weave_h1)     | OK
-4  | Pod_h1      | ClusterIP         | Pod_h1 | ip(weave_h1)     | **NOK**
+4  | Pod_h1      | ClusterIP         | Pod_h1 | ip(Pod_h1)       | OK
 5  | Pod_h1      | ClusterIP         | Pod_h2 | ip(weave_h1)     | OK
 6  | h1          | ip(h1):NodePort   | Pod_h1 | ip(weave_h1)     | **NOK**
 7  | h1          | ip(h1):NodePort   | Pod_h2 | ip(weave_h1)     | OK

docs/release-process.md

@@ -3,6 +3,7 @@
 
 * Up-to-date versions of `git`, `go` etc
 * Install GitHub release tool `go get github.com/weaveworks/github-release`
+* Install manifest tool `go get github.com/estesp/manifest-tool`
 * Create a [github token for
   github-release](https://help.github.com/articles/creating-an-access-token-for-command-line-use/); select the `repo` OAuth scope; set and export `$GITHUB_TOKEN` with this value
 
@@ -135,7 +136,7 @@ Now, you can validate whether images were published for all platforms:
     # Published platforms per image:
     for img in $(grep '^PUBLISH=' Makefile); do
         img="weaveworks/$(echo $img | cut -d_ -f2):${TAG#v}"
-        platforms=$(vendor/github.com/estesp/manifest-tool/manifest-tool \
+        platforms=$(manifest-tool \
                 inspect --raw "$img" | \
                 jq '.[].Platform | .os + "/" +.architecture' | \
                 tr '\n' ' ')

docs/weavenpc-design.md

@@ -9,7 +9,9 @@ destroyed automatically in response to Pod, Namespace and
 NetworkPolicy object updates from the k8s API server:
 
 * A `hash:ip` set per namespace, containing the IP addresses of all
-  pods in that namespace
+  pods in that namespace for which default ingress is allowed
+* A `hash:ip` set per namespace, containing the IP addresses of all
+  pods in that namespace for which default egress is allowed  
 * A `list:set` per distinct (across all network policies in all
   namespaces) namespace selector mentioned in a network policy,
   containing the names of any of the above hash:ip sets whose
@@ -18,6 +20,9 @@ NetworkPolicy object updates from the k8s API server:
   containing network policy's namespace) pod selector mentioned in a
   network policy, containing the IP addresses of all pods in the
   namespace whose labels match that selector
+* A `hash:net` set for each distinct (within the scope of the
+  containing network policy's namespace) `except` list of CIDR's mentioned in
+  the network policies
 
 IPsets are implemented by the kernel module `xt_set`, without which
 weave-npc will not work.
@@ -38,11 +43,21 @@ hashing to avoid clashes.
 
 # iptables chains
 
-The policy controller maintains two iptables chains in response to
-changes to pods, namespaces and network policies. One chain contains
-the ingress rules that implement the network policy specifications,
-and the other is used to bypass the ingress rules for namespaces which
-have an ingress isolation policy of `DefaultAllow`.
+The policy controller maintains several iptables chains in response to
+changes to pods, namespaces and network policies. 
+
+## Static `WEAVE-NPC` chain
+
+`WEAVE-NPC` chain contains static rules to ACCEPT traffic that is `RELATED,ESTABLISHED`
+and run `NEW` traffic through `WEAVE-NPC-DEFAULT` followed by `WEAVE-NPC-INGRESS` chains 
+
+Static configuration:
+
+```
+iptables -A WEAVE-NPC -m state --state RELATED,ESTABLISHED -j ACCEPT
+iptables -A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-DEFAULT
+iptables -A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-INGRESS
+```
 
 ## Dynamically maintained `WEAVE-NPC-DEFAULT` chain
 
@@ -63,23 +78,58 @@ For each namespace network policy ingress rule peer/port combination:
 iptables -A WEAVE-NPC-INGRESS -p $PROTO [-m set --match-set $SRCSET] -m set --match-set $DSTSET --dport $DPORT -j ACCEPT
 ```
 
-## Static `WEAVE-NPC` chain
+## Static `WEAVE-NPC-EGRESS` chain
+
+`WEAVE-NPC-EGRESS` chain contains static rules to ACCEPT traffic that is `RELATED,ESTABLISHED`
+and run `NEW` traffic through `WEAVE-NPC-EGRESS-DEFAULT` followed by `WEAVE-NPC-EGRESS-CUSTOM` chains 
 
 Static configuration:
 
 ```
-iptables -A WEAVE-NPC -m state --state RELATED,ESTABLISHED -j ACCEPT
-iptables -A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-DEFAULT
-iptables -A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-INGRESS
+iptables -A WEAVE-NPC-EGRESS -m state --state RELATED,ESTABLISHED -j ACCEPT
+iptables -A WEAVE-NPC-EGRESS -m state --state NEW -j WEAVE-NPC-EGRESS-DEFAULT
+iptables -A WEAVE-NPC-EGRESS -m state --state NEW -m mark ! --mark 0x40000/0x40000 -j WEAVE-NPC-EGRESS-CUSTOM
+iptables -A WEAVE-NPC-EGRESS -m mark ! --mark 0x40000/0x40000 -j DROP
 ```
 
+## Dynamically maintained `WEAVE-NPC-EGRESS-DEFAULT` chain
+
+The policy controller maintains a rule in this chain for every
+namespace whose egress isolation policy is `DefaultAllow`. The
+purpose of this rule is simply to ACCEPT any traffic originating from such namespace before it reaches the egress chain.
+
+```
+iptables -A WEAVE-NPC-EGRESS-DEFAULT -m set --match-set $NSIPSET src -j WEAVE-NPC-EGRESS-ACCEPT
+iptables -A WEAVE-NPC-EGRESS-DEFAULT -m set --match-set $NSIPSET src -j RETURN
+```
+
+## Static `WEAVE-NPC-EGRESS-ACCEPT` chain
+
+`WEAVE-NPC-EGRESS-ACCEPTS` chain contains static rules to mark traffic
+
+Static configuration:
+
+```
+iptables -A WEAVE-NPC-EGRESS-ACCEPT -j MARK --set-xmark 0x40000/0x40000
+```
+
+## Dynamically maintained `WEAVE-NPC-EGRESS-CUSTOM` chain
+
+For each namespace network policy egress rule peer/port combination:
+
+```
+iptables -A WEAVE-NPC-EGRESS-CUSTOM -p $PROTO [-m set --match-set $SRCSET] -m set --match-set $DSTSET --dport $DPORT -j ACCEPT
+```
+
+
 # Steering traffic into the policy engine
 
 To direct traffic into the policy engine:
 
 ```
-iptables -A FORWARD -o weave -m physdev ! --physdev-out vethwe-bridge -j WEAVE-NPC
-iptables -A FORWARD -o weave -m physdev ! --physdev-out vethwe-bridge -j DROP
+iptables -A INPUT -i weave -j WEAVE-NPC-EGRESS
+iptables -A FORWARD -i weave -j WEAVE-NPC-EGRESS
+iptables -A FORWARD -o weave -j WEAVE-NPC
 ```
 
 Note this only affects traffic which egresses the bridge on a physical

go.mod

@@ -0,0 +1,119 @@
+module github.com/rajch/weave
+
+go 1.23.1
+
+require (
+	github.com/andybalholm/go-bit v1.0.1
+	github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5
+	github.com/aws/aws-sdk-go v1.34.0
+	github.com/boltdb/bolt v1.3.1
+	github.com/containernetworking/cni v1.1.2
+	github.com/containernetworking/plugins v1.2.0
+	github.com/coreos/go-iptables v0.6.0
+	github.com/docker/docker v25.0.6+incompatible
+	github.com/docker/libnetwork v0.8.0-dev.2.0.20180212220201-119badbd737b
+	github.com/docker/machine v0.14.0
+	github.com/fsouza/go-dockerclient v1.10.0
+	github.com/google/gopacket v1.1.16
+	github.com/gorilla/mux v1.8.0
+	github.com/j-keck/arping v0.0.0-20160618110441-2cf9dc699c56
+	github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b
+	github.com/miekg/dns v1.1.52
+	github.com/pkg/errors v0.9.1
+	github.com/pkg/profile v1.2.1
+	github.com/prometheus/client_golang v1.14.0
+	github.com/sirupsen/logrus v1.9.3
+	github.com/stretchr/testify v1.8.4
+	github.com/vishvananda/netlink v1.2.1-beta.2
+	github.com/vishvananda/netns v0.0.0-20210104183010-2eb08e3e575f
+	github.com/weaveworks/common v0.0.0-20180321110636-ebab3a78900a
+	github.com/weaveworks/go-checkpoint v0.0.0-20170503165305-ebbb8b0518ab
+	github.com/weaveworks/go-odp v0.0.0-20181017121109-6b0aa22550d9
+	github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63
+	golang.org/x/crypto v0.31.0
+	golang.org/x/sys v0.28.0
+	golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d
+	k8s.io/api v0.26.2
+	k8s.io/apimachinery v0.26.2
+	k8s.io/client-go v0.26.2
+)
+
+require github.com/spf13/cobra v1.6.1
+
+require github.com/mattn/go-isatty v0.0.18 // indirect
+
+require (
+	github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
+	github.com/Microsoft/go-winio v0.6.1 // indirect
+	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/cespare/xxhash/v2 v2.2.0 // indirect
+	github.com/containerd/containerd v1.7.11 // indirect
+	github.com/containerd/log v0.1.0 // indirect
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/distribution/reference v0.6.0 // indirect
+	github.com/docker/go-connections v0.4.0 // indirect
+	github.com/docker/go-units v0.5.0 // indirect
+	github.com/emicklei/go-restful/v3 v3.10.1 // indirect
+	github.com/evanphx/json-patch v4.12.0+incompatible // indirect
+	github.com/felixge/httpsnoop v1.0.3 // indirect
+	github.com/go-logr/logr v1.2.4 // indirect
+	github.com/go-logr/stdr v1.2.2 // indirect
+	github.com/go-openapi/jsonpointer v0.19.5 // indirect
+	github.com/go-openapi/jsonreference v0.20.0 // indirect
+	github.com/go-openapi/swag v0.19.14 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/protobuf v1.5.3 // indirect
+	github.com/google/gnostic v0.5.7-v3refs // indirect
+	github.com/google/go-cmp v0.6.0 // indirect
+	github.com/google/gofuzz v1.2.0 // indirect
+	github.com/google/uuid v1.3.0 // indirect
+	github.com/hashicorp/go-cleanhttp v0.0.0-20171218145408-d5fe4b57a186 // indirect
+	github.com/inconshreveable/mousetrap v1.0.1 // indirect
+	github.com/jmespath/go-jmespath v0.3.0 // indirect
+	github.com/josharian/intern v1.0.0 // indirect
+	github.com/json-iterator/go v1.1.12 // indirect
+	github.com/klauspost/compress v1.16.0 // indirect
+	github.com/mailru/easyjson v0.7.6 // indirect
+	github.com/mattn/go-colorable v0.0.9 // indirect
+	github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
+	github.com/mdlayher/raw v0.0.0-20181016155347-fa5ef3332ca9 // indirect
+	github.com/moby/patternmatcher v0.6.0 // indirect
+	github.com/moby/sys/sequential v0.5.0 // indirect
+	github.com/moby/sys/user v0.3.0 // indirect
+	github.com/moby/term v0.0.0-20221205130635-1aeaba878587 // indirect
+	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
+	github.com/modern-go/reflect2 v1.0.2 // indirect
+	github.com/morikuni/aec v1.0.0 // indirect
+	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
+	github.com/opencontainers/go-digest v1.0.0 // indirect
+	github.com/opencontainers/image-spec v1.1.0-rc2.0.20221005185240-3a7f492d3f1b // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	github.com/prometheus/client_model v0.3.0 // indirect
+	github.com/prometheus/common v0.37.0 // indirect
+	github.com/prometheus/procfs v0.8.0 // indirect
+	github.com/safchain/ethtool v0.2.0 // indirect
+	github.com/spf13/pflag v1.0.5 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.45.0 // indirect
+	go.opentelemetry.io/otel v1.19.0 // indirect
+	go.opentelemetry.io/otel/metric v1.19.0 // indirect
+	go.opentelemetry.io/otel/trace v1.19.0 // indirect
+	golang.org/x/mod v0.17.0 // indirect
+	golang.org/x/net v0.33.0 // indirect
+	golang.org/x/oauth2 v0.10.0 // indirect
+	golang.org/x/sync v0.10.0 // indirect
+	golang.org/x/term v0.27.0 // indirect
+	golang.org/x/text v0.21.0 // indirect
+	golang.org/x/time v0.0.0-20220210224613-90d013bbcef8 // indirect
+	google.golang.org/appengine v1.6.7 // indirect
+	google.golang.org/protobuf v1.33.0 // indirect
+	gopkg.in/inf.v0 v0.9.1 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+	gotest.tools/v3 v3.4.0 // indirect
+	k8s.io/klog/v2 v2.90.1 // indirect
+	k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280 // indirect
+	k8s.io/utils v0.0.0-20230220204549-a5ecb0141aa5 // indirect
+	sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 // indirect
+	sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
+	sigs.k8s.io/yaml v1.3.0 // indirect
+)

go.sum

@@ -0,0 +1,772 @@
+cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
+cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=
+cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
+cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
+cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
+cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
+cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=
+cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=
+cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=
+cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
+cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
+cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
+cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
+cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
+cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
+cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
+cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
+cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
+cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
+cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
+cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
+cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
+cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
+cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
+cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=
+cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
+cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
+cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
+cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
+cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
+dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
+github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24 h1:bvDV9vkmnHYOMsOr4WLk+Vo07yKIzd94sVoIqshQ4bU=
+github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8=
+github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 h1:UQHMgLO+TxOElx5B5HZ4hJQsoJ/PvUvKRhJHDQXO8P8=
+github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
+github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
+github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
+github.com/Microsoft/go-winio v0.6.1 h1:9/kr64B9VUZrLm5YYwbGtUJnMgqWVOdUAXu6Migciow=
+github.com/Microsoft/go-winio v0.6.1/go.mod h1:LRdKpFKfdobln8UmuiYcKPot9D2v6svN5+sAH+4kjUM=
+github.com/Microsoft/hcsshim v0.11.4 h1:68vKo2VN8DE9AdN4tnkWnmdhqdbpUFM8OF3Airm7fz8=
+github.com/Microsoft/hcsshim v0.11.4/go.mod h1:smjE4dvqPX9Zldna+t5FG3rnoHhaB7QYxPRqGcpAD9w=
+github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
+github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
+github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
+github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
+github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
+github.com/andybalholm/go-bit v1.0.1 h1:GihKSzvvcJKJtMFvPov8ITgdDvHxHt5p5whGyptxzwc=
+github.com/andybalholm/go-bit v1.0.1/go.mod h1:ezUwZUxOwUxgSjWJxb7p3Lo5dTdBBvImVcb6EsHgg8I=
+github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=
+github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
+github.com/aws/aws-sdk-go v1.34.0 h1:brux2dRrlwCF5JhTL7MUT3WUwo9zfDHZZp3+g3Mvlmo=
+github.com/aws/aws-sdk-go v1.34.0/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
+github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
+github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
+github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
+github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
+github.com/boltdb/bolt v1.3.1 h1:JQmyP4ZBrce+ZQu0dY660FMfatumYDLun9hBCUVIkF4=
+github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps=
+github.com/cenkalti/backoff/v4 v4.2.1 h1:y4OZtCnogmCPw98Zjyt5a6+QwPLGkiQsYW5oUqylYbM=
+github.com/cenkalti/backoff/v4 v4.2.1/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
+github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
+github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
+github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
+github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
+github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
+github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
+github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
+github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
+github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
+github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
+github.com/containerd/containerd v1.7.11 h1:lfGKw3eU35sjV0aG2eYZTiwFEY1pCzxdzicHP3SZILw=
+github.com/containerd/containerd v1.7.11/go.mod h1:5UluHxHTX2rdvYuZ5OJTC5m/KJNs0Zs9wVoJm9zf5ZE=
+github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I=
+github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo=
+github.com/containernetworking/cni v1.1.2 h1:wtRGZVv7olUHMOqouPpn3cXJWpJgM6+EUl31EQbXALQ=
+github.com/containernetworking/cni v1.1.2/go.mod h1:sDpYKmGVENF3s6uvMvGgldDWeG8dMxakj/u+i9ht9vw=
+github.com/containernetworking/plugins v1.2.0 h1:SWgg3dQG1yzUo4d9iD8cwSVh1VqI+bP7mkPDoSfP9VU=
+github.com/containernetworking/plugins v1.2.0/go.mod h1:/VjX4uHecW5vVimFa1wkG4s+r/s9qIfPdqlLF4TW8c4=
+github.com/coreos/go-iptables v0.6.0 h1:is9qnZMPYjLd8LYqmm/qlE+wwEgJIkTYdhV3rfZo4jk=
+github.com/coreos/go-iptables v0.6.0/go.mod h1:Qe8Bv2Xik5FyTXwgIbLAnv2sWSBmvWdFETJConOQ//Q=
+github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
+github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
+github.com/creack/pty v1.1.18 h1:n56/Zwd5o6whRC5PMGretI4IdRLlmBXYNjScPaBgsbY=
+github.com/creack/pty v1.1.18/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
+github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
+github.com/docker/docker v25.0.6+incompatible h1:5cPwbwriIcsua2REJe8HqQV+6WlWc1byg2QSXzBxBGg=
+github.com/docker/docker v25.0.6+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
+github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
+github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
+github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
+github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
+github.com/docker/libnetwork v0.8.0-dev.2.0.20180212220201-119badbd737b h1:u7WZJnaLG9o39ZmpCJEwuFM6nCBuUQL0y/AbyE5sbaM=
+github.com/docker/libnetwork v0.8.0-dev.2.0.20180212220201-119badbd737b/go.mod h1:93m0aTqz6z+g32wla4l4WxTrdtvBRmVzYRkYvasA5Z8=
+github.com/docker/machine v0.14.0 h1:7zMwMAz0XbL27Q4NPqtW7I0wuhOaGFpq5WfvrUfZROM=
+github.com/docker/machine v0.14.0/go.mod h1:I8mPNDeK1uH+JTcUU7X0ZW8KiYz0jyAgNaeSJ1rCfDI=
+github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE=
+github.com/emicklei/go-restful/v3 v3.10.1 h1:rc42Y5YTp7Am7CS630D7JmhRjq4UlEUuEKfrDac4bSQ=
+github.com/emicklei/go-restful/v3 v3.10.1/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
+github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
+github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
+github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
+github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
+github.com/evanphx/json-patch v4.12.0+incompatible h1:4onqiflcdA9EOZ4RxV643DvftH5pOlLGNtQ5lPWQu84=
+github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
+github.com/felixge/httpsnoop v1.0.3 h1:s/nj+GCswXYzN5v2DpNMuMQYe+0DDwt5WVCU6CWBdXk=
+github.com/felixge/httpsnoop v1.0.3/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
+github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
+github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
+github.com/fsnotify/fsnotify v1.6.0 h1:n+5WquG0fcWoWp6xPWfHdbskMCQaFnG6PfBrh1Ky4HY=
+github.com/fsnotify/fsnotify v1.6.0/go.mod h1:sl3t1tCWJFWoRz9R8WJCbQihKKwmorjAbSClcnxKAGw=
+github.com/fsouza/go-dockerclient v1.10.0 h1:ppSBsbR60I1DFbV4Ag7LlHlHakHFRNLk9XakATW1yVQ=
+github.com/fsouza/go-dockerclient v1.10.0/go.mod h1:+iNzAW78AzClIBTZ6WFjkaMvOgz68GyCJ236b1opLTs=
+github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
+github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
+github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
+github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
+github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
+github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY=
+github.com/go-kit/log v0.2.0/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBjv0=
+github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
+github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
+github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=
+github.com/go-logfmt/logfmt v0.5.1/go.mod h1:WYhtIu8zTZfxdn5+rREduYbwxfcBr/Vr6KEVveWlfTs=
+github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
+github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
+github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ=
+github.com/go-logr/logr v1.2.4/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
+github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
+github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
+github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
+github.com/go-openapi/jsonpointer v0.19.5 h1:gZr+CIYByUqjcgeLXnQu2gHYQC9o73G2XUeOFYEICuY=
+github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
+github.com/go-openapi/jsonreference v0.20.0 h1:MYlu0sBgChmCfJxxUKZ8g1cPWFOB37YSZqewK7OKeyA=
+github.com/go-openapi/jsonreference v0.20.0/go.mod h1:Ag74Ico3lPc+zR+qjn4XBUmXymS4zJbYVCZmcgkasdo=
+github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
+github.com/go-openapi/swag v0.19.14 h1:gm3vOOXfiuw5i9p5N9xJvfjvuofpyvLA9Wr6QfK5Fng=
+github.com/go-openapi/swag v0.19.14/go.mod h1:QYRuS/SOXUCsnplDa677K7+DxSOj6IPNl/eQntq43wQ=
+github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
+github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
+github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE=
+github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
+github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
+github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
+github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
+github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
+github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
+github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
+github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
+github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
+github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
+github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
+github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
+github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
+github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
+github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
+github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
+github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
+github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
+github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
+github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
+github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
+github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
+github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
+github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
+github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
+github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
+github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
+github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
+github.com/google/gnostic v0.5.7-v3refs h1:FhTMOKj2VhjpouxvWJAV1TL304uMlb9zcDqkl6cEI54=
+github.com/google/gnostic v0.5.7-v3refs/go.mod h1:73MKFl6jIHelAJNaBGFzt3SPtZULs9dYrGFt8OiIsHQ=
+github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
+github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
+github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
+github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
+github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
+github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
+github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
+github.com/google/gopacket v1.1.16 h1:u6Afvia5C5srlLcbTwpHaFW918asLYPxieziOaWwz8M=
+github.com/google/gopacket v1.1.16/go.mod h1:UCLx9mCmAwsVbn6qQl1WIEt2SO7Nd2fD0th1TBAsqBw=
+github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
+github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
+github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
+github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
+github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
+github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
+github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
+github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
+github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
+github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
+github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
+github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
+github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
+github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
+github.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI=
+github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
+github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0 h1:YBftPWNWd4WwGqtY2yeZL2ef8rHAxPBD8KFhJpmcqms=
+github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0/go.mod h1:YN5jB8ie0yfIUg6VvR9Kz84aCaG7AsGZnLjhHbUqwPg=
+github.com/hashicorp/go-cleanhttp v0.0.0-20171218145408-d5fe4b57a186 h1:URgjUo+bs1KwatoNbwG0uCO4dHN4r1jsp4a5AGgHRjo=
+github.com/hashicorp/go-cleanhttp v0.0.0-20171218145408-d5fe4b57a186/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
+github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
+github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
+github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
+github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
+github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
+github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7Pgzkat/bFNc=
+github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
+github.com/j-keck/arping v0.0.0-20160618110441-2cf9dc699c56 h1:742eGXur0715JMq73aD95/FU0XpVKXqNuTnEfXsLOYQ=
+github.com/j-keck/arping v0.0.0-20160618110441-2cf9dc699c56/go.mod h1:ymszkNOg6tORTn+6F6j+Jc8TOr5osrynvN6ivFWZ2GA=
+github.com/jmespath/go-jmespath v0.3.0 h1:OS12ieG61fsCg5+qLJ+SsW9NicxNkg3b25OyT2yCeUc=
+github.com/jmespath/go-jmespath v0.3.0/go.mod h1:9QtRXoHjLGCJ5IBSaohpXITPlowMeeYCZ7fLUTSywik=
+github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
+github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
+github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4=
+github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
+github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
+github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
+github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
+github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
+github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
+github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
+github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
+github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
+github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
+github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
+github.com/klauspost/compress v1.16.0 h1:iULayQNOReoYUe+1qtKOqw9CwJv3aNQu8ivo7lw1HU4=
+github.com/klauspost/compress v1.16.0/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
+github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
+github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
+github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
+github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
+github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
+github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
+github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
+github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
+github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
+github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
+github.com/mailru/easyjson v0.7.6 h1:8yTIVnZgCoiM1TgqoeTl+LfU5Jg6/xL3QhGQnimLYnA=
+github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
+github.com/mattn/go-colorable v0.0.9 h1:UVL0vNpWh04HeJXV0KLcaT7r06gOH2l4OW6ddYRUIY4=
+github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
+github.com/mattn/go-isatty v0.0.18 h1:DOKFKCQ7FNG2L1rbrmstDN4QVRdS89Nkh85u68Uwp98=
+github.com/mattn/go-isatty v0.0.18/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
+github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
+github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo=
+github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
+github.com/mdlayher/raw v0.0.0-20181016155347-fa5ef3332ca9 h1:tOtO8DXiNGj9NshRKHWiZuGlSldPFzFCFYhNtsKTBCs=
+github.com/mdlayher/raw v0.0.0-20181016155347-fa5ef3332ca9/go.mod h1:rC/yE65s/DoHB6BzVOUBNYBGTg772JVytyAytffIZkY=
+github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b h1:j7+1HpAFS1zy5+Q4qx1fWh90gTKwiN4QCGoY9TWyyO4=
+github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
+github.com/miekg/dns v1.1.52 h1:Bmlc/qsNNULOe6bpXcUTsuOajd0DzRHwup6D9k1An0c=
+github.com/miekg/dns v1.1.52/go.mod h1:uInx36IzPl7FYnDcMeVWxj9byh7DutNykX4G9Sj60FY=
+github.com/moby/patternmatcher v0.6.0 h1:GmP9lR19aU5GqSSFko+5pRqHi+Ohk1O69aFiKkVGiPk=
+github.com/moby/patternmatcher v0.6.0/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
+github.com/moby/sys/sequential v0.5.0 h1:OPvI35Lzn9K04PBbCLW0g4LcFAJgHsvXsRyewg5lXtc=
+github.com/moby/sys/sequential v0.5.0/go.mod h1:tH2cOOs5V9MlPiXcQzRC+eEyab644PWKGRYaaV5ZZlo=
+github.com/moby/sys/user v0.3.0 h1:9ni5DlcW5an3SvRSx4MouotOygvzaXbaSrc/wGDFWPo=
+github.com/moby/sys/user v0.3.0/go.mod h1:bG+tYYYJgaMtRKgEmuueC0hJEAZWwtIbZTB+85uoHjs=
+github.com/moby/term v0.0.0-20221205130635-1aeaba878587 h1:HfkjXDfhgVaN5rmueG8cL8KKeFNecRCXFhaJ2qZ5SKA=
+github.com/moby/term v0.0.0-20221205130635-1aeaba878587/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y=
+github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
+github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
+github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
+github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
+github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
+github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
+github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
+github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
+github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
+github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
+github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
+github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
+github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
+github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs=
+github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
+github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
+github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE=
+github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=
+github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
+github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=
+github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0=
+github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=
+github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU=
+github.com/onsi/ginkgo/v2 v2.1.3/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c=
+github.com/onsi/ginkgo/v2 v2.4.0 h1:+Ig9nvqgS5OBSACXNk15PLdp0U9XPYROt9CFzVdFGIs=
+github.com/onsi/ginkgo/v2 v2.4.0/go.mod h1:iHkDK1fKGcBoEHT5W7YBq4RFWaQulw+caOMkAt4OrFo=
+github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=
+github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
+github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY=
+github.com/onsi/gomega v1.24.2 h1:J/tulyYK6JwBldPViHJReihxxZ+22FHs0piGjQAvoUE=
+github.com/onsi/gomega v1.24.2/go.mod h1:gs3J10IS7Z7r7eXRoNJIrNqU4ToQukCJhFtKrWgHWnk=
+github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
+github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
+github.com/opencontainers/image-spec v1.1.0-rc2.0.20221005185240-3a7f492d3f1b h1:YWuSjZCQAPM8UUBLkYUk1e+rZcvWHJmFb6i6rM44Xs8=
+github.com/opencontainers/image-spec v1.1.0-rc2.0.20221005185240-3a7f492d3f1b/go.mod h1:3OVijpioIKYWTqjiG0zfF6wvoJ4fAXGbjdZuI2NgsRQ=
+github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
+github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pkg/profile v1.2.1 h1:F++O52m40owAmADcojzM+9gyjmMOY/T4oYJkgFDH8RE=
+github.com/pkg/profile v1.2.1/go.mod h1:hJw3o1OdXxsrSjjVksARp5W95eeEaEfptyVZyv6JUPA=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
+github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
+github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
+github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0=
+github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY=
+github.com/prometheus/client_golang v1.14.0 h1:nJdhIvne2eSX/XRAFV9PcvFFRbrjbcTUj0VP62TMhnw=
+github.com/prometheus/client_golang v1.14.0/go.mod h1:8vpkKitgIVNcqrRBWh1C4TIUQgYNtG/XQE4E/Zae36Y=
+github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
+github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/prometheus/client_model v0.3.0 h1:UBgGFHqYdG/TPFD1B1ogZywDqEkwp3fBMvqdiQ7Xew4=
+github.com/prometheus/client_model v0.3.0/go.mod h1:LDGWKZIo7rky3hgvBe+caln+Dr3dPggB5dvjtD7w9+w=
+github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
+github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo=
+github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc=
+github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls=
+github.com/prometheus/common v0.37.0 h1:ccBbHCgIiT9uSoFY0vX8H3zsNR5eLt17/RQLUvn8pXE=
+github.com/prometheus/common v0.37.0/go.mod h1:phzohg0JFMnBEFGxTDbfu3QyL5GI8gTQJFhYO5B3mfA=
+github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
+github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
+github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
+github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
+github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
+github.com/prometheus/procfs v0.8.0 h1:ODq8ZFEaYeCaZOJlZZdJA2AbQR98dSHSM1KW/You5mo=
+github.com/prometheus/procfs v0.8.0/go.mod h1:z7EfXMXOkbkqb9IINtpCn86r/to3BnA0uaxHdg830/4=
+github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
+github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
+github.com/safchain/ethtool v0.2.0 h1:dILxMBqDnQfX192cCAPjZr9v2IgVXeElHPy435Z/IdE=
+github.com/safchain/ethtool v0.2.0/go.mod h1:WkKB1DnNtvsMlDmQ50sgwowDJV/hGbJSOvJoEXs1AJQ=
+github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
+github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
+github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
+github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
+github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
+github.com/spf13/cobra v1.6.1 h1:o94oiPyS4KD1mPy2fmcYYHHfCxLqYjJOhGsCHFZtEzA=
+github.com/spf13/cobra v1.6.1/go.mod h1:IOw/AERYS7UzyrGinqmz6HLUo219MORXGxhbaJUqzrY=
+github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
+github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
+github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
+github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
+github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
+github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
+github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
+github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
+github.com/vishvananda/netlink v1.2.1-beta.2 h1:Llsql0lnQEbHj0I1OuKyp8otXp0r3q0mPkuhwHfStVs=
+github.com/vishvananda/netlink v1.2.1-beta.2/go.mod h1:twkDnbuQxJYemMlGd4JFIcuhgX83tXhKS2B/PRMpOho=
+github.com/vishvananda/netns v0.0.0-20200728191858-db3c7e526aae/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0=
+github.com/vishvananda/netns v0.0.0-20210104183010-2eb08e3e575f h1:p4VB7kIXpOQvVn1ZaTIVp+3vuYAXFe3OJEvjbUYJLaA=
+github.com/vishvananda/netns v0.0.0-20210104183010-2eb08e3e575f/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0=
+github.com/weaveworks/common v0.0.0-20180321110636-ebab3a78900a h1:dIOYWkmKCq9tU3DY96p1D/8xXFWkqt9Jd1ipu6sTLSk=
+github.com/weaveworks/common v0.0.0-20180321110636-ebab3a78900a/go.mod h1:pSm+0KR57BG3pvGoJWFXJSAC7+sEPewcvdt5StevL3A=
+github.com/weaveworks/go-checkpoint v0.0.0-20170503165305-ebbb8b0518ab h1:mW+hgchD9qUUBqnuaDBj7BkcpFPk/FxeFcUFI5lvvUw=
+github.com/weaveworks/go-checkpoint v0.0.0-20170503165305-ebbb8b0518ab/go.mod h1:qkbvw5GPibQ/Nf7IZJL0UoLwmJ6858b4S/hUWRd+cH4=
+github.com/weaveworks/go-odp v0.0.0-20181017121109-6b0aa22550d9 h1:7OqrOqOJkneFGUT7CoR3uZnZP3ZyuUtzWNfv7GIc3RE=
+github.com/weaveworks/go-odp v0.0.0-20181017121109-6b0aa22550d9/go.mod h1:6/WClQKiVm/SQf6WJXv49l+9K9ONE2U8DYpeXEk3X7k=
+github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 h1:s0fUBZ8Vhtc3ruFmLIr3qVTQUb/j6ySkPLHoKKitHeM=
+github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63/go.mod h1:RZebXKv56dax5zXcLIJZm1Awk28sx0XODXF94Z8WssY=
+github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
+go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
+go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
+go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
+go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.45.0 h1:x8Z78aZx8cOF0+Kkazoc7lwUNMGy0LrzEMxTm4BbTxg=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.45.0/go.mod h1:62CPTSry9QZtOaSsE3tOzhx6LzDhHnXJ6xHeMNNiM6Q=
+go.opentelemetry.io/otel v1.19.0 h1:MuS/TNf4/j4IXsZuJegVzI1cwut7Qc00344rgH7p8bs=
+go.opentelemetry.io/otel v1.19.0/go.mod h1:i0QyjOq3UPoTzff0PJB2N66fb4S0+rSbSB15/oyH9fY=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.19.0 h1:Mne5On7VWdx7omSrSSZvM4Kw7cS7NQkOOmLcgscI51U=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.19.0/go.mod h1:IPtUMKL4O3tH5y+iXVyAXqpAwMuzC1IrxVS81rummfE=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.19.0 h1:IeMeyr1aBvBiPVYihXIaeIZba6b8E1bYp7lbdxK8CQg=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.19.0/go.mod h1:oVdCUtjq9MK9BlS7TtucsQwUcXcymNiEDjgDD2jMtZU=
+go.opentelemetry.io/otel/metric v1.19.0 h1:aTzpGtV0ar9wlV4Sna9sdJyII5jTVJEvKETPiOKwvpE=
+go.opentelemetry.io/otel/metric v1.19.0/go.mod h1:L5rUsV9kM1IxCj1MmSdS+JQAcVm319EUrDVLrt7jqt8=
+go.opentelemetry.io/otel/sdk v1.19.0 h1:6USY6zH+L8uMH8L3t1enZPR3WFEmSTADlqldyHtJi3o=
+go.opentelemetry.io/otel/sdk v1.19.0/go.mod h1:NedEbbS4w3C6zElbLdPJKOpJQOrGUJ+GfzpjUvI0v1A=
+go.opentelemetry.io/otel/trace v1.19.0 h1:DFVQmlVbfVeOuBRrwdtaehRrWiL1JoVs9CPIQ1Dzxpg=
+go.opentelemetry.io/otel/trace v1.19.0/go.mod h1:mfaSyvGyEJEI0nyV2I4qhNQnbBOUUmYZpYojqMnX2vo=
+go.opentelemetry.io/proto/otlp v1.0.0 h1:T0TX0tmXU8a3CbNXzEKGeU5mIVOdf0oykP+u2lIVU/I=
+go.opentelemetry.io/proto/otlp v1.0.0/go.mod h1:Sy6pihPLfYHkr3NkUbEhGHFhINUSI/v80hjKIs5JXpM=
+golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
+golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
+golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
+golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
+golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
+golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
+golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
+golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
+golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
+golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
+golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
+golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
+golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
+golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
+golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
+golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
+golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
+golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
+golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
+golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
+golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
+golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
+golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
+golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
+golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
+golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
+golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
+golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
+golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
+golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
+golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA=
+golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
+golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
+golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
+golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
+golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
+golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
+golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=
+golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
+golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
+golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
+golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
+golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
+golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
+golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
+golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
+golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
+golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
+golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc=
+golang.org/x/oauth2 v0.10.0 h1:zHCpF2Khkwy4mMB4bv0U37YtJdTGW8jI0glAApi0Kh8=
+golang.org/x/oauth2 v0.10.0/go.mod h1:kTpgurOux7LqtuxjuyZa4Gj2gdezIt/jQtGnNFfypQI=
+golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ=
+golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
+golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200217220822-9197077df867/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200728102440-3e129f6d46b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
+golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.27.0 h1:WP60Sv1nlK1T6SupCHbXzSaN0b9wUmsPoRS9b61A23Q=
+golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM=
+golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
+golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
+golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
+golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/time v0.0.0-20220210224613-90d013bbcef8 h1:vVKdlvoWBphwdxWKrFZEuM0kGgGLxUOYcY4U/2Vjg44=
+golang.org/x/time v0.0.0-20220210224613-90d013bbcef8/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
+golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
+golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
+golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
+golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
+golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
+golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
+golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
+golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
+golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
+golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
+golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
+golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
+golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
+golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg=
+golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
+golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
+google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
+google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
+google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
+google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
+google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
+google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
+google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
+google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
+google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
+google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
+google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
+google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
+google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
+google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=
+google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
+google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
+google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
+google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
+google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
+google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
+google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
+google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c=
+google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
+google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
+google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
+google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
+google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
+google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
+google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
+google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
+google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
+google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
+google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
+google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
+google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
+google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
+google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
+google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=
+google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=
+google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
+google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
+google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20201019141844-1ed22bb0c154/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20230711160842-782d3b101e98 h1:Z0hjGZePRE0ZBWotvtrwxFNrNE9CUAGtplaDK5NNI/g=
+google.golang.org/genproto/googleapis/api v0.0.0-20230711160842-782d3b101e98 h1:FmF5cCW94Ij59cfpoLiwTgodWmm60eEV0CjlsVg2fuw=
+google.golang.org/genproto/googleapis/api v0.0.0-20230711160842-782d3b101e98/go.mod h1:rsr7RhLuwsDKL7RmgDDCUc6yaGr1iqceVb5Wv6f6YvQ=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98 h1:bVf09lpb+OJbByTj913DRJioFFAjf/ZGxEz7MajTp2U=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98/go.mod h1:TUfxEVdsvPg18p6AslUXFoLdpED4oBnGwyqk3dV1XzM=
+google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
+google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
+google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
+google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
+google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
+google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
+google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
+google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
+google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
+google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
+google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
+google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
+google.golang.org/grpc v1.58.3 h1:BjnpXut1btbtgN/6sp+brB2Kbm2LjNXnidYujAVbSoQ=
+google.golang.org/grpc v1.58.3/go.mod h1:tgX3ZQDlNJGU96V6yHh1T/JeoBQ2TXdr43YbYSsCJk0=
+google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
+google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
+google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
+google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
+google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
+google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
+google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
+google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
+google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
+google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
+google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
+google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
+google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
+google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
+gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU=
+gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
+gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
+gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
+gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
+gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
+gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
+gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
+gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gotest.tools/v3 v3.4.0 h1:ZazjZUfuVeZGLAmlKKuyv3IKP5orXcwtOwDQH6YVr6o=
+gotest.tools/v3 v3.4.0/go.mod h1:CtbdzLSsqVhDgMtKsx03ird5YTGB3ar27v0u/yKBW5g=
+honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
+honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
+honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
+k8s.io/api v0.26.2 h1:dM3cinp3PGB6asOySalOZxEG4CZ0IAdJsrYZXE/ovGQ=
+k8s.io/api v0.26.2/go.mod h1:1kjMQsFE+QHPfskEcVNgL3+Hp88B80uj0QtSOlj8itU=
+k8s.io/apimachinery v0.26.2 h1:da1u3D5wfR5u2RpLhE/ZtZS2P7QvDgLZTi9wrNZl/tQ=
+k8s.io/apimachinery v0.26.2/go.mod h1:ats7nN1LExKHvJ9TmwootT00Yz05MuYqPXEXaVeOy5I=
+k8s.io/client-go v0.26.2 h1:s1WkVujHX3kTp4Zn4yGNFK+dlDXy1bAAkIl+cFAiuYI=
+k8s.io/client-go v0.26.2/go.mod h1:u5EjOuSyBa09yqqyY7m3abZeovO/7D/WehVVlZ2qcqU=
+k8s.io/klog/v2 v2.90.1 h1:m4bYOKall2MmOiRaR1J+We67Do7vm9KiQVlT96lnHUw=
+k8s.io/klog/v2 v2.90.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0=
+k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280 h1:+70TFaan3hfJzs+7VK2o+OGxg8HsuBr/5f6tVAjDu6E=
+k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280/go.mod h1:+Axhij7bCpeqhklhUTe3xmOn6bWxolyZEeyaFpjGtl4=
+k8s.io/utils v0.0.0-20230220204549-a5ecb0141aa5 h1:kmDqav+P+/5e1i9tFfHq1qcF3sOrDp+YEkVDAHu7Jwk=
+k8s.io/utils v0.0.0-20230220204549-a5ecb0141aa5/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
+rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
+rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
+rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
+sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 h1:iXTIw73aPyC+oRdyqqvVJuloN1p0AC/kzH07hu3NE+k=
+sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0=
+sigs.k8s.io/structured-merge-diff/v4 v4.2.3 h1:PRbqxJClWWYMNV1dhaG4NsibJbArud9kFxnAMREiWFE=
+sigs.k8s.io/structured-merge-diff/v4 v4.2.3/go.mod h1:qjx8mGObPmV2aSZepjQjbmb2ihdVs8cGKBraizNC69E=
+sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo=
+sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8=

ipam/allocate.go

@@ -3,8 +3,8 @@ package ipam
 import (
 	"fmt"
 
-	"github.com/weaveworks/weave/api"
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/api"
+	"github.com/rajch/weave/net/address"
 )
 
 type allocateResult struct {

ipam/allocator.go

@@ -9,13 +9,13 @@ import (
 
 	"github.com/weaveworks/mesh"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/db"
-	"github.com/weaveworks/weave/ipam/paxos"
-	"github.com/weaveworks/weave/ipam/ring"
-	"github.com/weaveworks/weave/ipam/space"
-	"github.com/weaveworks/weave/ipam/tracker"
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/db"
+	"github.com/rajch/weave/ipam/paxos"
+	"github.com/rajch/weave/ipam/ring"
+	"github.com/rajch/weave/ipam/space"
+	"github.com/rajch/weave/ipam/tracker"
+	"github.com/rajch/weave/net/address"
 )
 
 // Kinds of message we can unicast to other peers
@@ -207,21 +207,6 @@ func (alloc *Allocator) doOperation(op operation, ops *[]operation) {
 	}
 }
 
-// Given an operation, remove it from the pending queue
-//  Note the op may not be on the queue; it may have
-//  already succeeded.  If it is on the queue, we call
-//  cancel on it, allowing callers waiting for the resultChans
-//  to unblock.
-func (alloc *Allocator) cancelOp(opToCancel operation, ops *[]operation) {
-	for i, op := range *ops {
-		if op == opToCancel {
-			*ops = append((*ops)[:i], (*ops)[i+1:]...)
-			op.Cancel()
-			break
-		}
-	}
-}
-
 // Cancel all operations in a queue
 func (alloc *Allocator) cancelOps(ops *[]operation) {
 	for _, op := range *ops {
@@ -275,7 +260,7 @@ func (alloc *Allocator) spaceRequestDenied(sender mesh.PeerName, r address.Range
 	for i := 0; i < len(alloc.pendingClaims); {
 		claim := alloc.pendingClaims[i].(*claim)
 		if r.Contains(claim.cidr.Addr) {
-			claim.deniedBy(alloc, sender)
+			claim.deniedBy(alloc, sender, nil)
 			alloc.pendingClaims = append(alloc.pendingClaims[:i], alloc.pendingClaims[i+1:]...)
 			continue
 		}
@@ -556,7 +541,8 @@ func (alloc *Allocator) annotatePeernames(names []mesh.PeerName) []string {
 // in the ring. Async.
 //
 // NB: the function is invoked by the gossip library routines and should be
-//     registered manually.
+//
+//	registered manually.
 func (alloc *Allocator) PeerGone(peerName mesh.PeerName) {
 	alloc.debugf("PeerGone: peer %s", peerName)
 
@@ -814,6 +800,16 @@ func (alloc *Allocator) sendRingUpdate(dest mesh.PeerName) {
 	alloc.gossip.GossipUnicast(dest, msg)
 }
 
+func (alloc *Allocator) hasAllocations(rs []address.Range) bool {
+	for _, r := range rs {
+		alloc.debugf("hasAllocations: %v %v vs %v", r, alloc.space.NumFreeAddressesInRange(r), r.Size())
+		if alloc.space.NumFreeAddressesInRange(r) != r.Size() {
+			return true
+		}
+	}
+	return false
+}
+
 func (alloc *Allocator) update(sender mesh.PeerName, msg []byte) error {
 	reader := bytes.NewReader(msg)
 	decoder := gob.NewDecoder(reader)
@@ -832,7 +828,7 @@ func (alloc *Allocator) update(sender mesh.PeerName, msg []byte) error {
 	// If someone sent us a ring, merge it into ours. Note this will move us
 	// out of the awaiting-consensus state if we didn't have a ring already.
 	case data.Ring != nil:
-		updated, err := alloc.ring.Merge(*data.Ring)
+		updated, err := alloc.ring.Merge(*data.Ring, alloc.hasAllocations)
 		switch err {
 		case nil:
 			if updated {

ipam/allocator_test.go

@@ -9,16 +9,14 @@ import (
 
 	"github.com/stretchr/testify/require"
 
-	"github.com/weaveworks/weave/api"
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/address"
-	"github.com/weaveworks/weave/testing/gossip"
+	"github.com/rajch/weave/api"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/net/address"
+	"github.com/rajch/weave/testing/gossip"
 )
 
 const (
 	testStart1 = "10.0.1.0"
-	testStart2 = "10.0.2.0"
-	testStart3 = "10.0.3.0"
 )
 
 func returnFalse() bool { return false }

ipam/claim.go

@@ -5,13 +5,14 @@ import (
 
 	"github.com/weaveworks/mesh"
 
-	"github.com/weaveworks/weave/api"
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/api"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/net/address"
 )
 
 type claim struct {
 	resultChan       chan<- error
+	tryCount         int
 	ident            string       // a container ID, something like "weave:expose", or api.NoContainerID
 	cidr             address.CIDR // single address being claimed
 	isContainer      bool         // true if ident is a container ID
@@ -19,6 +20,8 @@ type claim struct {
 	hasBeenCancelled func() bool
 }
 
+const maxTryCount = 5
+
 // Send an error (or nil for success) back to caller listening on resultChan
 func (c *claim) sendResult(result error) {
 	// Make sure we only send a result once, since listener stops listening after that
@@ -40,6 +43,8 @@ func (c *claim) Try(alloc *Allocator) bool {
 		return true
 	}
 
+	c.tryCount++
+
 	addOwned := func() {
 		if c.ident == api.NoContainerID {
 			alloc.addOwned(c.cidr.Addr.String(), c.cidr, c.isContainer)
@@ -90,8 +95,10 @@ func (c *claim) Try(alloc *Allocator) bool {
 			if c.noErrorOnUnknown {
 				alloc.infof("Claim %s for %s: %s; will try later.", c.cidr, c.ident, err)
 				c.sendResult(nil)
-			} else { // just tell the user they can't do this.
-				c.deniedBy(alloc, owner)
+			} else if c.tryCount > maxTryCount {
+				// give up, tell the user they can't do this
+				c.deniedBy(alloc, owner, err)
+				return true
 			}
 		}
 		return false
@@ -128,12 +135,16 @@ func (c *claim) Try(alloc *Allocator) bool {
 	return true
 }
 
-func (c *claim) deniedBy(alloc *Allocator, owner mesh.PeerName) {
+func (c *claim) deniedBy(alloc *Allocator, owner mesh.PeerName, err error) {
 	name, found := alloc.nicknames[owner]
 	if found {
 		name = " (" + name + ")"
 	}
-	c.sendResult(fmt.Errorf("address %s is owned by other peer %s%s", c.cidr.String(), owner, name))
+	reason := ""
+	if err != nil {
+		reason = fmt.Sprintf(" - %s", err)
+	}
+	c.sendResult(fmt.Errorf("address %s is owned by other peer %s%s%s", c.cidr.String(), owner, name, reason))
 }
 
 func (c *claim) Cancel() {

ipam/http.go

@@ -1,15 +1,16 @@
 package ipam
 
 import (
+	"context"
 	"encoding/json"
 	"fmt"
 	"net/http"
 
 	"github.com/gorilla/mux"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/common/docker"
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/common/docker"
+	"github.com/rajch/weave/net/address"
 )
 
 func badRequest(w http.ResponseWriter, err error) {
@@ -41,7 +42,7 @@ func writeAddresses(w http.ResponseWriter, cidrs []address.CIDR) {
 	}
 }
 
-func hasBeenCancelled(dockerCli *docker.Client, closedChan <-chan bool, ident string, checkAlive bool) func() bool {
+func hasBeenCancelled(dockerCli *docker.Client, closedChan <-chan struct{}, ident string, checkAlive bool) func() bool {
 	return func() bool {
 		select {
 		case <-closedChan:
@@ -63,9 +64,9 @@ func cancellationErr(w http.ResponseWriter, err error) bool {
 	return false
 }
 
-func (alloc *Allocator) handleHTTPAllocate(dockerCli *docker.Client, w http.ResponseWriter, ident string, checkAlive bool, subnet address.CIDR) {
+func (alloc *Allocator) handleHTTPAllocate(ctx context.Context, dockerCli *docker.Client, w http.ResponseWriter, ident string, checkAlive bool, subnet address.CIDR) {
 	addr, err := alloc.Allocate(ident, subnet, checkAlive,
-		hasBeenCancelled(dockerCli, w.(http.CloseNotifier).CloseNotify(), ident, checkAlive))
+		hasBeenCancelled(dockerCli, ctx.Done(), ident, checkAlive))
 	if err != nil {
 		if !cancellationErr(w, err) {
 			badRequest(w, err)
@@ -75,9 +76,9 @@ func (alloc *Allocator) handleHTTPAllocate(dockerCli *docker.Client, w http.Resp
 	fmt.Fprintf(w, "%s/%d", addr, subnet.PrefixLen)
 }
 
-func (alloc *Allocator) handleHTTPClaim(dockerCli *docker.Client, w http.ResponseWriter, ident string, cidr address.CIDR, checkAlive, noErrorOnUnknown bool) {
+func (alloc *Allocator) handleHTTPClaim(ctx context.Context, dockerCli *docker.Client, w http.ResponseWriter, ident string, cidr address.CIDR, checkAlive, noErrorOnUnknown bool) {
 	err := alloc.Claim(ident, cidr, checkAlive, noErrorOnUnknown,
-		hasBeenCancelled(dockerCli, w.(http.CloseNotifier).CloseNotify(), ident, checkAlive))
+		hasBeenCancelled(dockerCli, ctx.Done(), ident, checkAlive))
 	if err != nil {
 		if !cancellationErr(w, err) {
 			badRequest(w, fmt.Errorf("Unable to claim: %s", err))
@@ -99,7 +100,7 @@ func (alloc *Allocator) HandleHTTP(router *mux.Router, defaultSubnet address.CID
 			ident := vars["id"]
 			checkAlive := r.FormValue("check-alive") == "true"
 			noErrorOnUnknown := r.FormValue("noErrorOnUnknown") == "true"
-			alloc.handleHTTPClaim(dockerCli, w, ident, cidr, checkAlive, noErrorOnUnknown)
+			alloc.handleHTTPClaim(r.Context(), dockerCli, w, ident, cidr, checkAlive, noErrorOnUnknown)
 		}
 	})
 
@@ -164,13 +165,13 @@ func (alloc *Allocator) HandleHTTP(router *mux.Router, defaultSubnet address.CID
 	router.Methods("POST").Path("/ip/{id}/{ip}/{prefixlen}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		vars := mux.Vars(r)
 		if subnet, ok := parseCIDR(w, vars["ip"]+"/"+vars["prefixlen"], true); ok {
-			alloc.handleHTTPAllocate(dockerCli, w, vars["id"], r.FormValue("check-alive") == "true", subnet)
+			alloc.handleHTTPAllocate(r.Context(), dockerCli, w, vars["id"], r.FormValue("check-alive") == "true", subnet)
 		}
 	})
 
 	router.Methods("POST").Path("/ip/{id}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		vars := mux.Vars(r)
-		alloc.handleHTTPAllocate(dockerCli, w, vars["id"], r.FormValue("check-alive") == "true", defaultSubnet)
+		alloc.handleHTTPAllocate(r.Context(), dockerCli, w, vars["id"], r.FormValue("check-alive") == "true", defaultSubnet)
 	})
 
 	router.Methods("DELETE").Path("/ip/{id}/{ip}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -203,6 +204,6 @@ func (alloc *Allocator) HandleHTTP(router *mux.Router, defaultSubnet address.CID
 		if alloc.tracker != nil {
 			tracker = alloc.tracker.String()
 		}
-		fmt.Fprintf(w, tracker)
+		fmt.Fprintln(w, tracker)
 	})
 }

ipam/http_test.go

@@ -1,6 +1,7 @@
 package ipam
 
 import (
+	"context"
 	"fmt"
 	"io"
 	"io/ioutil"
@@ -11,9 +12,9 @@ import (
 	"time"
 
 	"github.com/gorilla/mux"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/net/address"
 	"github.com/stretchr/testify/require"
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/address"
 )
 
 func HTTPPost(t *testing.T, url string) string {
@@ -158,11 +159,13 @@ func TestHTTPCancel(t *testing.T) {
 
 	// Ask the http server for a new address
 	req, _ := http.NewRequest("POST", allocURL(port, testCIDR1, containerID), nil)
+	ctx, cancel := context.WithCancel(context.Background())
+	req = req.WithContext(ctx)
 	// On another goroutine, wait for a bit then cancel the request
 	go func() {
 		time.Sleep(100 * time.Millisecond)
 		common.Log.Debug("Cancelling allocate")
-		http.DefaultTransport.(*http.Transport).CancelRequest(req)
+		cancel()
 	}()
 
 	res, _ := http.DefaultClient.Do(req)

ipam/paxos/paxos_test.go

@@ -56,16 +56,6 @@ func (m *Model) addLink(a, b *TestNode) {
 	m.readyLinks = append(m.readyLinks, &ab, &ba)
 }
 
-func (m *Model) linkExists(a, b *TestNode) bool {
-	for _, l := range a.links {
-		if l.to == b {
-			return true
-		}
-	}
-
-	return false
-}
-
 type TestParams struct {
 	// Number of nodes
 	nodeCount uint

ipam/ring/entry.go

@@ -1,12 +1,13 @@
 package ring
 
 import (
+	"fmt"
 	"sort"
 
 	"github.com/weaveworks/mesh"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/net/address"
 )
 
 // Entry represents entries around the ring
@@ -25,6 +26,7 @@ func (e *entry) Equal(e2 *entry) bool {
 func (e *entry) update(peername mesh.PeerName, free address.Count) {
 	e.Peer = peername
 	e.Version++
+	common.Log.Debugln(fmt.Sprintf("Updating ring token=%s current peer=%s updated peer=%s new version=%s", e.Token, e.Peer, peername, fmt.Sprint(e.Version)))
 	e.Free = free
 }
 
@@ -72,8 +74,9 @@ func (es entries) get(token address.Address) (*entry, bool) {
 // Is token between entries at i and j?
 // NB i and j can overflow and will wrap
 // NBB this function does not work very well if there is only one
-//     token on the ring; luckily an accurate answer is not needed
-//     by the call sites in this case.
+//
+//	token on the ring; luckily an accurate answer is not needed
+//	by the call sites in this case.
 func (es entries) between(token address.Address, i, j int) bool {
 	common.Assert(i < j)
 

ipam/ring/ring.go

@@ -14,8 +14,8 @@ import (
 
 	"github.com/weaveworks/mesh"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/net/address"
 )
 
 type OnUpdate func(prev, curr []address.Range, local bool)
@@ -213,7 +213,7 @@ func (r *Ring) GrantRangeToHost(start, end address.Address, peer mesh.PeerName)
 
 // Merge the given ring into this ring and indicate whether this ring
 // got updated as a result.
-func (r *Ring) Merge(gossip Ring) (bool, error) {
+func (r *Ring) Merge(gossip Ring, hasAllocations func(r []address.Range) bool) (bool, error) {
 	r.assertInvariants()
 	defer r.trackUpdates()()
 
@@ -238,12 +238,22 @@ func (r *Ring) Merge(gossip Ring) (bool, error) {
 		return false, ErrDifferentRange
 	}
 
-	result, updated, err := r.Entries.merge(gossip.Entries, r.Peer)
+	result, updated, err := r.Entries.merge(gossip.Entries, r.Peer, r, hasAllocations)
 
 	if err != nil {
 		return false, err
 	}
 
+	// reset the free space for entries if there is invalid free space
+	// due to accepting an unexpected update from the peers
+	for i := 0; i < len(result); i++ {
+		distance := r.distance(result.entry(i).Token, result.entry(i+1).Token)
+		if result.entry(i).Peer == r.Peer && result.entry(i).Free > distance {
+			// case that can arise when a range that we own that got split and had no allocations
+			result.entry(i).Free = distance
+		}
+	}
+
 	if err := r.checkEntries(result); err != nil {
 		return false, fmt.Errorf("Merge of incoming data causes: %s", err)
 	}
@@ -260,7 +270,7 @@ func (r *Ring) Merge(gossip Ring) (bool, error) {
 // entries belonging to ourPeer. Returns the merged entries and an
 // indication whether the merge resulted in any changes, i.e. the
 // result differs from the original.
-func (es entries) merge(other entries, ourPeer mesh.PeerName) (result entries, updated bool, err error) {
+func (es entries) merge(other entries, ourPeer mesh.PeerName, r *Ring, hasAllocations func(r []address.Range) bool) (result entries, updated bool, err error) {
 	var mine, theirs *entry
 	var previousOwner *mesh.PeerName
 	addToResult := func(e entry) { result = append(result, &e) }
@@ -274,6 +284,7 @@ func (es entries) merge(other entries, ourPeer mesh.PeerName) (result entries, u
 	var i, j int
 	for i < len(es) && j < len(other) {
 		mine, theirs = es[i], other[j]
+		common.Log.Debugf("[ring %s]: Merge mine.Token=%s theirs.Token=%s mine.Peer=%s theirs.Peer=%s mine.Version=%v theirs.Version=%v", ourPeer, mine.Token, theirs.Token, mine.Peer, theirs.Peer, mine.Version, theirs.Version)
 		switch {
 		case mine.Token < theirs.Token:
 			addToResult(*mine)
@@ -282,8 +293,11 @@ func (es entries) merge(other entries, ourPeer mesh.PeerName) (result entries, u
 		case mine.Token > theirs.Token:
 			// insert, checking that a range owned by us hasn't been split
 			if previousOwner != nil && *previousOwner == ourPeer && theirs.Peer != ourPeer {
-				err = errEntryInMyRange(theirs)
-				return
+				// check we have no allocations in the range that got split
+				if hasAllocations(r.makeRanges(theirs.Token, mine.Token)) {
+					err = errEntryInMyRange(theirs)
+					return
+				}
 			}
 			addTheirs(*theirs)
 			j++
@@ -292,17 +306,44 @@ func (es entries) merge(other entries, ourPeer mesh.PeerName) (result entries, u
 			switch {
 			case mine.Version >= theirs.Version:
 				if mine.Version == theirs.Version && !mine.Equal(theirs) {
-					err = errInconsistentEntry(mine, theirs)
-					return
+					if mine.Peer == ourPeer {
+						// if we own the entry and has allocations
+						if hasAllocations(r.makeRanges(theirs.Token, es.entry(i+1).Token)) {
+							err = errInconsistentEntry(mine, theirs)
+							return
+						}
+					}
+					// tie-break here, pick the entry with the highest free count
+					if mine.Free >= theirs.Free {
+						addToResult(*mine)
+						previousOwner = &mine.Peer
+					} else {
+						addTheirs(*theirs)
+					}
+				} else {
+					addToResult(*mine)
+					previousOwner = &mine.Peer
 				}
-				addToResult(*mine)
-				previousOwner = &mine.Peer
 			case mine.Version < theirs.Version:
-				if mine.Peer == ourPeer { // We shouldn't receive updates to our own tokens
-					err = errNewerVersion(mine, theirs)
-					return
+				if mine.Peer == ourPeer {
+					// We received update to our own tokens accept the received entry
+					// if either it belongs to a different peer and we do not have allocations
+					// in the range effectively given away, or it belongs to our own peer, in
+					// which case we should set our version to the one received plus one,
+					// effectively imposing our existing entry.
+					if theirs.Peer != ourPeer && !hasAllocations(r.makeRanges(mine.Token, es.entry(i+1).Token)) {
+						common.Log.Infof("ring: repair inconsistent data by accepting token %v from %v - no allocations up to %v", theirs.Token, theirs.Peer, es.entry(i+1).Token)
+						addTheirs(*theirs)
+					} else if theirs.Peer == ourPeer {
+						mine.Version = theirs.Version + 1
+						addToResult(*mine)
+					} else {
+						err = errNewerVersion(mine, theirs)
+						return
+					}
+				} else {
+					addTheirs(*theirs)
 				}
-				addTheirs(*theirs)
 			}
 			i++
 			j++
@@ -334,6 +375,11 @@ func (r *Ring) Empty() bool {
 	return len(r.Entries) == 0
 }
 
+// convenience function to normalise a single range that may span zero
+func (r *Ring) makeRanges(start, end address.Address) []address.Range {
+	return r.splitRangesOverZero([]address.Range{{Start: start, End: end}})
+}
+
 // Given a slice of ranges which are all in the right order except
 // possibly the last one spans zero, fix that up and return the slice
 func (r *Ring) splitRangesOverZero(ranges []address.Range) []address.Range {
@@ -384,10 +430,7 @@ type RangeInfo struct {
 
 func (r *Ring) AllRangeInfo() (result []RangeInfo) {
 	for i, entry := range r.Entries {
-		nextEntry := r.Entries.entry(i + 1)
-		ranges := []address.Range{{Start: entry.Token, End: nextEntry.Token}}
-		ranges = r.splitRangesOverZero(ranges)
-		for _, r := range ranges {
+		for _, r := range r.makeRanges(entry.Token, r.Entries.entry(i+1).Token) {
 			result = append(result, RangeInfo{entry.Peer, r, entry.Version})
 		}
 	}
@@ -485,6 +528,7 @@ func (r *Ring) ReportFree(freespace map[address.Address]address.Count) (updated
 
 		entries[i].Free = free
 		entries[i].Version++
+		common.Log.Debugf("[ring %s]: ReportFree token=%s peer=%s version=%v", r.Peer, entries[i].Token, entries[i].Peer, entries[i].Version)
 		updated = true
 	}
 	return
@@ -516,7 +560,7 @@ func (r *Ring) ChoosePeersToAskForSpace(start, end address.Address) []mesh.PeerN
 			break
 		}
 		// Ignore ranges with no free space
-		if entry.Free <= 0 {
+		if entry.Free == 0 {
 			continue
 		}
 
@@ -568,6 +612,7 @@ func (r *Ring) Transfer(from, to mesh.PeerName) []address.Range {
 			// bump version by large value so that transfer of range takes precedence (if there is any conflict in merging entry)
 			// over other opertaions on the entry that increments the version
 			entry.Version = entry.Version + 100
+			common.Log.Debugf("[ring %s]: Transfer token=%s from=%s to=%s version=%v", r.Peer, entry.Token, from, to, entry.Version)
 			newRanges = append(newRanges, address.Range{Start: entry.Token, End: r.Entries.entry(i + 1).Token})
 		}
 	}

ipam/ring/ring_test.go

@@ -10,8 +10,8 @@ import (
 	"github.com/stretchr/testify/require"
 	"github.com/weaveworks/mesh"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/net/address"
 )
 
 var (
@@ -32,7 +32,27 @@ func ParseIP(s string) address.Address {
 }
 
 func merge(r1, r2 *Ring) error {
-	_, err := r1.Merge(*r2)
+	distance := func(start, end address.Address) address.Count {
+		if end > start {
+			return address.Count(end - start)
+		}
+		return address.Count((r1.End - start) + (end - r1.Start))
+	}
+	check := func(rs []address.Range) bool {
+		for _, r := range rs {
+			size := distance(r.Start, r.End)
+			entry, found := r1.Entries.get(r.Start)
+			if !found {
+				continue
+			}
+			if entry.Free < size {
+				return true
+			}
+		}
+		return false
+	}
+
+	_, err := r1.Merge(*r2, check)
 	return err
 }
 
@@ -219,6 +239,61 @@ func TestMergeSimple(t *testing.T) {
 	require.Equal(t, ring2.Entries, ring1.Entries)
 }
 
+func TestMergeWithConflicts(t *testing.T) {
+
+	// received update to entry with a token and version identical to an entry we have
+	// but but holding different content, if there are no allocations, resolve the
+	// conflict by picking entry one with high free count
+	ring1 := NewRing(start, end, peer1name)
+	ring2 := NewRing(start, end, peer2name)
+	ring1.Entries = []*entry{{Token: start, Peer: peer1name, Free: 256, Version: 1}}
+	ring2.Entries = []*entry{{Token: start, Peer: peer2name, Free: 256, Version: 1}}
+	require.NoError(t, merge(ring1, ring2))
+
+	// received update to entry with a token and version identical to an entry we have
+	// but but holding different content, if there are allocations, then reject the received
+	// entry
+	ring1 = NewRing(start, end, peer1name)
+	ring2 = NewRing(start, end, peer2name)
+	ring1.Entries = []*entry{{Token: start, Peer: peer1name, Free: 128, Version: 1}}
+	ring2.Entries = []*entry{{Token: start, Peer: peer2name, Free: 256, Version: 1}}
+	require.Error(t, merge(ring1, ring2), "Expected error")
+
+	// received an entry with update to one of our own tokens and with new version,
+	// accept the received entry if its still going to belong to us
+	ring1 = NewRing(start, end, peer1name)
+	ring2 = NewRing(start, end, peer2name)
+	ring1.Entries = []*entry{{Token: start, Peer: peer1name}}
+	ring2.Entries = []*entry{{Token: start, Peer: peer1name, Version: 1}}
+	require.NoError(t, merge(ring1, ring2))
+
+	// received an entry with update to one of our own tokens and with new version,
+	// but belongs to a different peer, accept the received entry if we do not
+	// have allocations in the range
+	ring1 = NewRing(start, end, peer1name)
+	ring2 = NewRing(start, end, peer2name)
+	ring1.Entries = []*entry{{Token: start, Peer: peer1name, Free: 256}}
+	ring2.Entries = []*entry{{Token: start, Peer: peer2name, Free: 256, Version: 1}}
+	require.NoError(t, merge(ring1, ring2))
+
+	// received an entry with update to one of our own tokens and with new version,
+	// reject received entry if we have allocations in the range
+	ring1 = NewRing(start, end, peer1name)
+	ring2 = NewRing(start, end, peer2name)
+	ring1.Entries = []*entry{{Token: start, Peer: peer1name, Free: 128}}
+	ring2.Entries = []*entry{{Token: start, Peer: peer2name, Version: 1}}
+	require.Error(t, merge(ring1, ring2), "Expected error")
+
+	// we receive an entry that splits one of our ranges, giving some of it
+	// away to another peer. accept the entry provided that we have no allocations
+	// in the range that got given away.
+	ring1 = NewRing(start, end, peer1name)
+	ring2 = NewRing(start, end, peer2name)
+	ring1.Entries = []*entry{{Token: start, Peer: peer1name, Free: 256, Version: 1}}
+	ring2.Entries = []*entry{{Token: start, Peer: peer1name, Free: 128, Version: 2}, {Token: middle, Peer: peer2name, Free: 128}}
+	require.NoError(t, merge(ring1, ring2))
+}
+
 func TestMergeErrors(t *testing.T) {
 	// Cannot Merge in an invalid ring
 	ring1 := NewRing(start, end, peer1name)
@@ -231,13 +306,6 @@ func TestMergeErrors(t *testing.T) {
 	ring2.Entries = []*entry{}
 	require.True(t, merge(ring1, ring2) == ErrDifferentRange, "Expected ErrDifferentRange")
 
-	// Cannot Merge newer version of entry I own
-	ring2 = NewRing(start, end, peer2name)
-	ring1.Entries = []*entry{{Token: start, Peer: peer1name}}
-	ring2.Entries = []*entry{{Token: start, Peer: peer1name, Version: 1}}
-	fmt.Println(merge(ring1, ring2))
-	require.Error(t, merge(ring1, ring2), "Expected error")
-
 	// Cannot Merge two entries with same version but different hosts
 	ring1.Entries = []*entry{{Token: start, Peer: peer1name}}
 	ring2.Entries = []*entry{{Token: start, Peer: peer2name}}

ipam/space/space.go

@@ -5,8 +5,8 @@ import (
 	"fmt"
 	"sort"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/net/address"
 )
 
 type Space struct {

ipam/space/space_test.go

@@ -3,9 +3,9 @@ package space
 import (
 	"testing"
 
+	"github.com/rajch/weave/net/address"
+	wt "github.com/rajch/weave/testing"
 	"github.com/stretchr/testify/require"
-	"github.com/weaveworks/weave/net/address"
-	wt "github.com/weaveworks/weave/testing"
 )
 
 func makeSpace(start address.Address, size address.Offset) *Space {
@@ -73,7 +73,7 @@ func TestLowlevel(t *testing.T) {
 
 	// test Donate when addresses are scarce
 	s = New()
-	r, ok = s.Donate(address.NewRange(0, 1000))
+	_, ok = s.Donate(address.NewRange(0, 1000))
 	require.True(t, !ok, "donate on empty space should fail")
 	s.Add(0, 3)
 	require.NoError(t, s.Claim(0))
@@ -81,7 +81,7 @@ func TestLowlevel(t *testing.T) {
 	r, ok = s.Donate(address.NewRange(0, 1000))
 	require.True(t, ok, "donate")
 	require.Equal(t, address.NewRange(1, 1), r, "donate")
-	r, ok = s.Donate(address.NewRange(0, 1000))
+	_, ok = s.Donate(address.NewRange(0, 1000))
 	require.True(t, !ok, "donate should fail")
 }
 

ipam/status.go

@@ -2,8 +2,9 @@ package ipam
 
 import (
 	"fmt"
-	"github.com/weaveworks/weave/ipam/paxos"
-	"github.com/weaveworks/weave/net/address"
+
+	"github.com/rajch/weave/ipam/paxos"
+	"github.com/rajch/weave/net/address"
 )
 
 type Status struct {

ipam/testutils_test.go

@@ -9,9 +9,9 @@ import (
 
 	"github.com/weaveworks/mesh"
 
+	"github.com/rajch/weave/net/address"
+	"github.com/rajch/weave/testing/gossip"
 	"github.com/stretchr/testify/require"
-	"github.com/weaveworks/weave/net/address"
-	"github.com/weaveworks/weave/testing/gossip"
 )
 
 type mockMessage struct {
@@ -24,14 +24,6 @@ func (m *mockMessage) String() string {
 	return fmt.Sprintf("-> %s [%x]", m.dst, m.buf)
 }
 
-func toStringArray(messages []mockMessage) []string {
-	out := make([]string, len(messages))
-	for i := range out {
-		out[i] = messages[i].String()
-	}
-	return out
-}
-
 type mockGossipComms struct {
 	sync.RWMutex
 	*testing.T
@@ -65,6 +57,9 @@ func (m *mockGossipComms) GossipBroadcast(update mesh.GossipData) {
 	}
 }
 
+func (m *mockGossipComms) GossipNeighbourSubset(update mesh.GossipData) {
+	m.GossipBroadcast(update)
+}
 func equalByteBuffer(a, b []byte) bool {
 	if len(a) != len(b) {
 		return false
@@ -97,14 +92,6 @@ func (m *mockGossipComms) GossipUnicast(dstPeerName mesh.PeerName, buf []byte) e
 	return nil
 }
 
-func ExpectMessage(alloc *Allocator, dst string, msgType byte, buf []byte) {
-	m := alloc.gossip.(*mockGossipComms)
-	dstPeerName, _ := mesh.PeerNameFromString(dst)
-	m.Lock()
-	m.messages = append(m.messages, mockMessage{dstPeerName, msgType, buf})
-	m.Unlock()
-}
-
 func ExpectBroadcastMessage(alloc *Allocator, buf []byte) {
 	m := alloc.gossip.(*mockGossipComms)
 	m.Lock()
@@ -205,15 +192,6 @@ func AssertNothingSent(t *testing.T, ch <-chan bool) {
 	}
 }
 
-func AssertNothingSentErr(t *testing.T, ch <-chan error) {
-	select {
-	case val := <-ch:
-		require.FailNow(t, fmt.Sprintf("Unexpected value on channel: %v", val))
-	default:
-		// no message received
-	}
-}
-
 func makeNetworkOfAllocators(size int, cidr string, preClaims ...[]PreClaim) ([]*Allocator, *gossip.TestRouter, address.CIDR) {
 	gossipRouter := gossip.NewTestRouter(0.0)
 	allocs := make([]*Allocator, size)

ipam/tracker/awsvpc.go

@@ -29,8 +29,8 @@ import (
 	"github.com/aws/aws-sdk-go/service/ec2"
 	"github.com/vishvananda/netlink"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/net/address"
 )
 
 type AWSVPCTracker struct {
@@ -84,7 +84,7 @@ func NewAWSVPCTracker(bridgeName string) (*AWSVPCTracker, error) {
 func (t *AWSVPCTracker) HandleUpdate(prevRanges, currRanges []address.Range, local bool) error {
 	t.debugf("replacing %q by %q; local(%t)", prevRanges, currRanges, local)
 
-	prev, curr := RemoveCommon(address.NewCIDRs(Merge(prevRanges)), address.NewCIDRs(Merge(currRanges)))
+	prev, curr := address.RemoveCommon(address.NewCIDRs(address.Merge(prevRanges)), address.NewCIDRs(address.Merge(currRanges)))
 
 	// It might make sense to do the removal first and then add entries
 	// because of the 50 routes limit. However, in such case a container might

ipam/tracker/helpers.go

@@ -1,46 +0,0 @@
-package tracker
-
-import (
-	"github.com/weaveworks/weave/net/address"
-)
-
-// Merge merges adjacent range entries.
-// The given slice has to be sorted in increasing order.
-func Merge(r []address.Range) []address.Range {
-	var merged []address.Range
-
-	for i := range r {
-		if prev := len(merged) - 1; prev >= 0 && merged[prev].End == r[i].Start {
-			merged[prev].End = r[i].End
-		} else {
-			merged = append(merged, r[i])
-		}
-	}
-
-	return merged
-}
-
-// RemoveCommon filters out CIDR ranges which are contained in both a and b slices.
-// Both slices have to be sorted in increasing order.
-func RemoveCommon(a, b []address.CIDR) (newA, newB []address.CIDR) {
-	i, j := 0, 0
-
-	for i < len(a) && j < len(b) {
-		switch {
-		case a[i].Start() < b[j].Start() || a[i].End() < b[j].End():
-			newA = append(newA, a[i])
-			i++
-		case a[i].Start() > b[j].Start() || a[i].End() > b[j].End():
-			newB = append(newB, b[j])
-			j++
-		default:
-			i++
-			j++
-		}
-
-	}
-	newA = append(newA, a[i:]...)
-	newB = append(newB, b[j:]...)
-
-	return
-}

ipam/tracker/helpers_test.go

@@ -1,52 +0,0 @@
-package tracker
-
-import (
-	"testing"
-
-	"github.com/stretchr/testify/require"
-
-	"github.com/weaveworks/weave/net/address"
-)
-
-var (
-	r0to127     = cidr("10.0.0.0", "10.0.0.127")
-	r128to255   = cidr("10.0.0.128", "10.0.0.255")
-	r0to255     = cidr("10.0.0.0", "10.0.0.255")
-	r1dot0to255 = cidr("10.0.1.0", "10.0.1.255")
-	r2dot0to255 = cidr("10.0.2.0", "10.0.2.255")
-)
-
-func TestRemoveCommon(t *testing.T) {
-	a := []address.CIDR{r0to127, r1dot0to255}
-	b := []address.CIDR{r1dot0to255, r2dot0to255}
-	newA, newB := RemoveCommon(a, b)
-	require.Equal(t, []address.CIDR{r0to127}, newA)
-	require.Equal(t, []address.CIDR{r2dot0to255}, newB)
-}
-
-func TestMerge(t *testing.T) {
-	ranges := []address.Range{
-		r0to127.Range(),
-		r128to255.Range(),
-		r2dot0to255.Range(),
-	}
-	require.Equal(t, []address.Range{r0to255.Range(), r2dot0to255.Range()}, Merge(ranges))
-}
-
-// Helper
-
-// TODO(mp) DRY with helpers of other tests.
-
-func ip(s string) address.Address {
-	addr, _ := address.ParseIP(s)
-	return addr
-}
-
-// [start; end]
-func cidr(start, end string) address.CIDR {
-	c := address.Range{Start: ip(start), End: ip(end) + 1}.CIDRs()
-	if len(c) != 1 {
-		panic("invalid cidr")
-	}
-	return c[0]
-}

ipam/tracker/tracker.go

@@ -1,7 +1,7 @@
 package tracker
 
 import (
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/net/address"
 )
 
 // LocalRangeTracker is an interface for tracking changes in the IPAM ring.

lb/addresses.go

@@ -0,0 +1,87 @@
+package lb
+
+import (
+	"fmt"
+	"net"
+	"sort"
+	"strings"
+
+	"github.com/vishvananda/netlink"
+	"golang.org/x/sys/unix"
+)
+
+// Local is what this node publishes about itself, read from its interfaces.
+type Local struct {
+	Addresses []net.IP
+	WeaveIP   net.IP
+	WeaveMAC  net.HardwareAddr
+	PodNet    *net.IPNet // the weave allocation range: the bridge address's network
+}
+
+// overlay- and container-side interfaces: their addresses are never held ones
+var overlayPrefixes = []string{"weave", "datapath", "vxlan-", "veth", "docker", "cni"}
+
+func overlay(name string) bool {
+	for _, p := range overlayPrefixes {
+		if strings.HasPrefix(name, p) {
+			return true
+		}
+	}
+	return false
+}
+
+// ReadLocal lists the global IPv4 addresses on host interfaces, and the weave bridge's address and MAC.
+func ReadLocal() (Local, error) {
+	var l Local
+	links, err := netlink.LinkList()
+	if err != nil {
+		return l, err
+	}
+	for _, link := range links {
+		name := link.Attrs().Name
+		addrs, err := netlink.AddrList(link, unix.AF_INET)
+		if err != nil {
+			return l, fmt.Errorf("%s: %v", name, err)
+		}
+		if name == bridge {
+			for _, a := range addrs {
+				if a.IPNet != nil && a.IP.To4() != nil {
+					l.WeaveIP = a.IP.To4()
+					l.WeaveMAC = link.Attrs().HardwareAddr
+					l.PodNet = &net.IPNet{IP: a.IP.Mask(a.Mask), Mask: a.Mask}
+					break
+				}
+			}
+			continue
+		}
+		if overlay(name) || link.Attrs().Flags&net.FlagLoopback != 0 {
+			continue
+		}
+		for _, a := range addrs {
+			if a.Scope == unix.RT_SCOPE_UNIVERSE && a.IP.To4() != nil && a.IP.IsGlobalUnicast() {
+				l.Addresses = append(l.Addresses, a.IP.To4())
+			}
+		}
+	}
+	// an address inside the pod network is never a host one (whatever interface it is on)
+	if l.PodNet != nil {
+		kept := l.Addresses[:0]
+		for _, ip := range l.Addresses {
+			if !l.PodNet.Contains(ip) {
+				kept = append(kept, ip)
+			}
+		}
+		l.Addresses = kept
+	}
+	sort.Slice(l.Addresses, func(i, j int) bool { return l.Addresses[i].String() < l.Addresses[j].String() })
+	return l, nil
+}
+
+// Annotations is what this node's Node object should carry for l.
+func (l Local) Annotations() map[string]string {
+	a := map[string]string{AnnAddresses: FormatAddresses(l.Addresses)}
+	if l.WeaveIP != nil && l.WeaveMAC != nil {
+		a[AnnWeave] = l.WeaveIP.String() + "," + l.WeaveMAC.String()
+	}
+	return a
+}

lb/admit/admit.go

@@ -0,0 +1,88 @@
+// Package admit is the protocol between the weave CNI plugin and weave-npc's egress agent: at CNI ADD the
+// plugin tells the agent which pod got which IP, and the agent answers once the pod's egress rules are in
+// place, so an EgressService member's first packet already leaves from the Service's address (or not at all).
+//
+// It depends on the standard library only: the plugin binary must not grow a Kubernetes client.
+package admit
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net"
+	"net/http"
+	"strings"
+	"time"
+)
+
+// SocketPath is where the agent listens (a hostPath directory shared with the host, where the plugin runs).
+const SocketPath = "/run/weave-npc/egress.sock"
+
+// Request is one pod's admission: the IP the plugin just allocated to it.
+type Request struct {
+	Namespace string `json:"namespace"`
+	Name      string `json:"name"`
+	IP        string `json:"ip"`
+}
+
+// Response says whether the pod is an EgressService member (its rules are installed either way when 200).
+type Response struct {
+	Member bool `json:"member"`
+}
+
+// Timeout bounds one admission (the agent may have to fetch the pod from the API server).
+const Timeout = 20 * time.Second
+
+// PodFromArgs reads the pod's namespace and name from CNI_ARGS ("K8S_POD_NAMESPACE=..;K8S_POD_NAME=..").
+func PodFromArgs(args string) (namespace, name string) {
+	for _, kv := range strings.Split(args, ";") {
+		k, v, ok := strings.Cut(kv, "=")
+		if !ok {
+			continue
+		}
+		switch k {
+		case "K8S_POD_NAMESPACE":
+			namespace = v
+		case "K8S_POD_NAME":
+			name = v
+		}
+	}
+	return namespace, name
+}
+
+// Admit asks the agent at socket to admit req, and returns only once the agent has installed the pod's rules.
+func Admit(ctx context.Context, socket string, req Request) (Response, error) {
+	var resp Response
+	body, err := json.Marshal(req)
+	if err != nil {
+		return resp, err
+	}
+	client := &http.Client{Transport: &http.Transport{
+		DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
+			var d net.Dialer
+			return d.DialContext(ctx, "unix", socket)
+		},
+	}}
+	ctx, cancel := context.WithTimeout(ctx, Timeout)
+	defer cancel()
+	hreq, err := http.NewRequestWithContext(ctx, http.MethodPost, "http://weave-npc/admit", bytes.NewReader(body))
+	if err != nil {
+		return resp, err
+	}
+	hreq.Header.Set("Content-Type", "application/json")
+	hresp, err := client.Do(hreq)
+	if err != nil {
+		return resp, fmt.Errorf("egress admission (weave-npc at %s): %v", socket, err)
+	}
+	defer hresp.Body.Close()
+	out, _ := io.ReadAll(io.LimitReader(hresp.Body, 1<<16))
+	if hresp.StatusCode != http.StatusOK {
+		return resp, fmt.Errorf("egress admission refused (%s): %s", hresp.Status, strings.TrimSpace(string(out)))
+	}
+	if err := json.Unmarshal(out, &resp); err != nil {
+		return resp, fmt.Errorf("egress admission: bad answer: %v", err)
+	}
+	return resp, nil
+}

lb/controller.go

@@ -0,0 +1,460 @@
+package lb
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"net"
+	"net/http"
+	"os"
+	"path/filepath"
+	"sync"
+	"time"
+
+	coreapi "k8s.io/api/core/v1"
+	discovery "k8s.io/api/discovery/v1"
+	apierrors "k8s.io/apimachinery/pkg/api/errors"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/fields"
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/types"
+	"k8s.io/client-go/kubernetes"
+	"k8s.io/client-go/rest"
+	"k8s.io/client-go/tools/cache"
+
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/lb/admit"
+)
+
+// Controller watches Services, EndpointSlices, Nodes, EgressServices and this node's Pods, and keeps this
+// node's part of the picture true. It also answers the CNI plugin's egress admissions.
+type Controller struct {
+	self   string
+	client kubernetes.Interface
+	egress rest.Interface
+	dp     *Datapath
+	// readLocal is the node's own interfaces (swappable in tests)
+	readLocal func() (Local, error)
+
+	services, slices, nodes, pods, egresses cache.Store
+	synced                                  []cache.InformerSynced
+	egressSynced                            cache.InformerSynced // nil while the EgressService API is absent
+	kick                                    chan struct{}
+	ready                                   chan struct{} // closed once the first pass ran
+
+	mu           sync.Mutex // one pass at a time (the loop, or an admission)
+	lastProblems map[string]bool
+	// Services whose status this node wrote since it started (so a restart rewrites ipMode once)
+	written map[types.UID]string
+	// admitted: IP -> pod the CNI plugin gave it; fetched: pods admitted before the pod informer had them
+	admitted map[string]Admission
+	fetched  map[string]*coreapi.Pod
+	at       map[string]time.Time
+	// lastEgress is what was last applied, reused while the EgressService caches are not synced (never wipe
+	// members' rules on a partial view)
+	lastEgress EgressDesired
+}
+
+// Resync is how often the node re-reads its interfaces and re-applies everything even without API events.
+const Resync = 30 * time.Second
+
+// NewController runs nothing yet; Run does. egress is the EgressService API client.
+func NewController(self string, client kubernetes.Interface, egress rest.Interface, dp *Datapath) *Controller {
+	return &Controller{self: self, client: client, egress: egress, dp: dp, readLocal: ReadLocal,
+		kick: make(chan struct{}, 1), ready: make(chan struct{}), lastProblems: map[string]bool{},
+		written: map[types.UID]string{}, admitted: map[string]Admission{}, fetched: map[string]*coreapi.Pod{},
+		at: map[string]time.Time{}}
+}
+
+func (c *Controller) poke() {
+	select {
+	case c.kick <- struct{}{}:
+	default:
+	}
+}
+
+func newInformer(c *Controller, lw cache.ListerWatcher, obj runtime.Object) (cache.Store, cache.Controller) {
+	return cache.NewInformer(lw, obj, 0, cache.ResourceEventHandlerFuncs{
+		AddFunc:    func(interface{}) { c.poke() },
+		UpdateFunc: func(interface{}, interface{}) { c.poke() },
+		DeleteFunc: func(interface{}) { c.poke() },
+	})
+}
+
+// egressAPI reports whether the EgressService API is served (the CRD is installed).
+func (c *Controller) egressAPI() (bool, error) {
+	_, err := c.client.Discovery().ServerResourcesForGroupVersion(GroupVersion.String())
+	if err == nil {
+		return true, nil
+	}
+	if apierrors.IsNotFound(err) {
+		return false, nil
+	}
+	return false, err
+}
+
+func (c *Controller) startEgress(ctx context.Context) {
+	var ctl cache.Controller
+	c.egresses, ctl = newInformer(c, cache.NewListWatchFromClient(c.egress, EgressServices, "", fields.Everything()), &EgressService{})
+	go ctl.Run(ctx.Done())
+	c.egressSynced = ctl.HasSynced
+}
+
+// Run blocks until ctx ends.
+func (c *Controller) Run(ctx context.Context) {
+	var sc, ec, nc, pc cache.Controller
+	c.services, sc = newInformer(c, cache.NewListWatchFromClient(c.client.CoreV1().RESTClient(), "services", "", fields.Everything()), &coreapi.Service{})
+	c.slices, ec = newInformer(c, cache.NewListWatchFromClient(c.client.DiscoveryV1().RESTClient(), "endpointslices", "", fields.Everything()), &discovery.EndpointSlice{})
+	c.nodes, nc = newInformer(c, cache.NewListWatchFromClient(c.client.CoreV1().RESTClient(), "nodes", "", fields.Everything()), &coreapi.Node{})
+	c.pods, pc = newInformer(c, cache.NewListWatchFromClient(c.client.CoreV1().RESTClient(), "pods", "",
+		fields.OneTermEqualSelector("spec.nodeName", c.self)), &coreapi.Pod{})
+	for _, ctl := range []cache.Controller{sc, ec, nc, pc} {
+		go ctl.Run(ctx.Done())
+		c.synced = append(c.synced, ctl.HasSynced)
+	}
+	// with the API present, nothing is applied before EgressServices are known: an empty view would wipe
+	// members' rules (their traffic would leave from the node); without it, there can be no members
+	for {
+		present, err := c.egressAPI()
+		if err == nil {
+			if present {
+				c.startEgress(ctx)
+				c.synced = append(c.synced, c.egressSynced)
+			}
+			break
+		}
+		common.Log.Warnf("lb: cannot tell whether the EgressService API is served (retrying): %v", err)
+		select {
+		case <-ctx.Done():
+			return
+		case <-time.After(5 * time.Second):
+		}
+	}
+	go c.serveAdmission(ctx)
+	if !cache.WaitForCacheSync(ctx.Done(), c.synced...) {
+		return
+	}
+	tick := time.NewTicker(Resync)
+	defer tick.Stop()
+	c.poke()
+	first := true
+	for {
+		select {
+		case <-ctx.Done():
+			return
+		case <-tick.C:
+			if c.egressSynced == nil {
+				// the API may have been installed since
+				if present, err := c.egressAPI(); err == nil && present {
+					c.mu.Lock()
+					c.startEgress(ctx)
+					c.mu.Unlock()
+				}
+			}
+		case <-c.kick:
+			// let a burst of events settle into one pass
+			time.Sleep(200 * time.Millisecond)
+			select {
+			case <-c.kick:
+			default:
+			}
+		}
+		c.mu.Lock()
+		err := c.sync(ctx)
+		c.mu.Unlock()
+		if err != nil {
+			common.Log.Errorf("lb: %v", err)
+		}
+		if first {
+			first = false
+			close(c.ready)
+		}
+	}
+}
+
+// egressViewSynced: the EgressService view can be trusted (absent API: trivially empty; else synced).
+func (c *Controller) egressViewSynced() bool {
+	return c.egressSynced == nil || c.egressSynced()
+}
+
+// sync is one pass: publish this node, compute, apply, write statuses. Callers hold c.mu.
+func (c *Controller) sync(ctx context.Context) error {
+	local, err := c.readLocal()
+	if err != nil {
+		return fmt.Errorf("reading interfaces: %v", err)
+	}
+	if err := c.publish(ctx, local); err != nil {
+		return err
+	}
+	if local.PodNet == nil {
+		return fmt.Errorf("weave bridge has no address yet; not forwarding")
+	}
+
+	var nodes []Node
+	var problems []string
+	for _, o := range c.nodes.List() {
+		n, err := ParseNode(o.(*coreapi.Node))
+		if err != nil {
+			problems = append(problems, err.Error())
+		}
+		nodes = append(nodes, n)
+	}
+	var svcs []*coreapi.Service
+	for _, o := range c.services.List() {
+		svcs = append(svcs, o.(*coreapi.Service))
+	}
+	var slices []*discovery.EndpointSlice
+	for _, o := range c.slices.List() {
+		slices = append(slices, o.(*discovery.EndpointSlice))
+	}
+	d := Compute(c.self, nodes, svcs, slices)
+	// rewrite once after start what an earlier run may have written without ipMode
+	for _, f := range d.Forwards {
+		if s := c.service(f.Namespace, f.Name); s != nil && c.written[s.UID] != f.Address.String() && !hasStatusFor(d, f) {
+			d.Status = append(d.Status, Status{Namespace: f.Namespace, Name: f.Name, Address: f.Address})
+		}
+	}
+
+	e := c.lastEgress
+	e.Status = nil // reusing the datapath, never the writes of an earlier view
+	if c.egressViewSynced() {
+		e = c.computeEgress(nodes, svcs)
+	} else {
+		problems = append(problems, "EgressServices not synced yet: keeping the egress rules last applied")
+	}
+
+	current := map[string]bool{}
+	for _, p := range append(append(problems, d.Problems...), e.Problems...) {
+		current[p] = true
+		if !c.lastProblems[p] {
+			common.Log.Warnf("lb: %s", p)
+		}
+	}
+	c.lastProblems = current
+
+	if err := c.dp.Apply(d, e, c.self, local); err != nil {
+		return fmt.Errorf("applying datapath: %v", err)
+	}
+	c.lastEgress = e
+	for _, st := range d.Status {
+		if err := c.writeStatus(ctx, st); err != nil {
+			return err
+		}
+	}
+	for _, st := range e.Status {
+		if err := PatchEgressStatus(ctx, c.egress, st.Namespace, st.Name, st.Status); err != nil {
+			return fmt.Errorf("status of egress %s/%s: %v", st.Namespace, st.Name, err)
+		}
+		common.Log.Infof("lb: egress %s/%s: host %q, address %q, %d port blocks", st.Namespace, st.Name, st.Status.Host, st.Status.Address, len(st.Status.Blocks))
+	}
+	return nil
+}
+
+// computeEgress gathers this node's pods (informer, plus pods admitted before the informer had them) and computes.
+func (c *Controller) computeEgress(nodes []Node, svcs []*coreapi.Service) EgressDesired {
+	var egresses []*EgressService
+	if c.egresses != nil {
+		for _, o := range c.egresses.List() {
+			egresses = append(egresses, o.(*EgressService))
+		}
+	}
+	seen := map[string]bool{}
+	var pods []*coreapi.Pod
+	for _, o := range c.pods.List() {
+		p := o.(*coreapi.Pod)
+		seen[p.Namespace+"/"+p.Name] = true
+		pods = append(pods, p)
+	}
+	// forget admissions the API has caught up with or that are long gone
+	for ip, a := range c.admitted {
+		key := a.Namespace + "/" + a.Name
+		o, ok, _ := c.pods.GetByKey(key)
+		caughtUp := ok && o.(*coreapi.Pod).Status.PodIP == ip
+		if caughtUp || (!ok && time.Since(c.at[ip]) > 2*time.Minute) {
+			delete(c.admitted, ip)
+			delete(c.at, ip)
+		}
+	}
+	for key, p := range c.fetched {
+		stillAdmitted := false
+		for _, a := range c.admitted {
+			if a.Namespace+"/"+a.Name == key {
+				stillAdmitted = true
+			}
+		}
+		if !stillAdmitted {
+			delete(c.fetched, key)
+			continue
+		}
+		if !seen[key] {
+			pods = append(pods, p)
+		}
+	}
+	return ComputeEgress(c.self, nodes, svcs, egresses, pods, c.admitted)
+}
+
+func hasStatusFor(d Desired, f Forward) bool {
+	for _, s := range d.Status {
+		if s.Namespace == f.Namespace && s.Name == f.Name {
+			return true
+		}
+	}
+	return false
+}
+
+func (c *Controller) service(ns, name string) *coreapi.Service {
+	o, ok, err := c.services.GetByKey(ns + "/" + name)
+	if err != nil || !ok {
+		return nil
+	}
+	return o.(*coreapi.Service)
+}
+
+// serveAdmission answers the CNI plugin on admit.SocketPath: record the pod's IP, run a pass, answer once the
+// pod's rules are in place. A pod is refused (the plugin fails the ADD, kubelet retries) while this node cannot
+// yet tell whether it is a member.
+func (c *Controller) serveAdmission(ctx context.Context) {
+	if err := os.MkdirAll(filepath.Dir(admit.SocketPath), 0o700); err != nil {
+		common.Log.Errorf("lb: admission socket: %v", err)
+		return
+	}
+	_ = os.Remove(admit.SocketPath)
+	l, err := net.Listen("unix", admit.SocketPath)
+	if err != nil {
+		common.Log.Errorf("lb: admission socket: %v", err)
+		return
+	}
+	_ = os.Chmod(admit.SocketPath, 0o600)
+	mux := http.NewServeMux()
+	mux.HandleFunc("/admit", func(w http.ResponseWriter, r *http.Request) {
+		var req admit.Request
+		if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<16)).Decode(&req); err != nil {
+			http.Error(w, "bad request: "+err.Error(), http.StatusBadRequest)
+			return
+		}
+		member, code, err := c.admit(r.Context(), req)
+		if err != nil {
+			common.Log.Warnf("lb: admission of %s/%s (%s): %v", req.Namespace, req.Name, req.IP, err)
+			http.Error(w, err.Error(), code)
+			return
+		}
+		_ = json.NewEncoder(w).Encode(admit.Response{Member: member})
+	})
+	srv := &http.Server{Handler: mux, ReadHeaderTimeout: 5 * time.Second}
+	go func() {
+		<-ctx.Done()
+		_ = srv.Close()
+	}()
+	if err := srv.Serve(l); err != nil && err != http.ErrServerClosed {
+		common.Log.Errorf("lb: admission server: %v", err)
+	}
+}
+
+func (c *Controller) admit(ctx context.Context, req admit.Request) (bool, int, error) {
+	ip := net.ParseIP(req.IP).To4()
+	if req.Namespace == "" || req.Name == "" || ip == nil {
+		return false, http.StatusBadRequest, fmt.Errorf("need namespace, name and an IPv4 address")
+	}
+	select {
+	case <-c.ready:
+	case <-ctx.Done():
+		return false, http.StatusServiceUnavailable, ctx.Err()
+	case <-time.After(admit.Timeout - 5*time.Second):
+		return false, http.StatusServiceUnavailable, fmt.Errorf("not ready yet")
+	}
+	key := req.Namespace + "/" + req.Name
+	var pod *coreapi.Pod
+	if o, ok, _ := c.pods.GetByKey(key); ok {
+		pod = o.(*coreapi.Pod)
+	} else {
+		gctx, cancel := context.WithTimeout(ctx, 10*time.Second)
+		p, err := c.client.CoreV1().Pods(req.Namespace).Get(gctx, req.Name, metav1.GetOptions{})
+		cancel()
+		if err != nil {
+			return false, http.StatusServiceUnavailable, fmt.Errorf("fetching pod: %v", err)
+		}
+		pod = p
+	}
+	if pod.Spec.NodeName != c.self {
+		return false, http.StatusConflict, fmt.Errorf("pod is bound to %q, not this node", pod.Spec.NodeName)
+	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
+	if !c.egressViewSynced() {
+		return false, http.StatusServiceUnavailable, fmt.Errorf("EgressServices not synced yet")
+	}
+	c.admitted[ip.String()] = Admission{Namespace: req.Namespace, Name: req.Name}
+	c.at[ip.String()] = time.Now()
+	if _, ok, _ := c.pods.GetByKey(key); !ok {
+		c.fetched[key] = pod
+	}
+	if err := c.sync(ctx); err != nil {
+		// with no EgressService and no member rules in place there is nothing to protect or to clean up for
+		// this IP: a broken pass must not stop every pod on the node from starting
+		if len(c.lastEgress.Members) == 0 && (c.egresses == nil || len(c.egresses.List()) == 0) {
+			common.Log.Warnf("lb: admitting %s/%s despite a failed pass (no EgressServices): %v", req.Namespace, req.Name, err)
+			return false, http.StatusOK, nil
+		}
+		return false, http.StatusServiceUnavailable, err
+	}
+	for _, m := range c.lastEgress.Members {
+		if m.IP.Equal(ip) {
+			return true, http.StatusOK, nil
+		}
+	}
+	return false, http.StatusOK, nil
+}
+
+// publish keeps this node's annotations equal to what its interfaces say.
+func (c *Controller) publish(ctx context.Context, local Local) error {
+	want := local.Annotations()
+	o, ok, err := c.nodes.GetByKey(c.self)
+	if err != nil {
+		return err
+	}
+	if ok {
+		have := o.(*coreapi.Node).Annotations
+		same := true
+		for k, v := range want {
+			if have[k] != v {
+				same = false
+			}
+		}
+		if same {
+			return nil
+		}
+	}
+	patch, err := json.Marshal(map[string]interface{}{"metadata": map[string]interface{}{"annotations": want}})
+	if err != nil {
+		return err
+	}
+	if _, err := c.client.CoreV1().Nodes().Patch(ctx, c.self, types.MergePatchType, patch, metav1.PatchOptions{}); err != nil {
+		return fmt.Errorf("publishing node annotations: %v", err)
+	}
+	common.Log.Infof("lb: node %s holds [%s]", c.self, want[AnnAddresses])
+	return nil
+}
+
+// writeStatus sets (ipMode Proxy: kube-proxy leaves the address to us) or clears a Service's load-balancer status.
+func (c *Controller) writeStatus(ctx context.Context, st Status) error {
+	ingress := []map[string]string{}
+	if st.Address != nil {
+		ingress = append(ingress, map[string]string{"ip": st.Address.String(), "ipMode": "Proxy"})
+	}
+	patch, err := json.Marshal(map[string]interface{}{"status": map[string]interface{}{"loadBalancer": map[string]interface{}{"ingress": ingress}}})
+	if err != nil {
+		return err
+	}
+	s, err := c.client.CoreV1().Services(st.Namespace).Patch(ctx, st.Name, types.MergePatchType, patch, metav1.PatchOptions{}, "status")
+	if err != nil {
+		return fmt.Errorf("status of %s/%s: %v", st.Namespace, st.Name, err)
+	}
+	if st.Address != nil {
+		c.written[s.UID] = st.Address.String()
+		common.Log.Infof("lb: %s/%s at %s", st.Namespace, st.Name, st.Address)
+	} else {
+		delete(c.written, s.UID)
+		common.Log.Infof("lb: %s/%s has no address", st.Namespace, st.Name)
+	}
+	return nil
+}

lb/datapath.go

@@ -0,0 +1,336 @@
+package lb
+
+import (
+	"bytes"
+	"crypto/sha256"
+	"encoding/hex"
+	"fmt"
+	"net"
+	"os/exec"
+	"strings"
+
+	"github.com/coreos/go-iptables/iptables"
+	"github.com/vishvananda/netlink"
+	"golang.org/x/sys/unix"
+)
+
+const (
+	// ForwardChain (nat, from PREROUTING) DNATs held addresses' ports to endpoints.
+	ForwardChain = "REWEAVE-LB"
+	// PostChain (nat, from POSTROUTING, ahead of weave's WEAVE) keeps the client's address on what ForwardChain DNATed.
+	PostChain = "REWEAVE-LB-POST"
+	// MarkChain (mangle, from PREROUTING) marks holders' connections and steers their replies.
+	MarkChain = "REWEAVE-LB-MARK"
+	// servicePrefix names the per-port chains ForwardChain jumps to.
+	servicePrefix = "REWEAVE-LBS-"
+
+	markShift = 24
+	markMask  = uint32(0xff) << markShift
+	// RulePriority is where the reply-steering rules sit: before the main table (32766).
+	RulePriority = 9000
+	// TableBase + mark is the routing table that sends replies to that mark's holder.
+	TableBase = 52000
+
+	bridge = "weave"
+)
+
+func markValue(m uint32) uint32 { return m << markShift }
+
+// portChain is the per-(service, protocol, port) chain name: stable, short enough for iptables (28 chars).
+func portChain(f Forward, p Port) string {
+	h := sha256.Sum256([]byte(fmt.Sprintf("%s/%s/%s/%d", f.Namespace, f.Name, p.Protocol, p.Port)))
+	return servicePrefix + strings.ToUpper(hex.EncodeToString(h[:])[:12])
+}
+
+// Rules is the full content of the chains this package owns, in iptables-restore form.
+type Rules struct {
+	Nat, Mangle, Filter string
+}
+
+// Render turns the desired state (load-balancer forwards and egress) into iptables-restore input for the nat,
+// mangle and filter tables. podNet is the weave allocation range (pods and bridges).
+// Each payload declares every chain it owns, so a restore with --noflush replaces exactly those.
+func Render(d Desired, e EgressDesired, self string, podNet *net.IPNet, stale []string) Rules {
+	var nat bytes.Buffer
+	nat.WriteString("*nat\n")
+	for _, c := range []string{ForwardChain, PostChain, ClassChain} {
+		fmt.Fprintf(&nat, ":%s - [0:0]\n", c)
+	}
+	var body bytes.Buffer
+	seen := map[string]bool{}
+	for _, f := range d.Forwards {
+		a := f.Address.String()
+		if seen[a] {
+			continue
+		}
+		seen[a] = true
+		// weave masquerades everything entering the pod network from outside unless it is for a local pod
+		// (its expose rule); a connection we DNATed must keep the client's address, so it leaves the nat
+		// table here. One from a pod (hairpin) is masqueraded to this node, so its reply comes back here.
+		fmt.Fprintf(&body, "-A %s -m conntrack --ctstate DNAT --ctorigdst %s/32 -s %s -j MASQUERADE\n", PostChain, a, podNet)
+		fmt.Fprintf(&body, "-A %s -m conntrack --ctstate DNAT --ctorigdst %s/32 -j ACCEPT\n", PostChain, a)
+	}
+	for _, f := range d.Forwards {
+		for _, p := range f.Ports {
+			c := portChain(f, p)
+			fmt.Fprintf(&nat, ":%s - [0:0]\n", c)
+			proto := strings.ToLower(string(p.Protocol))
+			fmt.Fprintf(&body, "-A %s -d %s/32 -p %s -m %s --dport %d -m comment --comment %q -j %s\n",
+				ForwardChain, f.Address, proto, proto, p.Port, f.Namespace+"/"+f.Name, c)
+			n := len(p.Endpoints)
+			for i, e := range p.Endpoints {
+				// DNAT to an ip:port needs the protocol in the same rule
+				if i < n-1 {
+					// the i-th endpoint takes 1/(n-i) of what the earlier ones left: uniform overall
+					fmt.Fprintf(&body, "-A %s -p %s -m statistic --mode random --probability %.10f -j DNAT --to-destination %s\n",
+						c, proto, 1/float64(n-i), e)
+				} else {
+					fmt.Fprintf(&body, "-A %s -p %s -j DNAT --to-destination %s\n", c, proto, e)
+				}
+			}
+		}
+	}
+	for _, c := range stale {
+		// chains of ports no longer forwarded: emptied here, deleted after the restore
+		fmt.Fprintf(&nat, ":%s - [0:0]\n", c)
+	}
+
+	var mangle bytes.Buffer
+	mangle.WriteString("*mangle\n")
+	for _, c := range []string{MarkChain, MSSChain} {
+		fmt.Fprintf(&mangle, ":%s - [0:0]\n", c)
+	}
+	if len(d.Holders) > 0 {
+		// replies of a holder's connection carry its mark: the policy rule routes them back to it
+		fmt.Fprintf(&mangle, "-A %s -m conntrack --ctdir REPLY -m connmark ! --mark 0x0/0x%x -j CONNMARK --restore-mark --nfmask 0x%x --ctmask 0x%x\n",
+			MarkChain, markMask, markMask, markMask)
+		for _, h := range d.Holders {
+			// a new connection to a pod here, arriving from the holder's bridge, from outside the pod network:
+			// the holder DNATed it and must see its replies
+			fmt.Fprintf(&mangle, "-A %s -i %s -m mac --mac-source %s ! -s %s -d %s -m conntrack --ctstate NEW -m comment --comment %q -j CONNMARK --set-xmark 0x%x/0x%x\n",
+				MarkChain, bridge, h.Node.WeaveMAC, podNet, podNet, "holder "+h.Node.Name, markValue(h.Mark), markMask)
+		}
+	}
+
+	var filter bytes.Buffer
+	filter.WriteString("*filter\n")
+	for _, c := range []string{FwdChain, InChain} {
+		fmt.Fprintf(&filter, ":%s - [0:0]\n", c)
+	}
+	renderEgress(e, self, podNet, &body, &mangle, &filter)
+
+	nat.Write(body.Bytes())
+	nat.WriteString("COMMIT\n")
+	mangle.WriteString("COMMIT\n")
+	filter.WriteString("COMMIT\n")
+	return Rules{Nat: nat.String(), Mangle: mangle.String(), Filter: filter.String()}
+}
+
+// Datapath applies Desired to this node.
+type Datapath struct {
+	ipt *iptables.IPTables
+	// restore runs iptables-restore --noflush with the given input (swappable in tests)
+	restore func(input string) error
+}
+
+// NewDatapath uses the host's iptables (whichever backend the image's launch script selected).
+func NewDatapath(ipt *iptables.IPTables) *Datapath {
+	return &Datapath{ipt: ipt, restore: func(input string) error {
+		cmd := exec.Command("iptables-restore", "--noflush", "-w", "5")
+		cmd.Stdin = strings.NewReader(input)
+		if out, err := cmd.CombinedOutput(); err != nil {
+			return fmt.Errorf("iptables-restore: %v: %s", err, out)
+		}
+		return nil
+	}}
+}
+
+// Apply makes the node's iptables chains, tunnel, policy rules and routes exactly what d and e say.
+func (dp *Datapath) Apply(d Desired, e EgressDesired, self string, local Local) error {
+	podNet := local.PodNet
+	if err := dp.hooks(); err != nil {
+		return err
+	}
+	// the tunnels must exist before the routes into them (iptables rules may name a device that does not exist yet)
+	var peers []Node
+	if e.Tunnelled(self) {
+		peers = e.Peers(self)
+	}
+	if err := ensureTunnels(peers, local); err != nil {
+		return err
+	}
+	existing, err := dp.ipt.ListChains("nat")
+	if err != nil {
+		return err
+	}
+	want := map[string]bool{}
+	for _, f := range d.Forwards {
+		for _, p := range f.Ports {
+			want[portChain(f, p)] = true
+		}
+	}
+	var stale []string
+	for _, c := range existing {
+		if strings.HasPrefix(c, servicePrefix) && !want[c] {
+			stale = append(stale, c)
+		}
+	}
+	r := Render(d, e, self, podNet, stale)
+	// filter first: the guards are in place before the marks and SNATs that they guard
+	for _, payload := range []string{r.Filter, r.Nat, r.Mangle} {
+		if err := dp.restore(payload); err != nil {
+			return err
+		}
+	}
+	for _, c := range stale {
+		if err := dp.ipt.DeleteChain("nat", c); err != nil {
+			return fmt.Errorf("deleting stale chain %s: %v", c, err)
+		}
+	}
+	if err := dp.routes(d.Holders); err != nil {
+		return err
+	}
+	return egressNet(e, self)
+}
+
+// hooks makes sure the built-in chains jump to ours: first (kube-proxy's and weave's rules must not see these
+// connections first), except ClassChain, which must come after kube-proxy's KUBE-SERVICES (whose DNAT ends the
+// nat walk, so a connection to a Service is never classed as going outside).
+func (dp *Datapath) hooks() error {
+	for _, h := range []struct {
+		table, from, chain string
+		last               bool
+	}{
+		{"nat", "PREROUTING", ForwardChain, false},
+		{"nat", "POSTROUTING", PostChain, false},
+		{"nat", "PREROUTING", ClassChain, true},
+		{"mangle", "PREROUTING", MarkChain, false},
+		{"mangle", "FORWARD", MSSChain, false},
+		{"filter", "FORWARD", FwdChain, false},
+		{"filter", "INPUT", InChain, false},
+	} {
+		ok, err := dp.ipt.ChainExists(h.table, h.chain)
+		if err != nil {
+			return err
+		}
+		if !ok {
+			if err := dp.ipt.NewChain(h.table, h.chain); err != nil {
+				return err
+			}
+		}
+		rule := []string{"-m", "comment", "--comment", "reweave lb", "-j", h.chain}
+		rules, err := dp.ipt.List(h.table, h.from)
+		if err != nil {
+			return err
+		}
+		if h.last {
+			ours, kube := -1, -1
+			for i, r := range rules {
+				if strings.HasSuffix(r, "-j "+h.chain) {
+					ours = i
+				}
+				if strings.HasSuffix(r, "-j KUBE-SERVICES") {
+					kube = i
+				}
+			}
+			if ours > kube {
+				continue
+			}
+			if err := dp.ipt.DeleteIfExists(h.table, h.from, rule...); err != nil {
+				return err
+			}
+			if err := dp.ipt.Append(h.table, h.from, rule...); err != nil {
+				return err
+			}
+			continue
+		}
+		// rules[0] is the policy line; our jump must be rules[1]
+		if len(rules) > 1 && strings.HasSuffix(rules[1], "-j "+h.chain) {
+			continue
+		}
+		if err := dp.ipt.DeleteIfExists(h.table, h.from, rule...); err != nil {
+			return err
+		}
+		if err := dp.ipt.Insert(h.table, h.from, 1, rule...); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+// routes keeps one policy rule and one table per holder: marked replies go to the holder's bridge.
+func (dp *Datapath) routes(holders []Holder) error {
+	link, err := netlink.LinkByName(bridge)
+	if err != nil {
+		if len(holders) == 0 {
+			return nil
+		}
+		return fmt.Errorf("weave bridge: %v", err)
+	}
+	want := map[int]Holder{}
+	for _, h := range holders {
+		want[TableBase+int(h.Mark)] = h
+	}
+	rules, err := netlink.RuleList(unix.AF_INET)
+	if err != nil {
+		return err
+	}
+	have := map[int]bool{}
+	for _, r := range rules {
+		if r.Priority != RulePriority || r.Table < TableBase || r.Table > TableBase+MaxMarks {
+			continue
+		}
+		h, ok := want[r.Table]
+		if ok && uint32(r.Mark) == markValue(h.Mark) && uint32(r.Mask) == markMask {
+			have[r.Table] = true
+			continue
+		}
+		r := r
+		if err := netlink.RuleDel(&r); err != nil {
+			return fmt.Errorf("removing stale rule to table %d: %v", r.Table, err)
+		}
+	}
+	existing, err := routesByTable(TableBase+1, TableBase+MaxMarks)
+	if err != nil {
+		return err
+	}
+	for table := TableBase + 1; table <= TableBase+MaxMarks; table++ {
+		h, ok := want[table]
+		if !ok {
+			for _, rt := range existing[table] {
+				if err := delRoute(rt); err != nil {
+					return fmt.Errorf("removing stale route in table %d: %v", table, err)
+				}
+			}
+			continue
+		}
+		if err := netlink.RouteReplace(&netlink.Route{
+			LinkIndex: link.Attrs().Index,
+			Dst:       &net.IPNet{IP: net.IPv4zero, Mask: net.CIDRMask(0, 32)},
+			Gw:        h.Node.WeaveIP,
+			Table:     table,
+			Flags:     int(netlink.FLAG_ONLINK),
+		}); err != nil {
+			return fmt.Errorf("route to holder %s: %v", h.Node.Name, err)
+		}
+		if !have[table] {
+			rule := netlink.NewRule()
+			rule.Priority = RulePriority
+			rule.Table = table
+			rule.Mark = int(markValue(h.Mark))
+			rule.Mask = int(markMask)
+			if err := netlink.RuleAdd(rule); err != nil {
+				return fmt.Errorf("rule for holder %s: %v", h.Node.Name, err)
+			}
+		}
+	}
+	return nil
+}
+
+// delRoute deletes a route as RouteList returned it: a default route lists without Dst, which RouteDel refuses.
+func delRoute(rt netlink.Route) error {
+	if rt.Dst == nil {
+		rt.Dst = &net.IPNet{IP: net.IPv4zero, Mask: net.CIDRMask(0, 32)}
+	}
+	return netlink.RouteDel(&rt)
+}

lb/egress.go

@@ -0,0 +1,340 @@
+package lb
+
+import (
+	"fmt"
+	"net"
+	"sort"
+	"strings"
+
+	coreapi "k8s.io/api/core/v1"
+)
+
+// Port blocks: every node gets its own source ports on an EgressService's address, so the holder can route a
+// reply back to the member's node by port alone (no state on the holder). The range lies above the default
+// ephemeral range (32768-60999), so it never meets the holder's own connections from the address.
+const (
+	BlockBase  = 61000
+	BlockSize  = 512
+	BlockCount = 8 // 61000-65095
+)
+
+// Admission is a pod the CNI plugin announced at ADD, before the API shows its IP.
+type Admission struct {
+	Namespace, Name string
+}
+
+// Member is one pod on this node that an EgressService selects, and where its outside traffic goes.
+type Member struct {
+	IP      net.IP
+	Pod     string // namespace/name
+	Service string // namespace/name of its EgressService (empty if it is in several)
+	// Unreachable: its outside traffic is dropped (no holder, no block, several EgressServices, ...)
+	Unreachable bool
+	Address     net.IP // it leaves from Address:[From, To]
+	From, To    int32
+	Holder      Node // the node that emits it; Holder.Name == self: emitted here
+}
+
+// BlockRoute is, on the holder, the replies to Address:[From, To] going back to Peer.
+type BlockRoute struct {
+	Address  net.IP
+	From, To int32
+	Peer     Node
+}
+
+// EgressStatusWrite is a status this node writes (the holder, or the first node by name to clear it).
+type EgressStatusWrite struct {
+	Namespace, Name string
+	Status          EgressServiceStatus
+}
+
+// EgressDesired is this node's part of every EgressService.
+type EgressDesired struct {
+	Members   []Member
+	Routes    []BlockRoute
+	Status    []EgressStatusWrite
+	NodeAddrs []net.IP // the cluster's node addresses: members' traffic to them is not "outside"
+	Problems  []string
+}
+
+// Tunnelled reports whether this node sends or receives anything over the egress tunnel.
+func (e EgressDesired) Tunnelled(self string) bool {
+	if len(e.Routes) > 0 {
+		return true
+	}
+	for _, m := range e.Members {
+		if !m.Unreachable && m.Holder.Name != self {
+			return true
+		}
+	}
+	return false
+}
+
+// Peers are the nodes this node exchanges tunnelled packets with (sorted by name, deduplicated).
+func (e EgressDesired) Peers(self string) []Node {
+	seen := map[string]Node{}
+	for _, r := range e.Routes {
+		seen[r.Peer.Name] = r.Peer
+	}
+	for _, m := range e.Members {
+		if !m.Unreachable && m.Holder.Name != self {
+			seen[m.Holder.Name] = m.Holder
+		}
+	}
+	out := make([]Node, 0, len(seen))
+	for _, n := range seen {
+		out = append(out, n)
+	}
+	sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
+	return out
+}
+
+// AllocateBlocks keeps every valid block a node already has and gives each node without one the lowest free
+// block, in name order. A node beyond BlockCount gets none (its members are dropped, and that is reported).
+func AllocateBlocks(current []PortBlock, nodeNames []string) []PortBlock {
+	present := map[string]bool{}
+	for _, n := range nodeNames {
+		present[n] = true
+	}
+	used := map[int]bool{}
+	have := map[string]PortBlock{}
+	for _, b := range current {
+		k := int(b.From-BlockBase) / BlockSize
+		valid := present[b.Node] && b.From >= BlockBase && int(b.From-BlockBase)%BlockSize == 0 &&
+			k < BlockCount && b.To == b.From+BlockSize-1
+		if !valid || used[k] {
+			continue
+		}
+		if _, dup := have[b.Node]; dup {
+			continue
+		}
+		used[k] = true
+		have[b.Node] = b
+	}
+	names := append([]string(nil), nodeNames...)
+	sort.Strings(names)
+	for _, n := range names {
+		if _, ok := have[n]; ok {
+			continue
+		}
+		for k := 0; k < BlockCount; k++ {
+			if !used[k] {
+				used[k] = true
+				from := int32(BlockBase + k*BlockSize)
+				have[n] = PortBlock{Node: n, From: from, To: from + BlockSize - 1}
+				break
+			}
+		}
+	}
+	out := make([]PortBlock, 0, len(have))
+	for _, b := range have {
+		out = append(out, b)
+	}
+	sort.Slice(out, func(i, j int) bool { return out[i].Node < out[j].Node })
+	return out
+}
+
+func sameStatus(a, b EgressServiceStatus) bool {
+	if a.Host != b.Host || a.Address != b.Address || len(a.Blocks) != len(b.Blocks) {
+		return false
+	}
+	for i := range a.Blocks {
+		if a.Blocks[i] != b.Blocks[i] {
+			return false
+		}
+	}
+	return true
+}
+
+// holderOf is S1's rule for a Service's address: the one node that holds spec.loadBalancerIP.
+func holderOf(s *coreapi.Service, nodes []Node) (*Node, net.IP) {
+	want := net.ParseIP(s.Spec.LoadBalancerIP).To4()
+	if want == nil {
+		return nil, nil
+	}
+	var hs []Node
+	for _, n := range nodes {
+		if n.Holds(want) {
+			hs = append(hs, n)
+		}
+	}
+	if len(hs) != 1 {
+		return nil, want
+	}
+	return &hs[0], want
+}
+
+// selects reports whether a Service's selector (non-empty) matches the labels.
+func selects(s *coreapi.Service, labels map[string]string) bool {
+	if len(s.Spec.Selector) == 0 {
+		return false
+	}
+	for k, v := range s.Spec.Selector {
+		if labels[k] != v {
+			return false
+		}
+	}
+	return true
+}
+
+// ComputeEgress decides this node's part of every EgressService: the statuses it writes, the members it hosts and
+// where their traffic goes, and (as a holder) where replies to each node's port block go.
+// pods are this node's pods; admitted maps an IP to the pod the CNI plugin gave it (newer than the API may show).
+func ComputeEgress(self string, nodes []Node, services []*coreapi.Service, egresses []*EgressService,
+	pods []*coreapi.Pod, admitted map[string]Admission) EgressDesired {
+	var d EgressDesired
+	sorted := append([]Node(nil), nodes...)
+	sort.Slice(sorted, func(i, j int) bool { return sorted[i].Name < sorted[j].Name })
+	byName := map[string]Node{}
+	var names []string
+	for _, n := range sorted {
+		byName[n.Name] = n
+		names = append(names, n.Name)
+		d.NodeAddrs = append(d.NodeAddrs, n.Addresses...)
+	}
+	svcs := map[string]*coreapi.Service{}
+	for _, s := range services {
+		svcs[s.Namespace+"/"+s.Name] = s
+	}
+	es := append([]*EgressService(nil), egresses...)
+	sort.Slice(es, func(i, j int) bool { return es[i].Namespace+"/"+es[i].Name < es[j].Namespace+"/"+es[j].Name })
+
+	// valid[key]: the EgressService is in force: its Service is served from an address one node holds, and the
+	// holder has published that (status) - what members and holder both act on
+	valid := map[string]bool{}
+	for _, e := range es {
+		key := e.Namespace + "/" + e.Name
+		var want EgressServiceStatus
+		var holder *Node
+		var addr net.IP
+		s := svcs[key]
+		switch {
+		case e.Spec.Emit != EmitHolder:
+			d.Problems = append(d.Problems, fmt.Sprintf("egress %s: spec.emit must be %q (the only mode so far), got %q", key, EmitHolder, e.Spec.Emit))
+		case e.Spec.SourceIPBy != "" && e.Spec.SourceIPBy != SourceIPByLoadBalancerIP:
+			d.Problems = append(d.Problems, fmt.Sprintf("egress %s: spec.sourceIPBy must be %q, got %q", key, SourceIPByLoadBalancerIP, e.Spec.SourceIPBy))
+		case s == nil:
+			d.Problems = append(d.Problems, fmt.Sprintf("egress %s: no Service %s", key, key))
+		case !eligible(s):
+			d.Problems = append(d.Problems, fmt.Sprintf("egress %s: Service %s is not a LoadBalancer of class %s", key, key, Class))
+		default:
+			holder, addr = holderOf(s, sorted)
+			if holder == nil {
+				d.Problems = append(d.Problems, fmt.Sprintf("egress %s: no single node holds %s; members' outside traffic is dropped", key, s.Spec.LoadBalancerIP))
+			}
+		}
+		writer := ""
+		if holder != nil {
+			want = EgressServiceStatus{Host: holder.Name, Address: addr.String(), Blocks: AllocateBlocks(e.Status.Blocks, names)}
+			writer = holder.Name
+			if len(want.Blocks) < len(names) {
+				d.Problems = append(d.Problems, fmt.Sprintf("egress %s: only %d port blocks for %d nodes; members on nodes without one are dropped", key, BlockCount, len(names)))
+			}
+		} else if len(names) > 0 {
+			writer = names[0]
+		}
+		if writer == self && !sameStatus(e.Status, want) {
+			d.Status = append(d.Status, EgressStatusWrite{Namespace: e.Namespace, Name: e.Name, Status: want})
+		}
+		// in force only once the status in the API says what holds now (members and holder read the same thing)
+		if holder != nil && e.Status.Host == holder.Name && e.Status.Address == addr.String() {
+			valid[key] = true
+			if holder.Name == self {
+				for _, b := range e.Status.Blocks {
+					if b.Node == self {
+						continue
+					}
+					peer, ok := byName[b.Node]
+					if !ok || peer.WeaveIP == nil {
+						d.Problems = append(d.Problems, fmt.Sprintf("egress %s: node %s has no weave bridge published; its members' replies cannot reach it", key, b.Node))
+						continue
+					}
+					d.Routes = append(d.Routes, BlockRoute{Address: addr, From: b.From, To: b.To, Peer: peer})
+				}
+			}
+		}
+	}
+
+	// this node's pods, one per IP: the CNI plugin's latest word wins, then live pods over terminating ones
+	type candidate struct {
+		pod *coreapi.Pod
+		ip  string
+	}
+	byIP := map[string]*coreapi.Pod{}
+	podByKey := map[string]*coreapi.Pod{}
+	for _, p := range pods {
+		podByKey[p.Namespace+"/"+p.Name] = p
+	}
+	var cands []candidate
+	for _, p := range pods {
+		if p.Spec.NodeName != self || p.Spec.HostNetwork || p.Status.Phase == coreapi.PodSucceeded || p.Status.Phase == coreapi.PodFailed {
+			continue
+		}
+		if ip := net.ParseIP(p.Status.PodIP).To4(); ip != nil {
+			cands = append(cands, candidate{p, ip.String()})
+		}
+	}
+	for ip, a := range admitted {
+		if p := podByKey[a.Namespace+"/"+a.Name]; p != nil && p.Spec.NodeName == self && !p.Spec.HostNetwork {
+			cands = append(cands, candidate{p, ip})
+		}
+	}
+	for _, c := range cands {
+		if a, ok := admitted[c.ip]; ok && (a.Namespace != c.pod.Namespace || a.Name != c.pod.Name) {
+			continue // the IP was given to another pod since
+		}
+		cur := byIP[c.ip]
+		if cur == nil || (cur.DeletionTimestamp != nil && c.pod.DeletionTimestamp == nil) ||
+			(cur.DeletionTimestamp == nil) == (c.pod.DeletionTimestamp == nil) && c.pod.CreationTimestamp.After(cur.CreationTimestamp.Time) {
+			byIP[c.ip] = c.pod
+		}
+	}
+	ips := make([]string, 0, len(byIP))
+	for ip := range byIP {
+		ips = append(ips, ip)
+	}
+	sort.Strings(ips)
+	for _, ip := range ips {
+		p := byIP[ip]
+		var in []*EgressService
+		for _, e := range es {
+			if e.Namespace != p.Namespace {
+				continue
+			}
+			if s := svcs[e.Namespace+"/"+e.Name]; s != nil && selects(s, p.Labels) {
+				in = append(in, e)
+			}
+		}
+		if len(in) == 0 {
+			continue
+		}
+		m := Member{IP: net.ParseIP(ip).To4(), Pod: p.Namespace + "/" + p.Name, Unreachable: true}
+		if len(in) > 1 {
+			keys := make([]string, len(in))
+			for i, e := range in {
+				keys[i] = e.Namespace + "/" + e.Name
+			}
+			d.Problems = append(d.Problems, fmt.Sprintf("pod %s is selected by several EgressServices (%s); its outside traffic is dropped", m.Pod, strings.Join(keys, ", ")))
+			d.Members = append(d.Members, m)
+			continue
+		}
+		e := in[0]
+		m.Service = e.Namespace + "/" + e.Name
+		if valid[m.Service] {
+			holder := byName[e.Status.Host]
+			for _, b := range e.Status.Blocks {
+				if b.Node == self && (holder.Name == self || holder.WeaveIP != nil) {
+					m.Unreachable = false
+					m.Address = net.ParseIP(e.Status.Address).To4()
+					m.From, m.To = b.From, b.To
+					m.Holder = holder
+				}
+			}
+			if m.Unreachable {
+				d.Problems = append(d.Problems, fmt.Sprintf("pod %s: egress %s has no port block for %s (or its holder's bridge is unknown); its outside traffic is dropped", m.Pod, m.Service, self))
+			}
+		}
+		d.Members = append(d.Members, m)
+	}
+	return d
+}

lb/egress_api.go

@@ -0,0 +1,144 @@
+package lb
+
+import (
+	"context"
+	"encoding/json"
+
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/runtime/schema"
+	"k8s.io/apimachinery/pkg/runtime/serializer"
+	"k8s.io/apimachinery/pkg/types"
+	"k8s.io/client-go/rest"
+)
+
+// EgressService (reweave.levk.dev/v1alpha1) makes the pods a LoadBalancer Service selects leave the cluster
+// from the Service's address. It is named after the Service (same namespace, same name). Its semantics follow
+// OVN-Kubernetes' EgressService (k8s.ovn.org/v1, sourceIPBy: LoadBalancerIP) where they agree; they differ in
+// that membership is every pod the Service's selector matches (by pod IP, never by readiness), the holder is
+// the node that holds the address (no election), and a member's traffic to the outside leaves as the address
+// or not at all.
+type EgressService struct {
+	metav1.TypeMeta   `json:",inline"`
+	metav1.ObjectMeta `json:"metadata,omitempty"`
+	Spec              EgressServiceSpec   `json:"spec"`
+	Status            EgressServiceStatus `json:"status,omitempty"`
+}
+
+// EgressServiceSpec is what the operator asks for.
+type EgressServiceSpec struct {
+	// SourceIPBy is LoadBalancerIP: members leave from the Service's load-balancer address (the only mode in v1).
+	SourceIPBy string `json:"sourceIPBy,omitempty"`
+	// Emit is where members' packets leave the cluster. Explicit, no default: Holder (tunnelled to the node
+	// holding the address, which emits them) is the only mode in v1.
+	Emit string `json:"emit"`
+}
+
+// EgressServiceStatus is written by the holder alone.
+type EgressServiceStatus struct {
+	// Host is the node that holds the address and emits members' packets.
+	Host string `json:"host,omitempty"`
+	// Address is the address members leave from.
+	Address string `json:"address,omitempty"`
+	// Blocks gives every node its own source ports on Address, so the holder routes replies back by port.
+	Blocks []PortBlock `json:"blocks,omitempty"`
+}
+
+// PortBlock is the source-port range [From, To] on the address that one node's members use.
+type PortBlock struct {
+	Node string `json:"node"`
+	From int32  `json:"from"`
+	To   int32  `json:"to"`
+}
+
+// EgressServiceList is a list of EgressServices.
+type EgressServiceList struct {
+	metav1.TypeMeta `json:",inline"`
+	metav1.ListMeta `json:"metadata,omitempty"`
+	Items           []EgressService `json:"items"`
+}
+
+const (
+	// Group and Version of the EgressService API.
+	Group   = "reweave.levk.dev"
+	Version = "v1alpha1"
+	// EgressServices is the resource name.
+	EgressServices = "egressservices"
+
+	SourceIPByLoadBalancerIP = "LoadBalancerIP"
+	EmitHolder               = "Holder"
+)
+
+// GroupVersion of the EgressService API.
+var GroupVersion = schema.GroupVersion{Group: Group, Version: Version}
+
+// DeepCopyObject implements runtime.Object.
+func (e *EgressService) DeepCopyObject() runtime.Object { return e.DeepCopy() }
+
+// DeepCopy copies the whole object.
+func (e *EgressService) DeepCopy() *EgressService {
+	if e == nil {
+		return nil
+	}
+	out := *e
+	e.ObjectMeta.DeepCopyInto(&out.ObjectMeta)
+	out.Status.Blocks = append([]PortBlock(nil), e.Status.Blocks...)
+	return &out
+}
+
+// DeepCopyObject implements runtime.Object.
+func (l *EgressServiceList) DeepCopyObject() runtime.Object {
+	if l == nil {
+		return nil
+	}
+	out := *l
+	l.ListMeta.DeepCopyInto(&out.ListMeta)
+	out.Items = make([]EgressService, len(l.Items))
+	for i := range l.Items {
+		out.Items[i] = *l.Items[i].DeepCopy()
+	}
+	return &out
+}
+
+func addKnownTypes(s *runtime.Scheme) error {
+	s.AddKnownTypes(GroupVersion, &EgressService{}, &EgressServiceList{})
+	metav1.AddToGroupVersion(s, GroupVersion)
+	return nil
+}
+
+// NewEgressClient is a REST client for the EgressService API (no generated clientset: two plain types).
+func NewEgressClient(base *rest.Config) (*rest.RESTClient, error) {
+	scheme := runtime.NewScheme()
+	if err := addKnownTypes(scheme); err != nil {
+		return nil, err
+	}
+	cfg := *base
+	cfg.GroupVersion = &GroupVersion
+	cfg.APIPath = "/apis"
+	cfg.ContentType = runtime.ContentTypeJSON
+	cfg.NegotiatedSerializer = serializer.NewCodecFactory(scheme).WithoutConversion()
+	if cfg.UserAgent == "" {
+		cfg.UserAgent = rest.DefaultKubernetesUserAgent()
+	}
+	return rest.RESTClientFor(&cfg)
+}
+
+// PatchEgressStatus merge-patches an EgressService's status (nil fields are removed).
+func PatchEgressStatus(ctx context.Context, c rest.Interface, namespace, name string, st EgressServiceStatus) error {
+	body := map[string]interface{}{"host": nil, "address": nil, "blocks": nil}
+	if st.Host != "" {
+		body["host"] = st.Host
+	}
+	if st.Address != "" {
+		body["address"] = st.Address
+	}
+	if len(st.Blocks) > 0 {
+		body["blocks"] = st.Blocks
+	}
+	patch, err := json.Marshal(map[string]interface{}{"status": body})
+	if err != nil {
+		return err
+	}
+	return c.Patch(types.MergePatchType).Namespace(namespace).Resource(EgressServices).Name(name).
+		SubResource("status").Body(patch).Do(ctx).Error()
+}

lb/egress_datapath.go

@@ -0,0 +1,530 @@
+package lb
+
+import (
+	"bytes"
+	"fmt"
+	"hash/fnv"
+	"net"
+	"os"
+	"sort"
+	"strings"
+
+	"github.com/vishvananda/netlink"
+	"golang.org/x/sys/unix"
+)
+
+const (
+	// ClassChain (nat, appended to PREROUTING after kube-proxy's KUBE-SERVICES) marks members' new connections to
+	// the outside. kube-proxy's DNAT ends the nat walk, so a connection to a Service never gets here.
+	ClassChain = "REWEAVE-LB-CLASS"
+	// FwdChain (filter, first in FORWARD) drops members' outside packets that would not leave as their address.
+	FwdChain = "REWEAVE-LB-FWD"
+	// InChain (filter, first in INPUT) accepts the egress tunnels only from the cluster's nodes.
+	InChain = "REWEAVE-LB-IN"
+	// MSSChain (mangle, first in FORWARD) clamps TCP MSS into the tunnels.
+	MSSChain = "REWEAVE-LB-MSS"
+
+	// TunnelPrefix names the egress tunnels: one ipip device per peer node, between the two nodes' weave bridges,
+	// so a holder knows which node a member's packet came from by the device it arrives on.
+	TunnelPrefix = "rweg-"
+	// tunnelGlob matches every egress tunnel in an iptables rule.
+	tunnelGlob = "rweg+"
+	// legacyTunnel is the single shared tunnel of earlier builds (removed when found).
+	legacyTunnel = "reweave-eg"
+
+	// EgressMark marks a member's packets to the outside (connmark on its connections, fwmark on each packet).
+	EgressMark = uint32(0x100000)
+	// peerFlag, with a peer's index in the top byte, tags (on the holder) a connection that arrived from that peer's
+	// tunnel: replies and related ICMP errors carry it back there.
+	peerFlag = uint32(0x200000)
+	peerMask = markMask | peerFlag
+
+	// HolderRulePriority: replies to a block go to its node; ahead of the local table, which moves to LocalRulePriority,
+	// since the address is usually local on the holder.
+	HolderRulePriority = 90
+	LocalRulePriority  = 100
+	// MemberRulePriority: a member's marked packets go to its holder (or nowhere).
+	MemberRulePriority = 9100
+	// EgressTableBase + peer index routes into that peer's tunnel; EgressTableBase itself is unreachable.
+	EgressTableBase = 53000
+
+	ipipOverhead = 20
+)
+
+// portProtocols carry ports: their replies are routed by port block alone (stateless), and members' SNAT keeps
+// within the node's block. Everything else is routed back by the tag the holder put on the connection.
+var portProtocols = []struct {
+	name string
+	num  int
+}{{"tcp", unix.IPPROTO_TCP}, {"udp", unix.IPPROTO_UDP}, {"sctp", unix.IPPROTO_SCTP}}
+
+// TunnelName is the egress tunnel towards a peer node (stable, within the 15 characters a device name may have).
+func TunnelName(peer string) string {
+	h := fnv.New32a()
+	h.Write([]byte(peer))
+	return fmt.Sprintf("%s%08x", TunnelPrefix, h.Sum32())
+}
+
+func peerTag(idx uint32) uint32 { return markValue(idx) | peerFlag }
+
+// renderEgress appends the egress rules of each table to the bodies of S1's payloads.
+func renderEgress(e EgressDesired, self string, podNet *net.IPNet, nat, mangle, filter *bytes.Buffer) {
+	members := e.Members
+	if len(members) > 0 {
+		// what is not "outside": the pod network, multicast, the cluster's nodes
+		fmt.Fprintf(nat, "-A %s -d %s -j RETURN\n", ClassChain, podNet)
+		fmt.Fprintf(nat, "-A %s -d 224.0.0.0/4 -j RETURN\n", ClassChain)
+		seen := map[string]bool{}
+		for _, a := range e.NodeAddrs {
+			if !seen[a.String()] {
+				seen[a.String()] = true
+				fmt.Fprintf(nat, "-A %s -d %s/32 -j RETURN\n", ClassChain, a)
+			}
+		}
+		for _, m := range members {
+			fmt.Fprintf(nat, "-A %s -s %s/32 -m comment --comment %q -j CONNMARK --set-xmark 0x%x/0x%x\n", ClassChain, m.IP, m.Pod, EgressMark, EgressMark)
+			fmt.Fprintf(nat, "-A %s -s %s/32 -j MARK --set-xmark 0x%x/0x%x\n", ClassChain, m.IP, EgressMark, EgressMark)
+		}
+		// later packets of a member's outside connection carry the mark too (the nat table saw only the first)
+		fmt.Fprintf(mangle, "-A %s -m conntrack --ctdir ORIGINAL -m connmark --mark 0x%x/0x%x -j MARK --set-xmark 0x%x/0x%x\n",
+			MarkChain, EgressMark, EgressMark, EgressMark, EgressMark)
+	}
+	for _, m := range members {
+		if m.Unreachable {
+			fmt.Fprintf(filter, "-A %s -s %s/32 -m mark --mark 0x%x/0x%x -m comment --comment %q -j DROP\n", FwdChain, m.IP, EgressMark, EgressMark, m.Pod)
+			continue
+		}
+		out := "-o " + TunnelName(m.Holder.Name)
+		if m.Holder.Name == self {
+			// emitted here, from the address: never into the pod network unSNATed
+			out = "! -o " + bridge
+			fmt.Fprintf(filter, "-A %s -s %s/32 -o %s -m mark --mark 0x%x/0x%x -j DROP\n", FwdChain, m.IP, bridge, EgressMark, EgressMark)
+		}
+		// within this node's block for what has ports; the address alone for the rest (the holder tells those
+		// apart by the tunnel they arrive on)
+		for _, p := range portProtocols {
+			fmt.Fprintf(nat, "-A %s -s %s/32 %s -m mark --mark 0x%x/0x%x -p %s -j SNAT --to-source %s:%d-%d\n",
+				PostChain, m.IP, out, EgressMark, EgressMark, p.name, m.Address, m.From, m.To)
+		}
+		// ICMP queries (ping) too: NAT treats the echo identifier as a port, and without a block of their own two
+		// nodes' pods (whose small, alike PIDs make alike identifiers) collide on the holder
+		fmt.Fprintf(nat, "-A %s -s %s/32 %s -m mark --mark 0x%x/0x%x -p icmp -j SNAT --to-source %s:%d-%d\n",
+			PostChain, m.IP, out, EgressMark, EgressMark, m.Address, m.From, m.To)
+		fmt.Fprintf(nat, "-A %s -s %s/32 %s -m mark --mark 0x%x/0x%x -j SNAT --to-source %s\n",
+			PostChain, m.IP, out, EgressMark, EgressMark, m.Address)
+		fmt.Fprintf(filter, "-A %s -s %s/32 %s -m mark --mark 0x%x/0x%x -j RETURN\n", FwdChain, m.IP, out, EgressMark, EgressMark)
+		// a member's outside traffic leaves no other way
+		fmt.Fprintf(filter, "-A %s -s %s/32 -m mark --mark 0x%x/0x%x -m comment --comment %q -j DROP\n", FwdChain, m.IP, EgressMark, EgressMark, m.Pod)
+	}
+	if !e.Tunnelled(self) {
+		return
+	}
+	tags := peerIndex(e, self)
+
+	// as holder: tag each connection arriving from a peer's tunnel with that peer, once; replies and related ICMP
+	// errors restore the tag, and the policy rule for it sends them back into that tunnel
+	byPeer := map[string][]BlockRoute{}
+	var peerNames []string
+	for _, r := range e.Routes {
+		if _, ok := byPeer[r.Peer.Name]; !ok {
+			peerNames = append(peerNames, r.Peer.Name)
+		}
+		byPeer[r.Peer.Name] = append(byPeer[r.Peer.Name], r)
+	}
+	sort.Strings(peerNames)
+	if len(peerNames) > 0 {
+		fmt.Fprintf(mangle, "-A %s -m conntrack --ctdir REPLY -m connmark --mark 0x%x/0x%x -j CONNMARK --restore-mark --nfmask 0x%x --ctmask 0x%x\n",
+			MarkChain, peerFlag, peerFlag, peerMask, peerMask)
+	}
+	for _, name := range peerNames {
+		dev, tag := TunnelName(name), peerTag(tags[name])
+		fmt.Fprintf(mangle, "-A %s -i %s -m conntrack --ctstate NEW -m connmark --mark 0x0/0x%x -m comment --comment %q -j CONNMARK --set-xmark 0x%x/0x%x\n",
+			MarkChain, dev, peerMask, "from "+name, tag, peerMask)
+		// a connection another node already holds (a port-less protocol to the same remote): never cross-delivered
+		fmt.Fprintf(filter, "-A %s -i %s -m conntrack --ctdir ORIGINAL -m connmark ! --mark 0x%x/0x%x -j DROP\n", FwdChain, dev, tag, peerMask)
+		addrs := map[string]bool{}
+		for _, r := range byPeer[name] {
+			for _, p := range portProtocols {
+				fmt.Fprintf(filter, "-A %s -i %s -m conntrack --ctdir ORIGINAL -s %s/32 -p %s -m %s --sport %d:%d -m comment --comment %q -j RETURN\n",
+					FwdChain, dev, r.Address, p.name, p.name, r.From, r.To, "block of "+name)
+			}
+			addrs[r.Address.String()] = true
+		}
+		for _, a := range sortedSet(addrs) {
+			// outside its own block a node may not use the ports of others
+			for _, p := range portProtocols {
+				fmt.Fprintf(filter, "-A %s -i %s -m conntrack --ctdir ORIGINAL -s %s/32 -p %s -j DROP\n", FwdChain, dev, a, p.name)
+			}
+			fmt.Fprintf(filter, "-A %s -i %s -m conntrack --ctdir ORIGINAL -s %s/32 -j RETURN\n", FwdChain, dev, a)
+		}
+	}
+	// as a member's node: replies (and related errors) from the holder; nothing else comes back
+	addrs := map[string]bool{}
+	for _, m := range members {
+		if !m.Unreachable && m.Holder.Name != self {
+			addrs[m.Address.String()] = true
+		}
+	}
+	for _, a := range sortedSet(addrs) {
+		// still addressed to the address after nat: no member connection matched it
+		fmt.Fprintf(filter, "-A %s -i %s -d %s/32 -j DROP\n", FwdChain, tunnelGlob, a)
+	}
+	fmt.Fprintf(filter, "-A %s -i %s -m conntrack --ctstate ESTABLISHED,RELATED -j RETURN\n", FwdChain, tunnelGlob)
+	fmt.Fprintf(filter, "-A %s -i %s -j DROP\n", FwdChain, tunnelGlob)
+	// the tunnels themselves only from the cluster's nodes' bridges
+	for _, p := range e.Peers(self) {
+		fmt.Fprintf(filter, "-A %s -p 4 -s %s/32 -m mac --mac-source %s -j RETURN\n", InChain, p.WeaveIP, p.WeaveMAC)
+	}
+	fmt.Fprintf(filter, "-A %s -p 4 -j DROP\n", InChain)
+	fmt.Fprintf(mangle, "-A %s -o %s -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu\n", MSSChain, tunnelGlob)
+}
+
+func sortedSet(m map[string]bool) []string {
+	out := make([]string, 0, len(m))
+	for k := range m {
+		out = append(out, k)
+	}
+	sort.Strings(out)
+	return out
+}
+
+// peerIndex gives each tunnel peer a stable index (1..MaxMarks): its tag and its routing table.
+func peerIndex(e EgressDesired, self string) map[string]uint32 {
+	out := map[string]uint32{}
+	for _, h := range assignMarks(e.Peers(self)) {
+		out[h.Node.Name] = h.Mark
+	}
+	return out
+}
+
+// peerTables gives each tunnel peer its routing table (EgressTableBase + its index).
+func peerTables(e EgressDesired, self string) map[string]int {
+	out := map[string]int{}
+	for name, idx := range peerIndex(e, self) {
+		out[name] = EgressTableBase + int(idx)
+	}
+	return out
+}
+
+// egressRule is a policy rule this package owns, in a comparable form.
+type egressRule struct {
+	priority   int
+	src, dst   string
+	ipproto    int
+	from, to   uint16
+	mark, mask uint32
+	table      int
+}
+
+func (r egressRule) key() string {
+	return fmt.Sprintf("%d|%s|%s|%d|%d-%d|%x/%x|%d", r.priority, r.src, r.dst, r.ipproto, r.from, r.to, r.mark, r.mask, r.table)
+}
+
+// wantRules is every egress policy rule this node needs.
+func wantRules(e EgressDesired, self string) []egressRule {
+	tables := peerTables(e, self)
+	tags := peerIndex(e, self)
+	var out []egressRule
+	tagged := map[string]bool{}
+	for _, r := range e.Routes {
+		for _, p := range portProtocols {
+			out = append(out, egressRule{priority: HolderRulePriority, dst: r.Address.String() + "/32", ipproto: p.num,
+				from: uint16(r.From), to: uint16(r.To), table: tables[r.Peer.Name]})
+		}
+		if !tagged[r.Peer.Name] {
+			// whatever the tag says came from that peer (replies without ports, ICMP errors) goes back to it
+			tagged[r.Peer.Name] = true
+			out = append(out, egressRule{priority: HolderRulePriority, mark: peerTag(tags[r.Peer.Name]), mask: peerMask, table: tables[r.Peer.Name]})
+		}
+	}
+	for _, m := range e.Members {
+		if !m.Unreachable && m.Holder.Name == self {
+			continue // emitted here: the main table already sends it out
+		}
+		table := EgressTableBase
+		if !m.Unreachable {
+			table = tables[m.Holder.Name]
+		}
+		out = append(out, egressRule{priority: MemberRulePriority, src: m.IP.String() + "/32", mark: EgressMark, mask: EgressMark, table: table})
+	}
+	sort.Slice(out, func(i, j int) bool { return out[i].key() < out[j].key() })
+	return out
+}
+
+func ruleOf(r netlink.Rule) egressRule {
+	er := egressRule{priority: r.Priority, ipproto: r.IPProto, table: r.Table}
+	if r.Src != nil {
+		er.src = r.Src.String()
+	}
+	if r.Dst != nil {
+		er.dst = r.Dst.String()
+	}
+	if r.Dport != nil {
+		er.from, er.to = r.Dport.Start, r.Dport.End
+	}
+	if r.Mark > 0 {
+		er.mark = uint32(r.Mark)
+		er.mask = uint32(r.Mask)
+	}
+	if er.ipproto < 0 {
+		er.ipproto = 0
+	}
+	return er
+}
+
+// routesByTable lists, in one dump, the IPv4 routes of tables lo..hi.
+func routesByTable(lo, hi int) (map[int][]netlink.Route, error) {
+	all, err := netlink.RouteListFiltered(unix.AF_INET, &netlink.Route{Table: unix.RT_TABLE_UNSPEC}, netlink.RT_FILTER_TABLE)
+	if err != nil {
+		return nil, err
+	}
+	out := map[int][]netlink.Route{}
+	for _, rt := range all {
+		if rt.Table >= lo && rt.Table <= hi {
+			out[rt.Table] = append(out[rt.Table], rt)
+		}
+	}
+	return out, nil
+}
+
+// egressNet makes the local rule's place, the policy rules and the tunnel tables exactly what e needs (the
+// tunnels themselves are made first, by Apply).
+func egressNet(e EgressDesired, self string) error {
+	if len(e.Routes) > 0 {
+		if err := moveLocalRule(); err != nil {
+			return err
+		}
+	}
+	tables := peerTables(e, self)
+	peerOf := map[int]Node{}
+	for _, p := range e.Peers(self) {
+		peerOf[tables[p.Name]] = p
+	}
+	unreachable := false
+	for _, m := range e.Members {
+		unreachable = unreachable || m.Unreachable
+	}
+	existing, err := routesByTable(EgressTableBase, EgressTableBase+MaxMarks)
+	if err != nil {
+		return err
+	}
+	for table := EgressTableBase; table <= EgressTableBase+MaxMarks; table++ {
+		var want *netlink.Route
+		if p, ok := peerOf[table]; ok {
+			link, err := netlink.LinkByName(TunnelName(p.Name))
+			if err != nil {
+				return fmt.Errorf("egress tunnel to %s: %v", p.Name, err)
+			}
+			want = &netlink.Route{LinkIndex: link.Attrs().Index, Dst: &net.IPNet{IP: net.IPv4zero, Mask: net.CIDRMask(0, 32)},
+				Scope: netlink.SCOPE_LINK, Table: table}
+		} else if table == EgressTableBase && unreachable {
+			want = &netlink.Route{Dst: &net.IPNet{IP: net.IPv4zero, Mask: net.CIDRMask(0, 32)}, Table: table, Type: unix.RTN_UNREACHABLE}
+		}
+		keep := false
+		for _, rt := range existing[table] {
+			if want != nil && rt.Type == want.Type &&
+				(want.Type == unix.RTN_UNREACHABLE || rt.LinkIndex == want.LinkIndex && rt.Gw == nil) {
+				keep = true
+				continue
+			}
+			if err := delRoute(rt); err != nil {
+				return fmt.Errorf("removing stale route in table %d: %v", table, err)
+			}
+		}
+		if want != nil && !keep {
+			if err := netlink.RouteReplace(want); err != nil {
+				return fmt.Errorf("route in table %d: %v", table, err)
+			}
+		}
+	}
+	want := map[string]egressRule{}
+	for _, r := range wantRules(e, self) {
+		want[r.key()] = r
+	}
+	rules, err := netlink.RuleList(unix.AF_INET)
+	if err != nil {
+		return err
+	}
+	have := map[string]bool{}
+	for _, r := range rules {
+		if r.Priority != HolderRulePriority && r.Priority != MemberRulePriority {
+			continue
+		}
+		k := ruleOf(r).key()
+		if _, ok := want[k]; ok && !have[k] {
+			have[k] = true
+			continue
+		}
+		r := r
+		if err := netlink.RuleDel(&r); err != nil {
+			return fmt.Errorf("removing stale egress rule: %v", err)
+		}
+	}
+	for k, r := range want {
+		if have[k] {
+			continue
+		}
+		rule := netlink.NewRule()
+		rule.Priority = r.priority
+		rule.Table = r.table
+		if r.src != "" {
+			_, rule.Src, _ = net.ParseCIDR(r.src)
+		}
+		if r.dst != "" {
+			_, rule.Dst, _ = net.ParseCIDR(r.dst)
+		}
+		if r.ipproto != 0 {
+			rule.IPProto = r.ipproto
+			rule.Dport = netlink.NewRulePortRange(r.from, r.to)
+		}
+		if r.mark != 0 {
+			rule.Mark = int(r.mark)
+			rule.Mask = int(r.mask)
+		}
+		if err := netlink.RuleAdd(rule); err != nil {
+			return fmt.Errorf("adding egress rule %s: %v", k, err)
+		}
+	}
+	return nil
+}
+
+// moveLocalRule puts the local table's lookup after HolderRulePriority (it is at 0 by default), so the holder can
+// route replies to its own address onwards to their node instead of delivering them to itself.
+func moveLocalRule() error {
+	rules, err := netlink.RuleList(unix.AF_INET)
+	if err != nil {
+		return err
+	}
+	var at0, atLocal bool
+	for _, r := range rules {
+		if r.Table == unix.RT_TABLE_LOCAL && r.Src == nil && r.Dst == nil && r.Mark <= 0 {
+			switch {
+			case r.Priority <= 0:
+				// the kernel leaves out the priority attribute when it is 0: listed as unset (-1)
+				at0 = true
+			case r.Priority == LocalRulePriority:
+				atLocal = true
+			}
+		}
+	}
+	if !atLocal {
+		rule := netlink.NewRule()
+		rule.Priority = LocalRulePriority
+		rule.Table = unix.RT_TABLE_LOCAL
+		if err := netlink.RuleAdd(rule); err != nil {
+			return fmt.Errorf("adding local table rule at %d: %v", LocalRulePriority, err)
+		}
+	}
+	if at0 {
+		rule := netlink.NewRule()
+		rule.Priority = 0
+		rule.Table = unix.RT_TABLE_LOCAL
+		if err := netlink.RuleDel(rule); err != nil {
+			return fmt.Errorf("removing local table rule at 0: %v", err)
+		}
+	}
+	return nil
+}
+
+// ensureTunnels makes one ipip tunnel per peer (this node's bridge address to the peer's), and removes every
+// other egress tunnel (including the shared one of earlier builds).
+func ensureTunnels(peers []Node, local Local) error {
+	links, err := netlink.LinkList()
+	if err != nil {
+		return err
+	}
+	want := map[string]Node{}
+	for _, p := range peers {
+		want[TunnelName(p.Name)] = p
+	}
+	have := map[string]netlink.Link{}
+	for _, l := range links {
+		name := l.Attrs().Name
+		if name == legacyTunnel || strings.HasPrefix(name, TunnelPrefix) {
+			if _, ok := want[name]; ok {
+				have[name] = l
+				continue
+			}
+			if err := netlink.LinkDel(l); err != nil {
+				return fmt.Errorf("removing egress tunnel %s: %v", name, err)
+			}
+		}
+	}
+	if len(want) == 0 {
+		return nil
+	}
+	if local.WeaveIP == nil {
+		return fmt.Errorf("egress tunnels need the weave bridge's address")
+	}
+	br, err := netlink.LinkByName(bridge)
+	if err != nil {
+		return fmt.Errorf("weave bridge: %v", err)
+	}
+	mtu := br.Attrs().MTU - ipipOverhead
+	names := make([]string, 0, len(want))
+	for n := range want {
+		names = append(names, n)
+	}
+	sort.Strings(names)
+	for _, name := range names {
+		peer := want[name]
+		link := have[name]
+		if link != nil {
+			t, ok := link.(*netlink.Iptun)
+			if !ok || !t.Local.Equal(local.WeaveIP) || !t.Remote.Equal(peer.WeaveIP) || t.Attrs().MTU != mtu {
+				if err := netlink.LinkDel(link); err != nil {
+					return err
+				}
+				link = nil
+			}
+		}
+		if link == nil {
+			t := &netlink.Iptun{LinkAttrs: netlink.LinkAttrs{Name: name, MTU: mtu}, Local: local.WeaveIP, Remote: peer.WeaveIP, PMtuDisc: 1}
+			if err := netlink.LinkAdd(t); err != nil {
+				return fmt.Errorf("creating egress tunnel %s to %s: %v", name, peer.Name, err)
+			}
+			if link, err = netlink.LinkByName(name); err != nil {
+				return err
+			}
+		}
+		if err := tunnelSetup(link, local); err != nil {
+			return fmt.Errorf("%s: %v", name, err)
+		}
+	}
+	return nil
+}
+
+// tunnelSetup: the holder receives members' packets from its own address (accept_local); reverse-path filtering
+// rejects what arrives on a device with no IPv4 address unless the reverse route leaves by that very device, so
+// the tunnel carries this node's bridge address (never on the wire from here: the tunnel sets the outer source,
+// and what crosses it is forwarded).
+func tunnelSetup(link netlink.Link, local Local) error {
+	name := link.Attrs().Name
+	for k, v := range map[string]string{"accept_local": "1", "rp_filter": "2"} {
+		if err := os.WriteFile("/proc/sys/net/ipv4/conf/"+name+"/"+k, []byte(v), 0o644); err != nil {
+			return fmt.Errorf("sysctl %s: %v", k, err)
+		}
+	}
+	addrs, err := netlink.AddrList(link, unix.AF_INET)
+	if err != nil {
+		return err
+	}
+	has := false
+	for _, a := range addrs {
+		if a.IP.Equal(local.WeaveIP) {
+			has = true
+			continue
+		}
+		a := a
+		if err := netlink.AddrDel(link, &a); err != nil {
+			return fmt.Errorf("removing stale address %s: %v", a.IPNet, err)
+		}
+	}
+	if !has {
+		addr := &netlink.Addr{IPNet: &net.IPNet{IP: local.WeaveIP, Mask: net.CIDRMask(32, 32)}, Scope: unix.RT_SCOPE_LINK}
+		if err := netlink.AddrAdd(link, addr); err != nil {
+			return fmt.Errorf("address: %v", err)
+		}
+	}
+	return netlink.LinkSetUp(link)
+}

lb/egress_test.go

@@ -0,0 +1,355 @@
+package lb
+
+import (
+	"fmt"
+	"net"
+	"strings"
+	"testing"
+	"time"
+
+	coreapi "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+)
+
+// the S1 fixtures: cp, w1 (holds 86.107.168.15), w2
+
+func egressFor(name string, status EgressServiceStatus) *EgressService {
+	return &EgressService{ObjectMeta: metav1.ObjectMeta{Namespace: "gitea", Name: name},
+		Spec: EgressServiceSpec{Emit: EmitHolder}, Status: status}
+}
+
+func mailSvc() *coreapi.Service {
+	return svc("mail", "86.107.168.15", func(s *coreapi.Service) {
+		s.Spec.Selector = map[string]string{"app": "mail"}
+	})
+}
+
+func pod(name, node, ip string, labels map[string]string, mutate ...func(*coreapi.Pod)) *coreapi.Pod {
+	p := &coreapi.Pod{ObjectMeta: metav1.ObjectMeta{Namespace: "gitea", Name: name, Labels: labels,
+		CreationTimestamp: metav1.NewTime(time.Unix(1000, 0))},
+		Spec: coreapi.PodSpec{NodeName: node}, Status: coreapi.PodStatus{PodIP: ip, Phase: coreapi.PodRunning}}
+	for _, m := range mutate {
+		m(p)
+	}
+	return p
+}
+
+var mailLabels = map[string]string{"app": "mail"}
+
+// the status the holder (w1) writes for three nodes, blocks in name order
+func written() EgressServiceStatus {
+	return EgressServiceStatus{Host: "w1", Address: "86.107.168.15", Blocks: []PortBlock{
+		{Node: "cp", From: 61000, To: 61511}, {Node: "w1", From: 61512, To: 62023}, {Node: "w2", From: 62024, To: 62535}}}
+}
+
+func TestBlocksAreStableAndCapped(t *testing.T) {
+	got := AllocateBlocks(nil, []string{"w2", "cp", "w1"})
+	if !sameStatus(EgressServiceStatus{Blocks: got}, EgressServiceStatus{Blocks: written().Blocks}) {
+		t.Fatalf("fresh blocks: %+v", got)
+	}
+	// a node leaves, another joins: survivors keep their blocks, the newcomer takes the freed lowest one
+	got = AllocateBlocks(got, []string{"w1", "w2", "w3"})
+	want := []PortBlock{{"w1", 61512, 62023}, {"w2", 62024, 62535}, {"w3", 61000, 61511}}
+	if !sameStatus(EgressServiceStatus{Blocks: got}, EgressServiceStatus{Blocks: want}) {
+		t.Fatalf("after churn: %+v", got)
+	}
+	// misaligned or duplicate blocks are not kept
+	got = AllocateBlocks([]PortBlock{{"w1", 61001, 61512}, {"w2", 61000, 61511}, {"w3", 61000, 61511}}, []string{"w1", "w2", "w3"})
+	for _, b := range got {
+		if (b.From-BlockBase)%BlockSize != 0 || b.To != b.From+BlockSize-1 {
+			t.Fatalf("invalid block kept: %+v", got)
+		}
+	}
+	if got[1].Node != "w2" || got[1].From != 61000 {
+		t.Fatalf("first valid claim wins: %+v", got)
+	}
+	var many []string
+	for i := 0; i < BlockCount+2; i++ {
+		many = append(many, string(rune('a'+i)))
+	}
+	if n := len(AllocateBlocks(nil, many)); n != BlockCount {
+		t.Fatalf("more blocks than exist: %d", n)
+	}
+	for _, b := range AllocateBlocks(nil, many) {
+		if b.To > 65535 || b.From < 61000 {
+			t.Fatalf("block outside the range: %+v", b)
+		}
+	}
+}
+
+func TestHolderWritesStatusAndMembersWaitForIt(t *testing.T) {
+	e := egressFor("mail", EgressServiceStatus{})
+	pods := []*coreapi.Pod{pod("mail-0", "w2", "10.44.0.9", mailLabels)}
+	d := ComputeEgress("w1", nodes, []*coreapi.Service{mailSvc()}, []*EgressService{e}, nil, nil)
+	if len(d.Status) != 1 || !sameStatus(d.Status[0].Status, written()) {
+		t.Fatalf("holder's status: %+v", d.Status)
+	}
+	// until the status is in the API, a member's outside traffic goes nowhere (fail closed)
+	d = ComputeEgress("w2", nodes, []*coreapi.Service{mailSvc()}, []*EgressService{e}, pods, nil)
+	if len(d.Status) != 0 || len(d.Members) != 1 || !d.Members[0].Unreachable {
+		t.Fatalf("member before status: %+v", d)
+	}
+	if d.Tunnelled("w2") {
+		t.Fatalf("no tunnel for an unreachable member")
+	}
+}
+
+func TestMemberTunnelsToHolderWhichRoutesItsBlockBack(t *testing.T) {
+	e := egressFor("mail", written())
+	svcs := []*coreapi.Service{mailSvc()}
+	pods := []*coreapi.Pod{
+		pod("mail-0", "w2", "10.44.0.9", mailLabels),
+		pod("web", "w2", "10.44.0.10", map[string]string{"app": "web"}), // not a member (A4)
+	}
+	d := ComputeEgress("w2", nodes, svcs, []*EgressService{e}, pods, nil)
+	if len(d.Members) != 1 || d.Members[0].Pod != "gitea/mail-0" {
+		t.Fatalf("members: %+v", d.Members)
+	}
+	m := d.Members[0]
+	if m.Unreachable || !m.Address.Equal(net.ParseIP("86.107.168.15")) || m.From != 62024 || m.To != 62535 || m.Holder.Name != "w1" {
+		t.Fatalf("member path: %+v", m)
+	}
+	if !d.Tunnelled("w2") || len(d.Peers("w2")) != 1 || d.Peers("w2")[0].Name != "w1" {
+		t.Fatalf("w2 tunnels to w1 only: %+v", d.Peers("w2"))
+	}
+	if len(d.Status) != 0 || len(d.Routes) != 0 {
+		t.Fatalf("only the holder writes and routes: %+v", d)
+	}
+	// the holder: blocks of the other nodes route back to them; its own block is plain conntrack
+	h := ComputeEgress("w1", nodes, svcs, []*EgressService{e}, nil, nil)
+	if len(h.Status) != 0 {
+		t.Fatalf("status already right: %+v", h.Status)
+	}
+	if len(h.Routes) != 2 || h.Routes[0].Peer.Name != "cp" || h.Routes[1].Peer.Name != "w2" || h.Routes[1].From != 62024 {
+		t.Fatalf("holder routes: %+v", h.Routes)
+	}
+	// a member on the holder leaves from there
+	hm := ComputeEgress("w1", nodes, svcs, []*EgressService{e}, []*coreapi.Pod{pod("mail-1", "w1", "10.40.0.4", mailLabels)}, nil)
+	if len(hm.Members) != 1 || hm.Members[0].Holder.Name != "w1" || hm.Members[0].From != 61512 {
+		t.Fatalf("member on the holder: %+v", hm.Members)
+	}
+}
+
+func TestEgressFailsClosed(t *testing.T) {
+	pods := []*coreapi.Pod{pod("mail-0", "w2", "10.44.0.9", mailLabels)}
+	// no node holds the address: status cleared (by the first node by name), member dropped
+	unheld := svc("mail", "86.107.168.99", func(s *coreapi.Service) { s.Spec.Selector = mailLabels })
+	for _, self := range []string{"cp", "w2"} {
+		d := ComputeEgress(self, nodes, []*coreapi.Service{unheld}, []*EgressService{egressFor("mail", written())}, pods, nil)
+		clears := len(d.Status) == 1 && d.Status[0].Status.Host == ""
+		if clears != (self == "cp") {
+			t.Fatalf("%s clears=%v: %+v", self, clears, d.Status)
+		}
+		if self == "w2" && (len(d.Members) != 1 || !d.Members[0].Unreachable) {
+			t.Fatalf("member of an unheld address: %+v", d.Members)
+		}
+	}
+	// two EgressServices select the pod: dropped, reported
+	other := svc("mail2", "86.107.168.15", func(s *coreapi.Service) { s.Spec.Selector = mailLabels })
+	d := ComputeEgress("w2", nodes, []*coreapi.Service{mailSvc(), other},
+		[]*EgressService{egressFor("mail", written()), egressFor("mail2", written())}, pods, nil)
+	if len(d.Members) != 1 || !d.Members[0].Unreachable || !strings.Contains(strings.Join(d.Problems, "\n"), "several EgressServices") {
+		t.Fatalf("double member: %+v", d)
+	}
+	// a spec this agent does not implement is refused, not guessed at
+	bad := egressFor("mail", written())
+	bad.Spec.Emit = "Direct"
+	d = ComputeEgress("w2", nodes, []*coreapi.Service{mailSvc()}, []*EgressService{bad}, pods, nil)
+	if len(d.Members) != 1 || !d.Members[0].Unreachable {
+		t.Fatalf("unknown emit mode: %+v", d)
+	}
+	// hostNetwork pods are never members (their traffic is the node's)
+	hn := pod("mail-h", "w2", "86.107.168.88", mailLabels, func(p *coreapi.Pod) { p.Spec.HostNetwork = true })
+	d = ComputeEgress("w2", nodes, []*coreapi.Service{mailSvc()}, []*EgressService{egressFor("mail", written())}, []*coreapi.Pod{hn}, nil)
+	if len(d.Members) != 0 {
+		t.Fatalf("hostNetwork pod became a member: %+v", d.Members)
+	}
+}
+
+func TestAdmissionDecidesWhoOwnsAnIP(t *testing.T) {
+	e := egressFor("mail", written())
+	svcs := []*coreapi.Service{mailSvc()}
+	old := pod("mail-0", "w2", "10.44.0.9", mailLabels, func(p *coreapi.Pod) {
+		now := metav1.Now()
+		p.DeletionTimestamp = &now
+	})
+	newer := pod("web", "w2", "", map[string]string{"app": "web"}) // API has no IP for it yet
+	// the CNI plugin just gave 10.44.0.9 to web: the terminating member must not keep it (A4)
+	d := ComputeEgress("w2", nodes, svcs, []*EgressService{e}, []*coreapi.Pod{old, newer},
+		map[string]Admission{"10.44.0.9": {Namespace: "gitea", Name: "web"}})
+	if len(d.Members) != 0 {
+		t.Fatalf("reused IP kept the old member's rules: %+v", d.Members)
+	}
+	// and the other way round: a member admitted before the API shows its IP is protected at once (A1)
+	fresh := pod("mail-1", "w2", "", mailLabels)
+	d = ComputeEgress("w2", nodes, svcs, []*EgressService{e}, []*coreapi.Pod{fresh},
+		map[string]Admission{"10.44.0.11": {Namespace: "gitea", Name: "mail-1"}})
+	if len(d.Members) != 1 || d.Members[0].IP.String() != "10.44.0.11" || d.Members[0].Unreachable {
+		t.Fatalf("admitted member: %+v", d.Members)
+	}
+	// without an admission, a live pod wins a shared IP over a terminating one
+	live := pod("web", "w2", "10.44.0.9", map[string]string{"app": "web"})
+	d = ComputeEgress("w2", nodes, svcs, []*EgressService{e}, []*coreapi.Pod{old, live}, nil)
+	if len(d.Members) != 0 {
+		t.Fatalf("terminating member won the IP: %+v", d.Members)
+	}
+}
+
+func TestRenderedEgressRules(t *testing.T) {
+	_, podNet, _ := net.ParseCIDR("10.32.0.0/12")
+	e := egressFor("mail", written())
+	svcs := []*coreapi.Service{mailSvc()}
+	toW1, toW2, toCP := TunnelName("w1"), TunnelName("w2"), TunnelName("cp")
+	member := ComputeEgress("w2", nodes, svcs, []*EgressService{e}, []*coreapi.Pod{pod("mail-0", "w2", "10.44.0.9", mailLabels)}, nil)
+	r := Render(Desired{}, member, "w2", podNet, nil)
+	for _, want := range []string{
+		"-A REWEAVE-LB-CLASS -d 10.32.0.0/12 -j RETURN\n",
+		"-A REWEAVE-LB-CLASS -d 86.107.168.24/32 -j RETURN\n", // a node is not outside
+		"-A REWEAVE-LB-CLASS -s 10.44.0.9/32 -m comment --comment \"gitea/mail-0\" -j CONNMARK --set-xmark 0x100000/0x100000\n",
+		"-A REWEAVE-LB-CLASS -s 10.44.0.9/32 -j MARK --set-xmark 0x100000/0x100000\n",
+		// port protocols within this node's block, into the holder's own tunnel
+		"-A REWEAVE-LB-POST -s 10.44.0.9/32 -o " + toW1 + " -m mark --mark 0x100000/0x100000 -p tcp -j SNAT --to-source 86.107.168.15:62024-62535\n",
+		"-A REWEAVE-LB-POST -s 10.44.0.9/32 -o " + toW1 + " -m mark --mark 0x100000/0x100000 -p udp -j SNAT --to-source 86.107.168.15:62024-62535\n",
+		"-A REWEAVE-LB-POST -s 10.44.0.9/32 -o " + toW1 + " -m mark --mark 0x100000/0x100000 -p sctp -j SNAT --to-source 86.107.168.15:62024-62535\n",
+		// ping within the block too (the echo identifier is NAT's port for ICMP)
+		"-A REWEAVE-LB-POST -s 10.44.0.9/32 -o " + toW1 + " -m mark --mark 0x100000/0x100000 -p icmp -j SNAT --to-source 86.107.168.15:62024-62535\n",
+		// everything else (GRE, ...) from the address alone
+		"-A REWEAVE-LB-POST -s 10.44.0.9/32 -o " + toW1 + " -m mark --mark 0x100000/0x100000 -j SNAT --to-source 86.107.168.15\n",
+	} {
+		if !strings.Contains(r.Nat, want) {
+			t.Fatalf("member nat lacks %q:\n%s", want, r.Nat)
+		}
+	}
+	// the catch-all SNAT comes after the port-ranged ones (or ports would not stay in the block)
+	if strings.Index(r.Nat, "-j SNAT --to-source 86.107.168.15\n") < strings.Index(r.Nat, "-p icmp -j SNAT") {
+		t.Fatalf("SNAT order:\n%s", r.Nat)
+	}
+	for _, want := range []string{
+		"-A REWEAVE-LB-FWD -s 10.44.0.9/32 -o " + toW1 + " -m mark --mark 0x100000/0x100000 -j RETURN\n",
+		"-A REWEAVE-LB-FWD -s 10.44.0.9/32 -m mark --mark 0x100000/0x100000 -m comment --comment \"gitea/mail-0\" -j DROP\n",
+		"-A REWEAVE-LB-FWD -i rweg+ -d 86.107.168.15/32 -j DROP\n",
+		"-A REWEAVE-LB-FWD -i rweg+ -m conntrack --ctstate ESTABLISHED,RELATED -j RETURN\n",
+		"-A REWEAVE-LB-FWD -i rweg+ -j DROP\n",
+		"-A REWEAVE-LB-IN -p 4 -s 10.40.0.0/32 -m mac --mac-source aa:aa:aa:aa:aa:02 -j RETURN\n",
+		"-A REWEAVE-LB-IN -p 4 -j DROP\n",
+	} {
+		if !strings.Contains(r.Filter, want) {
+			t.Fatalf("member filter lacks %q:\n%s", want, r.Filter)
+		}
+	}
+	for _, want := range []string{
+		"-A REWEAVE-LB-MARK -m conntrack --ctdir ORIGINAL -m connmark --mark 0x100000/0x100000 -j MARK --set-xmark 0x100000/0x100000\n",
+		"-A REWEAVE-LB-MSS -o rweg+ -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu\n",
+	} {
+		if !strings.Contains(r.Mangle, want) {
+			t.Fatalf("member mangle lacks %q:\n%s", want, r.Mangle)
+		}
+	}
+	if strings.Contains(r.Mangle, "--set-xmark 0x") && strings.Contains(r.Mangle, "-i "+toW1) {
+		t.Fatalf("a member's node does not tag what its holder sends back:\n%s", r.Mangle)
+	}
+	// the guard's DROP comes after the member's own RETURN: nothing else of its leaves
+	if strings.Index(r.Filter, toW1+" -m mark --mark 0x100000/0x100000 -j RETURN") > strings.Index(r.Filter, "\"gitea/mail-0\" -j DROP") {
+		t.Fatalf("guard order:\n%s", r.Filter)
+	}
+
+	// the holder: tags each tunnel's connections with its node, accepts each node's block, nothing foreign
+	h := ComputeEgress("w1", nodes, svcs, []*EgressService{e}, []*coreapi.Pod{pod("mail-1", "w1", "10.40.0.4", mailLabels)}, nil)
+	r = Render(Desired{}, h, "w1", podNet, nil)
+	idx := peerIndex(h, "w1")
+	tagW2 := fmt.Sprintf("0x%x", peerTag(idx["w2"]))
+	for _, want := range []string{
+		"-A REWEAVE-LB-MARK -m conntrack --ctdir REPLY -m connmark --mark 0x200000/0x200000 -j CONNMARK --restore-mark --nfmask 0xff200000 --ctmask 0xff200000\n",
+		"-A REWEAVE-LB-MARK -i " + toW2 + " -m conntrack --ctstate NEW -m connmark --mark 0x0/0xff200000 -m comment --comment \"from w2\" -j CONNMARK --set-xmark " + tagW2 + "/0xff200000\n",
+	} {
+		if !strings.Contains(r.Mangle, want) {
+			t.Fatalf("holder mangle lacks %q:\n%s", want, r.Mangle)
+		}
+	}
+	for _, want := range []string{
+		"-A REWEAVE-LB-FWD -i " + toW2 + " -m conntrack --ctdir ORIGINAL -m connmark ! --mark " + tagW2 + "/0xff200000 -j DROP\n",
+		"-A REWEAVE-LB-FWD -i " + toW2 + " -m conntrack --ctdir ORIGINAL -s 86.107.168.15/32 -p tcp -m tcp --sport 62024:62535 -m comment --comment \"block of w2\" -j RETURN\n",
+		"-A REWEAVE-LB-FWD -i " + toCP + " -m conntrack --ctdir ORIGINAL -s 86.107.168.15/32 -p sctp -m sctp --sport 61000:61511 -m comment --comment \"block of cp\" -j RETURN\n",
+		"-A REWEAVE-LB-FWD -i " + toW2 + " -m conntrack --ctdir ORIGINAL -s 86.107.168.15/32 -p udp -j DROP\n",
+		"-A REWEAVE-LB-FWD -i " + toW2 + " -m conntrack --ctdir ORIGINAL -s 86.107.168.15/32 -j RETURN\n",
+		"-A REWEAVE-LB-FWD -s 10.40.0.4/32 -o weave -m mark --mark 0x100000/0x100000 -j DROP\n",
+	} {
+		if !strings.Contains(r.Filter, want) {
+			t.Fatalf("holder filter lacks %q:\n%s", want, r.Filter)
+		}
+	}
+	// foreign tags are refused before anything of that tunnel is accepted; a port outside the block before the
+	// catch-all accept
+	if strings.Index(r.Filter, "! --mark "+tagW2) > strings.Index(r.Filter, "\"block of w2\"") ||
+		strings.Index(r.Filter, "-i "+toW2+" -m conntrack --ctdir ORIGINAL -s 86.107.168.15/32 -p udp -j DROP") >
+			strings.Index(r.Filter, "-i "+toW2+" -m conntrack --ctdir ORIGINAL -s 86.107.168.15/32 -j RETURN") {
+		t.Fatalf("holder filter order:\n%s", r.Filter)
+	}
+	if !strings.Contains(r.Nat, "-A REWEAVE-LB-POST -s 10.40.0.4/32 ! -o weave -m mark --mark 0x100000/0x100000 -p tcp -j SNAT --to-source 86.107.168.15:61512-62023\n") {
+		t.Fatalf("holder's own member:\n%s", r.Nat)
+	}
+	for _, line := range strings.Split(r.Nat+r.Filter, "\n") {
+		if strings.Contains(line, "10.44.0.10") {
+			t.Fatalf("a rule names a non-member: %s", line)
+		}
+	}
+}
+
+func TestTunnelNames(t *testing.T) {
+	a, b := TunnelName("reweave-worker-fe19b040"), TunnelName("reweave-cp-94a8db43")
+	if a == b || len(a) > 15 || !strings.HasPrefix(a, TunnelPrefix) || a != TunnelName("reweave-worker-fe19b040") {
+		t.Fatalf("tunnel names must be distinct, stable, short: %q %q", a, b)
+	}
+}
+
+func TestPolicyRules(t *testing.T) {
+	e := egressFor("mail", written())
+	svcs := []*coreapi.Service{mailSvc()}
+	h := ComputeEgress("w1", nodes, svcs, []*EgressService{e}, nil, nil)
+	rules := wantRules(h, "w1")
+	tables := peerTables(h, "w1")
+	idx := peerIndex(h, "w1")
+	ports, tags := 0, 0
+	for _, r := range rules {
+		if r.priority != HolderRulePriority || r.src != "" {
+			t.Fatalf("holder rule: %+v", r)
+		}
+		switch {
+		case r.ipproto != 0:
+			ports++
+			if r.dst != "86.107.168.15/32" || r.mark != 0 {
+				t.Fatalf("port rule: %+v", r)
+			}
+			if (r.from == 62024) != (r.table == tables["w2"]) || (r.from == 61000) != (r.table == tables["cp"]) {
+				t.Fatalf("block routed to the wrong node: %+v %v", r, tables)
+			}
+		default:
+			tags++
+			if r.mask != peerMask || r.dst != "" {
+				t.Fatalf("tag rule: %+v", r)
+			}
+			for name, i := range idx {
+				if r.mark == peerTag(i) && r.table != tables[name] {
+					t.Fatalf("tag of %s routed to table %d, want %d", name, r.table, tables[name])
+				}
+			}
+		}
+	}
+	if ports != 2*len(portProtocols) || tags != 2 {
+		t.Fatalf("2 peers: %d port rules (tcp, udp, sctp each), %d tag rules: %+v", ports, tags, rules)
+	}
+	if HolderRulePriority >= LocalRulePriority {
+		t.Fatalf("holder rules must precede the local table")
+	}
+	m := ComputeEgress("w2", nodes, svcs, []*EgressService{e}, []*coreapi.Pod{pod("mail-0", "w2", "10.44.0.9", mailLabels)}, nil)
+	rules = wantRules(m, "w2")
+	if len(rules) != 1 || rules[0].src != "10.44.0.9/32" || rules[0].mark != EgressMark || rules[0].mask != EgressMark ||
+		rules[0].table != peerTables(m, "w2")["w1"] {
+		t.Fatalf("member rule: %+v", rules)
+	}
+	u := ComputeEgress("w2", nodes, svcs, []*EgressService{egressFor("mail", EgressServiceStatus{})}, []*coreapi.Pod{pod("mail-0", "w2", "10.44.0.9", mailLabels)}, nil)
+	rules = wantRules(u, "w2")
+	if len(rules) != 1 || rules[0].table != EgressTableBase {
+		t.Fatalf("unreachable member must hit the unreachable table: %+v", rules)
+	}
+}

lb/model.go

@@ -0,0 +1,308 @@
+// Package lb gives LoadBalancer Services an address that a node holds, and
+// forwards that address's traffic to the Service's endpoints with the
+// client's address intact.
+//
+// A Service opts in with spec.loadBalancerClass = Class and names its
+// address in spec.loadBalancerIP. The node that holds the address (has it on
+// one of its interfaces, as published in its AnnAddresses annotation) is the
+// address's holder: it alone writes the Service's status (ipMode Proxy, so
+// kube-proxy installs no rules for the address) and DNATs the address's
+// ports to ready endpoints, without SNAT. Every other node marks the
+// connections that the holder forwards to its pods, and routes their replies
+// back to the holder over weave, so that the holder's conntrack can undo the
+// DNAT and the reply leaves from the address the client connected to.
+//
+// Compute is the whole policy as a pure function of what the API shows; the
+// datapath and the controller only carry it out.
+package lb
+
+import (
+	"fmt"
+	"hash/fnv"
+	"net"
+	"sort"
+	"strings"
+
+	coreapi "k8s.io/api/core/v1"
+	discovery "k8s.io/api/discovery/v1"
+)
+
+const (
+	// Class is the spec.loadBalancerClass a Service sets to be served by this agent.
+	Class = "reweave.levk.dev/node-address"
+	// AnnAddresses lists the IPv4 addresses a node holds on its host interfaces (comma-separated).
+	AnnAddresses = "reweave.levk.dev/addresses"
+	// AnnWeave is "<ip>,<mac>" of the node's weave bridge: where replies to its forwards go.
+	AnnWeave = "reweave.levk.dev/weave"
+
+	serviceNameLabel = "kubernetes.io/service-name"
+)
+
+// Node is what one node publishes about itself.
+type Node struct {
+	Name      string
+	Addresses []net.IP
+	WeaveIP   net.IP
+	WeaveMAC  net.HardwareAddr
+}
+
+// Holds reports whether the node has ip on one of its host interfaces.
+func (n Node) Holds(ip net.IP) bool {
+	for _, a := range n.Addresses {
+		if a.Equal(ip) {
+			return true
+		}
+	}
+	return false
+}
+
+// ParseNode reads a Node's annotations. A node that has published nothing
+// (yet) holds nothing and cannot take replies.
+func ParseNode(n *coreapi.Node) (Node, error) {
+	out := Node{Name: n.Name}
+	if s := n.Annotations[AnnAddresses]; s != "" {
+		for _, f := range strings.Split(s, ",") {
+			ip := net.ParseIP(strings.TrimSpace(f)).To4()
+			if ip == nil {
+				return out, fmt.Errorf("node %s: %s: %q is not an IPv4 address", n.Name, AnnAddresses, f)
+			}
+			out.Addresses = append(out.Addresses, ip)
+		}
+	}
+	if s := n.Annotations[AnnWeave]; s != "" {
+		f := strings.Split(s, ",")
+		if len(f) != 2 {
+			return out, fmt.Errorf("node %s: %s: want \"<ip>,<mac>\", got %q", n.Name, AnnWeave, s)
+		}
+		ip := net.ParseIP(f[0]).To4()
+		mac, err := net.ParseMAC(f[1])
+		if ip == nil || err != nil {
+			return out, fmt.Errorf("node %s: %s: want \"<ip>,<mac>\", got %q", n.Name, AnnWeave, s)
+		}
+		out.WeaveIP, out.WeaveMAC = ip, mac
+	}
+	return out, nil
+}
+
+// FormatAddresses is the AnnAddresses value for a set of addresses (sorted, so it only changes when the set does).
+func FormatAddresses(ips []net.IP) string {
+	s := make([]string, 0, len(ips))
+	for _, ip := range ips {
+		s = append(s, ip.String())
+	}
+	sort.Strings(s)
+	return strings.Join(s, ",")
+}
+
+// Endpoint is one ready backend of one Service port.
+type Endpoint struct {
+	IP   net.IP
+	Port int32
+}
+
+func (e Endpoint) String() string { return fmt.Sprintf("%s:%d", e.IP, e.Port) }
+
+// Port is one Service port at the held address.
+type Port struct {
+	Protocol  coreapi.Protocol
+	Port      int32
+	Endpoints []Endpoint
+}
+
+// Forward is a Service whose address this node holds.
+type Forward struct {
+	Namespace, Name string
+	Address         net.IP
+	Ports           []Port
+}
+
+// Status is a write to a Service's status.loadBalancer: Address set, or cleared when nil.
+type Status struct {
+	Namespace, Name string
+	Address         net.IP
+}
+
+// Holder is another node that forwards held addresses' connections to pods; Mark identifies its connections here.
+type Holder struct {
+	Node Node
+	Mark uint32 // 1..MaxMarks, in the connmark's top byte
+}
+
+// MaxMarks is how many holders a node can tell apart (the top byte of the mark, 0 meaning none).
+const MaxMarks = 250
+
+// Desired is what one node must do.
+type Desired struct {
+	Forwards []Forward
+	Holders  []Holder
+	Status   []Status
+	Problems []string // what an operator should hear about (logged and evented by the controller)
+}
+
+// eligible reports whether the Service asks for this agent.
+func eligible(s *coreapi.Service) bool {
+	return s.Spec.Type == coreapi.ServiceTypeLoadBalancer && s.Spec.LoadBalancerClass != nil &&
+		*s.Spec.LoadBalancerClass == Class
+}
+
+// shown is the address status.loadBalancer currently shows (nil if none or not a single IP).
+func shown(s *coreapi.Service) net.IP {
+	if len(s.Status.LoadBalancer.Ingress) != 1 {
+		return nil
+	}
+	return net.ParseIP(s.Status.LoadBalancer.Ingress[0].IP).To4()
+}
+
+func hasStatus(s *coreapi.Service) bool { return len(s.Status.LoadBalancer.Ingress) != 0 }
+
+// Compute decides what node `self` does, given every node, every Service and every EndpointSlice.
+func Compute(self string, nodes []Node, services []*coreapi.Service, slices []*discovery.EndpointSlice) Desired {
+	var d Desired
+	sortedNodes := append([]Node(nil), nodes...)
+	sort.Slice(sortedNodes, func(i, j int) bool { return sortedNodes[i].Name < sortedNodes[j].Name })
+	holdersOf := func(ip net.IP) []Node {
+		var hs []Node
+		for _, n := range sortedNodes {
+			if ip != nil && n.Holds(ip) {
+				hs = append(hs, n)
+			}
+		}
+		return hs
+	}
+	// who may write a status that shows `ip`: its single holder, or (nobody holds it) the first node by name
+	writerFor := func(ip net.IP) string {
+		if hs := holdersOf(ip); len(hs) == 1 {
+			return hs[0].Name
+		}
+		if len(sortedNodes) > 0 {
+			return sortedNodes[0].Name
+		}
+		return ""
+	}
+	holding := map[string]bool{} // nodes that forward some address
+
+	svcs := append([]*coreapi.Service(nil), services...)
+	sort.Slice(svcs, func(i, j int) bool { return key(svcs[i]) < key(svcs[j]) })
+	for _, s := range svcs {
+		if !eligible(s) {
+			// loadBalancerClass is immutable and only allowed on type LoadBalancer, so a
+			// Service never leaves the class while keeping a status we wrote
+			continue
+		}
+		want := net.ParseIP(s.Spec.LoadBalancerIP).To4()
+		var holder *Node
+		switch hs := holdersOf(want); {
+		case want == nil:
+			d.Problems = append(d.Problems, fmt.Sprintf("%s: spec.loadBalancerIP must name an IPv4 address a node holds (got %q)", key(s), s.Spec.LoadBalancerIP))
+		case len(hs) == 0:
+			d.Problems = append(d.Problems, fmt.Sprintf("%s: no node holds %s", key(s), want))
+		case len(hs) > 1:
+			names := make([]string, len(hs))
+			for i, h := range hs {
+				names[i] = h.Name
+			}
+			d.Problems = append(d.Problems, fmt.Sprintf("%s: %s is held by several nodes (%s); serving it from none", key(s), want, strings.Join(names, ", ")))
+		default:
+			holder = &hs[0]
+		}
+		if holder == nil {
+			// fail closed: nobody forwards, and the status must not claim an address
+			if hasStatus(s) && writerFor(shown(s)) == self {
+				d.Status = append(d.Status, Status{Namespace: s.Namespace, Name: s.Name})
+			}
+			continue
+		}
+		holding[holder.Name] = true
+		if holder.Name != self {
+			continue
+		}
+		if cur := shown(s); cur == nil || !cur.Equal(want) {
+			d.Status = append(d.Status, Status{Namespace: s.Namespace, Name: s.Name, Address: want})
+		}
+		d.Forwards = append(d.Forwards, Forward{Namespace: s.Namespace, Name: s.Name, Address: want, Ports: ports(s, slices)})
+	}
+
+	var others []Node
+	for _, n := range sortedNodes {
+		if holding[n.Name] && n.Name != self {
+			if n.WeaveIP == nil {
+				d.Problems = append(d.Problems, fmt.Sprintf("node %s holds an address but has not published its weave bridge; replies to its forwards cannot reach it", n.Name))
+				continue
+			}
+			others = append(others, n)
+		}
+	}
+	d.Holders = assignMarks(others)
+	return d
+}
+
+// assignMarks gives each holder a mark from its name, probing past collisions in name order: stable while the set is.
+func assignMarks(nodes []Node) []Holder {
+	taken := map[uint32]bool{}
+	out := make([]Holder, 0, len(nodes))
+	for _, n := range nodes {
+		if len(out) == MaxMarks {
+			break
+		}
+		h := fnv.New32a()
+		h.Write([]byte(n.Name))
+		m := h.Sum32()%MaxMarks + 1
+		for taken[m] {
+			m = m%MaxMarks + 1
+		}
+		taken[m] = true
+		out = append(out, Holder{Node: n, Mark: m})
+	}
+	return out
+}
+
+func key(s *coreapi.Service) string { return s.Namespace + "/" + s.Name }
+
+// ports pairs each Service port with its ready endpoints.
+func ports(s *coreapi.Service, slices []*discovery.EndpointSlice) []Port {
+	var out []Port
+	for _, sp := range s.Spec.Ports {
+		proto := sp.Protocol
+		if proto == "" {
+			proto = coreapi.ProtocolTCP
+		}
+		seen := map[string]bool{}
+		var eps []Endpoint
+		for _, sl := range slices {
+			if sl.Namespace != s.Namespace || sl.Labels[serviceNameLabel] != s.Name || sl.AddressType != discovery.AddressTypeIPv4 {
+				continue
+			}
+			for _, p := range sl.Ports {
+				pname, pproto := "", coreapi.ProtocolTCP
+				if p.Name != nil {
+					pname = *p.Name
+				}
+				if p.Protocol != nil {
+					pproto = *p.Protocol
+				}
+				if pname != sp.Name || pproto != proto || p.Port == nil {
+					continue
+				}
+				for _, ep := range sl.Endpoints {
+					if ep.Conditions.Ready != nil && !*ep.Conditions.Ready {
+						continue
+					}
+					for _, a := range ep.Addresses {
+						ip := net.ParseIP(a).To4()
+						if ip == nil {
+							continue
+						}
+						e := Endpoint{IP: ip, Port: *p.Port}
+						if !seen[e.String()] {
+							seen[e.String()] = true
+							eps = append(eps, e)
+						}
+					}
+				}
+			}
+		}
+		sort.Slice(eps, func(i, j int) bool { return eps[i].String() < eps[j].String() })
+		out = append(out, Port{Protocol: proto, Port: sp.Port, Endpoints: eps})
+	}
+	return out
+}

lb/model_test.go

@@ -0,0 +1,253 @@
+package lb
+
+import (
+	"fmt"
+	"net"
+	"strings"
+	"testing"
+
+	coreapi "k8s.io/api/core/v1"
+	discovery "k8s.io/api/discovery/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+)
+
+func mac(s string) net.HardwareAddr {
+	m, err := net.ParseMAC(s)
+	if err != nil {
+		panic(err)
+	}
+	return m
+}
+
+var (
+	cp = Node{Name: "cp", Addresses: []net.IP{net.ParseIP("86.107.168.24").To4()},
+		WeaveIP: net.ParseIP("10.32.0.1").To4(), WeaveMAC: mac("aa:aa:aa:aa:aa:01")}
+	w1 = Node{Name: "w1", Addresses: []net.IP{net.ParseIP("86.107.168.15").To4()},
+		WeaveIP: net.ParseIP("10.40.0.0").To4(), WeaveMAC: mac("aa:aa:aa:aa:aa:02")}
+	w2 = Node{Name: "w2", Addresses: []net.IP{net.ParseIP("86.107.168.88").To4()},
+		WeaveIP: net.ParseIP("10.44.0.0").To4(), WeaveMAC: mac("aa:aa:aa:aa:aa:03")}
+	nodes = []Node{w2, cp, w1}
+)
+
+func svc(name, ip string, mutate ...func(*coreapi.Service)) *coreapi.Service {
+	class := Class
+	s := &coreapi.Service{
+		ObjectMeta: metav1.ObjectMeta{Namespace: "gitea", Name: name},
+		Spec: coreapi.ServiceSpec{
+			Type:              coreapi.ServiceTypeLoadBalancer,
+			LoadBalancerClass: &class,
+			LoadBalancerIP:    ip,
+			Ports: []coreapi.ServicePort{
+				{Name: "ssh", Protocol: coreapi.ProtocolTCP, Port: 22},
+				{Name: "https", Port: 443}, // protocol defaults to TCP
+			},
+		},
+	}
+	for _, m := range mutate {
+		m(s)
+	}
+	return s
+}
+
+func shows(ip string) func(*coreapi.Service) {
+	return func(s *coreapi.Service) {
+		s.Status.LoadBalancer.Ingress = []coreapi.LoadBalancerIngress{{IP: ip}}
+	}
+}
+
+func ptr[T any](v T) *T { return &v }
+
+func slice(svcName string, eps map[string]bool, ports map[string]int32) *discovery.EndpointSlice {
+	sl := &discovery.EndpointSlice{
+		ObjectMeta:  metav1.ObjectMeta{Namespace: "gitea", Name: svcName + "-x", Labels: map[string]string{serviceNameLabel: svcName}},
+		AddressType: discovery.AddressTypeIPv4,
+	}
+	for ip, ready := range eps {
+		sl.Endpoints = append(sl.Endpoints, discovery.Endpoint{Addresses: []string{ip}, Conditions: discovery.EndpointConditions{Ready: ptr(ready)}})
+	}
+	for name, port := range ports {
+		sl.Ports = append(sl.Ports, discovery.EndpointPort{Name: ptr(name), Protocol: ptr(coreapi.ProtocolTCP), Port: ptr(port)})
+	}
+	return sl
+}
+
+func TestHolderForwardsToReadyEndpointsAndWritesStatus(t *testing.T) {
+	s := svc("gitea", "86.107.168.15")
+	sl := slice("gitea", map[string]bool{"10.40.0.7": true, "10.44.0.3": true, "10.44.0.9": false},
+		map[string]int32{"ssh": 2222, "https": 3000})
+	d := Compute("w1", nodes, []*coreapi.Service{s}, []*discovery.EndpointSlice{sl})
+	if len(d.Problems) != 0 {
+		t.Fatalf("problems: %v", d.Problems)
+	}
+	if len(d.Forwards) != 1 || !d.Forwards[0].Address.Equal(net.ParseIP("86.107.168.15")) {
+		t.Fatalf("forwards: %+v", d.Forwards)
+	}
+	f := d.Forwards[0]
+	if len(f.Ports) != 2 || f.Ports[0].Port != 22 || f.Ports[1].Port != 443 || f.Ports[1].Protocol != coreapi.ProtocolTCP {
+		t.Fatalf("ports: %+v", f.Ports)
+	}
+	if got := f.Ports[0].Endpoints; len(got) != 2 || got[0].String() != "10.40.0.7:2222" || got[1].String() != "10.44.0.3:2222" {
+		t.Fatalf("ssh endpoints (not-ready one must be left out): %v", got)
+	}
+	if got := f.Ports[1].Endpoints; len(got) != 2 || got[0].Port != 3000 {
+		t.Fatalf("https endpoints: %v", got)
+	}
+	if len(d.Status) != 1 || !d.Status[0].Address.Equal(net.ParseIP("86.107.168.15")) {
+		t.Fatalf("status: %+v", d.Status)
+	}
+	if len(d.Holders) != 0 {
+		t.Fatalf("a holder does not steer its own replies: %+v", d.Holders)
+	}
+	// already shown: nothing to write
+	d = Compute("w1", nodes, []*coreapi.Service{svc("gitea", "86.107.168.15", shows("86.107.168.15"))}, nil)
+	if len(d.Status) != 0 {
+		t.Fatalf("status rewritten: %+v", d.Status)
+	}
+}
+
+func TestOtherNodesSteerRepliesToTheHolderAndNeverWrite(t *testing.T) {
+	s := svc("gitea", "86.107.168.15")
+	for _, self := range []string{"cp", "w2"} {
+		d := Compute(self, nodes, []*coreapi.Service{s}, nil)
+		if len(d.Forwards) != 0 || len(d.Status) != 0 {
+			t.Fatalf("%s: only the holder forwards and writes: %+v", self, d)
+		}
+		if len(d.Holders) != 1 || d.Holders[0].Node.Name != "w1" || d.Holders[0].Mark < 1 || d.Holders[0].Mark > MaxMarks {
+			t.Fatalf("%s: holders: %+v", self, d.Holders)
+		}
+	}
+}
+
+func TestUnheldOrContestedAddressFailsClosed(t *testing.T) {
+	// nobody holds it: no forward anywhere; the stale status is cleared by exactly one node (first by name)
+	s := svc("gitea", "86.107.168.99", shows("86.107.168.99"))
+	for _, self := range []string{"cp", "w1", "w2"} {
+		d := Compute(self, nodes, []*coreapi.Service{s}, nil)
+		if len(d.Forwards) != 0 || len(d.Holders) != 0 || len(d.Problems) != 1 || !strings.Contains(d.Problems[0], "no node holds") {
+			t.Fatalf("%s: %+v", self, d)
+		}
+		clears := len(d.Status) == 1 && d.Status[0].Address == nil
+		if clears != (self == "cp") {
+			t.Fatalf("%s: clears=%v", self, clears)
+		}
+	}
+	// two nodes claim it: neither serves it, its holder's view clears the status
+	both := w2
+	both.Addresses = append([]net.IP{net.ParseIP("86.107.168.15").To4()}, w2.Addresses...)
+	d := Compute("w1", []Node{cp, w1, both}, []*coreapi.Service{svc("gitea", "86.107.168.15", shows("86.107.168.15"))}, nil)
+	if len(d.Forwards) != 0 || len(d.Problems) != 1 || !strings.Contains(d.Problems[0], "several nodes") {
+		t.Fatalf("contested: %+v", d)
+	}
+	// a bad request is reported, not guessed at
+	d = Compute("w1", nodes, []*coreapi.Service{svc("gitea", "")}, nil)
+	if len(d.Forwards) != 0 || len(d.Problems) != 1 || !strings.Contains(d.Problems[0], "spec.loadBalancerIP") {
+		t.Fatalf("no address: %+v", d)
+	}
+}
+
+func TestOnlyOptedInLoadBalancersAreServed(t *testing.T) {
+	other := svc("a", "86.107.168.15", func(s *coreapi.Service) { s.Spec.LoadBalancerClass = ptr("example.com/other") })
+	noClass := svc("b", "86.107.168.15", func(s *coreapi.Service) { s.Spec.LoadBalancerClass = nil })
+	clusterIP := svc("c", "86.107.168.15", func(s *coreapi.Service) {
+		s.Spec.Type = coreapi.ServiceTypeClusterIP
+	})
+	d := Compute("w1", nodes, []*coreapi.Service{other, noClass, clusterIP}, nil)
+	if len(d.Forwards)+len(d.Status)+len(d.Holders)+len(d.Problems) != 0 {
+		t.Fatalf("touched a Service that did not opt in: %+v", d)
+	}
+}
+
+func TestParseNodeAndAnnotationsRoundTrip(t *testing.T) {
+	l := Local{Addresses: []net.IP{net.ParseIP("86.107.168.88").To4(), net.ParseIP("86.107.168.15").To4()},
+		WeaveIP: net.ParseIP("10.44.0.0").To4(), WeaveMAC: mac("aa:aa:aa:aa:aa:03")}
+	a := l.Annotations()
+	if a[AnnAddresses] != "86.107.168.15,86.107.168.88" || a[AnnWeave] != "10.44.0.0,aa:aa:aa:aa:aa:03" {
+		t.Fatalf("annotations: %v", a)
+	}
+	n, err := ParseNode(&coreapi.Node{ObjectMeta: metav1.ObjectMeta{Name: "w2", Annotations: a}})
+	if err != nil || !n.Holds(net.ParseIP("86.107.168.15")) || !n.WeaveIP.Equal(l.WeaveIP) || n.WeaveMAC.String() != "aa:aa:aa:aa:aa:03" {
+		t.Fatalf("parsed %+v, %v", n, err)
+	}
+	for _, bad := range []map[string]string{{AnnAddresses: "1.2.3.4,nope"}, {AnnWeave: "10.44.0.0"}, {AnnWeave: "x,aa:aa:aa:aa:aa:03"}} {
+		if _, err := ParseNode(&coreapi.Node{ObjectMeta: metav1.ObjectMeta{Name: "w2", Annotations: bad}}); err == nil {
+			t.Fatalf("accepted %v", bad)
+		}
+	}
+	if n, err := ParseNode(&coreapi.Node{ObjectMeta: metav1.ObjectMeta{Name: "new"}}); err != nil || len(n.Addresses) != 0 || n.WeaveIP != nil {
+		t.Fatalf("a node that published nothing holds nothing: %+v %v", n, err)
+	}
+}
+
+func TestMarksAreDistinctAndStable(t *testing.T) {
+	var ns []Node
+	for _, name := range []string{"a", "b", "c", "d", "e", "f", "g", "h"} {
+		ns = append(ns, Node{Name: name})
+	}
+	first := assignMarks(ns)
+	seen := map[uint32]bool{}
+	for _, h := range first {
+		if seen[h.Mark] || h.Mark < 1 || h.Mark > MaxMarks {
+			t.Fatalf("bad or repeated mark: %+v", first)
+		}
+		seen[h.Mark] = true
+	}
+	again := assignMarks(ns)
+	for i := range first {
+		if first[i].Mark != again[i].Mark {
+			t.Fatalf("marks moved between passes")
+		}
+	}
+}
+
+func TestRenderedRules(t *testing.T) {
+	_, podNet, _ := net.ParseCIDR("10.32.0.0/12")
+	s := svc("gitea", "86.107.168.15")
+	sl := slice("gitea", map[string]bool{"10.40.0.7": true, "10.44.0.3": true, "10.44.0.4": true}, map[string]int32{"ssh": 2222, "https": 3000})
+	d := Compute("w1", nodes, []*coreapi.Service{s}, []*discovery.EndpointSlice{sl})
+	r := Render(d, EgressDesired{}, "w1", podNet, []string{"REWEAVE-LBS-DEADBEEF0000"})
+	for _, want := range []string{
+		"*nat\n:REWEAVE-LB - [0:0]\n:REWEAVE-LB-POST - [0:0]\n:REWEAVE-LB-CLASS - [0:0]\n",
+		// the client's address survives weave's masquerade; a pod's own connection (hairpin) is masqueraded here
+		"-A REWEAVE-LB-POST -m conntrack --ctstate DNAT --ctorigdst 86.107.168.15/32 -s 10.32.0.0/12 -j MASQUERADE\n",
+		"-A REWEAVE-LB-POST -m conntrack --ctstate DNAT --ctorigdst 86.107.168.15/32 -j ACCEPT\n",
+		":REWEAVE-LBS-DEADBEEF0000 - [0:0]\n", // stale chain emptied
+		"-A REWEAVE-LB -d 86.107.168.15/32 -p tcp -m tcp --dport 22 -m comment --comment \"gitea/gitea\" -j REWEAVE-LBS-",
+		"-p tcp -m statistic --mode random --probability 0.3333333333 -j DNAT --to-destination 10.40.0.7:2222\n",
+		"-p tcp -m statistic --mode random --probability 0.5000000000 -j DNAT --to-destination 10.44.0.3:2222\n",
+		"-p tcp -j DNAT --to-destination 10.44.0.4:2222\n",
+		"COMMIT\n",
+	} {
+		if !strings.Contains(r.Nat, want) {
+			t.Fatalf("nat lacks %q:\n%s", want, r.Nat)
+		}
+	}
+	for _, line := range strings.Split(r.Nat, "\n") {
+		// iptables refuses a DNAT to ip:port without the protocol in the same rule
+		if strings.Contains(line, "-j DNAT") && !strings.Contains(line, " -p tcp ") {
+			t.Fatalf("DNAT without its protocol: %q", line)
+		}
+	}
+	if strings.Count(r.Nat, "MASQUERADE") != 1 || strings.Contains(r.Nat, "SNAT") {
+		t.Fatalf("only a pod's hairpin connection may be masqueraded:\n%s", r.Nat)
+	}
+	if r.Mangle != "*mangle\n:REWEAVE-LB-MARK - [0:0]\n:REWEAVE-LB-MSS - [0:0]\nCOMMIT\n" {
+		t.Fatalf("a holder marks nothing:\n%s", r.Mangle)
+	}
+	d = Compute("w2", nodes, []*coreapi.Service{s}, []*discovery.EndpointSlice{sl})
+	r = Render(d, EgressDesired{}, "w2", podNet, nil)
+	m := d.Holders[0].Mark << 24
+	for _, want := range []string{
+		"-A REWEAVE-LB-MARK -m conntrack --ctdir REPLY -m connmark ! --mark 0x0/0xff000000 -j CONNMARK --restore-mark --nfmask 0xff000000 --ctmask 0xff000000\n",
+		"-A REWEAVE-LB-MARK -i weave -m mac --mac-source aa:aa:aa:aa:aa:02 ! -s 10.32.0.0/12 -d 10.32.0.0/12 -m conntrack --ctstate NEW -m comment --comment \"holder w1\" -j CONNMARK --set-xmark " + fmt.Sprintf("0x%x", m) + "/0xff000000\n",
+	} {
+		if !strings.Contains(r.Mangle, want) {
+			t.Fatalf("mangle lacks %q:\n%s", want, r.Mangle)
+		}
+	}
+	if r.Nat != "*nat\n:REWEAVE-LB - [0:0]\n:REWEAVE-LB-POST - [0:0]\n:REWEAVE-LB-CLASS - [0:0]\nCOMMIT\n" {
+		t.Fatalf("a non-holder forwards nothing:\n%s", r.Nat)
+	}
+	if r.Filter != "*filter\n:REWEAVE-LB-FWD - [0:0]\n:REWEAVE-LB-IN - [0:0]\nCOMMIT\n" {
+		t.Fatalf("no EgressService, no filter rules:\n%s", r.Filter)
+	}
+}

nameserver/dns.go

@@ -13,7 +13,7 @@ import (
 
 	"github.com/miekg/dns"
 
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/net/address"
 )
 
 const (
@@ -90,7 +90,7 @@ type DNSServer struct {
 	ns      *Nameserver
 	domain  string
 	ttl     uint32
-	address string
+	address string // listen address and port, e.g. 1.2.3.4:53
 
 	servers   []*dns.Server
 	upstream  Upstream
@@ -99,6 +99,10 @@ type DNSServer struct {
 }
 
 func NewDNSServer(ns *Nameserver, domain, address string, upstream Upstream, ttl uint32, clientTimeout time.Duration) (*DNSServer, error) {
+	if _, _, err := net.SplitHostPort(address); err != nil {
+		return nil, fmt.Errorf("invalid DNS listen address %q: %v", address, err)
+	}
+
 	s := &DNSServer{
 		ns:        ns,
 		domain:    dns.Fqdn(domain),
@@ -269,7 +273,7 @@ func (h *handler) handleRecursive(w dns.ResponseWriter, req *dns.Msg) {
 		reqCopy := req.Copy()
 		reqCopy.Id = dns.Id()
 		response, _, err := h.client.Exchange(reqCopy, fmt.Sprintf("%s:%s", server, upstreamConfig.Port))
-		if (err != nil && err != dns.ErrTruncated) || response == nil {
+		if (err != nil) || response == nil {
 			h.ns.debugf("error trying %s: %v", server, err)
 			continue
 		}

nameserver/dns_test.go

@@ -11,8 +11,8 @@ import (
 	"github.com/miekg/dns"
 	"github.com/stretchr/testify/require"
 
+	"github.com/rajch/weave/net/address"
 	"github.com/weaveworks/mesh"
-	"github.com/weaveworks/weave/net/address"
 )
 
 type mockUpstream struct {
@@ -41,23 +41,20 @@ func TestTruncation(t *testing.T) {
 	defer dnsserver.Stop()
 
 	// Add 100 mappings to nameserver
-	addrs := []address.Address{}
 	for i := address.Address(0); i < 100; i++ {
-		addrs = append(addrs, i)
 		nameserver.AddEntry("foo.weave.local.", "", mesh.UnknownPeerName, i)
 	}
 
-	doRequest := func(client *dns.Client, request *dns.Msg, port int, expectedErr error) *dns.Msg {
+	doRequest := func(client *dns.Client, request *dns.Msg, port int) *dns.Msg {
 		request.SetQuestion("foo.weave.local.", dns.TypeA)
-		response, _, err := client.Exchange(request, fmt.Sprintf("127.0.0.1:%d", port))
-		require.Equal(t, expectedErr, err)
+		response, _, _ := client.Exchange(request, fmt.Sprintf("127.0.0.1:%d", port))
 		return response
 	}
 
 	// do a udp query, ensure we get a truncated response
 	{
 		udpClient := dns.Client{Net: "udp", UDPSize: minUDPSize}
-		response := doRequest(&udpClient, &dns.Msg{}, udpPort, dns.ErrTruncated)
+		response := doRequest(&udpClient, &dns.Msg{}, udpPort)
 		require.True(t, response.MsgHdr.Truncated)
 		require.True(t, len(response.Answer) < 100)
 	}
@@ -67,7 +64,7 @@ func TestTruncation(t *testing.T) {
 		udpClient := dns.Client{Net: "udp", UDPSize: 65535}
 		request := &dns.Msg{}
 		request.SetEdns0(65535, false)
-		response := doRequest(&udpClient, request, udpPort, nil)
+		response := doRequest(&udpClient, request, udpPort)
 		require.False(t, response.MsgHdr.Truncated)
 		require.Equal(t, len(response.Answer), 100)
 	}
@@ -75,7 +72,7 @@ func TestTruncation(t *testing.T) {
 	// do a tcp query, ensure we don't get a truncated response
 	{
 		tcpClient := dns.Client{Net: "tcp"}
-		response := doRequest(&tcpClient, &dns.Msg{}, tcpPort, nil)
+		response := doRequest(&tcpClient, &dns.Msg{}, tcpPort)
 		require.False(t, response.MsgHdr.Truncated)
 		require.Equal(t, len(response.Answer), 100)
 	}

nameserver/entry.go

@@ -10,7 +10,7 @@ import (
 
 	"github.com/weaveworks/mesh"
 
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/net/address"
 )
 
 var now = func() int64 { return time.Now().Unix() }

nameserver/entry_test.go

@@ -6,7 +6,7 @@ import (
 	"github.com/stretchr/testify/require"
 	"github.com/weaveworks/mesh"
 
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/net/address"
 )
 
 func l(es Entries) Entries {

nameserver/http.go

@@ -3,13 +3,14 @@ package nameserver
 import (
 	"encoding/json"
 	"fmt"
+	"net"
 	"net/http"
 
 	"github.com/gorilla/mux"
 	"github.com/miekg/dns"
 
-	"github.com/weaveworks/weave/common/docker"
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/common/docker"
+	"github.com/rajch/weave/net/address"
 )
 
 func (n *Nameserver) badRequest(w http.ResponseWriter, err error) {
@@ -84,3 +85,13 @@ func (n *Nameserver) HandleHTTP(router *mux.Router, dockerCli *docker.Client) {
 		}
 	})
 }
+
+func (d *DNSServer) HandleHTTP(router *mux.Router) {
+	router.Methods("GET").Path("/dns-address-port").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		fmt.Fprint(w, d.address)
+	})
+	router.Methods("GET").Path("/dns-address").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		host, _, _ := net.SplitHostPort(d.address) // address is validated in NewDNSServer()
+		fmt.Fprint(w, host)
+	})
+}

nameserver/nameserver.go

@@ -8,8 +8,8 @@ import (
 	"github.com/miekg/dns"
 	"github.com/weaveworks/mesh"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/address"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/net/address"
 )
 
 const (

nameserver/nameserver_test.go

@@ -12,8 +12,8 @@ import (
 	"github.com/stretchr/testify/require"
 	"github.com/weaveworks/mesh"
 
-	"github.com/weaveworks/weave/net/address"
-	"github.com/weaveworks/weave/testing/gossip"
+	"github.com/rajch/weave/net/address"
+	"github.com/rajch/weave/testing/gossip"
 )
 
 func makeNameserver(name mesh.PeerName) *Nameserver {

net/address/address.go

@@ -4,7 +4,7 @@ import (
 	"fmt"
 	"net"
 
-	"github.com/weaveworks/weave/common"
+	"github.com/rajch/weave/common"
 )
 
 // Using 32-bit integer to represent IPv4 address

net/address/helpers.go

@@ -0,0 +1,42 @@
+package address
+
+// Merge merges adjacent range entries.
+// The given slice has to be sorted in increasing order.
+func Merge(r []Range) []Range {
+	var merged []Range
+
+	for i := range r {
+		if prev := len(merged) - 1; prev >= 0 && merged[prev].End == r[i].Start {
+			merged[prev].End = r[i].End
+		} else {
+			merged = append(merged, r[i])
+		}
+	}
+
+	return merged
+}
+
+// RemoveCommon filters out CIDR ranges which are contained in both a and b slices.
+// Both slices have to be sorted in increasing order.
+func RemoveCommon(a, b []CIDR) (newA, newB []CIDR) {
+	i, j := 0, 0
+
+	for i < len(a) && j < len(b) {
+		switch {
+		case a[i].Start() < b[j].Start() || a[i].End() < b[j].End():
+			newA = append(newA, a[i])
+			i++
+		case a[i].Start() > b[j].Start() || a[i].End() > b[j].End():
+			newB = append(newB, b[j])
+			j++
+		default:
+			i++
+			j++
+		}
+
+	}
+	newA = append(newA, a[i:]...)
+	newB = append(newB, b[j:]...)
+
+	return
+}

net/address/helpers_test.go

@@ -0,0 +1,43 @@
+package address
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/require"
+)
+
+var (
+	r0to127     = cidr2("10.0.0.0", "10.0.0.127")
+	r128to255   = cidr2("10.0.0.128", "10.0.0.255")
+	r0to255     = cidr2("10.0.0.0", "10.0.0.255")
+	r1dot0to255 = cidr2("10.0.1.0", "10.0.1.255")
+	r2dot0to255 = cidr2("10.0.2.0", "10.0.2.255")
+)
+
+func TestRemoveCommon(t *testing.T) {
+	a := []CIDR{r0to127, r1dot0to255}
+	b := []CIDR{r1dot0to255, r2dot0to255}
+	newA, newB := RemoveCommon(a, b)
+	require.Equal(t, []CIDR{r0to127}, newA)
+	require.Equal(t, []CIDR{r2dot0to255}, newB)
+}
+
+func TestMerge(t *testing.T) {
+	ranges := []Range{
+		r0to127.Range(),
+		r128to255.Range(),
+		r2dot0to255.Range(),
+	}
+	require.Equal(t, []Range{r0to255.Range(), r2dot0to255.Range()}, Merge(ranges))
+}
+
+// Helper
+
+// [start; end]
+func cidr2(start, end string) CIDR {
+	c := Range{Start: ip(start), End: ip(end) + 1}.CIDRs()
+	if len(c) != 1 {
+		panic("invalid cidr")
+	}
+	return c[0]
+}

net/arp.go

@@ -41,3 +41,11 @@ func sysctl(procPath, variable, value string) error {
 
 	return nil
 }
+
+func sysctlIfExists(procPath, variable, value string) error {
+	err := sysctl(procPath, variable, value)
+	if err != nil && os.IsNotExist(err) {
+		err = nil // ignore 'not found' errors
+	}
+	return err
+}

net/bridge.go

@@ -11,14 +11,12 @@ import (
 	"github.com/pkg/errors"
 	"github.com/sirupsen/logrus"
 	"github.com/vishvananda/netlink"
-	"k8s.io/apimachinery/pkg/types"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/common/odp"
-	"github.com/weaveworks/weave/ipam/tracker"
-	"github.com/weaveworks/weave/net/address"
-	"github.com/weaveworks/weave/net/ipset"
-	"github.com/weaveworks/weave/npc"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/common/chains"
+	"github.com/rajch/weave/common/odp"
+	"github.com/rajch/weave/net/address"
+	"github.com/rajch/weave/net/ipset"
 )
 
 /* This code implements three possible configurations to connect
@@ -50,7 +48,7 @@ the "datapath" device, via "router/fastdp.go:fastDatapathBridge"
 
 "weave" is an Open vSwitch datapath, and capture/injection are as in
 BridgedFastdp. Not used by default due to missing conntrack support in
-datapath of old kernel versions (https://github.com/weaveworks/weave/issues/1577).
+datapath of old kernel versions (https://github.com/rajch/weave/issues/1577).
 */
 
 const (
@@ -63,10 +61,10 @@ const (
 )
 
 type Bridge interface {
-	init(config *BridgeConfig) error // create and initialise bridge device(s)
-	attach(veth *netlink.Veth) error // attach veth to bridge
-	IsFastdp() bool                  // does this bridge use fastdp?
-	String() string                  // human-readable type string
+	init(procPath string, config *BridgeConfig) error // create and initialise bridge device(s)
+	attach(veth *netlink.Veth) error                  // attach veth to bridge
+	IsFastdp() bool                                   // does this bridge use fastdp?
+	String() string                                   // human-readable type string
 }
 
 // Used to indicate a fallback to the Bridge type
@@ -221,6 +219,7 @@ type BridgeConfig struct {
 	MTU              int
 	Mac              string
 	Port             int
+	ControlPort      string
 	NoMasqLocal      bool
 }
 
@@ -250,7 +249,7 @@ func EnsureBridge(procPath string, config *BridgeConfig, log *logrus.Logger, ips
 	}
 
 	for {
-		if err := bridgeType.init(config); err != nil {
+		if err := bridgeType.init(procPath, config); err != nil {
 			if errors.Cause(err) == errBridgeNotSupported {
 				log.Warnf("Skipping bridge creation of %q due to: %s", bridgeType, err)
 				bridgeType = bridgeImpl{}
@@ -261,7 +260,7 @@ func EnsureBridge(procPath string, config *BridgeConfig, log *logrus.Logger, ips
 		break
 	}
 
-	if err := configureIPTables(config, ips); err != nil {
+	if err := ConfigureIPTables(config, ips); err != nil {
 		return bridgeType, errors.Wrap(err, "configuring iptables")
 	}
 
@@ -279,6 +278,10 @@ func EnsureBridge(procPath string, config *BridgeConfig, log *logrus.Logger, ips
 			return bridgeType, errors.Wrap(err, "setting proxy_arp")
 		}
 	}
+	// No ipv6 router advertisments please
+	if err := sysctlIfExists(procPath, "net/ipv6/conf/"+config.WeaveBridgeName+"/accept_ra", "0"); err != nil {
+		return bridgeType, errors.Wrap(err, "setting accept_ra to 0")
+	}
 
 	if err := linkSetUpByName(config.WeaveBridgeName); err != nil {
 		return bridgeType, err
@@ -292,9 +295,7 @@ func EnsureBridge(procPath string, config *BridgeConfig, log *logrus.Logger, ips
 		return bridgeType, errors.Wrapf(err, "configuring ARP cache on bridge %q", config.WeaveBridgeName)
 	}
 
-	// NB: No concurrent call to Expose is possible, as EnsureBridge is called
-	// before any service has been started.
-	if err := reexpose(config, log); err != nil {
+	if err := Reexpose(config, log); err != nil {
 		return bridgeType, err
 	}
 
@@ -345,11 +346,11 @@ func (b bridgeImpl) initPrep(config *BridgeConfig) error {
 	return nil
 }
 
-func (b bridgeImpl) init(config *BridgeConfig) error {
+func (b bridgeImpl) init(procPath string, config *BridgeConfig) error {
 	if err := b.initPrep(config); err != nil {
 		return err
 	}
-	if _, err := CreateAndAttachVeth(BridgeIfName, PcapIfName, config.WeaveBridgeName, config.MTU, true, false, func(veth netlink.Link) error {
+	if _, err := CreateAndAttachVeth(procPath, BridgeIfName, PcapIfName, config.WeaveBridgeName, config.MTU, true, false, func(veth netlink.Link) error {
 		return netlink.LinkSetUp(veth)
 	}); err != nil {
 		return errors.Wrap(err, "creating pcap veth pair")
@@ -361,7 +362,7 @@ func (b bridgeImpl) init(config *BridgeConfig) error {
 	return nil
 }
 
-func (f fastdpImpl) init(config *BridgeConfig) error {
+func (f fastdpImpl) init(procPath string, config *BridgeConfig) error {
 	odpSupported, err := odp.CreateDatapath(f.datapathName)
 	if !odpSupported {
 		msg := ""
@@ -392,14 +393,14 @@ func (f fastdpImpl) init(config *BridgeConfig) error {
 	return nil
 }
 
-func (bf bridgedFastdpImpl) init(config *BridgeConfig) error {
-	if err := bf.fastdpImpl.init(config); err != nil {
+func (bf bridgedFastdpImpl) init(procPath string, config *BridgeConfig) error {
+	if err := bf.fastdpImpl.init(procPath, config); err != nil {
 		return err
 	}
 	if err := bf.bridgeImpl.initPrep(config); err != nil {
 		return err
 	}
-	if _, err := CreateAndAttachVeth(BridgeIfName, DatapathIfName, config.WeaveBridgeName, config.MTU, true, false, func(veth netlink.Link) error {
+	if _, err := CreateAndAttachVeth(procPath, BridgeIfName, DatapathIfName, config.WeaveBridgeName, config.MTU, true, false, func(veth netlink.Link) error {
 		if err := netlink.LinkSetUp(veth); err != nil {
 			return errors.Wrapf(err, "setting link up on %q", veth.Attrs().Name)
 		}
@@ -426,10 +427,40 @@ func (f fastdpImpl) attach(veth *netlink.Veth) error {
 	return odp.AddDatapathInterfaceIfNotExist(f.datapathName, veth.Attrs().Name)
 }
 
-func configureIPTables(config *BridgeConfig, ips ipset.Interface) error {
+// ResetIPTables resets IPTables in case they're in a strange state from a previous run.
+func ResetIPTables(config *BridgeConfig, ips ipset.Interface) error {
 	ipt, err := iptables.New()
 	if err != nil {
-		return errors.Wrap(err, "creating iptables object")
+		return errors.Wrap(err, "creating iptables object while resetting")
+	}
+
+	if !config.NPC {
+		// Create/Flush a chain for allowing ingress traffic when the bridge is exposed
+		if err := ipt.ClearChain("filter", "WEAVE-EXPOSE"); err != nil {
+			return errors.Wrap(err, "failed to clear/create filter/WEAVE-EXPOSE chain")
+		}
+	}
+
+	if err := ipt.ClearChain("nat", "WEAVE"); err != nil {
+		return errors.Wrap(err, "failed to clear/create nat/WEAVE chain")
+	}
+
+	if config.NoMasqLocal {
+		ips := ipset.New(common.LogLogger(), 0)
+		_ = ips.Destroy(NoMasqLocalIpset)
+		if err := ips.Create(NoMasqLocalIpset, ipset.HashNet); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+// ConfigureIPTables idempotently configures all the iptables!
+func ConfigureIPTables(config *BridgeConfig, ips ipset.Interface) error {
+	ipt, err := iptables.New()
+	if err != nil {
+		return errors.Wrap(err, "creating iptables object while configuring")
 	}
 
 	// The order among weave filter/FORWARD rules is important!
@@ -465,15 +496,27 @@ func configureIPTables(config *BridgeConfig, ips ipset.Interface) error {
 		}
 	}
 
+	if config.ControlPort != "" {
+		if err = ipt.AppendUnique("filter", "INPUT", "-p", "tcp", "--dst", "127.0.0.1", "--dport", config.ControlPort,
+			"-m", "addrtype", "!", "--src-type", "LOCAL",
+			"-m", "conntrack", "!", "--ctstate", "RELATED,ESTABLISHED",
+			"-m", "comment", "--comment", "Block non-local access to Weave Net control port",
+			"-j", "DROP"); err != nil {
+			return err
+		}
+	}
+	// Remove the rule from Weave Net 2.6.3 which dropped too much.
+	_ = ipt.Delete("filter", "INPUT", "-p", "tcp", "--dst", "127.0.0.1", "-m", "addrtype", "!", "--src-type", "LOCAL", "-m", "conntrack", "!", "--ctstate", "RELATED,ESTABLISHED", "-j", "DROP")
+
 	if config.NPC {
 		// Steer traffic via the NPC.
 
-		if err = ensureChains(ipt, "filter", npc.MainChain, npc.EgressChain); err != nil {
+		if err = ensureChains(ipt, "filter", chains.MainChain, chains.EgressChain); err != nil {
 			return err
 		}
 
 		// Steer egress traffic destined to local node.
-		if err = ipt.AppendUnique("filter", "INPUT", "-i", config.WeaveBridgeName, "-j", npc.EgressChain); err != nil {
+		if err = ipt.AppendUnique("filter", "INPUT", "-i", config.WeaveBridgeName, "-j", chains.EgressChain); err != nil {
 			return err
 		}
 		fwdRules = append(fwdRules,
@@ -482,11 +525,11 @@ func configureIPTables(config *BridgeConfig, ips ipset.Interface) error {
 				// ACCEPT in WEAVE-NPC-EGRESS chain
 				{"-i", config.WeaveBridgeName,
 					"-m", "comment", "--comment", "NOTE: this must go before '-j KUBE-FORWARD'",
-					"-j", npc.EgressChain},
+					"-j", chains.EgressChain},
 				// The following rules are for ingress NPC processing
 				{"-o", config.WeaveBridgeName,
 					"-m", "comment", "--comment", "NOTE: this must go before '-j KUBE-FORWARD'",
-					"-j", npc.MainChain},
+					"-j", chains.MainChain},
 				{"-o", config.WeaveBridgeName, "-m", "state", "--state", "NEW", "-j", "NFLOG", "--nflog-group", "86"},
 				{"-o", config.WeaveBridgeName, "-j", "DROP"},
 			}...)
@@ -498,8 +541,8 @@ func configureIPTables(config *BridgeConfig, ips ipset.Interface) error {
 
 	if !config.NPC {
 		// Create/Flush a chain for allowing ingress traffic when the bridge is exposed
-		if err := ipt.ClearChain("filter", "WEAVE-EXPOSE"); err != nil {
-			return errors.Wrap(err, "failed to clear/create filter/WEAVE-EXPOSE chain")
+		if err := ensureChains(ipt, "filter", "WEAVE-EXPOSE"); err != nil {
+			return errors.Wrap(err, "failed to ensure existence of filter/WEAVE-EXPOSE chain")
 		}
 
 		fwdRules = append(fwdRules, []string{"-o", config.WeaveBridgeName, "-j", "WEAVE-EXPOSE"})
@@ -515,26 +558,22 @@ func configureIPTables(config *BridgeConfig, ips ipset.Interface) error {
 	}
 
 	// Create a chain for masquerading
-	if err := ipt.ClearChain("nat", "WEAVE"); err != nil {
-		return errors.Wrap(err, "failed to clear/create nat/WEAVE chain")
+	if err := ensureChains(ipt, "nat", "WEAVE"); err != nil {
+		return errors.Wrap(err, "failed to ensure existence of nat/WEAVE chain")
 	}
 	if err := ipt.AppendUnique("nat", "POSTROUTING", "-j", "WEAVE"); err != nil {
 		return err
 	}
 
 	// For the cases where the weave bridge is the default gateway for
-	// containers (e.g. Kubernetes): create the ipset to store CIDRs allocated
-	// by IPAM for local containers. In the case of Kubernetes, external traffic
-	// sent to these CIDRs avoids SNAT'ing so that NodePort with
-	// `"externalTrafficPolicy":"Local"` would receive packets with correct
-	// src IP addr.
+	// containers (e.g. Kubernetes): In `ResetIPTables` (which we assume
+	// to have been called at this point) we create an ipset to store CIDRs
+	// allocated by IPAM for local containers.
+	// In the case of Kubernetes, external traffic sent to these CIDRs
+	// avoids SNAT'ing so that NodePort with `"externalTrafficPolicy":"Local"`
+	// would receive packets with correct src IP addr.
 	if config.NoMasqLocal {
-		ips := ipset.New(common.LogLogger(), 0)
-		_ = ips.Destroy(NoMasqLocalIpset)
-		if err := ips.Create(NoMasqLocalIpset, ipset.HashNet); err != nil {
-			return err
-		}
-		if err := ipt.Insert("nat", "WEAVE", 1,
+		if err := ipt.AppendUnique("nat", "WEAVE",
 			"-m", "set", "--match-set", string(NoMasqLocalIpset), "dst",
 			"-m", "comment", "--comment", "Prevent SNAT to locally running containers",
 			"-j", "RETURN"); err != nil {
@@ -547,13 +586,13 @@ func configureIPTables(config *BridgeConfig, ips ipset.Interface) error {
 
 type NoMasqLocalTracker struct {
 	ips   ipset.Interface
-	owner types.UID
+	owner ipset.UID
 }
 
 func NewNoMasqLocalTracker(ips ipset.Interface) *NoMasqLocalTracker {
 	return &NoMasqLocalTracker{
 		ips:   ips,
-		owner: types.UID(0), // dummy ipset owner
+		owner: ipset.UID("0"), // dummy ipset owner
 	}
 }
 
@@ -566,9 +605,9 @@ func (t *NoMasqLocalTracker) HandleUpdate(prevRanges, currRanges []address.Range
 		return nil
 	}
 
-	prev, curr := tracker.RemoveCommon(
-		address.NewCIDRs(tracker.Merge(prevRanges)),
-		address.NewCIDRs(tracker.Merge(currRanges)))
+	prev, curr := address.RemoveCommon(
+		address.NewCIDRs(address.Merge(prevRanges)),
+		address.NewCIDRs(address.Merge(currRanges)))
 
 	for _, cidr := range curr {
 		if err := t.ips.AddEntry(t.owner, NoMasqLocalIpset, cidr.String(), ""); err != nil {
@@ -592,7 +631,7 @@ func linkSetUpByName(linkName string) error {
 	return netlink.LinkSetUp(link)
 }
 
-func reexpose(config *BridgeConfig, log *logrus.Logger) error {
+func Reexpose(config *BridgeConfig, log *logrus.Logger) error {
 	// Get existing IP addrs of the weave bridge.
 	// If the bridge hasn't been exposed, then this functions does nothing.
 	//

net/expose.go

@@ -103,13 +103,8 @@ func addNatRule(ipt *iptables.IPTables, rulespec ...string) error {
 	// Loop until we get an exit code other than "temporarily unavailable"
 	for {
 		if err := ipt.AppendUnique("nat", "WEAVE", rulespec...); err != nil {
-			if ierr, ok := err.(*iptables.Error); ok {
-				if status, ok := ierr.ExitError.Sys().(syscall.WaitStatus); ok {
-					// (magic exit code 4 found in iptables source code; undocumented)
-					if status.ExitStatus() == 4 {
-						continue
-					}
-				}
+			if isResourceError(err) {
+				continue
 			}
 			return err
 		}

net/if.go

@@ -21,7 +21,7 @@ func ensureInterface(ifaceName string) (*net.Interface, error) {
 	ch := make(chan netlink.LinkUpdate)
 	// NB: We do not supply (and eventually close) a 'done' channel
 	// here since that can cause incorrect file descriptor
-	// re-usage. See https://github.com/weaveworks/weave/issues/2120
+	// re-usage. See https://github.com/rajch/weave/issues/2120
 	if err := netlink.LinkSubscribe(ch, nil); err != nil {
 		return nil, err
 	}

net/ipsec/ipsec.go

@@ -85,7 +85,7 @@ func New(log *logrus.Logger) (*IPSec, error) {
 
 // InitSALocal initializes inbound ipsec from remotePeer and triggers
 // the initialization on remotePeer.
-func (ipsec *IPSec) InitSALocal(localPeer, remotePeer mesh.PeerName, connUID uint64, localIP, remoteIP net.IP, udpPort int, sessionKey *[32]byte, initRemote func([]byte) error) error {
+func (ipsec *IPSec) InitSALocal(localPeer, remotePeer mesh.PeerName, connUID uint64, localIP, remoteIP net.IP, udpPort int, sessionKey *[32]byte) ([]byte, error) {
 	// ID of inbound SPI
 	spiID := getSPIId(remotePeer, localPeer, connUID)
 
@@ -95,17 +95,17 @@ func (ipsec *IPSec) InitSALocal(localPeer, remotePeer mesh.PeerName, connUID uin
 	// Derive SA key
 	nonce, err := genNonce()
 	if err != nil {
-		return errors.Wrap(err, "generate nonce")
+		return nil, errors.Wrap(err, "generate nonce")
 	}
 	key, err := deriveKey(sessionKey[:], nonce, localPeer)
 	if err != nil {
-		return errors.Wrap(err, "derive key")
+		return nil, errors.Wrap(err, "derive key")
 	}
 
 	// Allocate SA
 	sa, err := netlink.XfrmStateAllocSpi(xfrmAllocSpiState(remoteIP, localIP))
 	if err != nil {
-		return errors.Wrap(err, fmt.Sprintf("ip xfrm state allocspi (in, %s, %s)", remoteIP, localIP))
+		return nil, errors.Wrap(err, fmt.Sprintf("ip xfrm state allocspi (in, %s, %s)", remoteIP, localIP))
 	}
 
 	// Use SPI generated by the kernel. The kernel ensures {dstIP, spi} to
@@ -117,28 +117,24 @@ func (ipsec *IPSec) InitSALocal(localPeer, remotePeer mesh.PeerName, connUID uin
 	// Create SA
 	if sa, err := xfrmState(remoteIP, localIP, spi, false, key); err == nil {
 		if err := netlink.XfrmStateUpdate(sa); err != nil {
-			return errors.Wrap(err, fmt.Sprintf("xfrm state update (in, %s, %s, 0x%x)", sa.Src, sa.Dst, sa.Spi))
+			return nil, errors.Wrap(err, fmt.Sprintf("xfrm state update (in, %s, %s, 0x%x)", sa.Src, sa.Dst, sa.Spi))
 		}
 	} else {
-		return errors.Wrap(err, "new xfrm state (in)")
+		return nil, errors.Wrap(err, "new xfrm state (in)")
 	}
 
 	// Install iptables rules
 	if err := ipsec.installDropNonEncrypted(localIP, remoteIP, udpPort, spi); err != nil {
-		return errors.Wrap(err, fmt.Sprintf("install protecting rules (%s, %s, %d, 0x%x)", localIP, remoteIP, udpPort, spi))
+		return nil, errors.Wrap(err, fmt.Sprintf("install protecting rules (%s, %s, %d, 0x%x)", localIP, remoteIP, udpPort, spi))
 	}
 
 	si := spiInfo{spi: spi, isDirOut: false}
 	ipsec.spiInfo[spiID] = si
 	ipsec.spis[spi] = &si
 
-	// Trigger the initialization on the remote peer
+	// Generate the message to trigger initialization on the remote peer
 	msg := &msgInitSARemote{nonce, spi}
-	if err := initRemote(msg.serialize()); err != nil {
-		return errors.Wrap(err, "send InitSARemote")
-	}
-
-	return nil
+	return msg.serialize(), nil
 }
 
 // InitSARemote initializes outbound ipsec to remotePeer.
@@ -223,9 +219,19 @@ func (ipsec *IPSec) Destroy(localPeer, remotePeer mesh.PeerName, connUID uint64,
 
 	if outSPIInfo, ok := ipsec.spiInfo[outSPIID]; ok {
 		ipsec.log.Infof("ipsec: destroy: out %s -> %s 0x%x", localIP, remoteIP, outSPIInfo.spi)
-
-		if err := netlink.XfrmPolicyDel(xfrmPolicy(localIP, remoteIP, outSPIInfo.spi)); err != nil {
-			ipsec.log.Warnf("ipsec: xfrm policy del (%s, %s, 0x%x) failed: %s", localIP, remoteIP, outSPIInfo.spi, err)
+		policy, err := netlink.XfrmPolicyGet(xfrmPolicy(localIP, remoteIP, outSPIInfo.spi))
+		if err != nil {
+			ipsec.log.Warnf("ipsec: xfrm policy get (%s, %s, 0x%x) failed: %s", localIP, remoteIP, outSPIInfo.spi, err)
+		} else {
+			if len(policy.Tmpls) == 1 {
+				if policy.Tmpls[0].Spi == int(outSPIInfo.spi) {
+					if err := netlink.XfrmPolicyDel(xfrmPolicy(localIP, remoteIP, outSPIInfo.spi)); err != nil {
+						ipsec.log.Warnf("ipsec: xfrm policy del (%s, %s, 0x%x) failed: %s", localIP, remoteIP, outSPIInfo.spi, err)
+					}
+				} else {
+					ipsec.log.Debugf("ipsec: xfrm not my policy (%s, %s, 0x%x) got 0x%x ", localIP, remoteIP, outSPIInfo.spi, policy.Tmpls[0].Spi)
+				}
+			}
 		}
 
 		outSA := &netlink.XfrmState{
@@ -424,14 +430,6 @@ func (ipsec *IPSec) removeDropNonEncrypted(srcIP, dstIP net.IP, udpPort int, inS
 	return ipsec.resetRules(rules, true)
 }
 
-func (ipsec *IPSec) removeDropNonEncryptedInbound(srcIP, dstIP net.IP, inSPI SPI) error {
-	r := ruleMarkInboundESP(srcIP, dstIP, inSPI)
-	if err := ipsec.ipt.Delete(r.table, r.chain, r.rulespec...); err != nil {
-		return errors.Wrap(err, fmt.Sprintf("iptables delete unique (%s, %s, %s)", r.table, r.chain, r.rulespec))
-	}
-	return nil
-}
-
 // xfrm
 
 func xfrmAllocSpiState(srcIP, dstIP net.IP) *netlink.XfrmState {

net/ipset/ipset.go

@@ -7,26 +7,31 @@ import (
 	"log"
 	"os/exec"
 	"strings"
+	"time"
 
 	"github.com/pkg/errors"
-	"k8s.io/apimachinery/pkg/types"
 )
 
 type Name string
 
 type Type string
 
+type UID string
+
 const (
 	ListSet = Type("list:set")
 	HashIP  = Type("hash:ip")
 	HashNet = Type("hash:net")
+
+	DestroyRetrySleepMs = 100
 )
 
 type Interface interface {
 	Create(ipsetName Name, ipsetType Type) error
-	AddEntry(user types.UID, ipsetName Name, entry string, comment string) error
-	DelEntry(user types.UID, ipsetName Name, entry string) error
-	Exist(user types.UID, ipsetName Name, entry string) bool
+	AddEntry(user UID, ipsetName Name, entry string, comment string) error
+	DelEntry(user UID, ipsetName Name, entry string) error
+	EntryExists(user UID, ipsetName Name, entry string) bool
+	Exists(ipsetName Name) (bool, error)
 	Flush(ipsetName Name) error
 	Destroy(ipsetName Name) error
 
@@ -48,8 +53,8 @@ type ipset struct {
 	// List of users per ipset entry. User is either a namespace or a pod.
 	// There might be multiple users for the same ipset & entry pair because
 	// events from k8s API server might be out of order causing duplicate IPs:
-	// https://github.com/weaveworks/weave/issues/2792.
-	users map[entryKey]map[types.UID]struct{}
+	// https://github.com/rajch/weave/issues/2792.
+	users map[entryKey]map[UID]struct{}
 }
 
 func New(logger *log.Logger, maxListSize int) Interface {
@@ -57,7 +62,7 @@ func New(logger *log.Logger, maxListSize int) Interface {
 		Logger:         logger,
 		enableComments: true,
 		maxListSize:    maxListSize,
-		users:          make(map[entryKey]map[types.UID]struct{}),
+		users:          make(map[entryKey]map[UID]struct{}),
 	}
 
 	// Check for comment support
@@ -93,7 +98,7 @@ func (i *ipset) Create(ipsetName Name, ipsetType Type) error {
 	return doExec(args...)
 }
 
-func (i *ipset) AddEntry(user types.UID, ipsetName Name, entry string, comment string) error {
+func (i *ipset) AddEntry(user UID, ipsetName Name, entry string, comment string) error {
 	i.Logger.Printf("adding entry %s to %s of %s", entry, ipsetName, user)
 
 	if !i.addUser(user, ipsetName, entry) { // already in the set
@@ -109,7 +114,7 @@ func (i *ipset) AddEntry(user types.UID, ipsetName Name, entry string, comment s
 	return doExec(args...)
 }
 
-func (i *ipset) DelEntry(user types.UID, ipsetName Name, entry string) error {
+func (i *ipset) DelEntry(user UID, ipsetName Name, entry string) error {
 	i.Logger.Printf("deleting entry %s from %s of %s", entry, ipsetName, user)
 
 	if !i.delUser(user, ipsetName, entry) { // still needed
@@ -121,28 +126,52 @@ func (i *ipset) DelEntry(user types.UID, ipsetName Name, entry string) error {
 	return doExec("del", string(ipsetName), entry)
 }
 
-func (i *ipset) Exist(user types.UID, ipsetName Name, entry string) bool {
+func (i *ipset) EntryExists(user UID, ipsetName Name, entry string) bool {
 	return i.existUser(user, ipsetName, entry)
 }
 
+// Dummy way to check whether a given ipset exists.
+func (i *ipset) Exists(name Name) (bool, error) {
+	sets, err := i.List(string(name))
+	if err != nil {
+		return false, err
+	}
+	for _, s := range sets {
+		if s == name {
+			return true, nil
+		}
+	}
+	return false, nil
+}
+
 func (i *ipset) Flush(ipsetName Name) error {
 	i.removeSetFromUsers(ipsetName)
 	return doExec("flush", string(ipsetName))
 }
 
 func (i *ipset) FlushAll() error {
-	i.users = make(map[entryKey]map[types.UID]struct{})
+	i.users = make(map[entryKey]map[UID]struct{})
 	return doExec("flush")
 }
 
 func (i *ipset) Destroy(ipsetName Name) error {
 	i.removeSetFromUsers(ipsetName)
-	return doExec("destroy", string(ipsetName))
+	err := doExec("destroy", string(ipsetName))
+	if err != nil {
+		time.Sleep(DestroyRetrySleepMs * time.Millisecond)
+		return doExec("destroy", string(ipsetName))
+	}
+	return err
 }
 
 func (i *ipset) DestroyAll() error {
-	i.users = make(map[entryKey]map[types.UID]struct{})
-	return doExec("destroy")
+	i.users = make(map[entryKey]map[UID]struct{})
+	err := doExec("destroy")
+	if err != nil {
+		time.Sleep(DestroyRetrySleepMs * time.Millisecond)
+		return doExec("destroy")
+	}
+	return err
 }
 
 // Fetch a list of all existing sets with a given prefix
@@ -164,12 +193,12 @@ func (i *ipset) List(prefix string) ([]Name, error) {
 }
 
 // Returns true if entry does not exist in ipset (entry has to be inserted into ipset).
-func (i *ipset) addUser(user types.UID, ipsetName Name, entry string) bool {
+func (i *ipset) addUser(user UID, ipsetName Name, entry string) bool {
 	k := entryKey{ipsetName, entry}
 	add := false
 
 	if i.users[k] == nil {
-		i.users[k] = make(map[types.UID]struct{})
+		i.users[k] = make(map[UID]struct{})
 	}
 	if len(i.users[k]) == 0 {
 		add = true
@@ -180,7 +209,7 @@ func (i *ipset) addUser(user types.UID, ipsetName Name, entry string) bool {
 }
 
 // Returns true if user is the last owner of entry (entry has to be removed from ipset).
-func (i *ipset) delUser(user types.UID, ipsetName Name, entry string) bool {
+func (i *ipset) delUser(user UID, ipsetName Name, entry string) bool {
 	k := entryKey{ipsetName, entry}
 
 	oneLeft := len(i.users[k]) == 1
@@ -193,7 +222,7 @@ func (i *ipset) delUser(user types.UID, ipsetName Name, entry string) bool {
 	return oneLeft && (len(i.users[k]) == 0)
 }
 
-func (i *ipset) existUser(user types.UID, ipsetName Name, entry string) bool {
+func (i *ipset) existUser(user UID, ipsetName Name, entry string) bool {
 	_, ok := i.users[entryKey{ipsetName, entry}][user]
 	return ok
 }

net/iptables.go

@@ -2,9 +2,13 @@ package net
 
 import (
 	"strings"
+	"syscall"
+	"time"
 
 	"github.com/coreos/go-iptables/iptables"
 	"github.com/pkg/errors"
+	"github.com/sirupsen/logrus"
+	utilwait "k8s.io/apimachinery/pkg/util/wait"
 )
 
 // AddChainWithRules creates a chain and appends given rules to it.
@@ -104,3 +108,119 @@ func ensureRulesAtTop(table, chain string, rulespecs [][]string, ipt *iptables.I
 
 	return nil
 }
+
+func chainExists(ipt *iptables.IPTables, table string, chain string) (bool, error) {
+	existingChains, err := ipt.ListChains(table)
+	if err != nil {
+		return false, errors.Wrapf(err, "ipt.ListChains(%s)", table)
+	}
+	chainMap := make(map[string]struct{})
+	for _, c := range existingChains {
+		chainMap[c] = struct{}{}
+	}
+
+	_, found := chainMap[chain]
+	return found, nil
+}
+
+const (
+	// Max time we wait for an iptables flush to complete after we notice it has started
+	iptablesFlushTimeout = 5 * time.Second
+	// How often we poll while waiting for an iptables flush to complete
+	iptablesFlushPollTime = 100 * time.Millisecond
+)
+
+// MonitorForIptablesFlush periodically checks for a canary chain in iptables. If this canary chain goes missing it calls the reloadFunc.
+// This is a more efficient way of detecting whether firewalld or another process has been removing rules that we rely on.
+// The reloadFunc can then check whether other chains that should exist are still there, fix things and restore the canary.
+func MonitorForIptablesFlush(log *logrus.Logger, canary string, tables []string, reloadFunc func(), interval time.Duration, stopCh <-chan struct{}) {
+	ipt, err := iptables.New()
+	if err != nil {
+		log.Errorf("creating iptables object while initializing iptable Monitoring %s", err)
+		return
+	}
+
+	for {
+		_ = PollImmediateUntil(interval, func() (bool, error) {
+			for _, table := range tables {
+				if err := ensureChains(ipt, table, canary); err != nil {
+					log.Warningf("Could not set up iptables canary %s/%s: %v", table, canary, err)
+					return false, nil
+				}
+			}
+			return true, nil
+		}, stopCh)
+
+		// Poll until stopCh is closed or iptables is flushed
+		err = utilwait.PollUntil(interval, func() (bool, error) {
+			if exists, err := chainExists(ipt, tables[0], canary); exists {
+				return false, nil
+			} else if isResourceError(err) {
+				log.Warningf("Could not check for iptables canary %s/%s: %v", tables[0], canary, err)
+				return false, nil
+			}
+			log.Infof("iptables canary %s/%s deleted", tables[0], canary)
+
+			// Wait for the other canaries to be deleted too before returning
+			// so we don't start reloading too soon.
+			err := utilwait.PollImmediate(iptablesFlushPollTime, iptablesFlushTimeout, func() (bool, error) {
+				for i := 1; i < len(tables); i++ {
+					if exists, err := chainExists(ipt, tables[i], canary); exists || isResourceError(err) {
+						return false, nil
+					}
+				}
+				return true, nil
+			})
+			if err != nil {
+				log.Warning("Inconsistent iptables state detected.")
+			}
+			return true, nil
+		}, stopCh)
+
+		if err != nil {
+			// stopCh was closed
+			for _, table := range tables {
+				_ = ipt.DeleteChain(table, canary)
+			}
+			return
+		}
+
+		log.Infof("Reloading after iptables flush")
+		reloadFunc()
+	}
+}
+
+const iptablesStatusResourceProblem = 4
+
+// isResourceError returns true if the error indicates that iptables ran into a "resource
+// problem" and was unable to attempt the request. In particular, this will be true if it
+// times out trying to get the iptables lock.
+func isResourceError(err error) bool {
+	if ierr, ok := err.(*iptables.Error); ok {
+		if status, ok := ierr.ExitError.Sys().(syscall.WaitStatus); ok {
+			return status.ExitStatus() == iptablesStatusResourceProblem
+		}
+	}
+
+	return false
+}
+
+// PollImmediateUntil tries a condition func until it returns true, an error or stopCh is closed.
+//
+// PollImmediateUntil runs the 'condition' before waiting for the interval.
+// 'condition' will always be invoked at least once.
+func PollImmediateUntil(interval time.Duration, condition utilwait.ConditionFunc, stopCh <-chan struct{}) error {
+	done, err := condition()
+	if err != nil {
+		return err
+	}
+	if done {
+		return nil
+	}
+	select {
+	case <-stopCh:
+		return utilwait.ErrWaitTimeout
+	default:
+		return utilwait.PollUntil(interval, condition, stopCh)
+	}
+}

net/netns.go

@@ -55,9 +55,9 @@ func WithNetNSByPath(path string, work func() error) error {
 }
 
 func NSPathByPid(pid int) string {
-	return NSPathByPidWithRoot("/", pid)
+	return NSPathByPidWithProc("/proc", pid)
 }
 
-func NSPathByPidWithRoot(root string, pid int) string {
-	return filepath.Join(root, fmt.Sprintf("/proc/%d/ns/net", pid))
+func NSPathByPidWithProc(procPath string, pid int) string {
+	return filepath.Join(procPath, fmt.Sprint(pid), "/ns/net")
 }

net/uuid.go

@@ -6,8 +6,8 @@ import (
 	"net"
 	"os"
 
+	"github.com/rajch/weave/db"
 	"github.com/weaveworks/mesh"
-	"github.com/weaveworks/weave/db"
 )
 
 func getOldStyleSystemUUID() ([]byte, error) {

net/veth.go

@@ -13,7 +13,7 @@ import (
 )
 
 // create and attach a veth to the Weave bridge
-func CreateAndAttachVeth(name, peerName, bridgeName string, mtu int, keepTXOn bool, errIfLinkExist bool, init func(peer netlink.Link) error) (*netlink.Veth, error) {
+func CreateAndAttachVeth(procPath, name, peerName, bridgeName string, mtu int, keepTXOn bool, errIfLinkExist bool, init func(peer netlink.Link) error) (*netlink.Veth, error) {
 	bridge, err := netlink.LinkByName(bridgeName)
 	if err != nil {
 		return nil, fmt.Errorf(`bridge "%s" not present; did you launch weave?`, bridgeName)
@@ -49,6 +49,13 @@ func CreateAndAttachVeth(name, peerName, bridgeName string, mtu int, keepTXOn bo
 	if err := bridgeType.attach(veth); err != nil {
 		return cleanup("attaching veth %q to %q: %s", name, bridgeName, err)
 	}
+	// No ipv6 router advertisments please
+	if err := sysctlIfExists(procPath, "net/ipv6/conf/"+name+"/accept_ra", "0"); err != nil {
+		return cleanup("setting accept_ra to 0: %s", err)
+	}
+	if err := sysctlIfExists(procPath, "net/ipv6/conf/"+peerName+"/accept_ra", "0"); err != nil {
+		return cleanup("setting accept_ra to 0: %s", err)
+	}
 	if !bridgeType.IsFastdp() && !keepTXOn {
 		if err := EthtoolTXOff(veth.PeerName); err != nil {
 			return cleanup(`unable to set tx off on %q: %s`, peerName, err)
@@ -112,6 +119,9 @@ func interfaceExistsInNamespace(netNSPath string, ifName string) bool {
 }
 
 func AttachContainer(netNSPath, id, ifName, bridgeName string, mtu int, withMulticastRoute bool, cidrs []*net.IPNet, keepTXOn bool, hairpinMode bool) error {
+	// AttachContainer expects to be called in host pid namespace
+	const procPath = "/proc"
+
 	ns, err := netns.GetFromPath(netNSPath)
 	if err != nil {
 		return err
@@ -124,12 +134,12 @@ func AttachContainer(netNSPath, id, ifName, bridgeName string, mtu int, withMult
 			id = id[:maxIDLen] // trim passed ID if too long
 		}
 		name, peerName := vethPrefix+"pl"+id, vethPrefix+"pg"+id
-		veth, err := CreateAndAttachVeth(name, peerName, bridgeName, mtu, keepTXOn, true, func(veth netlink.Link) error {
+		veth, err := CreateAndAttachVeth(procPath, name, peerName, bridgeName, mtu, keepTXOn, true, func(veth netlink.Link) error {
 			if err := netlink.LinkSetNsFd(veth, int(ns)); err != nil {
 				return fmt.Errorf("failed to move veth to container netns: %s", err)
 			}
 			if err := WithNetNS(ns, func() error {
-				return setupIface(peerName, ifName)
+				return setupIface(procPath, peerName, ifName)
 			}); err != nil {
 				return fmt.Errorf("error setting up interface: %s", err)
 			}
@@ -206,7 +216,7 @@ func setupIfaceAddrs(veth netlink.Link, withMulticastRoute bool, cidrs []*net.IP
 }
 
 // setupIface expects to be called in the container's netns
-func setupIface(ifaceName, newIfName string) error {
+func setupIface(procPath, ifaceName, newIfName string) error {
 	ipt, err := iptables.New()
 	if err != nil {
 		return err
@@ -219,15 +229,14 @@ func setupIface(ifaceName, newIfName string) error {
 	if err := netlink.LinkSetName(link, newIfName); err != nil {
 		return err
 	}
-	// This is only called by AttachContainer which is only called in host pid namespace
-	if err := configureARPCache("/proc", newIfName); err != nil {
+	if err := configureARPCache(procPath, newIfName); err != nil {
 		return err
 	}
 	return ipt.Append("filter", "INPUT", "-i", newIfName, "-d", "224.0.0.0/4", "-j", "DROP")
 }
 
 // configureARP is a helper for the Docker plugin which doesn't set the addresses itself
-func ConfigureARP(prefix, rootPath string) error {
+func ConfigureARP(prefix, procPath string) error {
 	links, err := netlink.LinkList()
 	if err != nil {
 		return err
@@ -235,7 +244,7 @@ func ConfigureARP(prefix, rootPath string) error {
 	for _, link := range links {
 		ifName := link.Attrs().Name
 		if strings.HasPrefix(ifName, prefix) {
-			configureARPCache(rootPath+"/proc", ifName)
+			configureARPCache(procPath, ifName)
 			if addrs, err := netlink.AddrList(link, netlink.FAMILY_V4); err == nil {
 				for _, addr := range addrs {
 					arping.GratuitousArpOverIfaceByName(addr.IPNet.IP, ifName)

npc/analyser.go

@@ -7,17 +7,19 @@ import (
 	networkingv1 "k8s.io/api/networking/v1"
 	"k8s.io/apimachinery/pkg/util/intstr"
 
-	"github.com/weaveworks/weave/net/ipset"
+	"github.com/rajch/weave/net/ipset"
 )
 
 func (ns *ns) analysePolicy(policy *networkingv1.NetworkPolicy) (
 	rules map[string]*ruleSpec,
-	nsSelectors, podSelectors map[string]*selectorSpec,
+	nsSelectors, podSelectors, namespacedPodSelectors map[string]*selectorSpec,
 	ipBlocks map[string]*ipBlockSpec,
 	err error) {
 
 	nsSelectors = make(map[string]*selectorSpec)
 	podSelectors = make(map[string]*selectorSpec)
+	namespacedPodSelectors = make(map[string]*selectorSpec)
+
 	ipBlocks = make(map[string]*ipBlockSpec)
 	rules = make(map[string]*ruleSpec)
 	policyTypes := make([]policyType, 0)
@@ -31,9 +33,9 @@ func (ns *ns) analysePolicy(policy *networkingv1.NetworkPolicy) (
 		}
 	}
 	// If empty, matches all pods in a namespace
-	targetSelector, err := newSelectorSpec(&policy.Spec.PodSelector, policyTypes, ns.name, ipset.HashIP)
+	targetSelector, err := newSelectorSpec(&policy.Spec.PodSelector, nil, policyTypes, ns.name, ipset.HashIP)
 	if err != nil {
-		return nil, nil, nil, nil, err
+		return nil, nil, nil, nil, nil, err
 	}
 
 	// To prevent targetSelector being overwritten by a subsequent selector with
@@ -52,7 +54,7 @@ func (ns *ns) analysePolicy(policy *networkingv1.NetworkPolicy) (
 
 			if !allPorts {
 				if err = checkForNamedPorts(ingressRule.Ports); err != nil {
-					return nil, nil, nil, nil, fmt.Errorf("named ports in network policies is not supported yet. "+
+					return nil, nil, nil, nil, nil, fmt.Errorf("named ports in network policies is not supported yet. "+
 						"Rejecting network policy: %s from further processing. "+err.Error(), policy.Name)
 				}
 			}
@@ -72,18 +74,24 @@ func (ns *ns) analysePolicy(policy *networkingv1.NetworkPolicy) (
 					var srcRuleHost ruleHost
 
 					// NetworkPolicyPeer describes a peer to allow traffic from.
-					// Exactly one of its fields must be specified.
-					if peer.PodSelector != nil {
-						srcSelector, err = newSelectorSpec(peer.PodSelector, nil, ns.name, ipset.HashIP)
+					if peer.PodSelector != nil && peer.NamespaceSelector != nil {
+						srcSelector, err = newSelectorSpec(peer.PodSelector, peer.NamespaceSelector, nil, "", ipset.HashIP)
 						if err != nil {
-							return nil, nil, nil, nil, err
+							return nil, nil, nil, nil, nil, err
+						}
+						addIfNotExist(srcSelector, namespacedPodSelectors)
+						srcRuleHost = srcSelector
+					} else if peer.PodSelector != nil {
+						srcSelector, err = newSelectorSpec(peer.PodSelector, nil, nil, ns.name, ipset.HashIP)
+						if err != nil {
+							return nil, nil, nil, nil, nil, err
 						}
 						addIfNotExist(srcSelector, podSelectors)
 						srcRuleHost = srcSelector
 					} else if peer.NamespaceSelector != nil {
-						srcSelector, err = newSelectorSpec(peer.NamespaceSelector, nil, "", ipset.ListSet)
+						srcSelector, err = newSelectorSpec(nil, peer.NamespaceSelector, nil, "", ipset.ListSet)
 						if err != nil {
-							return nil, nil, nil, nil, err
+							return nil, nil, nil, nil, nil, err
 						}
 						nsSelectors[srcSelector.key] = srcSelector
 						srcRuleHost = srcSelector
@@ -98,7 +106,7 @@ func (ns *ns) analysePolicy(policy *networkingv1.NetworkPolicy) (
 						rules[rule.key] = rule
 					} else {
 						withNormalisedProtoAndPort(ingressRule.Ports, func(proto, port string) {
-							rule := newRuleSpec(policyTypeIngress, &proto, srcSelector, targetSelector, &port)
+							rule := newRuleSpec(policyTypeIngress, &proto, srcRuleHost, targetSelector, &port)
 							rules[rule.key] = rule
 						})
 					}
@@ -117,7 +125,7 @@ func (ns *ns) analysePolicy(policy *networkingv1.NetworkPolicy) (
 
 			if !allPorts {
 				if err = checkForNamedPorts(egressRule.Ports); err != nil {
-					return nil, nil, nil, nil, fmt.Errorf("named ports in network policies is not supported yet. "+
+					return nil, nil, nil, nil, nil, fmt.Errorf("named ports in network policies is not supported yet. "+
 						"Rejecting network policy: %s from further processing. "+err.Error(), policy.Name)
 				}
 			}
@@ -137,19 +145,25 @@ func (ns *ns) analysePolicy(policy *networkingv1.NetworkPolicy) (
 					var dstRuleHost ruleHost
 
 					// NetworkPolicyPeer describes a peer to allow traffic to.
-					// Exactly one of its fields must be specified.
-					if peer.PodSelector != nil {
-						dstSelector, err = newSelectorSpec(peer.PodSelector, nil, ns.name, ipset.HashIP)
+					if peer.PodSelector != nil && peer.NamespaceSelector != nil {
+						dstSelector, err = newSelectorSpec(peer.PodSelector, peer.NamespaceSelector, nil, "", ipset.HashIP)
 						if err != nil {
-							return nil, nil, nil, nil, err
+							return nil, nil, nil, nil, nil, err
+						}
+						addIfNotExist(dstSelector, namespacedPodSelectors)
+						dstRuleHost = dstSelector
+					} else if peer.PodSelector != nil {
+						dstSelector, err = newSelectorSpec(peer.PodSelector, nil, nil, ns.name, ipset.HashIP)
+						if err != nil {
+							return nil, nil, nil, nil, nil, err
 						}
 						addIfNotExist(dstSelector, podSelectors)
 						dstRuleHost = dstSelector
 
 					} else if peer.NamespaceSelector != nil {
-						dstSelector, err = newSelectorSpec(peer.NamespaceSelector, nil, "", ipset.ListSet)
+						dstSelector, err = newSelectorSpec(nil, peer.NamespaceSelector, nil, "", ipset.ListSet)
 						if err != nil {
-							return nil, nil, nil, nil, err
+							return nil, nil, nil, nil, nil, err
 						}
 						nsSelectors[dstSelector.key] = dstSelector
 						dstRuleHost = dstSelector
@@ -173,7 +187,7 @@ func (ns *ns) analysePolicy(policy *networkingv1.NetworkPolicy) (
 		}
 	}
 
-	return rules, nsSelectors, podSelectors, ipBlocks, nil
+	return rules, nsSelectors, podSelectors, namespacedPodSelectors, ipBlocks, nil
 }
 
 func addIfNotExist(s *selectorSpec, ss map[string]*selectorSpec) {

npc/constants.go

@@ -2,16 +2,7 @@ package npc
 
 const (
 	TableFilter = "filter"
-
-	MainChain    = "WEAVE-NPC"
-	DefaultChain = "WEAVE-NPC-DEFAULT"
-	IngressChain = "WEAVE-NPC-INGRESS"
-
-	EgressChain        = "WEAVE-NPC-EGRESS"
-	EgressDefaultChain = "WEAVE-NPC-EGRESS-DEFAULT"
-	EgressCustomChain  = "WEAVE-NPC-EGRESS-CUSTOM"
-	EgressMarkChain    = "WEAVE-NPC-EGRESS-ACCEPT"
-	EgressMark         = "0x40000/0x40000"
+	EgressMark  = "0x40000/0x40000"
 
 	IpsetNamePrefix = "weave-"
 

npc/controller.go

@@ -1,30 +1,35 @@
 package npc
 
 import (
+	"context"
+	"fmt"
 	"sync"
 
 	"github.com/pkg/errors"
 	coreapi "k8s.io/api/core/v1"
 	extnapi "k8s.io/api/extensions/v1beta1"
 	networkingv1 "k8s.io/api/networking/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/client-go/kubernetes"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/ipset"
-	"github.com/weaveworks/weave/npc/iptables"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/common/chains"
+	"github.com/rajch/weave/net/ipset"
+	"github.com/rajch/weave/npc/iptables"
 )
 
 type NetworkPolicyController interface {
-	AddNamespace(ns *coreapi.Namespace) error
-	UpdateNamespace(oldObj, newObj *coreapi.Namespace) error
-	DeleteNamespace(ns *coreapi.Namespace) error
+	AddNamespace(ctx context.Context, ns *coreapi.Namespace) error
+	UpdateNamespace(ctx context.Context, oldObj, newObj *coreapi.Namespace) error
+	DeleteNamespace(ctx context.Context, ns *coreapi.Namespace) error
 
-	AddPod(obj *coreapi.Pod) error
-	UpdatePod(oldObj, newObj *coreapi.Pod) error
-	DeletePod(obj *coreapi.Pod) error
+	AddPod(ctx context.Context, obj *coreapi.Pod) error
+	UpdatePod(ctx context.Context, oldObj, newObj *coreapi.Pod) error
+	DeletePod(ctx context.Context, obj *coreapi.Pod) error
 
-	AddNetworkPolicy(obj interface{}) error
-	UpdateNetworkPolicy(oldObj, newObj interface{}) error
-	DeleteNetworkPolicy(obj interface{}) error
+	AddNetworkPolicy(ctx context.Context, obj interface{}) error
+	UpdateNetworkPolicy(ctx context.Context, oldObj, newObj interface{}) error
+	DeleteNetworkPolicy(ctx context.Context, obj interface{}) error
 }
 
 type controller struct {
@@ -32,31 +37,34 @@ type controller struct {
 
 	nodeName string // my node name
 
-	ipt iptables.Interface
-	ips ipset.Interface
-
-	nss         map[string]*ns // ns name -> ns struct
-	nsSelectors *selectorSet   // selector string -> nsSelector
+	ipt                    iptables.Interface
+	ips                    ipset.Interface
+	clientset              kubernetes.Interface
+	nss                    map[string]*ns // ns name -> ns struct
+	nsSelectors            *selectorSet   // selector string -> nsSelector
+	namespacedPodSelectors *selectorSet
+	defaultEgressDrop      bool // flag to track if base iptable rule to drop egress traffic is added or not
 }
 
-func New(nodeName string, ipt iptables.Interface, ips ipset.Interface) NetworkPolicyController {
+func New(nodeName string, ipt iptables.Interface, ips ipset.Interface, clientset kubernetes.Interface) NetworkPolicyController {
 	c := &controller{
-		nodeName: nodeName,
-		ipt:      ipt,
-		ips:      ips,
-		nss:      make(map[string]*ns)}
+		nodeName:  nodeName,
+		ipt:       ipt,
+		ips:       ips,
+		clientset: clientset,
+		nss:       make(map[string]*ns)}
 
 	doNothing := func(*selector, policyType) error { return nil }
 	c.nsSelectors = newSelectorSet(ips, c.onNewNsSelector, doNothing, doNothing)
-
+	c.namespacedPodSelectors = newSelectorSet(ips, c.onNewNamespacePodsSelector, doNothing, doNothing)
 	return c
 }
 
 func (npc *controller) onNewNsSelector(selector *selector) error {
 	for _, ns := range npc.nss {
-		if ns.namespace != nil {
-			if selector.matches(ns.namespace.ObjectMeta.Labels) {
-				if err := selector.addEntry(ns.namespace.ObjectMeta.UID, string(ns.allPods.ipsetName), namespaceComment(ns)); err != nil {
+		if ns.namespaceUID != "" {
+			if selector.matchesNamespaceSelector(ns.namespaceLabels) {
+				if err := selector.addEntry(ns.namespaceUID, string(ns.allPods.ipsetName), namespaceComment(ns)); err != nil {
 					return err
 				}
 			}
@@ -65,10 +73,32 @@ func (npc *controller) onNewNsSelector(selector *selector) error {
 	return nil
 }
 
-func (npc *controller) withNS(name string, f func(ns *ns) error) error {
+func (npc *controller) onNewNamespacePodsSelector(selector *selector) error {
+	for _, ns := range npc.nss {
+		if ns.namespaceUID != "" && len(ns.pods) > 0 {
+			for _, pod := range ns.pods {
+				if hasIP(pod) {
+					if selector.matchesNamespacedPodSelector(pod.ObjectMeta.Labels, ns.namespaceLabels) {
+						if err := selector.addEntry(uid(pod), pod.Status.PodIP, podComment(pod)); err != nil {
+							return err
+						}
+
+					}
+				}
+			}
+		}
+	}
+	return nil
+}
+
+func (npc *controller) withNS(ctx context.Context, name string, f func(ns *ns) error) error {
 	ns, found := npc.nss[name]
 	if !found {
-		newNs, err := newNS(name, npc.nodeName, npc.ipt, npc.ips, npc.nsSelectors)
+		namespace, err := npc.clientset.CoreV1().Namespaces().Get(ctx, name, metav1.GetOptions{})
+		if err != nil {
+			return err
+		}
+		newNs, err := newNS(name, npc.nodeName, npc.ipt, npc.ips, npc.nsSelectors, npc.namespacedPodSelectors, namespace)
 		if err != nil {
 			return err
 		}
@@ -87,52 +117,70 @@ func (npc *controller) withNS(name string, f func(ns *ns) error) error {
 	return nil
 }
 
-func (npc *controller) AddPod(obj *coreapi.Pod) error {
+func (npc *controller) AddPod(ctx context.Context, obj *coreapi.Pod) error {
 	npc.Lock()
 	defer npc.Unlock()
 
 	common.Log.Debugf("EVENT AddPod %s", js(obj))
-	return npc.withNS(obj.ObjectMeta.Namespace, func(ns *ns) error {
+	return npc.withNS(ctx, obj.ObjectMeta.Namespace, func(ns *ns) error {
 		return errors.Wrap(ns.addPod(obj), "add pod")
 	})
 }
 
-func (npc *controller) UpdatePod(oldObj, newObj *coreapi.Pod) error {
+func (npc *controller) UpdatePod(ctx context.Context, oldObj, newObj *coreapi.Pod) error {
 	npc.Lock()
 	defer npc.Unlock()
 
 	common.Log.Debugf("EVENT UpdatePod %s %s", js(oldObj), js(newObj))
-	return npc.withNS(oldObj.ObjectMeta.Namespace, func(ns *ns) error {
+	return npc.withNS(ctx, oldObj.ObjectMeta.Namespace, func(ns *ns) error {
 		return errors.Wrap(ns.updatePod(oldObj, newObj), "update pod")
 	})
 }
 
-func (npc *controller) DeletePod(obj *coreapi.Pod) error {
+func (npc *controller) DeletePod(ctx context.Context, obj *coreapi.Pod) error {
 	npc.Lock()
 	defer npc.Unlock()
 
 	common.Log.Debugf("EVENT DeletePod %s", js(obj))
-	return npc.withNS(obj.ObjectMeta.Namespace, func(ns *ns) error {
+	return npc.withNS(ctx, obj.ObjectMeta.Namespace, func(ns *ns) error {
 		return errors.Wrap(ns.deletePod(obj), "delete pod")
 	})
 }
 
-func (npc *controller) AddNetworkPolicy(obj interface{}) error {
+func (npc *controller) AddNetworkPolicy(ctx context.Context, obj interface{}) error {
 	npc.Lock()
 	defer npc.Unlock()
 
+	// lazily add default rule to drop egress traffic only when network policies are applied
+	if !npc.defaultEgressDrop {
+		egressNetworkPolicy, err := isEgressNetworkPolicy(obj)
+		if err != nil {
+			return err
+		}
+		if egressNetworkPolicy {
+			if err := npc.ipt.Append(TableFilter, chains.EgressChain,
+				"-m", "state", "--state", "NEW", "-m", "mark", "!", "--mark", EgressMark, "-j", "NFLOG", "--nflog-group", "86"); err != nil {
+				return fmt.Errorf("Failed to add iptable egress log rule: %w", err)
+			}
+			if err := npc.ipt.Append(TableFilter, chains.EgressChain,
+				"-m", "mark", "!", "--mark", EgressMark, "-j", "DROP"); err != nil {
+				return fmt.Errorf("Failed to add iptable rule to drop egress traffic from the pods by default due to %s", err.Error())
+			}
+			npc.defaultEgressDrop = true
+		}
+	}
+
 	nsName, err := nsName(obj)
 	if err != nil {
 		return err
 	}
-
 	common.Log.Infof("EVENT AddNetworkPolicy %s", js(obj))
-	return npc.withNS(nsName, func(ns *ns) error {
+	return npc.withNS(ctx, nsName, func(ns *ns) error {
 		return errors.Wrap(ns.addNetworkPolicy(obj), "add network policy")
 	})
 }
 
-func (npc *controller) UpdateNetworkPolicy(oldObj, newObj interface{}) error {
+func (npc *controller) UpdateNetworkPolicy(ctx context.Context, oldObj, newObj interface{}) error {
 	npc.Lock()
 	defer npc.Unlock()
 
@@ -142,12 +190,12 @@ func (npc *controller) UpdateNetworkPolicy(oldObj, newObj interface{}) error {
 	}
 
 	common.Log.Infof("EVENT UpdateNetworkPolicy %s %s", js(oldObj), js(newObj))
-	return npc.withNS(nsName, func(ns *ns) error {
+	return npc.withNS(ctx, nsName, func(ns *ns) error {
 		return errors.Wrap(ns.updateNetworkPolicy(oldObj, newObj), "update network policy")
 	})
 }
 
-func (npc *controller) DeleteNetworkPolicy(obj interface{}) error {
+func (npc *controller) DeleteNetworkPolicy(ctx context.Context, obj interface{}) error {
 	npc.Lock()
 	defer npc.Unlock()
 
@@ -157,37 +205,37 @@ func (npc *controller) DeleteNetworkPolicy(obj interface{}) error {
 	}
 
 	common.Log.Infof("EVENT DeleteNetworkPolicy %s", js(obj))
-	return npc.withNS(nsName, func(ns *ns) error {
+	return npc.withNS(ctx, nsName, func(ns *ns) error {
 		return errors.Wrap(ns.deleteNetworkPolicy(obj), "delete network policy")
 	})
 }
 
-func (npc *controller) AddNamespace(obj *coreapi.Namespace) error {
+func (npc *controller) AddNamespace(ctx context.Context, obj *coreapi.Namespace) error {
 	npc.Lock()
 	defer npc.Unlock()
 
 	common.Log.Infof("EVENT AddNamespace %s", js(obj))
-	return npc.withNS(obj.ObjectMeta.Name, func(ns *ns) error {
+	return npc.withNS(ctx, obj.ObjectMeta.Name, func(ns *ns) error {
 		return errors.Wrap(ns.addNamespace(obj), "add namespace")
 	})
 }
 
-func (npc *controller) UpdateNamespace(oldObj, newObj *coreapi.Namespace) error {
+func (npc *controller) UpdateNamespace(ctx context.Context, oldObj, newObj *coreapi.Namespace) error {
 	npc.Lock()
 	defer npc.Unlock()
 
 	common.Log.Infof("EVENT UpdateNamespace %s %s", js(oldObj), js(newObj))
-	return npc.withNS(oldObj.ObjectMeta.Name, func(ns *ns) error {
+	return npc.withNS(ctx, oldObj.ObjectMeta.Name, func(ns *ns) error {
 		return errors.Wrap(ns.updateNamespace(oldObj, newObj), "update namespace")
 	})
 }
 
-func (npc *controller) DeleteNamespace(obj *coreapi.Namespace) error {
+func (npc *controller) DeleteNamespace(ctx context.Context, obj *coreapi.Namespace) error {
 	npc.Lock()
 	defer npc.Unlock()
 
 	common.Log.Infof("EVENT DeleteNamespace %s", js(obj))
-	return npc.withNS(obj.ObjectMeta.Name, func(ns *ns) error {
+	return npc.withNS(ctx, obj.ObjectMeta.Name, func(ns *ns) error {
 		return errors.Wrap(ns.deleteNamespace(obj), "delete namespace")
 	})
 }
@@ -202,3 +250,28 @@ func nsName(obj interface{}) (string, error) {
 
 	return "", errInvalidNetworkPolicyObjType
 }
+
+func isEgressNetworkPolicy(obj interface{}) (bool, error) {
+	if policy, ok := obj.(*networkingv1.NetworkPolicy); ok {
+		if len(policy.Spec.PolicyTypes) > 0 {
+			for _, policyType := range policy.Spec.PolicyTypes {
+				if policyType == networkingv1.PolicyTypeEgress {
+					return true, nil
+				}
+			}
+		}
+		if policy.Spec.Egress != nil {
+			return true, nil
+		}
+		return false, nil
+	}
+	return false, errInvalidNetworkPolicyObjType
+}
+
+func uid(pod *coreapi.Pod) ipset.UID {
+	return ipset.UID(pod.UID)
+}
+
+func nsuid(ns *coreapi.Namespace) ipset.UID {
+	return ipset.UID(ns.UID)
+}

npc/controller_test.go

@@ -1,25 +1,27 @@
 package npc
 
 import (
+	"context"
 	"fmt"
 	"log"
 	"strings"
 	"testing"
 
 	"github.com/pkg/errors"
+	"github.com/rajch/weave/common/chains"
+	"github.com/rajch/weave/net/ipset"
 	"github.com/stretchr/testify/require"
-	"github.com/weaveworks/weave/net/ipset"
 	coreapi "k8s.io/api/core/v1"
 	networkingv1 "k8s.io/api/networking/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"k8s.io/apimachinery/pkg/types"
 	"k8s.io/apimachinery/pkg/util/intstr"
+	"k8s.io/client-go/kubernetes/fake"
 )
 
 type mockSet struct {
 	name    ipset.Name
 	setType ipset.Type
-	subSets map[string]map[types.UID]bool
+	subSets map[string]map[ipset.UID]bool
 }
 
 type mockIPSet struct {
@@ -34,17 +36,17 @@ func (i *mockIPSet) Create(ipsetName ipset.Name, ipsetType ipset.Type) error {
 	if _, ok := i.sets[string(ipsetName)]; ok {
 		return errors.Errorf("ipset %s already exists", ipsetName)
 	}
-	i.sets[string(ipsetName)] = mockSet{name: ipsetName, setType: ipsetType, subSets: make(map[string]map[types.UID]bool)}
+	i.sets[string(ipsetName)] = mockSet{name: ipsetName, setType: ipsetType, subSets: make(map[string]map[ipset.UID]bool)}
 	return nil
 }
 
-func (i *mockIPSet) AddEntry(user types.UID, ipsetName ipset.Name, entry string, comment string) error {
+func (i *mockIPSet) AddEntry(user ipset.UID, ipsetName ipset.Name, entry string, comment string) error {
 	log.Printf("adding entry %s to %s for %s", entry, ipsetName, user)
 	if _, ok := i.sets[string(ipsetName)]; !ok {
 		return errors.Errorf("%s does not exist", entry)
 	}
 	if i.sets[string(ipsetName)].subSets[entry] == nil {
-		i.sets[string(ipsetName)].subSets[entry] = make(map[types.UID]bool)
+		i.sets[string(ipsetName)].subSets[entry] = make(map[ipset.UID]bool)
 	}
 	if _, ok := i.sets[string(ipsetName)].subSets[entry][user]; ok {
 		return errors.Errorf("user %s already owns entry %s", user, entry)
@@ -54,7 +56,7 @@ func (i *mockIPSet) AddEntry(user types.UID, ipsetName ipset.Name, entry string,
 	return nil
 }
 
-func (i *mockIPSet) DelEntry(user types.UID, ipsetName ipset.Name, entry string) error {
+func (i *mockIPSet) DelEntry(user ipset.UID, ipsetName ipset.Name, entry string) error {
 	log.Printf("deleting entry %s from %s for %s", entry, ipsetName, user)
 	if _, ok := i.sets[string(ipsetName)]; !ok {
 		return errors.Errorf("ipset %s does not exist", ipsetName)
@@ -71,12 +73,17 @@ func (i *mockIPSet) DelEntry(user types.UID, ipsetName ipset.Name, entry string)
 	return nil
 }
 
-func (i *mockIPSet) Exist(user types.UID, ipsetName ipset.Name, entry string) bool {
+func (i *mockIPSet) EntryExists(user ipset.UID, ipsetName ipset.Name, entry string) bool {
 	_, ok := i.sets[string(ipsetName)].subSets[entry][user]
 	return ok
 }
 
-func (i *mockIPSet) entryExists(ipsetName ipset.Name, entry string) bool {
+func (i *mockIPSet) Exists(ipsetName ipset.Name) (bool, error) {
+	_, ok := i.sets[string(ipsetName)]
+	return ok, nil
+}
+
+func (i *mockIPSet) entriesExist(ipsetName ipset.Name, entry string) bool {
 	return len(i.sets[string(ipsetName)].subSets[entry]) > 0
 }
 
@@ -149,11 +156,12 @@ func (ipt *mockIPTables) Insert(table, chain string, pos int, rulespec ...string
 
 func TestRegressionPolicyNamespaceOrdering3059(t *testing.T) {
 	// Test for race condition between namespace and networkpolicy events
-	// https://github.com/weaveworks/weave/issues/3059
+	// https://github.com/rajch/weave/issues/3059
 
 	sourceNamespace := &coreapi.Namespace{
 		ObjectMeta: metav1.ObjectMeta{
 			Name: "source",
+			UID:  "source",
 			Labels: map[string]string{
 				"app": "source",
 			},
@@ -163,6 +171,7 @@ func TestRegressionPolicyNamespaceOrdering3059(t *testing.T) {
 	destinationNamespace := &coreapi.Namespace{
 		ObjectMeta: metav1.ObjectMeta{
 			Name: "destination",
+			UID:  "destination",
 		},
 	}
 
@@ -198,28 +207,32 @@ func TestRegressionPolicyNamespaceOrdering3059(t *testing.T) {
 
 	// Namespaces first
 	m := newMockIPSet()
-	controller := New("foo", newMockIPTables(), &m)
+	client := fake.NewSimpleClientset()
+	ctx := context.Background()
+	controller := New("foo", newMockIPTables(), &m, client)
+	client.CoreV1().Namespaces().Create(ctx, sourceNamespace, metav1.CreateOptions{})
+	client.CoreV1().Namespaces().Create(ctx, destinationNamespace, metav1.CreateOptions{})
 
 	const (
 		selectorIPSetName = "weave-I239Zp%sCvoVt*D6u=A!2]YEk"
 		sourceIPSetName   = "weave-HboJG1fGgG]/SR%k9H#hv5e96"
 	)
 
-	controller.AddNamespace(sourceNamespace)
-	controller.AddNamespace(destinationNamespace)
+	controller.AddNamespace(ctx, sourceNamespace)
+	controller.AddNamespace(ctx, destinationNamespace)
 
-	controller.AddNetworkPolicy(networkPolicy)
+	controller.AddNetworkPolicy(ctx, networkPolicy)
 
 	require.Equal(t, true, len(m.sets[selectorIPSetName].subSets[sourceIPSetName]) > 0)
 
 	// NetworkPolicy first
 	m = newMockIPSet()
-	controller = New("foo", newMockIPTables(), &m)
+	controller = New("foo", newMockIPTables(), &m, &fake.Clientset{})
 
-	controller.AddNetworkPolicy(networkPolicy)
+	controller.AddNetworkPolicy(ctx, networkPolicy)
 
-	controller.AddNamespace(sourceNamespace)
-	controller.AddNamespace(destinationNamespace)
+	controller.AddNamespace(ctx, sourceNamespace)
+	controller.AddNamespace(ctx, destinationNamespace)
 
 	require.Equal(t, true, len(m.sets[selectorIPSetName].subSets[sourceIPSetName]) > 0)
 }
@@ -235,14 +248,16 @@ func TestDefaultAllow(t *testing.T) {
 	)
 
 	m := newMockIPSet()
-	controller := New("bar", newMockIPTables(), &m)
+	controller := New("bar", newMockIPTables(), &m, &fake.Clientset{})
 
+	ctx := context.Background()
 	defaultNamespace := &coreapi.Namespace{
 		ObjectMeta: metav1.ObjectMeta{
 			Name: "default",
+			UID:  "default",
 		},
 	}
-	controller.AddNamespace(defaultNamespace)
+	controller.AddNamespace(ctx, defaultNamespace)
 
 	// Should create an ipset for default-allow
 	require.Contains(t, m.sets, ingressDefaultAllowIPSetName)
@@ -255,11 +270,11 @@ func TestDefaultAllow(t *testing.T) {
 			Name:      "foo",
 			Labels:    map[string]string{"run": "foo"}},
 		Status: coreapi.PodStatus{PodIP: fooPodIP}}
-	controller.AddPod(podFoo)
+	controller.AddPod(ctx, podFoo)
 
 	// Should add the foo pod to default-allow
-	require.True(t, m.entryExists(ingressDefaultAllowIPSetName, fooPodIP))
-	require.True(t, m.entryExists(egressDefaultAllowIPSetName, fooPodIP))
+	require.True(t, m.entriesExist(ingressDefaultAllowIPSetName, fooPodIP))
+	require.True(t, m.entriesExist(egressDefaultAllowIPSetName, fooPodIP))
 
 	podBar := &coreapi.Pod{
 		ObjectMeta: metav1.ObjectMeta{
@@ -269,13 +284,13 @@ func TestDefaultAllow(t *testing.T) {
 			Labels:    map[string]string{"run": "bar"}},
 		Status: coreapi.PodStatus{PodIP: barPodIP}}
 	podBarNoIP := &coreapi.Pod{ObjectMeta: podBar.ObjectMeta}
-	controller.AddPod(podBarNoIP)
+	controller.AddPod(ctx, podBarNoIP)
 
-	controller.UpdatePod(podBarNoIP, podBar)
+	controller.UpdatePod(ctx, podBarNoIP, podBar)
 
 	// Should add the bar pod to default-allow
-	require.True(t, m.entryExists(ingressDefaultAllowIPSetName, barPodIP))
-	require.True(t, m.entryExists(egressDefaultAllowIPSetName, barPodIP))
+	require.True(t, m.entriesExist(ingressDefaultAllowIPSetName, barPodIP))
+	require.True(t, m.entriesExist(egressDefaultAllowIPSetName, barPodIP))
 
 	// Allow access from the bar pod to the foo pod
 	netpol := &networkingv1.NetworkPolicy{
@@ -303,53 +318,53 @@ func TestDefaultAllow(t *testing.T) {
 			}},
 		},
 	}
-	controller.AddNetworkPolicy(netpol)
+	controller.AddNetworkPolicy(ctx, netpol)
 
 	// Should remove the foo pod from default-allow as the netpol selects it
-	require.False(t, m.entryExists(ingressDefaultAllowIPSetName, fooPodIP))
-	require.False(t, m.entryExists(egressDefaultAllowIPSetName, fooPodIP))
-	require.True(t, m.entryExists(ingressDefaultAllowIPSetName, barPodIP))
-	require.True(t, m.entryExists(egressDefaultAllowIPSetName, barPodIP))
+	require.False(t, m.entriesExist(ingressDefaultAllowIPSetName, fooPodIP))
+	require.False(t, m.entriesExist(egressDefaultAllowIPSetName, fooPodIP))
+	require.True(t, m.entriesExist(ingressDefaultAllowIPSetName, barPodIP))
+	require.True(t, m.entriesExist(egressDefaultAllowIPSetName, barPodIP))
 
 	podBarWithNewIP := *podBar
 	podBarWithNewIP.Status.PodIP = barPodNewIP
-	controller.UpdatePod(podBar, &podBarWithNewIP)
+	controller.UpdatePod(ctx, podBar, &podBarWithNewIP)
 
 	// Should update IP addr of the bar pod in default-allow
-	require.False(t, m.entryExists(ingressDefaultAllowIPSetName, barPodIP))
-	require.False(t, m.entryExists(egressDefaultAllowIPSetName, barPodIP))
-	require.True(t, m.entryExists(ingressDefaultAllowIPSetName, barPodNewIP))
-	require.True(t, m.entryExists(egressDefaultAllowIPSetName, barPodNewIP))
+	require.False(t, m.entriesExist(ingressDefaultAllowIPSetName, barPodIP))
+	require.False(t, m.entriesExist(egressDefaultAllowIPSetName, barPodIP))
+	require.True(t, m.entriesExist(ingressDefaultAllowIPSetName, barPodNewIP))
+	require.True(t, m.entriesExist(egressDefaultAllowIPSetName, barPodNewIP))
 
-	controller.UpdatePod(&podBarWithNewIP, podBarNoIP)
+	controller.UpdatePod(ctx, &podBarWithNewIP, podBarNoIP)
 	// Should remove the bar pod from default-allow as it does not have any IP addr
-	require.False(t, m.entryExists(ingressDefaultAllowIPSetName, barPodNewIP))
-	require.False(t, m.entryExists(egressDefaultAllowIPSetName, barPodNewIP))
+	require.False(t, m.entriesExist(ingressDefaultAllowIPSetName, barPodNewIP))
+	require.False(t, m.entriesExist(egressDefaultAllowIPSetName, barPodNewIP))
 
 	podFooWithNewLabel := *podFoo
 	podFooWithNewLabel.ObjectMeta.Labels = map[string]string{"run": "new-foo"}
-	controller.UpdatePod(podFoo, &podFooWithNewLabel)
+	controller.UpdatePod(ctx, podFoo, &podFooWithNewLabel)
 
 	// Should bring back the foo pod to default-allow as it does not match dst of any netpol
-	require.True(t, m.entryExists(ingressDefaultAllowIPSetName, fooPodIP))
-	require.True(t, m.entryExists(egressDefaultAllowIPSetName, fooPodIP))
+	require.True(t, m.entriesExist(ingressDefaultAllowIPSetName, fooPodIP))
+	require.True(t, m.entriesExist(egressDefaultAllowIPSetName, fooPodIP))
 
-	controller.UpdatePod(&podFooWithNewLabel, podFoo)
+	controller.UpdatePod(ctx, &podFooWithNewLabel, podFoo)
 	// Should remove from default-allow as it matches the netpol after the update
-	require.False(t, m.entryExists(ingressDefaultAllowIPSetName, fooPodIP))
-	require.False(t, m.entryExists(egressDefaultAllowIPSetName, fooPodIP))
+	require.False(t, m.entriesExist(ingressDefaultAllowIPSetName, fooPodIP))
+	require.False(t, m.entriesExist(egressDefaultAllowIPSetName, fooPodIP))
 
-	controller.DeleteNetworkPolicy(netpol)
+	controller.DeleteNetworkPolicy(ctx, netpol)
 	// Should bring back the foo pod to default-allow as no netpol selects it
-	require.True(t, m.entryExists(ingressDefaultAllowIPSetName, fooPodIP))
-	require.True(t, m.entryExists(egressDefaultAllowIPSetName, fooPodIP))
+	require.True(t, m.entriesExist(ingressDefaultAllowIPSetName, fooPodIP))
+	require.True(t, m.entriesExist(egressDefaultAllowIPSetName, fooPodIP))
 
-	controller.DeletePod(podFoo)
+	controller.DeletePod(ctx, podFoo)
 	// Should remove foo pod from default-allow
-	require.False(t, m.entryExists(ingressDefaultAllowIPSetName, fooPodIP))
-	require.False(t, m.entryExists(egressDefaultAllowIPSetName, fooPodIP))
+	require.False(t, m.entriesExist(ingressDefaultAllowIPSetName, fooPodIP))
+	require.False(t, m.entriesExist(egressDefaultAllowIPSetName, fooPodIP))
 
-	controller.DeleteNamespace(defaultNamespace)
+	controller.DeleteNamespace(ctx, defaultNamespace)
 	// Should remove default ipset
 	require.NotContains(t, m.sets, ingressDefaultAllowIPSetName)
 	require.NotContains(t, m.sets, egressDefaultAllowIPSetName)
@@ -363,14 +378,18 @@ func TestOutOfOrderPodEvents(t *testing.T) {
 	)
 
 	m := newMockIPSet()
-	controller := New("qux", newMockIPTables(), &m)
+	client := fake.NewSimpleClientset()
+	controller := New("qux", newMockIPTables(), &m, client)
+	ctx := context.Background()
 
 	defaultNamespace := &coreapi.Namespace{
 		ObjectMeta: metav1.ObjectMeta{
 			Name: "default",
+			UID:  "default",
 		},
 	}
-	controller.AddNamespace(defaultNamespace)
+	client.CoreV1().Namespaces().Create(ctx, defaultNamespace, metav1.CreateOptions{})
+	controller.AddNamespace(ctx, defaultNamespace)
 
 	podFoo := &coreapi.Pod{
 		ObjectMeta: metav1.ObjectMeta{
@@ -379,10 +398,10 @@ func TestOutOfOrderPodEvents(t *testing.T) {
 			Name:      "foo",
 			Labels:    map[string]string{"run": "foo"}},
 		Status: coreapi.PodStatus{PodIP: podIP}}
-	controller.AddPod(podFoo)
+	controller.AddPod(ctx, podFoo)
 
 	// Should be in default-allow as no netpol selects podFoo
-	require.True(t, m.entryExists(ingressDefaultAllowIPSetName, podIP))
+	require.True(t, m.entriesExist(ingressDefaultAllowIPSetName, podIP))
 
 	netpol := &networkingv1.NetworkPolicy{
 		ObjectMeta: metav1.ObjectMeta{
@@ -401,10 +420,10 @@ func TestOutOfOrderPodEvents(t *testing.T) {
 			}},
 		},
 	}
-	controller.AddNetworkPolicy(netpol)
+	controller.AddNetworkPolicy(ctx, netpol)
 
 	// Shouldn't be in default-allow as netpol above selects podFoo
-	require.False(t, m.entryExists(ingressDefaultAllowIPSetName, podIP))
+	require.False(t, m.entriesExist(ingressDefaultAllowIPSetName, podIP))
 
 	podBar := &coreapi.Pod{
 		ObjectMeta: metav1.ObjectMeta{
@@ -413,31 +432,31 @@ func TestOutOfOrderPodEvents(t *testing.T) {
 			Name:      "bar",
 			Labels:    map[string]string{"run": "bar"}},
 		Status: coreapi.PodStatus{PodIP: podIP}}
-	controller.AddPod(podBar)
+	controller.AddPod(ctx, podBar)
 
 	// Should be in default-allow as no netpol selects podBar
-	require.True(t, m.entryExists(ingressDefaultAllowIPSetName, podIP))
-	require.True(t, m.Exist(podBar.ObjectMeta.UID, ingressDefaultAllowIPSetName, podIP))
+	require.True(t, m.entriesExist(ingressDefaultAllowIPSetName, podIP))
+	require.True(t, m.EntryExists(uid(podBar), ingressDefaultAllowIPSetName, podIP))
 	// Should be in run=bar ipset
-	require.True(t, m.entryExists(runBarIPSetName, podIP))
+	require.True(t, m.entriesExist(runBarIPSetName, podIP))
 
-	controller.DeletePod(podFoo)
+	controller.DeletePod(ctx, podFoo)
 	// Multiple duplicate events should not affect npc state
-	controller.DeletePod(podFoo)
-	controller.DeletePod(podFoo)
+	controller.DeletePod(ctx, podFoo)
+	controller.DeletePod(ctx, podFoo)
 
 	// Should be in default-allow as no netpol selects podBar and podFoo removal
 	// should not affect podBar in default-allow
-	require.True(t, m.entryExists(ingressDefaultAllowIPSetName, podIP))
+	require.True(t, m.entriesExist(ingressDefaultAllowIPSetName, podIP))
 
-	controller.DeletePod(podBar)
+	controller.DeletePod(ctx, podBar)
 
 	// Should remove from default-allow and run=bar ipsets
 	require.Equal(t, 0, len(m.sets[ingressDefaultAllowIPSetName].subSets))
-	require.False(t, m.entryExists(runBarIPSetName, podIP))
+	require.False(t, m.entriesExist(runBarIPSetName, podIP))
 }
 
-// Test case for https://github.com/weaveworks/weave/issues/3222
+// Test case for https://github.com/rajch/weave/issues/3222
 func TestNewTargetSelector(t *testing.T) {
 	const (
 		ingressDefaultAllowIPSetName = "weave-;rGqyMIl1HN^cfDki~Z$3]6!N"
@@ -445,14 +464,18 @@ func TestNewTargetSelector(t *testing.T) {
 	)
 
 	m := newMockIPSet()
-	controller := New("baz", newMockIPTables(), &m)
+	client := fake.NewSimpleClientset()
+	controller := New("baz", newMockIPTables(), &m, client)
+	ctx := context.Background()
 
 	defaultNamespace := &coreapi.Namespace{
 		ObjectMeta: metav1.ObjectMeta{
 			Name: "default",
+			UID:  "default",
 		},
 	}
-	controller.AddNamespace(defaultNamespace)
+	client.CoreV1().Namespaces().Create(ctx, defaultNamespace, metav1.CreateOptions{})
+	controller.AddNamespace(ctx, defaultNamespace)
 
 	podFoo := &coreapi.Pod{
 		ObjectMeta: metav1.ObjectMeta{
@@ -461,9 +484,9 @@ func TestNewTargetSelector(t *testing.T) {
 			Name:      "foo",
 			Labels:    map[string]string{"run": "foo"}},
 		Status: coreapi.PodStatus{PodIP: podIP}}
-	controller.AddPod(podFoo)
+	controller.AddPod(ctx, podFoo)
 
-	require.True(t, m.entryExists(ingressDefaultAllowIPSetName, podIP))
+	require.True(t, m.entriesExist(ingressDefaultAllowIPSetName, podIP))
 
 	netpolBar := &networkingv1.NetworkPolicy{
 		ObjectMeta: metav1.ObjectMeta{
@@ -481,10 +504,10 @@ func TestNewTargetSelector(t *testing.T) {
 			}},
 		},
 	}
-	controller.AddNetworkPolicy(netpolBar)
+	controller.AddNetworkPolicy(ctx, netpolBar)
 
 	// netpolBar target selector selects podFoo
-	require.False(t, m.entryExists(ingressDefaultAllowIPSetName, podIP))
+	require.False(t, m.entriesExist(ingressDefaultAllowIPSetName, podIP))
 
 	netpolFoo := &networkingv1.NetworkPolicy{
 		ObjectMeta: metav1.ObjectMeta{
@@ -502,14 +525,14 @@ func TestNewTargetSelector(t *testing.T) {
 			}},
 		},
 	}
-	controller.AddNetworkPolicy(netpolFoo)
+	controller.AddNetworkPolicy(ctx, netpolFoo)
 
-	controller.DeleteNetworkPolicy(netpolBar)
+	controller.DeleteNetworkPolicy(ctx, netpolBar)
 	// netpolFoo target-selects podFoo
-	require.False(t, m.entryExists(ingressDefaultAllowIPSetName, podIP))
-	controller.DeleteNetworkPolicy(netpolFoo)
+	require.False(t, m.entriesExist(ingressDefaultAllowIPSetName, podIP))
+	controller.DeleteNetworkPolicy(ctx, netpolFoo)
 	// No netpol target-selects podFoo
-	require.True(t, m.entryExists(ingressDefaultAllowIPSetName, podIP))
+	require.True(t, m.entriesExist(ingressDefaultAllowIPSetName, podIP))
 }
 
 func TestEgressPolicyWithIPBlock(t *testing.T) {
@@ -521,7 +544,24 @@ func TestEgressPolicyWithIPBlock(t *testing.T) {
 
 	m := newMockIPSet()
 	ipt := newMockIPTables()
-	controller := New("foo", ipt, &m)
+	client := fake.NewSimpleClientset()
+	controller := New("foo", ipt, &m, client)
+	ctx := context.Background()
+
+	defaultNamespace := &coreapi.Namespace{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "default",
+			UID:  "default",
+		},
+	}
+	nonDefaultNamespace := &coreapi.Namespace{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "non-default",
+			UID:  "non-default",
+		},
+	}
+	client.CoreV1().Namespaces().Create(ctx, defaultNamespace, metav1.CreateOptions{})
+	client.CoreV1().Namespaces().Create(ctx, nonDefaultNamespace, metav1.CreateOptions{})
 
 	netpolFoo := &networkingv1.NetworkPolicy{
 		ObjectMeta: metav1.ObjectMeta{
@@ -547,16 +587,16 @@ func TestEgressPolicyWithIPBlock(t *testing.T) {
 			}},
 		},
 	}
-	err := controller.AddNetworkPolicy(netpolFoo)
+	err := controller.AddNetworkPolicy(ctx, netpolFoo)
 	require.NoError(t, err)
 
 	require.Equal(t, 2, len(m.sets[exceptIPSetName].subSets))
-	require.True(t, m.entryExists(exceptIPSetName, "192.168.48.1/32"))
-	require.True(t, m.entryExists(exceptIPSetName, "192.168.48.2/32"))
+	require.True(t, m.entriesExist(exceptIPSetName, "192.168.48.1/32"))
+	require.True(t, m.entriesExist(exceptIPSetName, "192.168.48.2/32"))
 
 	// Each egress rule is represented as two iptables rules (-J MARK and -J RETURN).
-	require.Equal(t, 2, len(ipt.rules[EgressCustomChain]))
-	for rule := range ipt.rules[EgressCustomChain] {
+	require.Equal(t, 2, len(ipt.rules[chains.EgressCustomChain]))
+	for rule := range ipt.rules[chains.EgressCustomChain] {
 		require.Contains(t, rule, "-d 192.168.48.0/24 -m set ! --match-set "+exceptIPSetName+" dst")
 	}
 
@@ -588,7 +628,85 @@ func TestEgressPolicyWithIPBlock(t *testing.T) {
 			}},
 		},
 	}
-	err = controller.AddNetworkPolicy(netpolBar)
+	err = controller.AddNetworkPolicy(ctx, netpolBar)
 	require.NoError(t, err)
 	require.Equal(t, 2, len(m.sets[exceptIPSetNameInNonDefault].subSets))
 }
+
+// Test case for https://github.com/rajch/weave/issues/3653
+func TestIngressPolicyWithIPBlockAndPortSpecified(t *testing.T) {
+	const (
+		barPodIP        = "10.32.0.11"
+		runBarIPSetName = "weave-bZ~x=yBgzH)Ht()K*Uv3z{M]Y"
+	)
+
+	m := newMockIPSet()
+	ipt := newMockIPTables()
+	client := fake.NewSimpleClientset()
+	controller := New("any", ipt, &m, client)
+	ctx := context.Background()
+
+	defaultNamespace := &coreapi.Namespace{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "default",
+			UID:  "default",
+		},
+	}
+
+	client.CoreV1().Namespaces().Create(ctx, defaultNamespace, metav1.CreateOptions{})
+
+	podBar := &coreapi.Pod{
+		ObjectMeta: metav1.ObjectMeta{
+			UID:       "bar",
+			Namespace: "default",
+			Name:      "bar",
+			Labels:    map[string]string{"run": "bar"}},
+		Status: coreapi.PodStatus{PodIP: barPodIP}}
+	controller.AddPod(ctx, podBar)
+	defer controller.DeletePod(ctx, podBar)
+
+	portProtocol := coreapi.ProtocolTCP
+	port := intstr.FromInt(80)
+	netpolicty := &networkingv1.NetworkPolicy{
+		ObjectMeta: metav1.ObjectMeta{
+			UID:       "ipblock-bar",
+			Name:      "allow-ipblock-to-bar",
+			Namespace: "default",
+		},
+		Spec: networkingv1.NetworkPolicySpec{
+			PolicyTypes: []networkingv1.PolicyType{
+				networkingv1.PolicyTypeIngress,
+			},
+			PodSelector: metav1.LabelSelector{MatchLabels: podBar.Labels},
+			Ingress: []networkingv1.NetworkPolicyIngressRule{
+				{
+					Ports: []networkingv1.NetworkPolicyPort{
+						{
+							Protocol: &portProtocol,
+							Port:     &port,
+						},
+					},
+					From: []networkingv1.NetworkPolicyPeer{
+						{
+							IPBlock: &networkingv1.IPBlock{
+								CIDR: "192.168.48.4/32",
+							},
+						},
+					},
+				},
+			},
+		},
+	}
+
+	err := controller.AddNetworkPolicy(ctx, netpolicty)
+	require.NoError(t, err)
+	defer controller.DeleteNetworkPolicy(ctx, netpolicty)
+
+	require.Equal(t, 1, len(m.sets[runBarIPSetName].subSets))
+	require.True(t, m.entriesExist(runBarIPSetName, barPodIP))
+
+	require.Equal(t, 1, len(ipt.rules[chains.IngressChain]))
+	for rule := range ipt.rules[chains.IngressChain] {
+		require.Contains(t, rule, "-s 192.168.48.4/32 -m set --match-set "+runBarIPSetName+" dst --dport 80")
+	}
+}

npc/ipblock.go

@@ -6,10 +6,9 @@ import (
 	"strings"
 
 	networkingv1 "k8s.io/api/networking/v1"
-	"k8s.io/apimachinery/pkg/types"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/ipset"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/net/ipset"
 )
 
 type ipBlockSpec struct {
@@ -51,17 +50,17 @@ func (spec *ipBlockSpec) getRuleSpec(src bool) ([]string, string) {
 
 type ipBlockSet struct {
 	ips   ipset.Interface
-	users map[string]map[types.UID]struct{}
+	users map[string]map[ipset.UID]struct{}
 }
 
 func newIPBlockSet(ips ipset.Interface) *ipBlockSet {
 	return &ipBlockSet{
 		ips:   ips,
-		users: make(map[string]map[types.UID]struct{}),
+		users: make(map[string]map[ipset.UID]struct{}),
 	}
 }
 
-func (s *ipBlockSet) deprovision(user types.UID, current, desired map[string]*ipBlockSpec) error {
+func (s *ipBlockSet) deprovision(user ipset.UID, current, desired map[string]*ipBlockSpec) error {
 	for key, spec := range current {
 		if key == "" {
 			continue
@@ -83,7 +82,7 @@ func (s *ipBlockSet) deprovision(user types.UID, current, desired map[string]*ip
 	return nil
 }
 
-func (s *ipBlockSet) provision(user types.UID, current, desired map[string]*ipBlockSpec) (err error) {
+func (s *ipBlockSet) provision(user ipset.UID, current, desired map[string]*ipBlockSpec) (err error) {
 	for key, spec := range desired {
 		if key == "" {
 			// No need to provision an ipBlock with empty list of excepted CIDRs
@@ -106,7 +105,7 @@ func (s *ipBlockSet) provision(user types.UID, current, desired map[string]*ipBl
 					}
 				}
 
-				s.users[key] = make(map[types.UID]struct{})
+				s.users[key] = make(map[ipset.UID]struct{})
 			}
 			s.users[key][user] = struct{}{}
 		}

npc/metrics/metrics.go

@@ -11,7 +11,7 @@ import (
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promhttp"
 
-	"github.com/weaveworks/weave/common"
+	"github.com/rajch/weave/common"
 )
 
 var (
@@ -22,6 +22,12 @@ var (
 		},
 		[]string{"protocol", "dport"},
 	)
+	PolicyEnforcementErrors = prometheus.NewCounter(
+		prometheus.CounterOpts{
+			Name: "weavenpc_policy_enforcement_errors",
+			Help: "Errors occured in enforcing the network policies.",
+		},
+	)
 )
 
 func gatherMetrics() {
@@ -95,6 +101,9 @@ func Start(addr string) error {
 	if err := prometheus.Register(blockedConnections); err != nil {
 		return err
 	}
+	if err := prometheus.Register(PolicyEnforcementErrors); err != nil {
+		return err
+	}
 
 	http.Handle("/metrics", promhttp.Handler())
 

npc/namespace.go

@@ -8,12 +8,12 @@ import (
 	extnapi "k8s.io/api/extensions/v1beta1"
 	networkingv1 "k8s.io/api/networking/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"k8s.io/apimachinery/pkg/types"
 	"k8s.io/apimachinery/pkg/util/uuid"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/ipset"
-	"github.com/weaveworks/weave/npc/iptables"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/common/chains"
+	"github.com/rajch/weave/net/ipset"
+	"github.com/rajch/weave/npc/iptables"
 )
 
 var errInvalidNetworkPolicyObjType = errors.New("invalid NetworkPolicy object type")
@@ -22,13 +22,14 @@ type ns struct {
 	ipt iptables.Interface // interface to iptables
 	ips ipset.Interface    // interface to ipset
 
-	name      string                     // k8s Namespace name
-	nodeName  string                     // my node name
-	namespace *coreapi.Namespace         // k8s Namespace object
-	pods      map[types.UID]*coreapi.Pod // k8s Pod objects by UID
-	policies  map[types.UID]interface{}  // k8s NetworkPolicy objects by UID
+	name            string                     // k8s Namespace name
+	nodeName        string                     // my node name
+	namespaceUID    ipset.UID                  // k8s Namespace UID
+	namespaceLabels map[string]string          // k8s Namespace labels
+	pods            map[ipset.UID]*coreapi.Pod // k8s Pod objects by UID
+	policies        map[ipset.UID]interface{}  // k8s NetworkPolicy objects by UID
 
-	uid     types.UID     // surrogate UID to own allPods selector
+	uid     ipset.UID     // surrogate UID to own allPods selector
 	allPods *selectorSpec // hash:ip ipset of all pod IPs in this namespace
 
 	// stores IP addrs of pods which are not selected by any target podSelector of
@@ -37,30 +38,34 @@ type ns struct {
 	ingressDefaultAllowIPSet ipset.Name
 	egressDefaultAllowIPSet  ipset.Name
 
-	nsSelectors  *selectorSet
-	podSelectors *selectorSet
-	ipBlocks     *ipBlockSet
-	rules        *ruleSet
+	nsSelectors             *selectorSet // reference to global selectorSet that is shared across the `ns`. Used to represent all pods in the matching namespaces
+	podSelectors            *selectorSet // used to represent the matching pods in namespace respresented by this `ns`
+	namespacedPodsSelectors *selectorSet // reference to global selectorSet that is shared across the `ns`. Used to represent matching pods in matching namespace
+	ipBlocks                *ipBlockSet
+	rules                   *ruleSet
 }
 
-func newNS(name, nodeName string, ipt iptables.Interface, ips ipset.Interface, nsSelectors *selectorSet) (*ns, error) {
-	allPods, err := newSelectorSpec(&metav1.LabelSelector{}, nil, name, ipset.HashIP)
+func newNS(name, nodeName string, ipt iptables.Interface, ips ipset.Interface, nsSelectors *selectorSet, namespacedPodsSelectors *selectorSet, namespaceObj *coreapi.Namespace) (*ns, error) {
+	allPods, err := newSelectorSpec(&metav1.LabelSelector{}, nil, nil, name, ipset.HashIP)
 	if err != nil {
 		return nil, err
 	}
 
 	ns := &ns{
-		ipt:         ipt,
-		ips:         ips,
-		name:        name,
-		nodeName:    nodeName,
-		pods:        make(map[types.UID]*coreapi.Pod),
-		policies:    make(map[types.UID]interface{}),
-		uid:         uuid.NewUUID(),
-		allPods:     allPods,
-		nsSelectors: nsSelectors,
-		ipBlocks:    newIPBlockSet(ips),
-		rules:       newRuleSet(ipt),
+		ipt:                     ipt,
+		ips:                     ips,
+		name:                    name,
+		namespaceUID:            nsuid(namespaceObj),
+		namespaceLabels:         namespaceObj.ObjectMeta.Labels,
+		nodeName:                nodeName,
+		pods:                    make(map[ipset.UID]*coreapi.Pod),
+		policies:                make(map[ipset.UID]interface{}),
+		uid:                     ipset.UID(uuid.NewUUID()),
+		allPods:                 allPods,
+		nsSelectors:             nsSelectors,
+		namespacedPodsSelectors: namespacedPodsSelectors,
+		ipBlocks:                newIPBlockSet(ips),
+		rules:                   newRuleSet(ipt),
 	}
 
 	ns.podSelectors = newSelectorSet(ips, ns.onNewPodSelector, ns.onNewTargetPodSelector, ns.onDestroyTargetPodSelector)
@@ -85,7 +90,7 @@ func newNS(name, nodeName string, ipt iptables.Interface, ips ipset.Interface, n
 }
 
 func (ns *ns) empty() bool {
-	return len(ns.pods) == 0 && len(ns.policies) == 0 && ns.namespace == nil
+	return len(ns.pods) == 0 && len(ns.policies) == 0 && ns.namespaceUID == ""
 }
 
 func (ns *ns) destroy() error {
@@ -95,8 +100,8 @@ func (ns *ns) destroy() error {
 func (ns *ns) onNewPodSelector(selector *selector) error {
 	for _, pod := range ns.pods {
 		if hasIP(pod) {
-			if selector.matches(pod.ObjectMeta.Labels) {
-				if err := selector.addEntry(pod.ObjectMeta.UID, pod.Status.PodIP, podComment(pod)); err != nil {
+			if selector.matchesPodSelector(pod.ObjectMeta.Labels) {
+				if err := selector.addEntry(uid(pod), pod.Status.PodIP, podComment(pod)); err != nil {
 					return err
 				}
 
@@ -111,8 +116,8 @@ func (ns *ns) onNewTargetPodSelector(selector *selector, policyType policyType)
 		if hasIP(pod) {
 			// Remove the pod from default-allow if dst podselector matches the pod
 			ipset := ns.defaultAllowIPSetName(policyType)
-			if selector.matches(pod.ObjectMeta.Labels) {
-				if err := ns.ips.DelEntry(pod.ObjectMeta.UID, ipset, pod.Status.PodIP); err != nil {
+			if selector.matchesPodSelector(pod.ObjectMeta.Labels) {
+				if err := ns.ips.DelEntry(uid(pod), ipset, pod.Status.PodIP); err != nil {
 					return err
 				}
 			}
@@ -125,7 +130,7 @@ func (ns *ns) onNewTargetPodSelector(selector *selector, policyType policyType)
 func (ns *ns) onDestroyTargetPodSelector(selector *selector, policyType policyType) error {
 	for _, pod := range ns.pods {
 		if hasIP(pod) {
-			if selector.matches(pod.ObjectMeta.Labels) {
+			if selector.matchesPodSelector(pod.ObjectMeta.Labels) {
 				if err := ns.addToDefaultAllowIfNoMatching(pod, policyType); err != nil {
 					return err
 				}
@@ -141,88 +146,93 @@ func (ns *ns) addToDefaultAllowIfNoMatching(pod *coreapi.Pod, policyType policyT
 	found := false
 	// TODO(mp) optimize (avoid iterating over selectors) by ref counting IP addrs.
 	for _, s := range ns.podSelectors.entries {
-		if ns.podSelectors.targetSelectorExist(s, policyType) && s.matches(pod.ObjectMeta.Labels) {
+		if ns.podSelectors.targetSelectorExist(s, policyType) && s.matchesPodSelector(pod.ObjectMeta.Labels) {
 			found = true
 			break
 		}
 	}
 	if !found {
 		ipset := ns.defaultAllowIPSetName(policyType)
-		if err := ns.ips.AddEntry(pod.ObjectMeta.UID, ipset, pod.Status.PodIP, podComment(pod)); err != nil {
+		if err := ns.ips.AddEntry(uid(pod), ipset, pod.Status.PodIP, podComment(pod)); err != nil {
 			return err
 		}
 	}
 	return nil
 }
 
-func (ns *ns) checkLocalPod(obj *coreapi.Pod) bool {
-	if obj.Spec.NodeName != ns.nodeName {
-		return false
-	}
-	return true
-}
-
 func (ns *ns) addPod(obj *coreapi.Pod) error {
-	ns.pods[obj.ObjectMeta.UID] = obj
+	ns.pods[uid(obj)] = obj
 
 	if !hasIP(obj) {
 		return nil
 	}
 
-	foundIngress, foundEgress, err := ns.podSelectors.addToMatching(obj.ObjectMeta.UID, obj.ObjectMeta.Labels, obj.Status.PodIP, podComment(obj))
+	foundIngress, foundEgress, err := ns.podSelectors.addToMatchingPodSelector(uid(obj), obj.ObjectMeta.Labels, obj.Status.PodIP, podComment(obj))
 	if err != nil {
 		return err
 	}
 	// If there are no matching target selectors, add the pod to default-allow
 	if !foundIngress {
-		if err := ns.ips.AddEntry(obj.ObjectMeta.UID, ns.ingressDefaultAllowIPSet, obj.Status.PodIP, podComment(obj)); err != nil {
+		if err := ns.ips.AddEntry(uid(obj), ns.ingressDefaultAllowIPSet, obj.Status.PodIP, podComment(obj)); err != nil {
 			return err
 		}
 	}
 	if !foundEgress {
-		if err := ns.ips.AddEntry(obj.ObjectMeta.UID, ns.egressDefaultAllowIPSet, obj.Status.PodIP, podComment(obj)); err != nil {
+		if err := ns.ips.AddEntry(uid(obj), ns.egressDefaultAllowIPSet, obj.Status.PodIP, podComment(obj)); err != nil {
 			return err
 		}
 	}
 
+	err = ns.namespacedPodsSelectors.addToMatchingNamespacedPodSelector(uid(obj), obj.ObjectMeta.Labels, ns.namespaceLabels, obj.Status.PodIP, podComment(obj))
+	if err != nil {
+		return err
+	}
+
 	return nil
 }
 
 func (ns *ns) updatePod(oldObj, newObj *coreapi.Pod) error {
-	delete(ns.pods, oldObj.ObjectMeta.UID)
-	ns.pods[newObj.ObjectMeta.UID] = newObj
+	delete(ns.pods, uid(oldObj))
+	ns.pods[uid(newObj)] = newObj
 
 	if !hasIP(oldObj) && !hasIP(newObj) {
 		return nil
 	}
 
 	if hasIP(oldObj) && !hasIP(newObj) {
-		if err := ns.ips.DelEntry(oldObj.ObjectMeta.UID, ns.ingressDefaultAllowIPSet, oldObj.Status.PodIP); err != nil {
+		if err := ns.ips.DelEntry(uid(oldObj), ns.ingressDefaultAllowIPSet, oldObj.Status.PodIP); err != nil {
 			return err
 		}
-		if err := ns.ips.DelEntry(oldObj.ObjectMeta.UID, ns.egressDefaultAllowIPSet, oldObj.Status.PodIP); err != nil {
+		if err := ns.ips.DelEntry(uid(oldObj), ns.egressDefaultAllowIPSet, oldObj.Status.PodIP); err != nil {
+			return err
+		}
+		if err := ns.namespacedPodsSelectors.delFromMatchingNamespacedPodSelector(uid(oldObj), oldObj.ObjectMeta.Labels, ns.namespaceLabels, oldObj.Status.PodIP); err != nil {
 			return err
 		}
 
-		return ns.podSelectors.delFromMatching(oldObj.ObjectMeta.UID, oldObj.ObjectMeta.Labels, oldObj.Status.PodIP)
+		return ns.podSelectors.delFromMatchingPodSelector(uid(oldObj), oldObj.ObjectMeta.Labels, oldObj.Status.PodIP)
 	}
 
 	if !hasIP(oldObj) && hasIP(newObj) {
-		foundIngress, foundEgress, err := ns.podSelectors.addToMatching(newObj.ObjectMeta.UID, newObj.ObjectMeta.Labels, newObj.Status.PodIP, podComment(newObj))
+		foundIngress, foundEgress, err := ns.podSelectors.addToMatchingPodSelector(uid(newObj), newObj.ObjectMeta.Labels, newObj.Status.PodIP, podComment(newObj))
 		if err != nil {
 			return err
 		}
 
 		if !foundIngress {
-			if err := ns.ips.AddEntry(newObj.ObjectMeta.UID, ns.ingressDefaultAllowIPSet, newObj.Status.PodIP, podComment(newObj)); err != nil {
+			if err := ns.ips.AddEntry(uid(newObj), ns.ingressDefaultAllowIPSet, newObj.Status.PodIP, podComment(newObj)); err != nil {
 				return err
 			}
 		}
 		if !foundEgress {
-			if err := ns.ips.AddEntry(newObj.ObjectMeta.UID, ns.egressDefaultAllowIPSet, newObj.Status.PodIP, podComment(newObj)); err != nil {
+			if err := ns.ips.AddEntry(uid(newObj), ns.egressDefaultAllowIPSet, newObj.Status.PodIP, podComment(newObj)); err != nil {
 				return err
 			}
 		}
+		err = ns.namespacedPodsSelectors.addToMatchingNamespacedPodSelector(uid(newObj), newObj.ObjectMeta.Labels, ns.namespaceLabels, newObj.Status.PodIP, podComment(newObj))
+		if err != nil {
+			return err
+		}
 
 		return nil
 	}
@@ -240,18 +250,18 @@ func (ns *ns) updatePod(oldObj, newObj *coreapi.Pod) error {
 		oldObj.Status.PodIP != newObj.Status.PodIP {
 
 		for _, ps := range ns.podSelectors.entries {
-			oldMatch := ps.matches(oldObj.ObjectMeta.Labels)
-			newMatch := ps.matches(newObj.ObjectMeta.Labels)
+			oldMatch := ps.matchesPodSelector(oldObj.ObjectMeta.Labels)
+			newMatch := ps.matchesPodSelector(newObj.ObjectMeta.Labels)
 			if oldMatch == newMatch && oldObj.Status.PodIP == newObj.Status.PodIP {
 				continue
 			}
 			if oldMatch {
-				if err := ps.delEntry(oldObj.ObjectMeta.UID, oldObj.Status.PodIP); err != nil {
+				if err := ps.delEntry(uid(oldObj), oldObj.Status.PodIP); err != nil {
 					return err
 				}
 			}
 			if newMatch {
-				if err := ps.addEntry(newObj.ObjectMeta.UID, newObj.Status.PodIP, podComment(newObj)); err != nil {
+				if err := ps.addEntry(uid(newObj), newObj.Status.PodIP, podComment(newObj)); err != nil {
 					return err
 				}
 			}
@@ -263,6 +273,23 @@ func (ns *ns) updatePod(oldObj, newObj *coreapi.Pod) error {
 				return err
 			}
 		}
+		for _, ps := range ns.namespacedPodsSelectors.entries {
+			oldMatch := ps.matchesNamespacedPodSelector(oldObj.ObjectMeta.Labels, ns.namespaceLabels)
+			newMatch := ps.matchesNamespacedPodSelector(newObj.ObjectMeta.Labels, ns.namespaceLabels)
+			if oldMatch == newMatch && oldObj.Status.PodIP == newObj.Status.PodIP {
+				continue
+			}
+			if oldMatch {
+				if err := ps.delEntry(uid(oldObj), oldObj.Status.PodIP); err != nil {
+					return err
+				}
+			}
+			if newMatch {
+				if err := ps.addEntry(uid(newObj), newObj.Status.PodIP, podComment(newObj)); err != nil {
+					return err
+				}
+			}
+		}
 	}
 
 	return nil
@@ -273,7 +300,7 @@ func (ns *ns) addOrRemoveToDefaultAllowIPSet(ps *selector, oldObj, newObj *corea
 	if ns.podSelectors.targetSelectorExist(ps, policyType) {
 		switch {
 		case !oldMatch && newMatch:
-			if err := ns.ips.DelEntry(oldObj.ObjectMeta.UID, ipset, oldObj.Status.PodIP); err != nil {
+			if err := ns.ips.DelEntry(uid(oldObj), ipset, oldObj.Status.PodIP); err != nil {
 				return err
 			}
 		case oldMatch && !newMatch:
@@ -286,26 +313,31 @@ func (ns *ns) addOrRemoveToDefaultAllowIPSet(ps *selector, oldObj, newObj *corea
 }
 
 func (ns *ns) deletePod(obj *coreapi.Pod) error {
-	delete(ns.pods, obj.ObjectMeta.UID)
+	delete(ns.pods, uid(obj))
 
 	if !hasIP(obj) {
 		return nil
 	}
 
-	if err := ns.ips.DelEntry(obj.ObjectMeta.UID, ns.ingressDefaultAllowIPSet, obj.Status.PodIP); err != nil {
+	if err := ns.ips.DelEntry(uid(obj), ns.ingressDefaultAllowIPSet, obj.Status.PodIP); err != nil {
 		return err
 	}
-	if err := ns.ips.DelEntry(obj.ObjectMeta.UID, ns.egressDefaultAllowIPSet, obj.Status.PodIP); err != nil {
+	if err := ns.ips.DelEntry(uid(obj), ns.egressDefaultAllowIPSet, obj.Status.PodIP); err != nil {
 		return err
 	}
-
-	return ns.podSelectors.delFromMatching(obj.ObjectMeta.UID, obj.ObjectMeta.Labels, obj.Status.PodIP)
+	if err := ns.podSelectors.delFromMatchingPodSelector(uid(obj), obj.ObjectMeta.Labels, obj.Status.PodIP); err != nil {
+		return err
+	}
+	if err := ns.namespacedPodsSelectors.delFromMatchingNamespacedPodSelector(uid(obj), obj.ObjectMeta.Labels, ns.namespaceLabels, obj.Status.PodIP); err != nil {
+		return err
+	}
+	return nil
 }
 
 func (ns *ns) addNetworkPolicy(obj interface{}) error {
 	// Analyse policy, determine which rules and ipsets are required
 
-	uid, rules, nsSelectors, podSelectors, ipBlocks, err := ns.analyse(obj)
+	uid, rules, nsSelectors, podSelectors, namespacedPodsSelectors, ipBlocks, err := ns.analyse(obj)
 	if err != nil {
 		return err
 	}
@@ -317,6 +349,9 @@ func (ns *ns) addNetworkPolicy(obj interface{}) error {
 	if err := ns.podSelectors.provision(uid, nil, podSelectors); err != nil {
 		return err
 	}
+	if err := ns.namespacedPodsSelectors.provision(uid, nil, namespacedPodsSelectors); err != nil {
+		return err
+	}
 	if err := ns.ipBlocks.provision(uid, nil, ipBlocks); err != nil {
 		return err
 	}
@@ -325,11 +360,11 @@ func (ns *ns) addNetworkPolicy(obj interface{}) error {
 
 func (ns *ns) updateNetworkPolicy(oldObj, newObj interface{}) error {
 	// Analyse the old and the new policy so we can determine differences
-	oldUID, oldRules, oldNsSelectors, oldPodSelectors, oldIPBlocks, err := ns.analyse(oldObj)
+	oldUID, oldRules, oldNsSelectors, oldPodSelectors, oldNamespacedPodsSelectors, oldIPBlocks, err := ns.analyse(oldObj)
 	if err != nil {
 		return err
 	}
-	newUID, newRules, newNsSelectors, newPodSelectors, newIPBlocks, err := ns.analyse(newObj)
+	newUID, newRules, newNsSelectors, newPodSelectors, newNamespacedPodsSelectors, newIPBlocks, err := ns.analyse(newObj)
 	if err != nil {
 		return err
 	}
@@ -347,6 +382,9 @@ func (ns *ns) updateNetworkPolicy(oldObj, newObj interface{}) error {
 	if err := ns.podSelectors.deprovision(oldUID, oldPodSelectors, newPodSelectors); err != nil {
 		return err
 	}
+	if err := ns.namespacedPodsSelectors.deprovision(oldUID, oldNamespacedPodsSelectors, newNamespacedPodsSelectors); err != nil {
+		return err
+	}
 	if err := ns.ipBlocks.deprovision(oldUID, oldIPBlocks, newIPBlocks); err != nil {
 		return err
 	}
@@ -356,6 +394,9 @@ func (ns *ns) updateNetworkPolicy(oldObj, newObj interface{}) error {
 	if err := ns.podSelectors.provision(oldUID, oldPodSelectors, newPodSelectors); err != nil {
 		return err
 	}
+	if err := ns.namespacedPodsSelectors.provision(oldUID, oldNamespacedPodsSelectors, newNamespacedPodsSelectors); err != nil {
+		return err
+	}
 	if err := ns.ipBlocks.provision(oldUID, oldIPBlocks, newIPBlocks); err != nil {
 		return err
 	}
@@ -364,7 +405,7 @@ func (ns *ns) updateNetworkPolicy(oldObj, newObj interface{}) error {
 
 func (ns *ns) deleteNetworkPolicy(obj interface{}) error {
 	// Analyse network policy to free resources
-	uid, rules, nsSelectors, podSelectors, ipBlocks, err := ns.analyse(obj)
+	uid, rules, nsSelectors, podSelectors, namespacedPodsSelectors, ipBlocks, err := ns.analyse(obj)
 	if err != nil {
 		return err
 	}
@@ -381,6 +422,9 @@ func (ns *ns) deleteNetworkPolicy(obj interface{}) error {
 	if err := ns.podSelectors.deprovision(uid, podSelectors, nil); err != nil {
 		return err
 	}
+	if err := ns.namespacedPodsSelectors.deprovision(uid, namespacedPodsSelectors, nil); err != nil {
+		return err
+	}
 	if err := ns.ipBlocks.deprovision(uid, ipBlocks, nil); err != nil {
 		return err
 	}
@@ -392,12 +436,12 @@ func (ns *ns) updateDefaultAllowIPSetEntry(oldObj, newObj *coreapi.Pod, ipsetNam
 	// Instead of iterating over all selectors we check whether old pod IP
 	// has been inserted into default-allow ipset to decide whether the IP
 	// in the ipset has to be updated.
-	if ns.ips.Exist(oldObj.ObjectMeta.UID, ipsetName, oldObj.Status.PodIP) {
+	if ns.ips.EntryExists(uid(oldObj), ipsetName, oldObj.Status.PodIP) {
 
-		if err := ns.ips.DelEntry(oldObj.ObjectMeta.UID, ipsetName, oldObj.Status.PodIP); err != nil {
+		if err := ns.ips.DelEntry(uid(oldObj), ipsetName, oldObj.Status.PodIP); err != nil {
 			return err
 		}
-		if err := ns.ips.AddEntry(newObj.ObjectMeta.UID, ipsetName, newObj.Status.PodIP, podComment(newObj)); err != nil {
+		if err := ns.ips.AddEntry(uid(newObj), ipsetName, newObj.Status.PodIP, podComment(newObj)); err != nil {
 			return err
 		}
 	}
@@ -406,12 +450,12 @@ func (ns *ns) updateDefaultAllowIPSetEntry(oldObj, newObj *coreapi.Pod, ipsetNam
 
 func bypassRules(namespace string, ingress, egress ipset.Name) map[string][][]string {
 	return map[string][][]string{
-		DefaultChain: {
+		chains.DefaultChain: {
 			{"-m", "set", "--match-set", string(ingress), "dst", "-j", "ACCEPT",
 				"-m", "comment", "--comment", "DefaultAllow ingress isolation for namespace: " + namespace},
 		},
-		EgressDefaultChain: {
-			{"-m", "set", "--match-set", string(egress), "src", "-j", EgressMarkChain,
+		chains.EgressDefaultChain: {
+			{"-m", "set", "--match-set", string(egress), "src", "-j", chains.EgressMarkChain,
 				"-m", "comment", "--comment", "DefaultAllow egress isolation for namespace: " + namespace},
 			{"-m", "set", "--match-set", string(egress), "src", "-j", "RETURN",
 				"-m", "comment", "--comment", "DefaultAllow egress isolation for namespace: " + namespace},
@@ -446,7 +490,8 @@ func (ns *ns) deleteBypassRules() error {
 }
 
 func (ns *ns) addNamespace(obj *coreapi.Namespace) error {
-	ns.namespace = obj
+	ns.namespaceUID = ipset.UID(obj.ObjectMeta.UID)
+	ns.namespaceLabels = obj.ObjectMeta.Labels
 
 	// Insert a rule to bypass policies
 	if err := ns.ensureBypassRules(); err != nil {
@@ -454,28 +499,28 @@ func (ns *ns) addNamespace(obj *coreapi.Namespace) error {
 	}
 
 	// Add namespace ipset to matching namespace selectors
-	_, _, err := ns.nsSelectors.addToMatching(obj.ObjectMeta.UID, obj.ObjectMeta.Labels, string(ns.allPods.ipsetName), namespaceComment(ns))
+	err := ns.nsSelectors.addToMatchingNamespaceSelector(nsuid(obj), obj.ObjectMeta.Labels, string(ns.allPods.ipsetName), namespaceComment(ns))
 	return err
 }
 
 func (ns *ns) updateNamespace(oldObj, newObj *coreapi.Namespace) error {
-	ns.namespace = newObj
-
 	// Re-evaluate namespace selector membership if labels have changed
 	if !equals(oldObj.ObjectMeta.Labels, newObj.ObjectMeta.Labels) {
+		ns.namespaceLabels = newObj.ObjectMeta.Labels
+
 		for _, selector := range ns.nsSelectors.entries {
-			oldMatch := selector.matches(oldObj.ObjectMeta.Labels)
-			newMatch := selector.matches(newObj.ObjectMeta.Labels)
+			oldMatch := selector.matchesNamespaceSelector(oldObj.ObjectMeta.Labels)
+			newMatch := selector.matchesNamespaceSelector(newObj.ObjectMeta.Labels)
 			if oldMatch == newMatch {
 				continue
 			}
 			if oldMatch {
-				if err := selector.delEntry(ns.namespace.ObjectMeta.UID, string(ns.allPods.ipsetName)); err != nil {
+				if err := selector.delEntry(ns.namespaceUID, string(ns.allPods.ipsetName)); err != nil {
 					return err
 				}
 			}
 			if newMatch {
-				if err := selector.addEntry(ns.namespace.ObjectMeta.UID, string(ns.allPods.ipsetName), namespaceComment(ns)); err != nil {
+				if err := selector.addEntry(ns.namespaceUID, string(ns.allPods.ipsetName), namespaceComment(ns)); err != nil {
 					return err
 				}
 			}
@@ -486,7 +531,7 @@ func (ns *ns) updateNamespace(oldObj, newObj *coreapi.Namespace) error {
 }
 
 func (ns *ns) deleteNamespace(obj *coreapi.Namespace) error {
-	ns.namespace = nil
+	ns.namespaceUID = ""
 
 	// Remove bypass rule
 	if err := ns.deleteBypassRules(); err != nil {
@@ -494,7 +539,7 @@ func (ns *ns) deleteNamespace(obj *coreapi.Namespace) error {
 	}
 
 	// Remove namespace ipset from any matching namespace selectors
-	err := ns.nsSelectors.delFromMatching(obj.ObjectMeta.UID, obj.ObjectMeta.Labels, string(ns.allPods.ipsetName))
+	err := ns.nsSelectors.delFromMatchingNamespaceSelector(nsuid(obj), obj.ObjectMeta.Labels, string(ns.allPods.ipsetName))
 	if err != nil {
 		return err
 	}
@@ -544,17 +589,17 @@ func (ns *ns) defaultAllowIPSetName(pt policyType) ipset.Name {
 }
 
 func (ns *ns) analyse(obj interface{}) (
-	uid types.UID,
+	uid ipset.UID,
 	rules map[string]*ruleSpec,
-	nsSelectors, podSelectors map[string]*selectorSpec,
+	nsSelectors, podSelectors, namespacedPodsSelectors map[string]*selectorSpec,
 	ipBlocks map[string]*ipBlockSpec,
 	err error) {
 
 	switch p := obj.(type) {
 	case *extnapi.NetworkPolicy:
-		uid = p.ObjectMeta.UID
+		uid = ipset.UID(p.ObjectMeta.UID)
 	case *networkingv1.NetworkPolicy:
-		uid = p.ObjectMeta.UID
+		uid = ipset.UID(p.ObjectMeta.UID)
 	default:
 		err = errInvalidNetworkPolicyObjType
 		return
@@ -562,7 +607,7 @@ func (ns *ns) analyse(obj interface{}) (
 	ns.policies[uid] = obj
 
 	// Analyse policy, determine which rules and ipsets are required
-	rules, nsSelectors, podSelectors, ipBlocks, err = ns.analysePolicy(obj.(*networkingv1.NetworkPolicy))
+	rules, nsSelectors, podSelectors, namespacedPodsSelectors, ipBlocks, err = ns.analysePolicy(obj.(*networkingv1.NetworkPolicy))
 	if err != nil {
 		return
 	}

npc/rule.go

@@ -5,10 +5,10 @@ import (
 	"reflect"
 	"strings"
 
-	"k8s.io/apimachinery/pkg/types"
-
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/npc/iptables"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/common/chains"
+	"github.com/rajch/weave/net/ipset"
+	"github.com/rajch/weave/npc/iptables"
 )
 
 type ruleHost interface {
@@ -55,9 +55,9 @@ func newRuleSpec(policyType policyType, proto *string, srcHost, dstHost ruleHost
 
 func (spec *ruleSpec) iptChain() string {
 	if spec.policyType == policyTypeEgress {
-		return EgressCustomChain
+		return chains.EgressCustomChain
 	}
-	return IngressChain
+	return chains.IngressChain
 }
 
 func (spec *ruleSpec) iptRuleSpecs() [][]string {
@@ -71,7 +71,7 @@ func (spec *ruleSpec) iptRuleSpecs() [][]string {
 	// policyTypeEgress
 	ruleMark := make([]string, len(spec.args))
 	copy(ruleMark, spec.args)
-	ruleMark = append(ruleMark, "-j", EgressMarkChain)
+	ruleMark = append(ruleMark, "-j", chains.EgressMarkChain)
 	ruleReturn := make([]string, len(spec.args))
 	copy(ruleReturn, spec.args)
 	ruleReturn = append(ruleReturn, "-j", "RETURN")
@@ -80,14 +80,14 @@ func (spec *ruleSpec) iptRuleSpecs() [][]string {
 
 type ruleSet struct {
 	ipt   iptables.Interface
-	users map[string]map[types.UID]struct{}
+	users map[string]map[ipset.UID]struct{}
 }
 
 func newRuleSet(ipt iptables.Interface) *ruleSet {
-	return &ruleSet{ipt, make(map[string]map[types.UID]struct{})}
+	return &ruleSet{ipt, make(map[string]map[ipset.UID]struct{})}
 }
 
-func (rs *ruleSet) deprovision(user types.UID, current, desired map[string]*ruleSpec) error {
+func (rs *ruleSet) deprovision(user ipset.UID, current, desired map[string]*ruleSpec) error {
 	for key, spec := range current {
 		if _, found := desired[key]; !found {
 			delete(rs.users[key], user)
@@ -108,7 +108,7 @@ func (rs *ruleSet) deprovision(user types.UID, current, desired map[string]*rule
 	return nil
 }
 
-func (rs *ruleSet) provision(user types.UID, current, desired map[string]*ruleSpec) error {
+func (rs *ruleSet) provision(user ipset.UID, current, desired map[string]*ruleSpec) error {
 	for key, spec := range desired {
 		if _, found := current[key]; !found {
 			if _, found := rs.users[key]; !found {
@@ -119,7 +119,7 @@ func (rs *ruleSet) provision(user types.UID, current, desired map[string]*ruleSp
 						return err
 					}
 				}
-				rs.users[key] = make(map[types.UID]struct{})
+				rs.users[key] = make(map[ipset.UID]struct{})
 			}
 			rs.users[key][user] = struct{}{}
 		}

npc/rule_test.go

@@ -1,13 +1,14 @@
 package npc
 
 import (
-	"github.com/stretchr/testify/require"
 	"testing"
+
+	"github.com/stretchr/testify/require"
 )
 
 func TestRegressionHandleNil3095(t *testing.T) {
 	// Test for handling nil values
-	// https://github.com/weaveworks/weave/issues/3095
+	// https://github.com/rajch/weave/issues/3095
 
 	rs := newRuleSpec(policyTypeIngress, nil, nil, nil, nil)
 

npc/selector.go

@@ -5,32 +5,45 @@ import (
 
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/labels"
-	"k8s.io/apimachinery/pkg/types"
 
-	"github.com/weaveworks/weave/common"
-	"github.com/weaveworks/weave/net/ipset"
+	"github.com/rajch/weave/common"
+	"github.com/rajch/weave/net/ipset"
 )
 
 type selectorSpec struct {
-	key         string          // string representation (for hash keying/equality comparison)
-	selector    labels.Selector // k8s Selector object (for matching)
-	policyTypes []policyType    // If non-empty, then selectorSpec is a target selector for given policyTypes.
+	key               string          // string representation (for hash keying/equality comparison)
+	podSelector       labels.Selector // k8s Selector object (for matching pods)
+	namespaceSelector labels.Selector // k8s Selector object (for matching namespaces)
+	policyTypes       []policyType    // If non-empty, then selectorSpec is a target selector for given policyTypes.
 
 	ipsetType ipset.Type // type of ipset to provision
 	ipsetName ipset.Name // generated ipset name
 	nsName    string     // Namespace name
 }
 
-func newSelectorSpec(json *metav1.LabelSelector, policyType []policyType, nsName string, ipsetType ipset.Type) (*selectorSpec, error) {
-	selector, err := metav1.LabelSelectorAsSelector(json)
-	if err != nil {
-		return nil, err
+func newSelectorSpec(podJSON *metav1.LabelSelector, namespaceJSON *metav1.LabelSelector, policyType []policyType, nsName string, ipsetType ipset.Type) (*selectorSpec, error) {
+	var podSelector, namespaceSelector labels.Selector
+	var err error
+	var key string
+	if namespaceJSON != nil {
+		namespaceSelector, err = metav1.LabelSelectorAsSelector(namespaceJSON)
+		if err != nil {
+			return nil, err
+		}
+		key = namespaceSelector.String()
+	}
+	if podJSON != nil {
+		podSelector, err = metav1.LabelSelectorAsSelector(podJSON)
+		if err != nil {
+			return nil, err
+		}
+		key = key + podSelector.String()
 	}
-	key := selector.String()
 	return &selectorSpec{
-		key:         key,
-		selector:    selector,
-		policyTypes: policyType,
+		key:               key,
+		podSelector:       podSelector,
+		namespaceSelector: namespaceSelector,
+		policyTypes:       policyType,
 		// We prefix the selector string with the namespace name when generating
 		// the shortname because you can specify the same selector in multiple
 		// namespaces - we need those to map to distinct ipsets
@@ -46,7 +59,7 @@ func (spec *selectorSpec) getRuleSpec(src bool) ([]string, string) {
 	}
 	rule := []string{"-m", "set", "--match-set", string(spec.ipsetName), dir}
 
-	comment := "anywhere"
+	var comment string
 	if spec.nsName != "" {
 		comment = fmt.Sprintf("pods: namespace: %s, selector: %s", spec.nsName, spec.key)
 	} else {
@@ -61,15 +74,23 @@ type selector struct {
 	spec *selectorSpec
 }
 
-func (s *selector) matches(labelMap map[string]string) bool {
-	return s.spec.selector.Matches(labels.Set(labelMap))
+func (s *selector) matchesPodSelector(podsLabelMap map[string]string) bool {
+	return s.spec.podSelector.Matches(labels.Set(podsLabelMap))
 }
 
-func (s *selector) addEntry(user types.UID, entry string, comment string) error {
+func (s *selector) matchesNamespaceSelector(namespaceLabelMap map[string]string) bool {
+	return s.spec.namespaceSelector.Matches(labels.Set(namespaceLabelMap))
+}
+
+func (s *selector) matchesNamespacedPodSelector(podsLabelMap, namespaceLabelMap map[string]string) bool {
+	return s.spec.podSelector.Matches(labels.Set(podsLabelMap)) && s.spec.namespaceSelector.Matches(labels.Set(namespaceLabelMap))
+}
+
+func (s *selector) addEntry(user ipset.UID, entry string, comment string) error {
 	return s.ips.AddEntry(user, s.spec.ipsetName, entry, comment)
 }
 
-func (s *selector) delEntry(user types.UID, entry string) error {
+func (s *selector) delEntry(user ipset.UID, entry string) error {
 	return s.ips.DelEntry(user, s.spec.ipsetName, entry)
 }
 
@@ -85,7 +106,7 @@ type selectorSet struct {
 	// invoked after the last instance of target selector has been deprovisioned
 	onDestroyTargetSelector selectorWithPolicyTypeFn
 
-	users   map[string]map[types.UID]struct{} // list of users per selector
+	users   map[string]map[ipset.UID]struct{} // list of users per selector
 	entries map[string]*selector
 
 	// We need to keep track of target selector instances to be able to invoke
@@ -100,16 +121,16 @@ func newSelectorSet(ips ipset.Interface, onNewSelector selectorFn, onNewTargetSe
 		onNewSelector:           onNewSelector,
 		onNewTargetSelector:     onNewTargetSelector,
 		onDestroyTargetSelector: onDestroyTargetSelector,
-		users:                make(map[string]map[types.UID]struct{}),
-		entries:              make(map[string]*selector),
-		targetSelectorsCount: make(map[string]map[policyType]int)}
+		users:                   make(map[string]map[ipset.UID]struct{}),
+		entries:                 make(map[string]*selector),
+		targetSelectorsCount:    make(map[string]map[policyType]int)}
 }
 
-func (ss *selectorSet) addToMatching(user types.UID, labelMap map[string]string, entry string, comment string) (bool, bool, error) {
+func (ss *selectorSet) addToMatchingPodSelector(user ipset.UID, podLabelsMap map[string]string, entry string, comment string) (bool, bool, error) {
 	foundIngress := false
 	foundEgress := false
 	for _, s := range ss.entries {
-		if s.matches(labelMap) {
+		if s.matchesPodSelector(podLabelsMap) {
 			if ss.targetSelectorExist(s, policyTypeIngress) {
 				foundIngress = true
 			}
@@ -124,9 +145,53 @@ func (ss *selectorSet) addToMatching(user types.UID, labelMap map[string]string,
 	return foundIngress, foundEgress, nil
 }
 
-func (ss *selectorSet) delFromMatching(user types.UID, labelMap map[string]string, entry string) error {
+func (ss *selectorSet) addToMatchingNamespaceSelector(user ipset.UID, namespaceLabelsMap map[string]string, entry string, comment string) error {
 	for _, s := range ss.entries {
-		if s.matches(labelMap) {
+		if s.matchesNamespaceSelector(namespaceLabelsMap) {
+			if err := s.addEntry(user, entry, comment); err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}
+
+func (ss *selectorSet) addToMatchingNamespacedPodSelector(user ipset.UID, podLabelsMap map[string]string, namespaceLabelsMap map[string]string, entry string, comment string) error {
+	for _, s := range ss.entries {
+		if s.matchesNamespacedPodSelector(podLabelsMap, namespaceLabelsMap) {
+			if err := s.addEntry(user, entry, comment); err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}
+
+func (ss *selectorSet) delFromMatchingPodSelector(user ipset.UID, podLabelsMap map[string]string, entry string) error {
+	for _, s := range ss.entries {
+		if s.matchesPodSelector(podLabelsMap) {
+			if err := s.delEntry(user, entry); err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}
+
+func (ss *selectorSet) delFromMatchingNamespaceSelector(user ipset.UID, namespaceLabelsMap map[string]string, entry string) error {
+	for _, s := range ss.entries {
+		if s.matchesNamespaceSelector(namespaceLabelsMap) {
+			if err := s.delEntry(user, entry); err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}
+
+func (ss *selectorSet) delFromMatchingNamespacedPodSelector(user ipset.UID, podLabelsMap map[string]string, namespaceLabelsMap map[string]string, entry string) error {
+	for _, s := range ss.entries {
+		if s.matchesNamespacedPodSelector(podLabelsMap, namespaceLabelsMap) {
 			if err := s.delEntry(user, entry); err != nil {
 				return err
 			}
@@ -139,7 +204,7 @@ func (ss *selectorSet) targetSelectorExist(s *selector, policyType policyType) b
 	return ss.targetSelectorsCount[s.spec.key][policyType] > 0
 }
 
-func (ss *selectorSet) deprovision(user types.UID, current, desired map[string]*selectorSpec) error {
+func (ss *selectorSet) deprovision(user ipset.UID, current, desired map[string]*selectorSpec) error {
 	for key, spec := range current {
 		if _, found := desired[key]; !found {
 			delete(ss.users[key], user)
@@ -167,7 +232,7 @@ func (ss *selectorSet) deprovision(user types.UID, current, desired map[string]*
 	return nil
 }
 
-func (ss *selectorSet) provision(user types.UID, current, desired map[string]*selectorSpec) error {
+func (ss *selectorSet) provision(user ipset.UID, current, desired map[string]*selectorSpec) error {
 	for key, spec := range desired {
 		if _, found := current[key]; !found {
 			selector := &selector{ss.ips, spec}
@@ -180,7 +245,7 @@ func (ss *selectorSet) provision(user types.UID, current, desired map[string]*se
 				if err := ss.onNewSelector(selector); err != nil {
 					return err
 				}
-				ss.users[key] = make(map[types.UID]struct{})
+				ss.users[key] = make(map[ipset.UID]struct{})
 				ss.entries[key] = selector
 			}
 			ss.users[key][user] = struct{}{}

npc/ulogd/ulogd.go

@@ -6,7 +6,7 @@ import (
 	"os/exec"
 	"syscall"
 
-	"github.com/weaveworks/weave/common"
+	"github.com/rajch/weave/common"
 )
 
 func waitForExit(cmd *exec.Cmd) {

original/.circleci/config.yml

@@ -1,7 +1,7 @@
 version: 2
 defaults: &defaults
   docker:
-    - image: weaveworks/weavebuild:circle20-2e6504d3
+    - image: weaveworks/weavebuild:master-1dfe1f3b
   working_directory: /go/src/github.com/weaveworks/weave
 
 jobs:

original/Makefile

@@ -0,0 +1,430 @@
+.DEFAULT: all
+.PHONY: all exes testrunner update tests lint publish-one-arch $(PUBLISH) clean clean-bin clean-work-dir prerequisites build run-smoketests
+
+# If you can use docker without being root, you can do "make SUDO="
+SUDO=$(shell docker info >/dev/null 2>&1 || echo "sudo -E")
+BUILD_IN_CONTAINER=true
+RM=--rm
+RUN_FLAGS=-ti
+COVERAGE=
+
+# This specifies the architecture we're building for
+ARCH?=amd64
+
+# We're using QEMU to be able to build Docker images for other architectures
+# from amd64 machines.
+QEMU_VERSION=v2.7.0
+
+# A list of all supported architectures here. Should be named as Go is naming platforms
+# All supported architectures must have an "ifeq" block below that customizes the parameters
+ALL_ARCHITECTURES=amd64 arm arm64 ppc64le s390x
+ML_PLATFORMS=linux/amd64,linux/arm,linux/arm64,linux/ppc64le,linux/s390x
+
+ifeq ($(ARCH),amd64)
+# The architecture to use when downloading the docker binary
+	WEAVEEXEC_DOCKER_ARCH?=x86_64
+
+# The name of the alpine baseimage to use as the base for weave images
+	ALPINE_BASEIMAGE?=alpine:3.10
+
+# The extension for the made images
+# Specifying none means for example weaveworks/weave:latest
+	ARCH_EXT?=
+
+# The name of the gcc we're using for compiling C code
+	CC=gcc
+
+# Optional parameters that can be passed when linking C code into the Go binary
+	CGO_LDFLAGS=
+endif
+ifeq ($(ARCH),arm)
+# The architecture to use when downloading the docker binary
+	WEAVEEXEC_DOCKER_ARCH?=armel
+
+# Using the (semi-)official alpine image
+	ALPINE_BASEIMAGE?=arm32v6/alpine:3.8
+
+# arm images have the -arm suffix, for instance weaveworks/weave-arm:latest
+	ARCH_EXT?=-arm
+
+# The name of the gcc binary
+	CC=arm-linux-gnueabihf-gcc
+
+# The architecture name to use when downloading a prebuilt QEMU binary
+	QEMUARCH=arm
+
+# In the weaveworks/build image; libpcap libraries for arm are placed here
+# Tell the gcc linker to search for libpcap here
+	CGO_LDFLAGS="-L/usr/local/lib/$(CC)"
+endif
+ifeq ($(ARCH),arm64)
+# Use the arm 32-bit docker client variant
+	WEAVEEXEC_DOCKER_ARCH?=armel
+
+# Using the (semi-)official alpine image
+	ALPINE_BASEIMAGE?=arm64v8/alpine:3.8
+
+# arm64 images have the -arm64 suffix, for instance weaveworks/weave-arm64:latest
+	ARCH_EXT?=-arm64
+
+# The name of the gcc binary
+	CC=aarch64-linux-gnu-gcc
+
+# The architecture name to use when downloading a prebuilt QEMU binary
+	QEMUARCH=aarch64
+
+# In the weaveworks/build image; libpcap libraries for arm64 are placed here
+# Tell the gcc linker to search for libpcap here
+	CGO_LDFLAGS="-L/usr/local/lib/$(CC)"
+endif
+ifeq ($(ARCH),ppc64le)
+# The architecture to use when downloading the docker binary
+	WEAVEEXEC_DOCKER_ARCH?=ppc64le
+
+# Using the (semi-)official alpine image
+	ALPINE_BASEIMAGE?=ppc64le/alpine:3.8
+
+# ppc64le images have the -ppc64le suffix, for instance weaveworks/weave-ppc64le:latest
+	ARCH_EXT?=-ppc64le
+
+# The name of the gcc binary
+	CC=powerpc64le-linux-gnu-gcc
+
+# The architecture name to use when downloading a prebuilt QEMU binary
+	QEMUARCH=ppc64le
+
+# In the weaveworks/build image; libpcap libraries for ppc64le are placed here
+# Tell the gcc linker to search for libpcap here
+	CGO_LDFLAGS="-L/usr/local/lib/$(CC)"
+endif
+ifeq ($(ARCH),s390x)
+# The architecture to use when downloading the docker binary
+	WEAVEEXEC_DOCKER_ARCH?=s390x
+
+# Using the (semi-)official alpine image
+	ALPINE_BASEIMAGE?=s390x/alpine:3.6
+
+# s390x images have the -s390x suffix, for instance weaveworks/weave-s390x:latest
+	ARCH_EXT?=-s390x
+
+# The name of the gcc binary
+	CC=s390x-linux-gnu-gcc
+
+# The architecture name to use when downloading a prebuilt QEMU binary
+	QEMUARCH=s390x
+
+# In the weaveworks/build image; libpcap libraries for s390x are placed here
+# Tell the gcc linker to search for libpcap here
+	CGO_LDFLAGS="-L/usr/local/lib/$(CC)"
+endif
+
+# The name of the user that this Makefile should produce image artifacts for. Can/should be overridden
+DOCKERHUB_USER?=weaveworks
+# The default version that's chosen when pushing the images. Can/should be overridden
+GIT_REVISION=$(shell git rev-parse HEAD)
+WEAVE_VERSION?=git-$(shell git rev-parse --short=12 HEAD)
+# Docker Store does not allow the "latest" tag.
+NET_PLUGIN_LATEST=latest_release
+
+# Paths to all relevant binaries that should be compiled
+WEAVER_EXE=prog/weaver/weaver
+SIGPROXY_EXE=prog/sigproxy/sigproxy
+KUBEUTILS_EXE=prog/kube-utils/kube-utils
+WEAVENPC_EXE=prog/weave-npc/weave-npc
+WEAVEWAIT_EXE=prog/weavewait/weavewait
+WEAVEWAIT_NOOP_EXE=prog/weavewait/weavewait_noop
+WEAVEWAIT_NOMCAST_EXE=prog/weavewait/weavewait_nomcast
+WEAVEUTIL_EXE=prog/weaveutil/weaveutil
+RUNNER_EXE=tools/runner/runner
+# Get this from https://github.com/estesp/manifest-tool/releases
+MANIFEST_TOOL_CMD=manifest-tool
+TEST_TLS_EXE=test/tls/tls
+NETWORKTESTER_EXE=test/images/network-tester/webserver
+
+# All binaries together in a list
+EXES=$(WEAVER_EXE) $(SIGPROXY_EXE) $(KUBEUTILS_EXE) $(WEAVENPC_EXE) $(WEAVEWAIT_EXE) $(WEAVEWAIT_NOOP_EXE) $(WEAVEWAIT_NOMCAST_EXE) $(WEAVEUTIL_EXE) $(RUNNER_EXE) $(TEST_TLS_EXE) $(NETWORKTESTER_EXE)
+
+# These stamp files are used to mark the current state of the build; whether an image has been built or not
+BUILD_UPTODATE=.build.uptodate
+WEAVER_UPTODATE=.weaver$(ARCH_EXT).uptodate
+WEAVEEXEC_UPTODATE=.weaveexec$(ARCH_EXT).uptodate
+PLUGIN_UPTODATE=.net-plugin$(ARCH_EXT).uptodate
+WEAVEKUBE_UPTODATE=.weavekube$(ARCH_EXT).uptodate
+WEAVENPC_UPTODATE=.weavenpc$(ARCH_EXT).uptodate
+WEAVEDB_UPTODATE=.weavedb.uptodate
+NETWORKTESTER_UPTODATE=.networktester$(ARCH_EXT).uptodate
+
+IMAGES_UPTODATE=$(WEAVER_UPTODATE) $(WEAVEEXEC_UPTODATE) $(WEAVEKUBE_UPTODATE) $(WEAVENPC_UPTODATE) $(NETWORKTESTER_UPTODATE)
+
+# The names of the images. Note that the images for other architectures than amd64 have a suffix in the image name.
+WEAVER_IMAGE=$(DOCKERHUB_USER)/weave$(ARCH_EXT)
+WEAVEEXEC_IMAGE=$(DOCKERHUB_USER)/weaveexec$(ARCH_EXT)
+WEAVEKUBE_IMAGE=$(DOCKERHUB_USER)/weave-kube$(ARCH_EXT)
+WEAVENPC_IMAGE=$(DOCKERHUB_USER)/weave-npc$(ARCH_EXT)
+BUILD_IMAGE=weaveworks/weavebuild
+WEAVEDB_IMAGE=$(DOCKERHUB_USER)/weavedb
+PLUGIN_IMAGE=$(DOCKERHUB_USER)/net-plugin
+NETWORKTESTER_IMAGE=$(DOCKERHUB_USER)/network-tester
+
+IMAGES=$(WEAVER_IMAGE) $(WEAVEEXEC_IMAGE) $(WEAVEKUBE_IMAGE) $(WEAVENPC_IMAGE) $(WEAVEDB_IMAGE) $(NETWORKTESTER_IMAGE)
+
+PLUGIN_WORK_DIR="prog/net-plugin/rootfs"
+PLUGIN_BUILD_IMG="plugin-builder"
+
+PUBLISH=publish_weave publish_weaveexec publish_weave-kube publish_weave-npc
+PUSH_ML=push_ml_weave push_ml_weaveexec push_ml_weave-kube push_ml_weave-npc
+
+WEAVE_EXPORT=weave$(ARCH_EXT).tar.gz
+
+# The flags we are passing to go build. -extldflags -static for making a static binary, 
+# -X main.version for telling the Go binary which version it is,
+# -s -w drop symbol tables used by debuggers (not Go's internal symbol info)
+# -tags "osusergo netgo" to use native Go UID and DNS implementations
+BUILD_FLAGS=-ldflags "-extldflags -static -X main.version=$(WEAVE_VERSION) -s -w" -tags "osusergo netgo"
+
+PACKAGE_BASE=$(shell go list -e ./)
+
+all: $(WEAVE_EXPORT)
+testrunner: $(RUNNER_EXE) $(TEST_TLS_EXE)
+
+$(WEAVER_EXE) $(WEAVEUTIL_EXE): common/*.go common/*/*.go net/*.go net/*/*.go
+$(WEAVER_EXE): router/*.go ipam/*.go ipam/*/*.go db/*.go nameserver/*.go prog/weaver/*.go
+$(WEAVER_EXE): api/*.go plugin/*.go plugin/*/*
+$(WEAVER_EXE):  proxy/*.go
+$(WEAVEUTIL_EXE): prog/weaveutil/*.go net/*.go plugin/net/*.go plugin/ipam/*.go db/*.go
+$(SIGPROXY_EXE): prog/sigproxy/*.go
+$(KUBEUTILS_EXE): prog/kube-utils/*.go
+$(WEAVENPC_EXE): prog/weave-npc/*.go npc/*.go npc/*/*.go
+$(TEST_TLS_EXE): test/tls/*.go
+$(RUNNER_EXE): tools/runner/*.go
+$(WEAVEWAIT_NOOP_EXE): prog/weavewait/*.go
+$(WEAVEWAIT_EXE): prog/weavewait/*.go net/*.go
+$(WEAVEWAIT_NOMCAST_EXE): prog/weavewait/*.go net/*.go
+$(NETWORKTESTER_EXE): test/images/network-tester/*.go
+tests: tools/.git
+lint: tools/.git
+
+ifeq ($(BUILD_IN_CONTAINER),true)
+
+# This make target compiles all binaries inside of the weaveworks/build container
+# It bind-mounts the source into the container and passes all important variables
+exes $(EXES) tests lint: $(BUILD_UPTODATE)
+	$(SUDO) docker run $(RM) $(RUN_FLAGS) \
+	    -v $(shell pwd):/go/src/github.com/weaveworks/weave \
+		-e GOARCH=$(ARCH) -e CGO_ENABLED=1 -e GOOS=linux -e CIRCLECI -e CIRCLE_BUILD_NUM -e CIRCLE_NODE_TOTAL -e CIRCLE_NODE_INDEX -e COVERDIR -e SLOW -e DEBUG \
+		$(BUILD_IMAGE) COVERAGE=$(COVERAGE) WEAVE_VERSION=$(WEAVE_VERSION) CC=$(CC) QEMUARCH=$(QEMUARCH) CGO_LDFLAGS=$(CGO_LDFLAGS) $@
+	touch $@
+
+else
+
+exes: $(EXES)
+
+$(WEAVER_EXE):
+ifeq ($(COVERAGE),true)
+	$(eval COVERAGE_MODULES := $(shell (go list ./$(@D); go list -f '{{join .Deps "\n"}}' ./$(@D) | grep "^$(PACKAGE_BASE)/") | grep -v "^$(PACKAGE_BASE)/vendor/" | paste -s -d,))
+	go test -c -o ./$@ $(BUILD_FLAGS) -v -covermode=atomic -coverpkg $(COVERAGE_MODULES) ./$(@D)/
+else
+	go build $(BUILD_FLAGS) -o $@ ./$(@D)
+endif
+
+$(WEAVEUTIL_EXE) $(KUBEUTILS_EXE) $(WEAVENPC_EXE) $(NETWORKTESTER_EXE):
+	go build $(BUILD_FLAGS) -o $@ ./$(@D)
+
+$(WEAVEWAIT_EXE):
+	go build $(BUILD_FLAGS) -tags "osusergo netgo iface mcast" -o $@ ./$(@D)
+
+$(WEAVEWAIT_NOMCAST_EXE):
+	go build $(BUILD_FLAGS) -tags "osusergo netgo iface" -o $@ ./$(@D)
+
+# These programs need a separate rule as they fail the netgo check in
+# the main build stanza due to not importing net package
+$(SIGPROXY_EXE) $(TEST_TLS_EXE) $(WEAVEWAIT_NOOP_EXE) $(RUNNER_EXE):
+	go build $(BUILD_FLAGS) -o $@ ./$(@D)
+
+tests:
+	./tools/test -no-go-get -netgo -timeout 8m
+
+lint:
+	./tools/lint -nocomment -notestpackage
+
+endif
+
+# This rule makes sure the build image is up-to-date.
+# It also makes sure the multiarch hooks are reqistered in the kernel so the QEMU emulation works
+$(BUILD_UPTODATE): build/* tools/.git
+	$(SUDO) docker build -t $(BUILD_IMAGE) build/
+	$(SUDO) docker tag $(BUILD_IMAGE) $(BUILD_IMAGE):$(shell tools/image-tag)
+	$(SUDO) docker run --rm --privileged multiarch/qemu-user-static:register --reset
+	touch $@
+
+# Creates the Dockerfile.your-user-here file from the template
+# Also replaces all placeholders with real values
+# If the architecture is amd64, it deletes all CROSS_BUILD lines
+# but otherwise, it only removes the "CROSS_BUILD_" placeholder and downloads QEMU
+%/Dockerfile.$(DOCKERHUB_USER): %/Dockerfile.template
+	echo "DOCKERHUB_USER|$(DOCKERHUB_USER)|g;s|ARCH_EXT|$(ARCH_EXT)|g;s|ALPINE_BASEIMAGE|$(ALPINE_BASEIMAGE)|g;s|QEMUARCH|$(QEMUARCH)"
+	sed -e "s|DOCKERHUB_USER|$(DOCKERHUB_USER)|g;s|ARCH_EXT|$(ARCH_EXT)|g;s|ALPINE_BASEIMAGE|$(ALPINE_BASEIMAGE)|g;s|QEMUARCH|$(QEMUARCH)|g" $^ > $@
+ifeq ($(ARCH),amd64)
+# When building "normally" for amd64, remove the whole line, it has no part in the amd64 image
+	sed -i.bak "/CROSS_BUILD_/d" $@
+else
+# When cross-building, only the placeholder "CROSS_BUILD_" should be removed
+# Register /usr/bin/qemu-ARCH-static as the handler for ARM and ppc64le binaries in the kernel
+	curl -sSL https://github.com/multiarch/qemu-user-static/releases/download/$(QEMU_VERSION)/x86_64_qemu-$(QEMUARCH)-static.tar.gz | tar -xz -C $(shell dirname $@)
+	cd $(shell dirname $@) && sha256sum -c $(shell pwd)/build/shasums/qemu-$(QEMUARCH)-static.sha256sum
+	sed -i.bak "s/CROSS_BUILD_//g" $@
+endif
+
+# The targets below builds the weave images
+$(WEAVER_UPTODATE): prog/weaver/Dockerfile.$(DOCKERHUB_USER) $(WEAVER_EXE) weave $(WEAVEUTIL_EXE)
+	cp $(WEAVEUTIL_EXE) prog/weaver/weaveutil
+	cp weave prog/weaver/weave
+	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker build --build-arg=revision=$(GIT_REVISION) -f prog/weaver/Dockerfile.$(DOCKERHUB_USER) -t $(WEAVER_IMAGE) prog/weaver
+	touch $@
+
+$(WEAVEEXEC_UPTODATE): prog/weaveexec/Dockerfile.$(DOCKERHUB_USER) prog/weaveexec/symlink $(SIGPROXY_EXE) $(WEAVEWAIT_EXE) $(WEAVEWAIT_NOOP_EXE) $(WEAVEWAIT_NOMCAST_EXE) $(WEAVER_UPTODATE)
+	cp $(SIGPROXY_EXE) prog/weaveexec/sigproxy
+	cp $(WEAVEWAIT_EXE) prog/weaveexec/weavewait
+	cp $(WEAVEWAIT_NOOP_EXE) prog/weaveexec/weavewait_noop
+	cp $(WEAVEWAIT_NOMCAST_EXE) prog/weaveexec/weavewait_nomcast
+	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker build --build-arg=revision=$(GIT_REVISION) -f prog/weaveexec/Dockerfile.$(DOCKERHUB_USER) -t $(WEAVEEXEC_IMAGE) prog/weaveexec
+	touch $@
+
+# Builds Docker plugin.
+$(PLUGIN_UPTODATE): prog/net-plugin/launch.sh prog/net-plugin/config.json $(WEAVER_UPTODATE)
+	-$(SUDO) docker rm -f $(PLUGIN_BUILD_IMG) 2>/dev/null
+	$(SUDO) docker create --name=$(PLUGIN_BUILD_IMG) $(WEAVER_IMAGE) true
+	rm -rf $(PLUGIN_WORK_DIR)
+	mkdir $(PLUGIN_WORK_DIR)
+	$(SUDO) docker export $(PLUGIN_BUILD_IMG) | tar -x -C $(PLUGIN_WORK_DIR)
+	$(SUDO) docker rm -f $(PLUGIN_BUILD_IMG)
+	cp prog/net-plugin/launch.sh $(PLUGIN_WORK_DIR)/home/weave/launch.sh
+	-$(SUDO) docker plugin disable $(PLUGIN_IMAGE):$(WEAVE_VERSION) 2>/dev/null
+	-$(SUDO) docker plugin rm $(PLUGIN_IMAGE):$(WEAVE_VERSION) 2>/dev/null
+	$(SUDO) docker plugin create $(PLUGIN_IMAGE):$(WEAVE_VERSION) prog/net-plugin
+	-$(SUDO) docker plugin disable $(PLUGIN_IMAGE):$(NET_PLUGIN_LATEST) 2>/dev/null
+	-$(SUDO) docker plugin rm $(PLUGIN_IMAGE):$(NET_PLUGIN_LATEST) 2>/dev/null
+	$(SUDO) docker plugin create $(PLUGIN_IMAGE):$(NET_PLUGIN_LATEST) prog/net-plugin
+	touch $@
+
+$(WEAVEKUBE_UPTODATE): prog/weave-kube/Dockerfile.$(DOCKERHUB_USER) prog/weave-kube/launch.sh $(KUBEUTILS_EXE) $(WEAVER_UPTODATE)
+	cp $(KUBEUTILS_EXE) prog/weave-kube/
+	$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) -f prog/weave-kube/Dockerfile.$(DOCKERHUB_USER) -t $(WEAVEKUBE_IMAGE) prog/weave-kube
+	touch $@
+
+$(WEAVENPC_UPTODATE): prog/weave-npc/Dockerfile.$(DOCKERHUB_USER) $(WEAVENPC_EXE) prog/weave-npc/ulogd.conf
+	$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) -f prog/weave-npc/Dockerfile.$(DOCKERHUB_USER) -t $(WEAVENPC_IMAGE) prog/weave-npc
+	touch $@
+
+$(WEAVEDB_UPTODATE): prog/weavedb/Dockerfile
+	$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) -t $(WEAVEDB_IMAGE) prog/weavedb
+	touch $@
+
+$(NETWORKTESTER_UPTODATE): test/images/network-tester/Dockerfile $(NETWORKTESTER_EXE)
+	$(SUDO) docker build -t $(NETWORKTESTER_IMAGE) test/images/network-tester
+	touch $@
+
+$(WEAVE_EXPORT): $(IMAGES_UPTODATE) $(WEAVEDB_UPTODATE)
+	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker save $(addsuffix :latest,$(IMAGES)) | gzip > $@
+
+tools/.git:
+	git submodule update --init
+
+# CODE FOR PUBLISHING THE IMAGES
+
+# Push plugin
+plugin_publish: $(PLUGIN_UPTODATE)
+	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker plugin push $(PLUGIN_IMAGE):$(WEAVE_VERSION)
+# "$(NET_PLUGIN_LATEST)" means "stable release" here, so only push that if explicitly told to
+ifeq ($(UPDATE_LATEST),true)
+	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker plugin push $(PLUGIN_IMAGE):$(NET_PLUGIN_LATEST)
+endif
+
+# This target first runs "make publish" for each architecture
+# Then it pushes the manifest lists
+publish: $(addprefix sub-publish-,$(ALL_ARCHITECTURES)) $(WEAVEDB_UPTODATE)
+	$(MAKE) DOCKER_HOST=$(DOCKER_HOST) DOCKERHUB_USER=$(DOCKERHUB_USER) WEAVE_VERSION=$(WEAVE_VERSION) UPDATE_LATEST=$(UPDATE_LATEST) $(PUSH_ML)
+ifeq ($(PUBLISH_WEAVEDB),true)
+	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker push $(DOCKERHUB_USER)/weavedb:latest
+endif
+
+sub-publish-%:
+	$(MAKE) ARCH=$* DOCKER_HOST=$(DOCKER_HOST) clean-bin
+	$(MAKE) ARCH=$* DOCKER_HOST=$(DOCKER_HOST) DOCKERHUB_USER=$(DOCKERHUB_USER) WEAVE_VERSION=$(WEAVE_VERSION) UPDATE_LATEST=$(UPDATE_LATEST) $(IMAGES_UPTODATE)
+	$(MAKE) ARCH=$* DOCKER_HOST=$(DOCKER_HOST) DOCKERHUB_USER=$(DOCKERHUB_USER) WEAVE_VERSION=$(WEAVE_VERSION) UPDATE_LATEST=$(UPDATE_LATEST) publish-one-arch
+
+publish-one-arch: $(PUBLISH)
+
+# This rule handles the pushing of the built images
+$(PUBLISH): publish_%: $(IMAGES_UPTODATE)
+# Tag :latest with the real version
+	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker tag  $(DOCKERHUB_USER)/$*$(ARCH_EXT) $(DOCKERHUB_USER)/$*$(ARCH_EXT):$(WEAVE_VERSION)
+# Push the image with the arch suffix for arm, arm64 and ppc64le, and without suffix for amd64
+	$(MAKE) DOCKER_HOST=$(DOCKER_HOST) DOCKERHUB_USER=$(DOCKERHUB_USER) WEAVE_VERSION=$(WEAVE_VERSION) UPDATE_LATEST=$(UPDATE_LATEST) push_$*$(ARCH_EXT)
+
+ifeq ($(ARCH),amd64)
+# If the architecture is amd64, add the -amd64 suffix.
+	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker tag  $(DOCKERHUB_USER)/$*:$(WEAVE_VERSION) $(DOCKERHUB_USER)/$*-amd64:$(WEAVE_VERSION)
+# If the version is latest, tag the -amd64 with latest as well
+ifneq ($(UPDATE_LATEST),false)
+	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker tag  $(DOCKERHUB_USER)/$*-amd64:$(WEAVE_VERSION) $(DOCKERHUB_USER)/$*-amd64:latest
+endif
+# Push the image with the -amd64-suffix so BINARY-ARCH-named images exist for all arches, so manifest lists may be made and ARCH is replaceable in scripts
+	$(MAKE) DOCKER_HOST=$(DOCKER_HOST) DOCKERHUB_USER=$(DOCKERHUB_USER) WEAVE_VERSION=$(WEAVE_VERSION) UPDATE_LATEST=$(UPDATE_LATEST) push_$*-amd64
+endif
+
+# This target pushes an image, and if UPDATE_LATEST is anything but false, also updates the latest tag
+# It takes one parameter, the image name
+push_%:
+	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker push $(DOCKERHUB_USER)/$*:$(WEAVE_VERSION)
+ifneq ($(UPDATE_LATEST),false)
+	$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker push $(DOCKERHUB_USER)/$*:latest
+endif
+
+# This target pushes a manifest list; it takes one parameter, the image name.
+# The variable UPDATE_LATEST controls whether it updates the versioned tag and/or the latest tag
+$(PUSH_ML): push_ml_%:
+ifneq ($(UPDATE_LATEST),latest-only)
+	$(MANIFEST_TOOL_CMD) push from-args --platforms $(ML_PLATFORMS) --template $(DOCKERHUB_USER)/$*-ARCH:$(WEAVE_VERSION) --target $(DOCKERHUB_USER)/$*:$(WEAVE_VERSION)
+endif
+ifneq ($(UPDATE_LATEST),false)
+# Push the manifest list to :latest as well
+	$(MANIFEST_TOOL_CMD) push from-args --platforms $(ML_PLATFORMS) --template $(DOCKERHUB_USER)/$*-ARCH:latest --target $(DOCKERHUB_USER)/$*:latest
+endif
+
+clean-work-dir:
+	rm -rf $(PLUGIN_WORK_DIR)
+
+clean-bin:
+	-$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker rmi $(IMAGES)
+	find prog -type f -name "Dockerfile.*" -not -name "Dockerfile.template" -print | xargs rm -f
+	find prog -type f -name "*qemu-*" -print | xargs rm -f
+	go clean ./...
+	rm -rf $(EXES) $(IMAGES_UPTODATE) $(WEAVEDB_UPTODATE) weave*.tar.gz .cache
+
+clean: clean-bin clean-work-dir
+	-$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker rmi $(BUILD_IMAGE)
+	rm -rf test/tls/*.pem test/coverage.* test/coverage $(BUILD_UPTODATE)
+
+build:
+	$(MAKE)
+
+run-smoketests: all testrunner
+	cd test && ./setup.sh && ./run_all.sh
+
+integration-tests: all testrunner
+# Usage:
+#   $ make \
+#     NAME="<prefix used to name VMs and other resources>" \
+#     PROVIDER="<provider among {vagrant|gcp|aws|do}>" \
+#     NUM_HOSTS="<# test machines>" \
+#     PLAYBOOK="<filename>" \
+#     RUNNER_ARGS="<...>" \
+#     TESTS="<...>" \  # Can be set to only run one or a few tests instead of the full test suite.
+#     DOCKER_VERSION=<...> \
+#     KUBERNETES_VERSION=<...> \
+#     KUBERNETES_CNI_VERSION=<...> \
+#     <...> # See also run-integration-test.sh for all variables and individual functions.
+#     integration-tests
+#
+	RUNNER_ARGS="-parallel" ./test/run-integration-tests.sh

original/README.md

@@ -0,0 +1,37 @@
+# Weave Net - Weaving Containers into Applications
+
+[![Integration Tests](https://circleci.com/gh/weaveworks/weave/tree/master.svg?style=shield)](https://circleci.com/gh/weaveworks/weave)
+[![Coverage Status](https://coveralls.io/repos/weaveworks/weave/badge.svg)](https://coveralls.io/r/weaveworks/weave)
+[![Go Report Card](https://goreportcard.com/badge/github.com/weaveworks/weave)](https://goreportcard.com/report/github.com/weaveworks/weave)
+[![Docker Pulls](https://img.shields.io/docker/pulls/weaveworks/weave.svg?maxAge=604800)](https://hub.docker.com/r/weaveworks/weave/)
+
+# About Weaveworks
+
+[Weaveworks](https://www.weave.works) is the company that delivers the most productive way for developers to connect, observe and control Docker containers.
+
+This repository contains [Weave Net](https://www.weave.works/products/weave-net/), the first product developed by Weaveworks, with over 8 million downloads to date. Weave Net enables you to get started with Docker clusters and portable apps in a fraction of the time required by other solutions.
+
+# Weave Net
+
+Weave Net creates a virtual network that connects Docker containers across multiple hosts and enables their automatic discovery. With Weave Net, portable microservices-based applications consisting of multiple containers can run anywhere: on one host, multiple hosts or even across cloud providers and data centers. Applications use the network just as if the containers were all plugged into the same network switch, without having to configure port mappings, ambassadors or links.
+
+Services provided by application containers on the Weave network can be exposed to the outside world, regardless of where they are running. Similarly, existing internal systems can be opened to accept connections from application containers irrespective of their location.
+
+## <a name="help"></a>Getting help
+
+If you have any questions about, feedback for or a problem with Weave Net:
+
+- Read [the Weave Net docs](https://www.weave.works/docs/net/latest/overview).
+- Invite yourself to the <a href="https://slack.weave.works/" target="_blank">Weave Users Slack</a>.
+  - The channel for contributors and developers of Weave Net is [#net-dev](https://weave-community.slack.com/messages/net-dev/).
+- Join the <a href="https://www.meetup.com/pro/Weave/"> Weave User Group </a> and get invited to online talks, hands-on training and meetups in your area.
+- [File an issue](https://github.com/weaveworks/weave/issues/new).
+
+We follow the [CNCF Code of Conduct](code-of-conduct.md).
+
+Your feedback is always welcome!
+
+## Further Information
+
+* [Installing Weave Net](https://www.weave.works/docs/net/latest/install/installing-weave)
+* [Weave Net Product Overview](https://www.weave.works/products/weave-net)

original/SECURITY.md

@@ -0,0 +1,12 @@
+# Security Policy
+
+At Weaveworks we take security very seriously, and value our close relationship with members of the security community.
+
+## Supported Versions
+
+As an Open Source project, only the latest code has all fixes and security patches.
+If you require a higher level of commitment, please [contact Weaveworks](https://www.weave.works/contact/) to discuss.
+
+## Reporting a Vulnerability
+
+To submit a security bug report please e-mail us at security@weave.works.

original/bin/provision_test_vms.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+
+set -e
+
+# Signal failures in lock file, in order to fail fast:
+function signal_failure() {
+    echo "KO" >"$TEST_VMS_PROV_AND_CONF_LOCK_FILE"
+    exit 1
+}
+trap signal_failure ERR
+
+function install_terraform() {
+    curl -fsS https://releases.hashicorp.com/terraform/0.11.14/terraform_0.11.14_linux_amd64.zip | gunzip >terraform && chmod +x terraform && sudo mv terraform /usr/bin
+}
+
+[ -n "$SECRET_KEY" ] || {
+    echo "Cannot run smoke tests: no secret key"
+    exit 1
+}
+
+source "$SRCDIR/bin/circle-env"
+
+install_terraform
+
+# Only attempt to create GCP image in first container, wait for it to be created otherwise:
+[ "$CIRCLE_NODE_INDEX" != "0" ] && export CREATE_IMAGE=0
+
+# Provision and configure testing VMs:
+cd "$SRCDIR/test" # Ensures we generate Terraform state files in the right folder, for later use by integration tests.
+./run-integration-tests.sh configure
+echo "OK" >"$TEST_VMS_PROV_AND_CONF_LOCK_FILE"
+echo "Test VMs now provisioned and configured. $(date)."

original/bin/release

@@ -89,10 +89,9 @@ build() {
     ## Inject the version numbers and build the distributables
     ## (library versions?)
     sed -i.bak "/SCRIPT_VERSION=/ c\SCRIPT_VERSION=\"$VERSION\"" ./weave
-    sed -i.bak -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset.yaml
-    sed -i.bak -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset-k8s-1.6.yaml
-    sed -i.bak -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset-k8s-1.7.yaml
     sed -i.bak -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset-k8s-1.8.yaml
+    sed -i.bak -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset-k8s-1.9.yaml
+    sed -i.bak -e "s/:latest/:$VERSION/" -e "/imagePullPolicy: Always/d" ./prog/weave-kube/weave-daemonset-k8s-1.11.yaml
     make SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER
 
     if make tests; then
@@ -161,34 +160,27 @@ draft() {
         --name "weave" \
         --file "./weave"
 
-    github-release upload \
-        --user $GITHUB_USER \
-        --repo weave \
-        --tag $LATEST_TAG \
-        --name "weave-daemonset.yaml" \
-        --file "./prog/weave-kube/weave-daemonset.yaml"
-
-    github-release upload \
-        --user $GITHUB_USER \
-        --repo weave \
-        --tag $LATEST_TAG \
-        --name "weave-daemonset-k8s-1.6.yaml" \
-        --file "./prog/weave-kube/weave-daemonset-k8s-1.6.yaml"
-
-    github-release upload \
-        --user $GITHUB_USER \
-        --repo weave \
-        --tag $LATEST_TAG \
-        --name "weave-daemonset-k8s-1.7.yaml" \
-        --file "./prog/weave-kube/weave-daemonset-k8s-1.7.yaml"
-
     github-release upload \
         --user $GITHUB_USER \
         --repo weave \
         --tag $LATEST_TAG \
         --name "weave-daemonset-k8s-1.8.yaml" \
         --file "./prog/weave-kube/weave-daemonset-k8s-1.8.yaml"
-    
+
+    github-release upload \
+        --user $GITHUB_USER \
+        --repo weave \
+        --tag $LATEST_TAG \
+        --name "weave-daemonset-k8s-1.9.yaml" \
+        --file "./prog/weave-kube/weave-daemonset-k8s-1.9.yaml"
+ 
+    github-release upload \
+        --user $GITHUB_USER \
+        --repo weave \
+        --tag $LATEST_TAG \
+        --name "weave-daemonset-k8s-1.11.yaml" \
+        --file "./prog/weave-kube/weave-daemonset-k8s-1.11.yaml"
+
     echo "** Draft $TYPE $RELEASE_NAME $VERSION created at"
     echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/$LATEST_TAG"
 }
@@ -276,27 +268,6 @@ publish() {
             --name "weave" \
             --file "./weave"
 
-        github-release upload \
-            --user $GITHUB_USER \
-            --repo weave \
-            --tag latest_release \
-            --name "weave-daemonset.yaml" \
-            --file "./prog/weave-kube/weave-daemonset.yaml"
-
-        github-release upload \
-            --user $GITHUB_USER \
-            --repo weave \
-            --tag latest_release \
-            --name "weave-daemonset-k8s-1.6.yaml" \
-            --file "./prog/weave-kube/weave-daemonset-k8s-1.6.yaml"
-
-        github-release upload \
-            --user $GITHUB_USER \
-            --repo weave \
-            --tag latest_release \
-            --name "weave-daemonset-k8s-1.7.yaml" \
-            --file "./prog/weave-kube/weave-daemonset-k8s-1.7.yaml"
-
         github-release upload \
             --user $GITHUB_USER \
             --repo weave \
@@ -304,6 +275,20 @@ publish() {
             --name "weave-daemonset-k8s-1.8.yaml" \
             --file "./prog/weave-kube/weave-daemonset-k8s-1.8.yaml"
 
+         github-release upload \
+            --user $GITHUB_USER \
+            --repo weave \
+            --tag latest_release \
+            --name "weave-daemonset-k8s-1.9.yaml" \
+            --file "./prog/weave-kube/weave-daemonset-k8s-1.9.yaml"
+
+         github-release upload \
+            --user $GITHUB_USER \
+            --repo weave \
+            --tag latest_release \
+            --name "weave-daemonset-k8s-1.11.yaml" \
+            --file "./prog/weave-kube/weave-daemonset-k8s-1.11.yaml"
+
         echo "** Release $RELEASE_NAME $VERSION published at"
         echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/$LATEST_TAG"
         echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/latest_release"

original/build/Dockerfile

@@ -0,0 +1,82 @@
+FROM golang:1.15.6-buster
+
+# Support Raspberry Pi 2 and newer
+ENV GOARM 7
+
+# The names of the architectures that we're fetching cross-compilers for from the Debian packages
+ENV DEB_CROSSPLATFORMS \
+	armhf \
+	arm64 \
+	ppc64el \
+	s390x
+
+# The go platforms we are supporting to cross-compile
+ENV GO_CROSSPLATFORMS \
+	linux/arm \
+	linux/arm64 \
+	linux/ppc64le \
+	linux/s390x
+
+# The names of the gcc cross-compilers we have installed
+ENV GCC_CROSSCOMPILERS \
+	arm-linux-gnueabihf \
+	aarch64-linux-gnu \
+	powerpc64le-linux-gnu \
+	s390x-linux-gnu
+
+# Install the build-essential and crossbuild-essential-ARCH packages
+RUN for platform in ${DEB_CROSSPLATFORMS}; do dpkg --add-architecture $platform; done \
+  && apt-get update \
+  && apt-get install -y build-essential \
+  && for platform in ${DEB_CROSSPLATFORMS}; do apt-get install -y crossbuild-essential-${platform}; done \
+  && apt-get clean && rm -rf /var/lib/apt/lists/*
+
+# Install some required packages
+# libpcap is required because we're linking against its C libraries from the prog/weaver binary
+# flex and bison are required packages for compiling libpcap manually later
+RUN apt-get update \
+	&& apt-get install -y \
+		libpcap-dev \
+		python-requests \
+		time \
+		file \
+		flex \
+		bison
+
+RUN curl -fsSLo shfmt https://github.com/mvdan/sh/releases/download/v1.3.0/shfmt_v1.3.0_linux_amd64 && \
+	echo "b1925c2c405458811f0c227266402cf1868b4de529f114722c2e3a5af4ac7bb2  shfmt" | sha256sum -c && \
+	chmod +x shfmt && \
+	mv shfmt /usr/bin
+
+# Install common Go tools
+RUN go get \
+    github.com/weaveworks/build-tools/cover \
+    github.com/mattn/goveralls \
+	golang.org/x/lint/golint \
+	github.com/fzipp/gocyclo \
+	github.com/fatih/hclfmt \
+	github.com/client9/misspell/cmd/misspell
+
+# Allow full write access to the Go folders for anyone
+RUN chmod -R a+w /usr/local/go
+
+# The libpcap version from Debian packages is 1.6.2, matching that version here although newer versions of libpcap have been released
+# We have to cross-compile the libpcap library for the new architectures, that's what we're doing here
+ENV LIBPCAP_CROSS_VERSION=1.6.2
+RUN curl -sSL https://www.tcpdump.org/release/libpcap-${LIBPCAP_CROSS_VERSION}.tar.gz | tar -xz \
+	&& cd libpcap-${LIBPCAP_CROSS_VERSION} \
+	&& for crosscompiler in ${GCC_CROSSCOMPILERS}; do \
+		CC=${crosscompiler}-gcc ac_cv_linux_vers=2 ./configure --host=${crosscompiler} --with-pcap=linux \
+		&& make \
+		&& export LIB_DIR="/usr/local/lib/${crosscompiler}-gcc" VER="$(cat ./VERSION)" MAJOR_VER="$(sed 's/\([0-9][0-9]*\)\..*/\1/' ./VERSION)" \
+		&& mkdir -p ${LIB_DIR} \
+		&& cp -f libpcap.a libpcap.so.${VER} ${LIB_DIR} \
+		&& ln -sf libpcap.so.${VER} /usr/local/lib/libpcap.so.${MAJOR_VER} \
+		&& ln -sf libpcap.so.${MAJOR_VER} /usr/local/lib/libpcap.so \
+		&& make clean; done
+
+# Install Docker
+RUN curl -fsSL https://get.docker.com | VERSION=18.06.0-ce /bin/sh
+
+COPY build.sh /
+ENTRYPOINT ["sh", "/build.sh"]