A Service of type LoadBalancer with loadBalancerClass
reweave.levk.dev/node-address names its address in spec.loadBalancerIP.
Each node's weave-npc publishes the IPv4 addresses on its host interfaces
and its weave bridge (Node annotations reweave.levk.dev/addresses and
reweave.levk.dev/weave). The one node that holds a Service's address is
its holder: it alone writes status.loadBalancer (ipMode Proxy, so
kube-proxy installs nothing for the address) and DNATs the address's ports
to ready endpoints (EndpointSlices), without SNAT, from nat PREROUTING
ahead of kube-proxy. Every other node marks the connections the holder
forwards to its pods (new, from the holder's bridge MAC, from outside the
pod network) and routes their replies back to the holder's bridge through
a per-holder policy rule and table, so the holder's conntrack undoes the
DNAT and the reply leaves from the address the client connected to.
Fail closed: an address no node holds, or that several nodes hold, is
served by no node, and its status is cleared (by its holder, or by the
first node by name if none holds it). Compute is the policy as a pure
function; Datapath replaces exactly the chains it owns with
iptables-restore --noflush and keeps its rules and tables exact.
weave-npc runs it unless --lb=false; the ClusterRole gains services
(watch), services/status (patch), endpointslices (watch), nodes (patch).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UHXzGrsxfkGk8LmwrBbHNZ
* Module name changed to `github.com/rajch/weave`
* Added environment variable `CHECKPOINT_DISABLE=1` to default manifests, to bypass weaveworks telemetry
This allows the main container to run without write access to the host
filesystem, except for two specific areas, thus trimming the attack
surface.
We lose the behaviour that the CNI plugin is only installed if Weave Net
gets up and running at least once, so the error reports from kubelet
will be slightly different in the case that it never manages to run.
This setting is passed by Kubelet to the Linux scheduler to ask for a share of
available CPU, so we should ask for a bit more than 1% of 1 CPU. Otherwise if a
CPU is contended Weave Net will get throttled frequently.
Weave Net in fastdp mode shouldn't need a lot of CPU most of the time, but at
startup there is quite a lot to do.
The setting is also used by the Kubernetes scheduler to decide how much will fit
on a node, so setting it much higher will reserve CPU that we shouldn't need most
of the time.
* fixes#3691
* `weave-daemonset-k8s-1.11.yaml` is `weave-daemonset-k8s-1.9.yaml` plus `priorityClassName: system-node-critical`
* notice that a Kubernetes cluster created with `kubeadm` has its `kube-proxy`s marked as `system-node-critical` (and several other pods, such as `kube-apiserver` marked as `system-cluster-critical`)
The liveness probe has the not-so-great side effect of killing the weave pods
if anything goes wrong, making debugging really difficult.
Not only that, we also bumped into the livenessProbe killing pods before they
had a chance to start (see below for the full story).
The readinessProbe is only used for Kubernetes Services: no traffic will be
routed to non-Ready pods when using the Service IP. We can use to to signal the
user something in wrong with Weave Net while not having Kubernetes delete the
pods behind our backs.
I removed the ininitialDelaySeconds field as Kurbernetes won't kill the pod if
not ready.
Notes from a debugging session:
Some nodes may have a vast amount of iptables rules, making each iptables
command take a few seconds (~5-10s). At startup, when the weave container
creates its chains, it runs quite a few iptables commands and the total time of
execution was > 30s.
It's important to note that, at this point, the weave process hasn't started as
those iptables commands are run by an initial script.
Kubernetes was trying to check for the container liveness, was failing, and
then proceeded to kill the weave pods before they had a chance to fully start.