We expect the loopback address 127.0.0.1 to be accessible only from
the local host.
We need to allow RELATED,ESTABLISHED traffic which may be the return
path of a Kubernetes HostPort.
This setting is passed by Kubelet to the Linux scheduler to ask for a share of
available CPU, so we should ask for a bit more than 1% of 1 CPU. Otherwise if a
CPU is contended Weave Net will get throttled frequently.
Weave Net in fastdp mode shouldn't need a lot of CPU most of the time, but at
startup there is quite a lot to do.
The setting is also used by the Kubernetes scheduler to decide how much will fit
on a node, so setting it much higher will reserve CPU that we shouldn't need most
of the time.
We were seeing flakes in test 320, seemingly because the 'docker run'
command would sometimes send a delete request for the container it had
started, before dying.
We need the container to carry on running so we can check it does get
attached when the cluster reaches quorum. Sending signal 9 means the
command dies immediately and does not get a chance to delete the
container.
Now we have a callback when a node is removed, there should be no
reason to wait 90 seconds.
The reclaim does wait up to 5 seconds before starting, so allowing 10
seconds in the script should be sufficient.
Turn some Sprint() calls into `%v` in the outer format string.
Remove one debug log line on duplicate token, since that info is already logged.
Add an Info log line when we accept another peer's token, since that
should be a rare event, and it's important if we've made a mistake.
Ring wraps around at zero but Space doesn't know about this, so we
have to normalise ranges coming out of Ring. In turn we may need to
split one range into two, across the end of the Ring.
Docker imposes a limit that events have to be retrieved within 100ms
or they are dropped on the sending side. Adding buffer space within
the Weave Net process allows those events to be collected from Docker
more quickly.
1024 is an arbitrary number, much higher than we expect to need.
* fixes#3691
* `weave-daemonset-k8s-1.11.yaml` is `weave-daemonset-k8s-1.9.yaml` plus `priorityClassName: system-node-critical`
* notice that a Kubernetes cluster created with `kubeadm` has its `kube-proxy`s marked as `system-node-critical` (and several other pods, such as `kube-apiserver` marked as `system-cluster-critical`)
Fixes 3675
It is possible for ipsec.InitSARemote to be called after ipsec.Destroy.
If an xfrm policy for that source/destination pair exists it will be
updated to the SPI from the Destroyed connection causing IPSEC to stop
working.
Added a check to make sure the xfrm policy has not yet been updated to
a new spi. The ipsec mutex is held at this point so nothing else should
be modifying this policy.
in network policies
- enhance selectorSpec to accomodate both pod and namespace selectors
- enhance analysePolicy to handle policies with both selectors
Fixes#3312
ask yourself: does `enables` refer to `this repository` or `weave net`, and it should become clear that it's unclear at best. if you take out the parentheticals, it reads:
> This repository contains Weave Net enables you to get started...
breaking into two sentences helps smooth this bumpiness.
Prior "bridge: Change local fdb entries whenever mac address of bridge device changes" [1],
FDB entry for a bridge was not inserted when creating the bridge and it could
only be inserted when a hw address of the bridge was changed. Therefore, on
an older than the 3.14 kernel containers were not able to reach
the "weave" bridge w/o setting the promiscuous mode on.
This should have hit hard Kubernetes users as the weave bridge in their
case is used as a default gateway, and thus it works only after manually
enabling the promiscuous mode.
Additionally, w/o the promiscuous mode a packet sent from a container and
destined to the gateway is flooded on all ports, so it can reach
the "datapath" device resulting in the "Vetoed installation of hairpin flow" error.
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a4b816d8ba1c1917842dc3de97cbf8ef116e043e
The version of manifest-tool in `weshigbee/manifest-tool` is version
0.3.0 and is erroring on every build; latest is 0.8.0 and doesn't error.
Since we are no longer limited to what will run in a container at
CircleCI, also build all architectures.
by-pass ingress network policy chains and accept the packet if
destination is not one of the local pods
return the packets so the egress network policy chains are skipped
in case packet's source is not one of the local pods
Fixes#3344
The liveness probe has the not-so-great side effect of killing the weave pods
if anything goes wrong, making debugging really difficult.
Not only that, we also bumped into the livenessProbe killing pods before they
had a chance to start (see below for the full story).
The readinessProbe is only used for Kubernetes Services: no traffic will be
routed to non-Ready pods when using the Service IP. We can use to to signal the
user something in wrong with Weave Net while not having Kubernetes delete the
pods behind our backs.
I removed the ininitialDelaySeconds field as Kurbernetes won't kill the pod if
not ready.
Notes from a debugging session:
Some nodes may have a vast amount of iptables rules, making each iptables
command take a few seconds (~5-10s). At startup, when the weave container
creates its chains, it runs quite a few iptables commands and the total time of
execution was > 30s.
It's important to note that, at this point, the weave process hasn't started as
those iptables commands are run by an initial script.
Kubernetes was trying to check for the container liveness, was failing, and
then proceeded to kill the weave pods before they had a chance to fully start.
from transient errors (heartbeast misses). In case of transient
errors OverlayForwarder is marked to be unhealty. When overlay forwarder
is unhealty its hept on hold so chooseBest() will skip selecting the forwrder
Once OverlayForwarder is considered healthy run chooseBest() again so the
forwarder can be selected if its the best forwarder
Fixes#1737
If we retain the Kubernetes list for a long time we might see a peer
appear in the Weave list and conclude it has been deleted. We must
always fetch the Kubernetes list after fetching the Weave list, during
deletion.
- In parallel on several goroutines, randomly add and remove peers.
- Maintain the test's own idea of current state.
- Check that the test's state and kube-utils' state are consistent.
Change the map used to reclaim peers so it is indexed by PeerName
which should be unique over time, rather than NodeName which can be
re-used. Create an auxilliary set of NodeNames to help.
This way, if we can see that a different peer was previously running
with the same node name, it can be included for reclaim.
is found to be in DOWN state
Fixes#3133 Do something more useful when the weave bridge is DOWN
On Weave restart there is already logic in place to create the bridge interface
and bring it up, this fix only monotors the weave bridge interface and logs error if
it is in DOWN state
When using ipBlock in Ingress policy, one should be aware
that source IP addr of packet destined to Pod in many cases
is mangled, see docs/k8s-src-ip.md for the cases.
However, the mangling in some cases can be prevented with
setting externalTrafficPolicy to Local.
Run build, lint and unit-test steps in a container, using a pre-built
image to save rebuilding at CircleCI
Move a couple of scripts up to config.yml for clarity
Otherwise, the same ipBlock ipset can be used among different namespaces which
results in panic as refcounting of ipsets happens only per namespace, so
npc will try to create the same ipset twice.
Egress does not work in the legacy netpol mode, so in the legacy case
we don't have to create the egress traffic steering rules. Otherwise,
some traffic will be blocked.
First check whether the chain exists with the required rules, and
if not - create it.
If we flush each time we start weave-npc then we create a (short)
time window during which all egress traffic is allowed.
We need two rules:
* `-A INPUT -i weave -j WEAVE-NPC-EGRESS`: applies to packets destined
to a local node.
* `-A FORWARD -i weave -j WEAVE-NPC-EGRESS`: the rest. It can match
an ingress packet.
dstSelector becomes ambiguous once we introduce the egress.
So, from now on, we call selector a target selector if it selects
pods to which a netpol applies (aka NetworkPolicySpec.podSelector).
This ensures two rules with the same rulespec but of different
policyType will end up having different keys.
Sharing the same key would prevent from provisioning the rules.
The following rules are provisioned by weave-npc:
-A WEAVE-NPC-EGRESS -m state --state RELATED,ESTABLISHED -j RETURN
-A WEAVE-NPC-EGRESS -m set ! --match-set weave-local-pods src -j RETURN
-A WEAVE-NPC-EGRESS -d 224.0.0.0/4 -j RETURN
-A WEAVE-NPC-EGRESS -m state --state NEW -j WEAVE-NPC-EGRESS-DEFAULT
-A WEAVE-NPC-EGRESS -m state --state NEW -m mark ! --mark 0x40000/0x40000 -j WEAVE-NPC-EGRESS-CUSTOM
-A WEAVE-NPC-EGRESS -m state --state NEW -m mark ! --mark 0x40000/0x40000 -j NFLOG --nflog-group 86
-A WEAVE-NPC-EGRESS -m mark ! --mark 0x40000/0x40000 -j DROP
-A WEAVE-NPC-EGRESS-CUSTOM <rulespec> -j MARK --set-xmark 0x40000/0x40000
-A WEAVE-NPC-EGRESS-CUSTOM <rulespec> -j RETURN
-A WEAVE-NPC-EGRESS-DEFAULT <rulespec> -j MARK --set-xmark 0x40000/0x40000
-A WEAVE-NPC-EGRESS-DEFAULT <rulespec> -j RETURN
For each rule we create two (mark and return). We cannot just accept
a packet if it matches any rule, as a packet might need to traverse
the ingress npc as well which happens later in the chain (in some cases
we cannot detect whether packet is ingress or egress, so we need to
check both chains).
Some logs from users show an entry of X with annotation X, which
suggests we have gone down the path of removing ourselves. This change
avoids that possibility.
As ipset.New(...) can be called by two processes (weaver and weave-npc)
at the same time, there is a possibility for a race when they both
check the comment support. To prevent the race, we append a random
nonce to the test ipset name.
- Track only local changes.
- Add comment to the iptables rule which prevents SNAT'ing.
- Rename the ipset to weaver-no-masq-local to prevent NPC from destroying it.
- Fix comment about create/flush WEAVE-EXPOSE.
- Add comment about reexpose being no-op if bridge hasn't been exposed.
- Add comment that the feature is not only for Kubernetes.
- Reuse the ipset instance.
This ensures that all exposing-related iptables rules are created.
As we do the re-expose, we need to clear the WEAVE-EXPOSE chain
if such exists. Thus, we use ClearChain instead of NewChain to
create the chain.
Otherwise test 150 would fail - Gorilla mux was not handling a newline
in the format parameter. This way you also get a better error if you
have an error in the parameter - instead of a 404.
A quick fix.
LG generates secretKeyRef.key w/o the ".txt" extension which makes it
inaccessible to weave-kube when passwd is stored in "weave-passwd.txt".
There was a bug in the npc analyser when adding a netpol with
dstSelector == srcSelector. The analyser was overwriting
dstSelector with srcSelector which resulted in Pod selected
by dstSelector not being removed from a corresponding
"default-allow" ipset.
This applies only to non-legacy netpols.
Remove the rule to ensure that api-server is not accessible while
kube-proxy is not ready.
Otherwise, we have a race between "-j KUBE-FORWARD" and "-j WEAVE-NPC"
which makes the NodePort isolation test case to fail.
Make the rule creation to use the same mechanism (via ensureRules)
as other rules from filter/FORWARD. Otherwise, the rule is preceded
by "-j WEAVE-EXPOSE" rule which can break the subnet isolation.
EnsureBridge used to skip creation and initialization of the weave
bridge and friends if such existed. This caused problems when
EnsureBridge failed before completing all init steps and so, subsequent
weaver could not work without "weave reset".
This change makes EnsureBridge idempotent.
"weave attach $CIDR $CONTAINER_ID" is no longer possible w/o
running weaver. This feature was undocumented and complicated
the weave script, as "weave attach" required to call "create_bridge".
Removing the feature allows us to eliminate the bridge creation
from the script.
Running "weave attach" w/o weaver results in the following err:
bridge "weave" not present; did you launch weave?
Also, this commit simplifies the "attach-container" cmd from
weaveutil by removing "mtu" as a param. The cmd is used
only by "weave attach", and a mtu is value for an attached
container interface is exactly the same as one of the weave bridge.
Both cmds depend on running weaver which creates the bridge
during its initialization. So, calling "create_bridge"
during execution of the cmds is redundant.
e.g. if we get a clash because someone else has updated the data, it
returns the new data so we should work off that next time.
Also re-fetch the Annotations every time round the main loop for safety.
Remove things like "args" and "env" which bulk the logs, aren't
relevant to the function of weave-npc, and may include sensitive
information.
Use a black-list rather than a white-list because we can't be sure
what information will be necessary in future.
There was a bug in handling "default-allow" ipsets - it could not
cope with out of order receive of Pod updates. E.g:
1) AddPod(Pod_A, PodIP_A),
2) AddPod(Pod_B, PodIP_A),
3) DeletePod(Pod_A),
would result with PodIP_A being removed from "default-allow" ipset,
as such ipset entries were not refcounted. Bypassing of refcounting
was due to the fact that adding a netpol which dst-selects a Pod which
IP addr is in "default-allow" ipset has to remove the IP addr from
the ipset regardless of refcount value of the IP addr entry in the
ipset. So, refcounting in such case didn't make sense and it was
bypassed.
This commit changes the way entries are refcounted in ipsets, and
it makes "default-allow" ipset the same citizen as other ipsets, so
from now "default-allow" ipset is refcounted.
Now we don't bundle the docker command inside weaveexec, we need to do
an explicit 'docker pull' to ensure the images are there.
Also set the weavedb tag to :latest to remove a warning from docker
Also tag kube-peer log messages with "[kube-peers]": in a standard
deployment they end up interspersed with weaver log messages so this
makes it easier to pick them out.
- Test that pods not matching ingress rules are not able to access
the nettest pods.
- Test upgrading of NPC (from legacy to non-legacy).
- Test new semantics of k8s netpol.
The main difference between new and legacy (pre-1.7) netpol semantics
are the following:
* The DenyDefault ingress isolation annotation is obsoleted.
* Traffic is allowed to a pod if there are no netpol selecting
the pod.
See https://kubernetes.io/docs/api-reference/v1.6/#networkpolicyspec-v1beta1-extensions
vs https://kubernetes.io/docs/api-reference/v1.7/#networkpolicyspec-v1-networking
for more details.
weave-npc can operate either in the legacy or the new mode (default). To
use the legacy, pass "--use-legacy-netpol" arg when starting the npc.
In non-legacy mode we create for each Namespace a default-allow ipset
which stores IP addrs of such Pods which are not selected by any
NetworkPolicy.The ipsets are dst's in rules of the WEAVE-NPC-DEFAULT chain.
Docker 1.13 has changed a default policy of FORWARD chain to DROP
(https://github.com/moby/moby/pull/28257) which makes containers
inaccessible from a remote host when the bridge is exposed.
The change breaks e.g. the AWSVPC mode. To overcome this we install
an explicit rule for accepting forwarded ingress traffic to an
exposed subnet which is appended to the WEAVE-EXPOSE chain.
The chain is a target of the rule "-t filter -A FORWARD -o weave".
The change makes "weave expose" to call the "/expose" HTTP
endpoint implemented in router which exposes the bridge.
Such architectural change required as we need to know whether NPC is
enabled when exposing the bridge and weaver knows this fact the best.
Unfortunately, this change makes impossible to "weave expose"
when weaver is not running.
It's sad because this changes a regular pattern into an irregular one,
which will create spurious diffs if more work is added at the end of
the function. Still, this is easier than fixing the lint runner to
ignore the report.
Mount iptables lock file in weave-kube
iptables uses the file `/run/xtables.lock` so that if changes are
under way, another instance of the iptables command can wait till it
is finished.
We need to mount the lock file from the host to the container in order
for this mechanism to work at all.
Note we rely on kubelet touching the file before running any pods -
see https://github.com/kubernetes/kubernetes/pull/47212
`weaveexec` no longer mounting '/' so we can't access the weavedb file
to get persistent peer ID. If `weaver` was running, we couldn't have
read the file anyway because BoltDB does not support concurrent access
from two processes.
Consequence is that if someone really does need the bridge creation on
'weave expose' the peer ID may change if created pre-1.9.0. We'll just
live with that (per @bboreham).
Fixes#2957
- replace `docker stop` with `weaveuitl stop-container`
- replace `docker kill` with `weaveutil kill-container`
- replace `docker rm` with `weaveutil remove-container`
- replace `docker run` with minimal `weaveutil container-run`
- replace `docker inspect -f {{.ID}}` with `weaveutil container-id`
- replace data-only container `docker create` with `weaveutil create-volume-container`
- remove `docker inspect` dependencies from `weave` script
`rewrite_etc_hosts` and `with_container_fqdn`
- rework `weave` script `check_running` and `check_not_running` to use `weaveutil`
- rework `weave version` to use `weaver` /report API or `weaver --version` if not running
- rework `weave setup`'s `docker pull` to run locally rather than in `weaveexec` container
Otherwise, the plugin is not able to load kernel modules
(openvswitch, vxlan, vport_vxlan) required by fastdp which
makes weave to fallback to sleeve.
CAP_SYS_MODULE is required in order to load kernel modules.
- "You cannot call `weave rmpeer` on more than one host" - yes you
can, but you must not call it on more than one host *for the same
peer*
- `weave forget` has nothing to do with this
- "results in duplicate IPs" - no, it causes duplicate address range
assignments
- "the address is reassigned" - it's the address *range* that is
reassigned
- "claim" is a new and confusing term -> drop
- "address space" -> "address range" (that's what it's called
elsewhere)
It was a hack for a badly-configured Kubernetes install: it shouldn't
be necessary, it will only work on an unsecured install, and it
generates confusing error messages when something is really wrong.
If we have namespaces Source and Destination, and Destination
contains a NetworkPolicy allowing access from Source, we need
to provision the ipsets for Source and add them to relevant
selectors, even though Source is DefaultAllow.
This also fixes an issue where if the Namespace labels are updated,
that would be ignored.
Fixes#3059
As discussed internally, this could just create more confusion for the end-user, than bringing benefits for previous users who would potentially wonder why the URLs have changed.
GitHub only allows git.io to point to content hosted under github.com, therefore it was not possible to have the short URLs point to Weave Cloud's Launch Generator.
Currently:
- there is no easy way to change the way we install Docker from Net
- if build-tools' default was to change, Net could break.
This change addresses this situation.
Before this change, when using Docker 17.06 we get:
tests/integration/container_test.py:121: in test_create_with_restart_policy
'You cannot remove a running container', err
E AssertionError: 'You cannot remove a running container' not found in u'You cannot remove a restarting container
The modified image simply checks for "You cannot remove a".
There are 3 ways to get help:
• weave --help
• weave help
• weave <command> --help
At the moment, only the first one prints the usage instantaneously, the
others go much further in the script and take quite some time, which is
not idea of impatient users.
Note that "help" only works because it's not a recognized command and
ends up in the default case of "case $COMMAND" (and indeed, "Unknown
weave command 'help'" is displayed). This is because handle_help_arg()
always returns true and so the second part of:
handle_help_arg "$1" || handle_help_arg "--$1"
is never evaluated.
This commit changes things a bit so all 3 ways to get help are
recognized early, print the usage and exit.
Currently our `Makefile` and `bin/release` scripts are such that we effectively do the following when releasing:
```
publish:
for arch in [amd64 arm arm64]:
sub-publish:
clean_bin
publish
plugin_publish
```
which leads the Docker v2 plugin to contain `arm64` binaries even though we target `amd64`.
Swaping `publish` and `plugin_publish` is a simple way to avoid this.
The fix ensures that if odp is not supported (usually due to
missing openvswitch, vxlan or vport-vxlan kernel modules),
weaver will create the pcap based bridge instead of crashing.
`proxy_parse_args` is currently called:
- once, all the time
- again, if Docker is configured with TLS.
If a second call to `proxy_parse_args` happens, given the way we append to `PROXY_ARGS` and merge proxy and remaining args, genuine arguments like peers can be mixed with flags/options/etc. hence forming an invalid peers list and crashing Weave Net.
Just no-op instead.
This change is necessary because previously a recent build-tools
change no longer excludes prog/* from unit testing since other
projects actually do have genuine tests under there.
This behaviour was breaking the use of pod selectors for the 'from'
part of a rule.
Move checking of local pods into namespace.go so we can maintain
LocalIpSet correctly
If it's destined for a pod on another node it will be checked again
when it reaches that other node. The previous version was rejecting
traffic coming in from outside the cluster.
It is necessary in case when kubelet's args includes `--hostname_override=`.
Without this change, pods on hostname overriding kubelets will never be local,
and all pod's traffic will be blocked.
Details:
- Update weave-kube docs to avoid explicit versioning by leveraging the Launch Generator.
- Update weave-kube docs to mention YAML customisation via the Launch Generator.
- Warn about potential backward incompatibility of the generated YAML, and how to ensure YAML and Net versions are compatible.
Before this change, `go get`-ting` would pick the latest version of `shfmt` which keeps changing, hence frequently breaking our builds.
After this change, we download the `shfmt` binary from its GitHub releases page which is fast and locks the version we use. Note that `RUN` does not like herestrings, hence `echo "..." | sha256sum -c` for checksum.
Alternatives considered:
- `go get gopkg.in/mvdan/sh.v1/cmd/shfmt` -- but `gopkg.in` currently doesn't behave well for this tool
- `go build` from a `git` tag -- but this is much more involved.
We don't get a very useful error object back from go-iptables and I
didn't feel like parsing the message, so just ignore all errors.
It would be racy to try to check if WEAVE-NPC existed before calling
NewChain.
EmptyDir has the same lifetime as the pod, whereas we want our
persistence to extend past that, e.g. when Weave Net is upgraded.
`weave reset` cleans out the file.
We assume that /var/lib is writable.
Since it is only called in contexts where we have already checked that.
We don't need the detail of whether plugin is enabled; if it's not
running then it's ok to remove those socket files.
If the plugin wasn't enabled then that network shouldn't exist.
If the network 'weave' doesn't exist then util_op remove-plugin-network will do nothing.
If it does exist for some reason then docker will pause for a while and remove it anyway.
For some reason the previous version was de-newlined by the shell and
the new version isn't. Since I can't figure out the exact reason the
shell is now behaving differently (man page says it depends on the
quoting that is in effect) I brute-force it with 'tr'.
The standard case for --local is when the weave script in the weaveexec container
is run by the weave script, which has just checked the version.
--local is also used from weave-kube where we don't mount the docker socket and
therefore cannot check.
Since --local is undocumented we will not worry about any other cases; the worst
that can happen is a confusing error message.
The problem is that the plugin name is prefixed with a registry name.
So, if the plugin is fetched from Docker Store, its name will be
"store/weaveworks/net-plugin".
The detection function does an exact match on "weaveworks/net-plugin",
so in the Docker Store case, the presence of the plugin cannot be
detected.
Weave Net initially had the ability to start a single component at a time with `launch-proxy` and `launch-router`.
This change enables this again via `--proxy=false` despite the merge of the proxy and the router.
Note that by default:
- `launch` passes `--proxy` which enables the proxy and keeps behaviour consistent with `launch` pre-merge,
- `weaver` defaults `--proxy` to `false` which disables the proxy and keeps behaviour consistent with `weaver` pre-merge (as it was `weaveproxy`'s responsibility).
Remove tests of situations that don't exist any more and fix the following tests:
- Fix test 660: pass arguments in correct order.
- Fix test 620: no double launch, weave stop instead of kill -9.
- Now that router and proxy have been merged, we should only call `weave launch` once.
- `sudo kill -9` was introduced because we previously had `docker kill` (see: http://github.com/weaveworks/weave/commit/cb73b738621d57727072ae998cdeaec337f23a1a), because we had no `weave stop-proxy` yet (see: http://github.com/weaveworks/weave/commit/e31f21a6d224d61de1d8a940ea4b0b45cebc9723). Pre-proxy-merge, this should just have been `weave stop-proxy`, and now that router and proxy have been merged, we should just call `weave stop`.
- Fix test 650: no double launch.
- Fix test 630: remove redundant case & assertions.
- Fix test 090: remove irrelevant assertions.
PING
Given `proxy` & `router` processes have been merged, `launch-proxy`,
`launch-router`, `stop-proxy`, `stop-router` do not make sense any more, and
should just be `launch` and `stop` with the appropriate flags.
Note: we no longer need to detect awsvpc when launching.
The "AWS VPC" mode is now set via the `--awsvpc` flag.
Before we had to "detect" it when the proxy was potentially started
later, after the router had started.
We drop WEAVEPROXY_DOCKER_ARGS as obsolete.
If this function dies in the middle, the leftover weave-kube-test set
causes this function and weave-npc to break. In particular
this means we can never restart weave with the launch script.
Here we clean up in advance, in case there's leftovers from a previous run
Fixes#2934
"docker.sock" is used to access Docker API which is required to:
1. subscribe to Docker events to configure-arp,
2. query Docker network info to detect whether multicast should be
enabled.
3. get list of Swarm peers before starting the plugin.
(1) is currently not implemented.
(2) is replaced by WEAVE_MULTICAST env var which
can enable multicast for all Weave networks created by
the plugin (not ideal, but better than nothing).
(3) is fixed by not removing "restart.sentinel".
Fixes#2921, as all errors are now ignored, including `ENOENT`.
This command typically attempts to delete the network interface `eth0` of a container being destroyed by Kubernetes.
If this interface has already been destroyed or is missing for whatever reason, we still want to remove this container's entry from IPAM in order to free associated resources.
Fixes:
- `test/run-integration-tests.sh:160:75: arithmetic expressions must consist of names and numbers`
- `test/run-integration-tests.sh:265:56: arithmetic expressions must consist of names and numbers`
- `test/run-integration-tests.sh:295:61: arithmetic expressions must consist of names and numbers`
- `test/run-integration-tests.sh:309:49: arithmetic expressions must consist of names and numbers`
- `test/run-integration-tests.sh:316:41: arithmetic expressions must consist of names and numbers`
- `test/run-integration-tests.sh: run shfmt -i 4 -w test/run-integration-tests.sh`
See also: https://circleci.com/gh/weaveworks/weave/8685
Why:
* In some environments Nodes have an ExternalIP but no InternalIP
This change addresses the need by:
* Falling back to ExternalIP addresses if there isn't an InternalIP for a node
Fixes:
- `test/run-integration-tests.sh:160:75: arithmetic expressions must consist of names and numbers`
- `test/run-integration-tests.sh:265:56: arithmetic expressions must consist of names and numbers`
- `test/run-integration-tests.sh:295:61: arithmetic expressions must consist of names and numbers`
- `test/run-integration-tests.sh:309:49: arithmetic expressions must consist of names and numbers`
- `test/run-integration-tests.sh:316:41: arithmetic expressions must consist of names and numbers`
- `test/run-integration-tests.sh: run shfmt -i 4 -w test/run-integration-tests.sh`
See also: https://circleci.com/gh/weaveworks/weave/8685
This logic was added for backward compatibility and to warn users about changes made ~2 years ago.
Given we are planning a 2.0 release for Weave Net, such logic is now irrelevant and can be removed.
For V2 plugin, "latest" means "latest stable release", to make
instructions simple for end-users. So we don't want to overwrite
"latest" with a random build by accident.
Docker Plugin V2 does not expose a plugin name to the plugin, so we
check for "net-plugin" within driver name of a network to detect
whether network is ours.
- "noDNS" controls whether upon ContainerStarted event a container
should be registered within the Weave DNS.
- "isPluginV2" will be used to determine a path to host's /proc.
Renamed to "is_plugin_v1_enabled" and "is_plugin_v2_enabled".
Improve weave status and weave launch* err message when plugin-v2 is running
```
$ weave launch
Docker plugin "weaveworks/net-plugin" is running; Aborting.
$ weave status
<..>
Service: plugin (v2)
DriverName: weaveworks/net-plugin
```
"docker plugin" does not support tagging plugin images, therefore
we need to build plugin images two times - one for :latest and
for :$(WEAVE_VERSION).
Pushing of :latest happens only if $UPDATE_LATEST is not false.
For example:
```
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
$ docker plugin ls
ID NAME DESCRIPTION ENABLED
e80beb4367a6 weaveworks/pluginv2:latest Weave Net plugin for Docker true
$ weave --local launch
weaveworks/pluginv2 is already running.
$ weave --local launch-proxy
weaveworks/pluginv2 is already running.
$ weave --local launch-router
weaveworks/pluginv2 is already running.
$ weave --local launch-plugin
```
NetworkAllocate is called on Swarm master node which creates a network.
The return option map is used to describe the network, and is passed
to CreateNetwork which is called on all nodes when a service is
created.
Returning nil instead the actual options made Docker to omit
the "works.weave.multicast" flag, thus no multicast could be used
in Swarm.
This simplifies the following:
- plugin-v2 does not need to start "plugin" as a separate process, thus
Docker is able to properly restart plugin-v2 in a case of any child
process failing.
- No need for weaveplugin container.
plugin is enabled by default when running "weave launch". when starting
only the router, you should pass "--plugin" parameter to enable it.
The env var WEAVE_NO_PLUGIN is no more.
Since 1.12.0, `docker inspect -f "{{ .NetworkSettings.Networks.<network>.Aliases }}"` also returns the container's short ID.
See also: https://github.com/docker/docker/blob/master/CHANGELOG.md
> 1.12.0 (2016-07-28)
> Networking
> Add container's short-id as default network alias #21901
Upgrading `docker/docker-py` to 1.9.0-rc2 effectively relaxes test assertions for:
- `tests/integration/network_test.py#test_connect_with_aliases`
- `tests/integration/network_test.py#test_create_with_aliases`
which do something equivalent to:
```
$ docker network create net1
$ docker run --name c1 -d busybox top
$ docker network connect --alias foo --alias bar net1 c1
$ docker inspect -f "{{ .NetworkSettings.Networks.net1.Aliases }}" c1
[foo bar e52bf50e5ee7]
```
allowing us to be consistent with the above change.
In 1.8.1, the below fails as the container's short ID would be present:
```
self.assertEqual(
container_data['NetworkSettings']['Networks'][net_name]['Aliases'],
['foo', 'bar'])
```
In 1.9.0-rc2, the below passes as the container's short ID is ignored:
```
aliases = (
container_data['NetworkSettings']['Networks'][net_name]['Aliases']
)
assert 'foo' in aliases
assert 'bar' in aliases
```
The vars can be customized before enabling the plugin with e.g.:
docker plugin set $PLUGIN_NAME set WEAVE_MTU=8160
The ARP settings apply to "weave" bridge. In order to manipulate them,
we bind mount the host's /proc to /host/proc.
(The mount type of /proc changed to bind, Otherwise (when type =
"proc"), the kernel does not propagate pids of host processes as the
plugin runs in a separate PID ns.)
Persistence relies on weave.db which is stored under
/var/lib/weave/weave.db on the host.
NOTE: we do not persist peer list for plugin-v2, as it gets
restored from existing alive master nodes during the peer discovery.
The peer discovery relies on "GET /info" API call which returns
list of master nodes. The peer list is passed to weaver when
launching it.
If node is a master, then we pass "--ipalloc-alloc
consensus=$(MASTER_NODES_COUNT)" to weaver. Otherwise, we pass
"--ipalloc-alloc observer".
Without this commit, `kubeadm` typically upgrades whichever version of Kubernetes is installed, sometimes leading to conflicts in between components' version.
This fixes the issue behind #2876 and should avoid build failures like [weave/8373](https://circleci.com/gh/weaveworks/weave/8373).
They seem to stop Docker using its build cache.
Removing the line entirely stops weaveproxy from working, as it
assumes it can exec `./weave` from the current directory.
The executables have dependencies that are not apparent to `go build`;
notably the build container. If Go doesn't see anything is out of date
then it will do nothing, hence touch the file so next time we run
`make` it is up to date.
It is possible to receive FastDatapathCryptoInitSARemote control plane
message before Confirm'ing a connection. In such case, resetting
heartbeatTimer leads to null pointer dereference, as the timer is
initialized during Confirm'ation.
In this situation, skipping the reset is non-harmful, because
Confirm'ing will reset it.
NB: the reset to 0 should only happen after establishing the connection
on both ends.
weave-kube is run by alpine image which uses busybox. modprobe of
busybox will return the "module not found" error [1], if a module is
built-in, which is an opposite behavior to modprobe distributed in
mainstream distros (e.g. ubuntu).
[1]: https://bugs.busybox.net/show_bug.cgi?id=5270
Previously, the first line of the Dockerfile would have been selected, regardless of its value.
What we want is to actually select the first line matching the regex, and then extract the group corresponding to the Go version out of it.
- One unique file to list all dependencies and dependencies' versions (except Go, currently defined in `build/Dockerfile`).
- Custom versions can now be injected into Vagrant VMs.
- Vagrant dev VMs, Vagrant test VMs, and `run-integration-tests.sh` all source from `DEPENDENCIES`, to keep things 'DRY'.
- Vagrant versions can be overriden using environment variables, like it was possible in `run-integration-tests.sh`.
The intention, per the comments, is that host1 should claim from host2.
But host2 and host3 can carve up the space either way round. If host1
claims from host3 then there is a race whether anyone finds out about
it.
Adding a 'prime' on host2 ensures the space is all owned by host2 at that point.
doing it from the script
Note we need to run weaver in the host pid namespace, so it can search
for existing container IPs. Also now we are firing off sub-processes
it's better to not be PID 1 so Linux doesn't expect us to reap zombies.
This should be useful when we just want to type one make integration-tests / ./run-integration-tests command to make/start VMs/configure them/test, but also investigate potential failures afterwards.
This should make development easier by allowing developers to set HOSTS in their calling shell, and re-use the value for this env. var. when calling individual test scripts, e.g.:
```
cd test
eval $(./run-integration-tests.sh setup)
./200_dns_test.sh
```
CircleCI keeps 10 nodes in order to speed up the build, but the default is 3, as this is the minimum required to run any test. This should make more sense as a default for a developer to quickly start an environment and iterate.
There may be situations when Terraform/Ansible/etc. fail to provision test machines.
In these cases, we want to fail fast instead of waiting until we time out in circle-test-pre and circle-test-smoke.
This way, Vagrant VMs and cloud-provided VMs are configured in the exact same way, hence avoiding configuration drift and subtle bugs associated to different versions of software.
Provisioning and configuration being expensive time-wise, we create Google Cloud Platform images for each Docker/Kubernetes/Kubernetes CNI versions triple, and configure these once (instead of for each build), and then re-use these in subsequent builds, effectively implementing a 'cache' of images.
Reason: test is periodically failing because of the timeout being too aggressive, especially when using Google Cloud Platform together with Terraform and Ansible.
If building against the latest 'build-tools', you would typically get:
tools/runner/runner.go:18:2: cannot find package github.com/weaveworks/common/mflag
This change fixes the issue by upgrading both dependencies.
Makes what the build does explicitly and helps for potential debugging.
Also reminds us that we are skipping these files, so that we ultimately shfmt them and stop lint-ignoring them.
Previous commit upgrades Weave Net to the latest version of `build-tools`.
However, `./tools/test`'s default behaviour has changed, so we need to explicitly pass new options/arguments, in order to keep it behaving like before fd568682c3.
Lint-ing all of Weave Net is currently not desirable because:
* it would create conflicts on many branches,
* it would pollute the git history,
* etc.
and because `lint` currently lints all files except `./vendor`, this breaks Weave Net's build.
As a result, Weave Net is not using the latest commit in `build-tools`'s `master` branch, which is also source of problems:
* it limits standardisation of projects across Weaveworks,
* it limits the ability to use the latest features of `build-tools`,
* it complexifies merging changes in `build-tools` (e.g. #53 and weaveworks/weave#2694 for weaveworks/weave#2647),
* etc.
This change:
* brings Weave Net to the latest version of `build-tools` (which now supports `.lintignore` files),
* adds such a `.lintignore` file to skip all bash scripts (in order to avoid breaking the build).
`GLOB`s in `.lintignore` can then, over time and as scripts are `shfmt`-ed, be relaxed and maybe even removed.
If the machine coordinating the test runs does not have jq, then test #175 fails with:
> test #1 "weave_on test-0.lon1.weave-net report | jq -r '.Router.Targets[] | tostring' | sort" failed:
> expected "test-1.lon1.weave-net
> test-2.lon1.weave-net"
> got nothing
This change completely removes the dependency on jq to make the test more robust and avoid potential confusion.
Test hard-codes a string containing '\n' and then expects echo
to interpret it as a newline, which it does not without the -e option.
This fixes flaky test runs like the below:
>>> Running /root/src/github.com/weaveworks/weave/test/150_connect_forget_2_test.sh on [138.68.153.213 138.68.149.186]
>>> Test /root/src/github.com/weaveworks/weave/test/150_connect_forget_2_test.sh finished after 12.4 secs with error: exit status 1
test #5 "direct_peers 138.68.149.186 | sort" failed:
expected "138.68.153.213\n138.68.149.186"
got "138.68.149.186\n138.68.153.213"
So we don't accidentally read or write the hosts's /etc.
Also move the other volumes under /host to be more consistent with
what the weave script expects.
This is a combination of 4 commits.
The first commit's message is:
2677 docu, how to publish prometheus metric endpoint on all interfaces
This is the 2nd commit message:
2677 change docu on how to publish metrics, be more careful with publishing on all interfaces
This is the 3rd commit message:
2677 fix spelling
This is the 4th commit message:
2677 fix wrong port title and grammar
Enabled hairpin mode on vethwe-bridge was one of the root causes
for installing looping flows in the fastdp mode.
Also, it confused the weave custom L2 bridge, as the bridge treats
a bounced back packet, e.g. ARP brodcast request, as a local one
(the case for "Captured a packet associated with remote host").
Previous fix in b4a43f1b85 did not work
when built with code coverage.
Running a test script with -v causes `weave_on` to output progress
messages, so we don't want to use those commands inside `$()`
So something waiting for the config (e.g. Kubernetes) doesn't try to
talk to Weave before then (this only helps on first install, but that
tends to be when things are busiest).
- Rewrote conditions to keep c.ident immutable and hopefully makes
things clearer.
- Added a case to tolerate reclaim after we have used CNI (passed CID
is “_” and existing CID is a real container ID).
test #4 "[
3e52b548e6df304c61ea5bee0af4b3515e025e7e25152c65ecae3d03c71cf52d ==
Unable to claim: address 10.32.0.3/12 is already owned by
fbe411facc8b657c5850591ab56fc40fb56160d43db0d7363a29ba3edad61134
Unable to claim: address 10.32.0.1/12 is already owned by
cf16a2bcdee8dd1ef7e5f0a611127611e67ea734f45d7526678fc3b1fc0c6f5d
3e52b548e6df304c61ea5bee0af4b3515e025e7e25152c65ecae3d03c71cf52d ]"
failed:
program terminated with code 2 instead of 0
1 of 6 tests failed in 6.499s.
The metrics include:
- weave_packets_total
- weave_bytes_total
The problem with those metrics is that fastdp periodically flushes
flows including their stats thus makes the current implementation
of the metrics unreliable.
Serve metrics+status on status socket and api+status+metrics on api
socket; the reasoning is that metrics or status are read-only hence
may be scraped from an off-machine connection, while we don't want to
open the control port up unprotected.
This implementation should be more efficient because it only uses the
names under /proc, whereas the previous implementation fetched the
full directory entry for each one, and sorted them too.
- our help page, to choose the best channel (Slack, etc.) to reach out: https://www.weave.works/help/
** Is this a FEATURE REQUEST? **
If so, please search existing feature requests, and if you find a similar one, up-vote it and/or add your comments to it instead.
If you did not find a similar one, please describe in details:
- why: your use-case, specific constraints you may have, etc.
- what: the feature/behaviour/change you would like to see in Weave Net
Do not hesitate, when appropriate, to share the exact commands or API you would like, and/or to share a diagram (e.g.: asciiflow.com): "a picture is worth a thousand words".
** Is this a BUG REPORT? **
Please fill in as much of the template below as you can.
Thank you!
-->
## What you expected to happen?
## What happened?
<!-- Error message, actual behaviour, etc. -->
## How to reproduce it?
<!-- Specific steps, as minimally and precisely as possible. -->
## Anything else we need to know?
<!-- Cloud provider? Hardware? How did you configure your cluster? Kubernetes YAML, KOPS, etc. -->
## Versions:
<!-- Please paste in the output of these commands; 'kubectl' only if using Kubernetes -->
[Full list of changes](https://github.com/weaveworks/weave/milestone/78?closed=1)
## Release 2.5.2
This release fixes several bugs causing inconsistencies in IPAM and fixes a
panic in daemon that reclaims and forgets deleted nodes in Kubernetes clusters.
### Bug fixes
* leaking IP addresses in IPAM when weave-net hangs or restarting #3587, #3638
* Panic in kube-utils after config map update error resulting in reclaim daemon
not performing weave forget on deleted node #3613, #3623
* preventing inconsistent IPAM entries and resolving the conflict in case of
conflict during IPAM ring merge #3629, #3635, #3632, #3444
[Full list of changes](https://github.com/weaveworks/weave/milestone/79?closed=1)
## Release 2.5.1
This release fixes bugs reported for 2.5 release and small improvements.
### Bug fixes
* prevent warnings in kernal logs due to use of physdev model for
non-bridged traffic #3449, #3453
* Check and report the error occurred while getting the list of Kubernetes
peers #3581, #3582
### Other improvements
* Update Ruby version to match Weaveworks website #3458
* Fix the link to "Troubleshooting the Kubernetes Addon" section in the docs #3461
* Document concept of connection limit in Weave #3395, #3413
[Full list of changes](https://github.com/weaveworks/weave/milestone/77?closed=1)
## Release 2.5.0
This release adds support for Kubernetes `hostPort` mapping (#3016,#3356)
and the `ipBlock` NetworkPolicy feature (#3168,#3367)
### Bug fixes
* Fix a crash at start-up on Docker for Mac #3405, #3408
* Network policy: block ingress traffic when no namespaceSelector or
podSelector is specified #3347
* Reclaim IP addresses which are locked by a non-existent peer #3386, #3416
* Fix a crash when blank IP data was loaded #3067, #3415
### Other improvements
* If a connection is downgraded to the slower "sleeve" mode, Weave Net
will now periodically try to upgrade it to "fast datapath" again. #1737, #3385
* Reclaim removed Kubernetes nodes' IP space when they are deleted,
rather than on next restart #3372, #3399
* Replace Kubernetes livenessProbe with readinessProbe, so the pod is
not killed if it runs slowly #3471, #3421
* In Kubernetes NetworkPolicy controller, remove the need to maintain
a set of local pod IP addresses #3344, #3423
* Don't crash on Kubernetes named port in NetworkPolicy, just report as unsupported #3375
* Ensure the `weave` network bridge is accessible on Linux kernels
older than 3.14 #3442, #3297, #3239
* Better reporting in the logs if the `weave` network device is in the Down state #3133, #3381
* Change log-level to debug of calls through the Docker proxy, to reduce noise #3439
* Add `--without-masquerade` option to `weave expose`, so external
services can see the original container IP address #3388
* Include Kubernetes cluster information in checkpoint call #3324,#3431
* Bump go-odp dependency, so that `fastdp` works on the 4.19 kernel #3430
### Build and Testing
* CI builds on master branch now publish images for all platforms
* Fix golint path and use https for download of libpcap #3435
* Update Kubernetes client-go to v8.0.0, removing code licenced under LGPL3 #3358,#3366
* Migrate CircleCI to V2, which is much faster #3255,#3270
### External Contributors
Thanks to the following contributors:
*@Ashiroq
*@leprechau
*@lkpdn
[Full list of changes](https://github.com/weaveworks/weave/milestone/74?closed=1)
## Release 2.4.1
This release fixes several bugs causing inconsistencies in IPAM for
Kubernetes users whose clusters scale up and down over time.
### Bug fixes
* Nodes unable to connect after Kubernetes addon erroneously reclaimed
node without any IP addresses #3392, #3393
* Kubernetes addon could have run out of free IP addresses after nodes
are deleted #3384, #3400
* Kubernetes addon had reduced free IP addresses due to not reclaiming
IP addresses when node name is re-used #3397
### Other improvements
* Support `--label` in `WEAVE_DOCKER_ARGS` when starting Weave #3370,#3371
* Add missing `--token` argument in help for `weave launch`#3226, #3379
* Print defunct processes after smoke-tests #3362
[Full list of changes](https://github.com/weaveworks/weave/milestone/75?closed=1)
## Release 2.4.0
This release introduces a support for Kubernetes Egress Network Policy (#2624, #3313)
and adds a mechanism for preserving the client source IP address to enable
`externalTrafficPolicy: Local` on Kubernetes (#2924, #3298).
### Bug fixes
* Increase the ipset list size which prevents weave-npc from crashing on older
kernels when more than eight Kubernetes Namespaces are used (#3289, #3305).
* Avoid a possible livelock when reclaiming IP address space in weave-kube (#3317).
* Ensure `xtables.lock` is mounted as a file so that kube-proxy can take the lock
if it has started after Weave Net (#3351, #3353).
* Upgrade the CNI plugin symlinks only if the plugin has changed (#3337, #3345).
### Other improvements
* Manipulate the Kubernetes node status `NetworkUnavailable` so that Pods can be
scheduled on nodes when the GCE cloud provider is in use (#3249, #3307, #3332, #3334).
* Refrain from creating a subprocess for configuring a network interface in
a container network namespace (#3291).
* Protect against handling the CNI plugin request with the host namespace which
prevents Weave Net from misconfiguring the host network (#3206, #3346).
* Weave Net can be run on minikube VM (#3124).
* Add `org.opencontainers.image.*` labels to Dockerfiles to improve association
of the container images with git revisions (#3299).
* Improve the error message when running `weave reset` on Kubernetes (#3319).
### Build and Testing
* Use `dep` instead of `git submodules` for managing external packages (#3268).
* Fix usage of `manifest-tool` in Makefile (#3320).
* Update Kubernetes to 1.11 for the integration tests (#3340).
### External Contributors
Thanks to the following contributors:
*@kitt1987
*@louismunro
*@Nodraak
*@stevenjohnstone
[Full list of changes](https://github.com/weaveworks/weave/milestone/73?closed=1)
## Release 2.3.0
### Security fixes
* By default, do not expose Weave "/status" and "/report" to all (0.0.0.0) when
running on Kubernetes #3271
### Other improvements
* Increase the default connection limit for Weave peers (from 30 to 100) when
running on Kubernetes, so that more peers could directly connect #3265
## Build and test
* Build Weave Net with Go 1.10.1 #3273
* Run integration tests against Kubernetes 1.10.0 #3266
[Full list of changes](https://github.com/weaveworks/weave/milestone/70?closed=1)
## Release 2.2.1
### Bug fixes
* Fix a bug in weave-npc which would allow ingress traffic to Kubernetes Pods selected
by a NetworkPolicy in which source and destination selectors were the same #3222,#3237
* Fix a bug in weave-npc which would crash if a previously deleted Kubernetes Namespace
has been created again #3247,#3250
### Other improvements
* Increase the default connection limit for Weave peers (from 30 to 100), so that
more peers could directly connect #3234
* When doing a rolling update of Weave Net on Kubernetes, allow each node five seconds
to initialize before rolling next Weave Net Pod, so that issues at startup will halt
the rollout and not spread across the whole cluster #3235
* Install common CA certificates from Alpine Linux package instead of copying
them manually #3236
### External contributors
Thanks to the following contributors:
*@alok87
[Full list of changes](https://github.com/weaveworks/weave/milestone/71?closed=1)
## Release 2.2.0
This release improves the way Weave Net configures Linux network
devices and network filter rules, so that it is more robust in the face
of unexpected changes in its environment. #3204,#3224
As a consequence of these changes, the `weave attach` command will now
fail unless the Weave Net daemon is up and running - previously it was
possible to run independently as long as you managed all IP addresses
yourself.
## Other improvements
* Update library miekg/dns for CVE-2017-15133 (details under embargo) #3223,#3227
* Reduce the volume of logging from weave-npc #3183
* Add ability to set log level for Docker "v2" plugin, and change
default log level from DEBUG to INFO #3197
* Downgrade log messages about Discovery and Expiration to DEBUG level #3202,#3203
* Use command-line parameter for WeaveDNS address in Docker proxy #3196
## Bug fixes
* Ensure that rules to block traffic for NetworkPolicy are placed
ahead of rules that Kubernetes has added to allow other traffic #3209,#3210
## Build and test
* Update CI tests to use Kubernetes 1.9.2 #3229
* Remove "daily update" from test VMs that only run for a few minutes #3224
## External Contributors
Thanks to the following contributors:
@vetal4444
[Full list of changes](https://github.com/weaveworks/weave/milestone/67?closed=1)
## Release 2.1.3
This release fixes a race-condition in the IP reclaim code for weave-kube
where two nodes could end up fighting over the same space and break
connectivity #3190, #3192
## Release 2.1.2
This release fixes a couple of bugs discovered since the release of Weave Net 2.1.0
##Bug fixes
* Fix crash seen when starting 10-15 nodes simultaneously #3184,#3186
* Fix NetworkPolicy blocking traffic if updates come out of order from Kubernetes #3177,#3181
Thanks to the following contributors:
@zignig
[Full list of changes](https://github.com/weaveworks/weave/milestone/66?closed=1)
## Release 2.1.1
As 2.1.0, but fixing a couple of installation glitches. #3175,#3176
## Release 2.1.0
##New Features
Improved Kubernetes Network Policy - Weave Net now supports the
'v1' policies introduced in Kubernetes 1.7 as well as the 'beta'
policies supported previously. See [Kubernetes 1.7 changelog](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.7.md#network)
for differences. To use old policies, `--use-legacy-netpol` argument
should be passed to `weave-npc`. #3105,#3141,#3151,#3169
Weave Net now reclaims IP addresses owned by Kubernetes nodes which
have been deleted from the cluster - this avoids running out of IP
addresses when many nodes are added and deleted over a long period.
#2797,#3149,#3170,#3172
##Other improvements
* Export a Prometheus-style metric giving count of unreachable peers #3119
* Update 'gopacket' library to reduce memory use by approx 15MB #3160
* Replace bundling the 'docker' binary with our own code to avoid
security vulnerability alerts and save space #2957,#3110
##Bug fixes
* When `weave expose` is used, allow traffic into the Weave network -
up till version 1.12 Docker would do this for us, but in 1.13 they
stopped so now we do it. This change makes `weave expose` require
Weave Net to be running. #2758,#3122
* Arm64 build now works on non-kubernetes installs #2832,#3110
* TX offload was being disabled in 'awsvpc' mode, which slows down packet sending #3089
* Removed spurious 'nil' in logs from CNI DEL operation #3143
##Build and test
* Images are now also built for the ppc64le platform #3129
* Tweak build scripts to run on OSX as well as Linux #3135
##External Contributors
Thanks to the following contributors:
@caarlos0
@dtshepherd
[Full list of changes](https://github.com/weaveworks/weave/milestone/59?closed=1)
## Release 2.0.5
Bug fixes
* Fix /etc/hosts inside containers so the container's name resolves to
its Weave Net address #3136,#3138
* New weave-kube config for Kubernetes 1.7 and 1.8 which resolves an
intermittent conflict with kube-proxy that could break Weave Net
until reboot #2998,#3134
* Remove persistence file in `weave reset` when using Docker plugin V2 #3103,#3114
Build and test
* Some trivial changes to placate go-lint #3137
[Full list of changes](https://github.com/weaveworks/weave/milestone/63?closed=1).
## Release 2.0.4
Bug fixes
* weave-npc failed on Centos 7, due to older 'ipset' version in kernel #3099,#3100
[Full list of changes](https://github.com/weaveworks/weave/milestone/62?closed=1).
## Release 2.0.3
Bug fixes
* Weave-npc would crash on a policy with no 'from' part - regression introduced in 2.0.2 #3095,#3096,#3097
[Full list of changes](https://github.com/weaveworks/weave/milestone/61?closed=1).
## Release 2.0.2
Bug fixes and minor improvements
* Fix race condition in weave-npc which would intermittently block all traffic for a namespace #3057,#3059
* Ensure Fast Datapath works on machines that need to mount the kernel module dynamically #3080
* Regression: weave-npc would block everything if `kubelet --hostname-override` was used #3049,#3051
* Fix netfilter rules to block containers from accessing the Weave Net control endpoint #3093
* If DNS server is off then disable proxy DNS registration, to avoid spurious errors #3054,#3088
* Add comments to each iptables rule and ipset, to help when troubleshooting #3064
* Remove code that checked for an outdated fallback address for Kubernetes api-server #3071
* Add a label to the weavedb image so it can be filtered out by tools #3066
* Fix various build and continuous-integration failures #3061
* Print 'help' text faster in the weave script #3056
* Add an option to create continuous integration hosts in different ways #3060
* Remove remnants of the pre-2.0 proxy and plugin from build and test #3035,#3036
[Full list of changes](https://github.com/weaveworks/weave/milestone/60?closed=1).
## Release 2.0.1
Bug fixes and minor improvements
* Fall back to slower data path (`sleeve`), rather than crashing, when the machine lacks VXLAN support (required for “fast data path”, `fastdp`) #3043
* Fix bug in processing of arguments when Docker has TLS enabled, rather than crashing with invalid peers list, e.g. `lookup --tlsverify: no such host`#3039
* Add `kube-system` namespace back to `weave-kube`'s YAMLs, preventing omissions leading to errors like `error contacting APIServer: the server does not allow access to the requested resource`#3033,#3042
* Fix release script to prevent ARM64 binaries to end up in AMD64 `net-plugin`, leading to `Error response from daemon: dial unix /run/docker/plugins/<id>/weave.sock: connect: no such file or directory` when installing `net-plugin`#3045
*`weave reset` and `weave rmpeer` now only contact Weave Cloud when Weave Net is configured with a Weave Cloud token, preventing unnecessary requests and potentially confusing `401 Unauthorized` errors in Weave Net’s logs #3044
[Full list of changes](https://github.com/weaveworks/weave/milestone/58?closed=1).
## Release 2.0.0
New Features
Peer Discovery via Weave Cloud
You can now get all your Weave Net peers to find each other via the
Weave Cloud service, instead of maintaining a list of peers at
@@ -24,6 +24,25 @@ taking the time to read this guide.
* Keep pull requests small so core developers can review them quickly.
* Keep each pull request focused on a specific topic. If you have two things to change, create two pull requests. This helps reviewers to understand the meat of your contribution.
## Managing Vendored Dependencies
We copy all Go dependencies into the `vendor` subdirectory and use
[`dep`](https://github.com/golang/dep) to manage them.
If you add a dependency in the code, run `dep ensure` to include it in
the config and vendor dir. Same if you remove a dependency from the code.
To update a single dependency run `dep ensure -update github.com/foo/bar`
To override `dep`'s choice of library version, you can add
# If the architecture is amd64, add the -amd64 suffix.
$(SUDO)DOCKER_HOST=$(DOCKER_HOST) docker tag $(DOCKERHUB_USER)/$*:$(WEAVE_VERSION)$(DOCKERHUB_USER)/$*-amd64:$(WEAVE_VERSION)
# If the version is latest, tag the -amd64 with latest as well
ifneq($(UPDATE_LATEST),false)
$(SUDO) DOCKER_HOST=$(DOCKER_HOST) docker tag $(DOCKERHUB_USER)/$*-amd64:$(WEAVE_VERSION)$(DOCKERHUB_USER)/$*-amd64:latest
endif
# Push the image with the -amd64-suffix so BINARY-ARCH-named images exist for all arches, so manifest lists may be made and ARCH is replaceable in scripts
[Weaveworks](https://www.weave.works) is the company that delivers the most productive way for developers to connect, observe and control
Docker containers.
[Weaveworks](https://www.weave.works) is the company that delivers the most productive way for developers to connect, observe and control Docker containers.
This repository contains [Weave Net](https://www.weave.works/products/weave-net/), the first product developed by Weaveworks, and with nearly 5 million downloads to date, it enables you to get started with Docker clusters and portable apps in a fraction of the time compared with other solutions.
This repository contains [Weave Net](https://www.weave.works/products/weave-net/), the first product developed by Weaveworks, with over 8 million downloads to date. Weave Net enables you to get started with Docker clusters and portable apps in a fraction of the time required by other solutions.
Other products from Weaveworks include [Weave Scope](https://www.weave.works/products/weave-scope/), a powerful container monitoring tool that automatically maps Docker containers and their interactions, and [Weave Flux](https://www.weave.works/products/weave-flux/), a microservice router that automates the access of containers as services.
## Weave Cloud:
[Weave Cloud](https://www.weave.works/product/cloud/) is software-as-a-service that simplifies deployment, monitoring and management for containers and microservices. It extends and complements popular orchestrators, and enables developers and DevOps to realize faster deployments, insightful monitoring, visualization and networking.
Weave Cloud is built using these Open Source projects: [Weave Scope](https://www.weave.works/products/weave-scope/), a powerful container visualization tool that automatically maps Docker containers and their interactions, [Weave Cortex](https://github.com/weaveworks/cortex), a horizontally-scalable version of Prometheus, and [Weave Flux](https://www.weave.works/products/weave-flux/), a continuous deployment tool that works with Kubernetes.
# Weave Net
@@ -21,5 +23,19 @@ Weave Net creates a virtual network that connects Docker containers across multi
Services provided by application containers on the Weave network can be exposed to the outside world, regardless of where they are running. Similarly, existing internal systems can be opened to accept connections from application containers irrespective of their location.
* [Find out more about Weave Net](https://www.weave.works/products/weave-net/)
## <a name="help"></a>Getting help
If you have any questions about, feedback for or a problem with Weave Net:
- Read [the Weave Net docs](https://www.weave.works/docs/net/latest/overview).
- Invite yourself to the <a href="https://slack.weave.works/" target="_blank">Weave Users Slack</a>.
- Ask a question on the [#net-dev](https://weave-community.slack.com/messages/net-dev/) slack channel.
- Join the <a href="https://www.meetup.com/pro/Weave/"> Weave User Group </a> and get invited to online talks, hands-on training and meetups in your area.
- [File an issue](https://github.com/weaveworks/weave/issues/new).
- go get github.com/weaveworks/wordepress && cd /home/ubuntu/src/github.com/weaveworks/wordepress && git checkout v1.0.0 && cd cmd/wordepress && go get
- cd $SRCDIR; bin/publish-site "$WP_LIVE_URL" "$WP_LIVE_USER" "$WP_LIVE_PASSWORD"
# - go get github.com/weaveworks/wordepress && cd /home/ubuntu/src/github.com/weaveworks/wordepress && git checkout v1.0.0 && cd cmd/wordepress && go get
# - cd $SRCDIR; bin/publish-site "$WP_DEV_URL" "$WP_DEV_USER" "$WP_DEV_PASSWORD"
# issues:
# branch: /.*/
# owner: weaveworks
# commands:
# - go get github.com/weaveworks/wordepress && cd /home/ubuntu/src/github.com/weaveworks/wordepress && git checkout v1.0.0 && cd cmd/wordepress && go get
# - cd $SRCDIR; bin/publish-site "$WP_DEV_URL" "$WP_DEV_USER" "$WP_DEV_PASSWORD"
host1# ip xfrm state add src ${IP1} dst ${IP2} proto esp spi 0x4c856ffc replay-window 256 flag esn reqid 0 mode transport aead 'rfc4106(gcm(aes))' ${KEY1} 128
host1# ip xfrm state add src ${IP2} dst ${IP1} proto esp spi 0x9b0830bc replay-window 256 flag esn reqid 0 mode transport aead 'rfc4106(gcm(aes))' ${KEY2} 128
host1# ip xfrm policy add src ${IP1}/32 dst ${IP2}/32 dir out tmpl src ${IP1} dst ${IP2} proto esp spi 0x4c856ffc reqid 0 mode transport
host2# ip xfrm state add src ${IP2} dst ${IP1} proto esp spi 0x9b0830bc reqid 0 replay-window 256 flag esn mode transport aead 'rfc4106(gcm(aes))' ${KEY2} 128
host2# ip xfrm state add src ${IP1} dst ${IP2} proto esp spi 0x4c856ffc reqid 0 replay-window 256 flag esn mode transport aead 'rfc4106(gcm(aes))' ${KEY1} 128
host2# ip xfrm policy add src ${IP2}/32 dst ${IP1}/32 dir out tmpl src ${IP1} dst ${IP2} proto esp spi 0x9b0830bc reqid 0 mode transport
This document describes the various ways that weave as a binary gets launched, and where and how it processes options.
The goal is to enable maintainers to modify functionality without having to reverse engineer the code to find all of the places you can start a weave binary and what it depends upon.
## Components
### weaver
`weaver` is the primary binary: it sets up the weave bridge to which containers are attached, manages IP address allocation, serves DNS requests and does many more things.
Since version 2.0, `weaver` also bundles the Docker Plugin (legacy version) and Docker API Proxy which used to be shipped as separate containers.
### weaveutil
weaveutil is a binary which provides a number of functions for managing an existing weave network. It gets information about the network, attaches and detaches containers, etc.
For the majority of interactions with an existing weave network, you will launch `weaveutil` in some manner or another.
Almost all options can be passed as `--option` to `weaveutil`. These options are created in `prog/weaveutil/main.go` with a table mapping each command to a dedicated golang function.
In addition to operating in normal mode, `weaveutil` has several additional operating modes. Before processing any commands, `weaveutil` checks the filename with which it was called.
* If it was `weave-ipam` it delegates responsibility to the `cniIPAM()` function in `prog/weaveutil/cni.go`, which, in turn, calls the standard CNI plugin function `cni.PluginMain()`, passing it weave's IPAM implementation from `plugin/ipam`.
* If it was `weave-net` it delegates responsibility to the `cniNet()` function in `prog/weaveutil/cni.go`, which, in turn, calls the standard CNI plugin function `cni.PluginMain()`, passing it weave's net plugin implementation from `plugin/net`.
Finally, `weaveutil` is called by `weaver` when it needs to operate in a different network namespace.
Go programs cannot safely switch namespaces; see this [issue](http://github.com/vishvananda/netns/issues/17).
The call chain is `weaver`->`nsenter`->`weaveutil`.
### weave
Wrapping `weaver` and `weaveutil` is `weave`, a `sh` script that provides help information and calls `weaveutil` as relevant.
### weave-kube
weave-kube is an image with the weave binaries and a wrapper script installed. It is responsible for:
1. Setting up the weave network
2. Connecting to peers
3. Copy the weave CNI-compatible binary plugin `weaveutil` to `/opt/cni/bin/weave-net` and weave config in `/etc/cni/net.d/10-weave.conflist`
4. Running the weave network policy controller (weave-npc) to implement kubernetes' `NetworkPolicy`
Once installation is complete, each network-relevant change in a container leads `kubelet` to:
1. Set certain environment variables to configure the CNI plugin, mostly related to the container ID
2. Launch `/opt/cni/bin/weave-net`
3. Pass the CNI config - the contents of `/etc/cni/net.d/10-weave.conflist` to `weave-net`
Thus, when each container is changed, and `kubelet` calls weave as a CNI plugin, it really just is launching `weaveutil` as `weave-net`.
The installation and setup of all of the above - and therefore the entrypoint to the weave-kube image - is the script `prog/weave-kube/launch.sh`. `launch.sh` does the following:
1. Read configuration variables from the environment. When the documentation for `weave-kube` describes configuring the weave network by changing the environment variables in the daemonset in the `.yml` file, `launch.sh` reads these environment variables.
2. Set up the config file at `/etc/cni/net.d/10-weave.conflist`
3. Run the `weave` initialization script
4. Run `weaver` with the correct configuration passed as command-line options
## Adding Options
To add new options to how weave should run with each invocation, you would do the following:
1. determine to which `weave` command you want to add the option(s). `weave` normally is launched as `weave <command> <options> ...`.
2. Add the option to `weave` script help for each `weave` command you wish to make it available. As of this writing, all of the commands and their options are listed in the function `usage_no_exist()`.
3. Add the option to `weave` script option processing for the `weave` command, under the `case $COMMAND in` in the main `weave` script.
4. In the function for the `weave` command, determine if the command should be passed on to `weaveutil` or `weaver`. Pass the option on to `weaveutil` or `weaver`, as appropriate, in the format `--option`, in the function for the `weave` command.
5. Add a command-line option `--option` to `weaveutil` or `weaver` as appropriate.
6. If the option can be configured for CNI:
* have `prog/weave-kube/launch.sh` read it as an environment variable and set inside
* set a default for the environment variable in `prog/weave-kube/weave-daemonset-k8s-N.N.yaml`
* if it should be set via `weaver` globally on its one-time initialization invocation, pass it on in `launch.sh`
* if it needs to be set via `weaveutil` on each invocation of `weave-net`, have `launch.sh` save it as an option in the CNI config file `/etc/cni/net.d/10-weave.conflist` and then have the CNI code in `plugin/net/cni.go` in `weaveutil` read it and use where appropriate
- it will continue to ask for space until it receives some, or its
copy of the ring tells it all peers are full in that subnet.
### Data persistence
Key IPAM data is saved to disk, in a [BoltDB](https://github.com/boltdb/bolt)
file, stored within a [data volume container](https://docs.docker.com/engine/userguide/containers/dockervolumes/#creating-and-mounting-a-data-volume-container)
named `weavedb`.
This file is used to persist data for various Weave Net components; for IPAM it contains:
* the division of the IP allocation range amongst peers, and
* allocation of addresses to containers on the local peer,
so that it is immediately available when the peer restarts.
### Claiming an address
If a Weave process is restarted, in most cases it will hear from
another peer which ranges it used to own, but it needs to know which
individual IP addresses are assigned to containers in order to avoid
giving the same address out in subsequent allocation requests. The
weave script invokes the `claim` command in `populate_ipam` to do
this.
If a Weave process is restarted,
- if `weavedb` is present, then it loads persisted IPAM data from there, as
described, in the previous section;
- else, it learns from other peers which ranges it used to own.
Weave Net then consults the allocator to know which individual IP addresses
are assigned to containers, and therefore avoid giving the same address out in
subsequent allocation requests.
When the Allocator is told to claim an address, there are four
github-release](https://help.github.com/articles/creating-an-access-token-for-command-line-use/); select the `repo` OAuth scope; set and export `$GITHUB_TOKEN` with this value
## Release Types
@@ -128,16 +128,59 @@ Finally, for **Mainline** releases only:
* Images tagged `latest` are updated on DockerHub
Now, you can validate whether images were published for all platforms:
Weave Net is [available in Docker Store](https://store.docker.com/plugins/weave-net-plugin) and should also be released there.
* Go to https://store.docker.com/ and log in.
* Go to "[_Publish_](https://store.docker.com/publisher/center)".
* Under "_My Products_", select "_owners (weaveworks)_". Weave Net should be listed among our products.
* Click "_Actions_" > "_Edit Product_".
* Go to "_Plans & Pricing_" > "_Free Tier_".
* Under "_Source Repositories & Tags_", click on "_Add Source_", select "_net-plugin_" under "_Repository_" and the version to release under "_Tag_".
* Under "_Resources_" > "_Installation Instructions_", update the lines containing `export NET_VERSION=<version>`.
* Click "_Save_"
* Click "_Submit For Review_". You should see a message like: "_Your product has been submitted for approval. [...] We'll be in touch with next steps soon!_".
* You should receive an email saying:
> This email confirms that we received your submission on \<date\> of Weave Net to the Docker Store.
>
> We're reviewing your submission to ensure that it meets our [security guidelines](https://success.docker.com/Store#Security_and_Audit_Policies) and complies with our [best practices](https://success.docker.com/Store#Create_Great_Content). Don't worry! We'll let you know if there's anything you need to change before we publish your submission. You should hear back from us within the next 14 days.
>
> Thanks for submitting your content to the Docker Store!
* Hope Docker eventually performs the release or contacts us. If nothing happens within 14 days, contact their support team: publisher-support@docker.com.
### Finish up
* If not on master, merge branch into master and push to GitHub.
* Close the [milestone](https://github.com/weaveworks/weave/milestones) in GitHub and create the next milestone
* Update the `#weavenetwork` topic heading on freenode (requires 'chanops' permission)
* For a mainline release vX.Y.0, create a release branch X.Y from the
tag and push to GitHub - this will result in X.Y.0 site docs being
published to https://www.weave.works
tag, push to GitHub and set [WEAVE_NET_REV](https://github.com/weaveworks/website-next/blob/master/Makefile)
to X.Y via a PR - this will result in X.Y.0 site docs being published to https://www.weave.works
* Add the new version of `weave-net` to the checkpoint system at
https://checkpoint-api.weave.works/admin
* File a PR to update the version of the daemonset at
common.Log.Infof("ring: repair inconsistent data by accepting token %v from %v - no allocations up to %v",theirs.Token,theirs.Peer,es.entry(i+1).Token)
Loaded 100 of 3714 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Bryan Boreham