Commit Graph
81 Commits
Author SHA1 Message Date
Tanishq Dubey 959d0bf2e2 Add tolerations for nodes tainted with NoExecute 2019-06-30 00:27:57 -04:00
Murali Reddy 17b45d9efa handle single node case where return peer list from kube-utils is empty
fixes side affet introduced in #3454
2019-01-18 01:06:37 +05:30
Murali Reddy df467be26e watch for node delete events and call reclaimRemovedPeers 2018-10-24 13:45:35 +05:30
Damien Lespiau 8d44f49edd kubernetes: Morph the livenessProbe into readinessProbe
The liveness probe has the not-so-great side effect of killing the weave pods
if anything goes wrong, making debugging really difficult.

Not only that, we also bumped into the livenessProbe killing pods before they
had a chance to start (see below for the full story).

The readinessProbe is only used for Kubernetes Services: no traffic will be
routed to non-Ready pods when using the Service IP. We can use to to signal the
user something in wrong with Weave Net while not having Kubernetes delete the
pods behind our backs.

I removed the ininitialDelaySeconds field as Kurbernetes won't kill the pod if
not ready.

Notes from a debugging session:

Some nodes may have a vast amount of iptables rules, making each iptables
command take a few seconds (~5-10s). At startup, when the weave container
creates its chains, it runs quite a few iptables commands and the total time of
execution was > 30s.

It's important to note that, at this point, the weave process hasn't started as
those iptables commands are run by an initial script.

Kubernetes was trying to check for the container liveness, was failing, and
then proceeded to kill the weave pods before they had a chance to fully start.
2018-09-28 15:50:39 +01:00
Martynas Pumputis 07d31a8558 npc: Get rid of the legacy netpol 2018-07-24 10:06:55 +02:00
Munro, Louis b262a1eedc Fix xtables.lock race condition between weave and kube-proxy.
Fixes #3351
2018-07-22 13:12:00 -04:00
Bryan Boreham d513e68fab Add permissions to set node status
This is needed to update the node NetworkUnavailable conditon
see https://github.com/weaveworks/weave/issues/3249
2018-06-21 11:08:27 +00:00
Martynas Pumputis 49216ce365 Merge pull request #3307 from weaveworks/set-kube-netstatus
Set Kubernetes NodeNetworkUnavailable to false when starting
2018-06-11 15:20:14 +02:00
Bryan Boreham 812c58297f Merge pull request #3298 from weaveworks/issues/2924-preserve-src-ip-on-local
2924: Preserve the client source IP
2018-06-11 13:21:12 +01:00
Bryan Boreham 2a04df1049 Rename "kube-peers" to "kube-utils"
It's now doing things like network status that are outside of the
scope of "peers".
2018-06-11 11:22:07 +00:00
Martynas Pumputis 6dbe95a0b5 Add NO_MASQ_LOCAL to weave-kube/launch.sh
One can enable the feature by setting NO_MASQ_LOCAL=1 among
env vars of the weave-kube DaemonSet.
2018-06-04 14:08:24 +02:00
Bryan Boreham 9b0d1bd054 Set Kubernetes NodeNetworkUnavailable to false when starting 2018-05-26 14:06:57 +00:00
Marc Carré 9f207f161a Add org.opencontainers.image.* labels to Dockerfiles
- This should ultimately help for image-to-code back references.
- `org.label-schema.*` labels are now deprecated, in favour of `org.opencontainers.image.*` labels.
  See also: https://github.com/opencontainers/image-spec/blob/master/annotations.md#back-compatibility-with-label-schema
- Git revision is now injected at `docker build` time.
2018-05-14 11:51:40 +01:00
Bryan Boreham f6fdaf6492 Only expose metrics by default in weave-kube 2018-04-04 15:00:53 +01:00
Bryan Boreham c528303841 Merge pull request #3265 from weaveworks/raise-kube-conn-limit
Raise weave-kube default connection limit from 30 to 100
2018-03-28 10:20:30 +01:00
Bryan Boreham a439288765 Raise weave-kube default connection limit from 30 to 100
This was missed from 265bfcf
2018-03-27 21:06:41 +00:00
Bryan Boreham ea3aba8971 Wait 5 seconds to let pod connect before rolling next pod 2018-01-31 10:54:07 +00:00
Bryan Boreham 3b499ce27c Rearrange peer reclaim checks in weave-kube launch script
This is to make it behave better on fresh start, and in the case where
the manifest has not been updated to give it permissions.
2017-11-16 14:14:54 +00:00
Bryan Boreham 570ab0b10b Avoid deleting persisted data on weave-kube upgrade
Add a feature to weaveutil to get and set flags, then use this feature
to mark the DB when this version runs
2017-11-15 11:26:43 +00:00
Bryan Boreham 214cf6cb2e Flip result code when checking for new peer
This avoids mistaking an error condition for the result we should act on
2017-11-15 10:11:02 +00:00
Bryan Boreham c67bc704e1 Temporarily raise kube-peers log level to debug 2017-11-14 14:16:01 +00:00
Bryan Boreham 3501ad45e6 Cope with a Kubernetes node being deleted and coming back
Add features to weaver and kube-peers to find the peer name and check
if it is in the list, then in launch.sh delete persisted data if not.
2017-11-14 14:15:31 +00:00
Bryan Boreham 71b39489da Maintain a list of peers in a Kubernetes annotation
This is prework for a feature which will remove 'dead' peers
2017-11-14 14:06:33 +00:00
Bryan Boreham ed8d92cfc6 RBAC to read and write ConfigMaps 2017-11-14 14:06:33 +00:00
Martynas Pumputis b605b832ed npc: Extend k8s NetworkPolicy smoke tests
- Test that pods not matching ingress rules are not able to access
  the nettest pods.
- Test upgrading of NPC (from legacy to non-legacy).
- Test new semantics of k8s netpol.
2017-11-07 14:55:03 +02:00
Martynas Pumputis 3c3e0b0352 Enable access to "networkpolicies" of "networking.k8s.io" API group
Since 1.7, k8s netpol is no longer in beta, so NPC can start subscribing
from the non-beta API group.
2017-10-22 19:32:52 +02:00
Bryan Boreham c16653a396 Mount iptables lock file in weave-kube (#3134)
Mount iptables lock file in weave-kube

iptables uses the file `/run/xtables.lock` so that if changes are
under way, another instance of the iptables command can wait till it
is finished.
We need to mount the lock file from the host to the container in order
for this mechanism to work at all.
Note we rely on kubelet touching the file before running any pods -
see https://github.com/kubernetes/kubernetes/pull/47212
2017-10-03 14:52:49 +01:00
Bryan Boreham 6b4c6c84b4 Add back HOSTNAME env var for weave-npc container 2017-07-03 16:41:07 +00:00
Marc CARRE a0064f7535 Update YAMLs to be what the Launch Generator serves.
See also:
- https://github.com/weaveworks/launch-generator/issues/133
- https://github.com/weaveworks/launch-generator/pull/134

Command used:
```
$ curl -fsSo prog/weave-kube/weave-daemonset-k8s-1.6.yaml "http://localhost:8080/k8s/v1.6/net?v=latest"
$ curl -fsSo prog/weave-kube/weave-daemonset.yaml "http://localhost:8080/k8s/v1.5/net?v=latest"
```
Annotations then removed.
2017-06-26 17:54:17 +01:00
Bryan Boreham d958cff58e Put kube-system namespace back in the weave-kube yaml
Save people getting in to trouble when they don't know to add `-n kube-system`
2017-06-23 16:07:55 +00:00
Marc CARRE 0d083077ab Update YAMLs to be what the Launch Generator serves.
See also:
- weaveworks/launch-generator/pull/130
- weaveworks/launch-generator/pull/125

Command used:
```
$ curl -fsSo prog/weave-kube/weave-daemonset-k8s-1.6.yaml "https://localhost:8080/k8s/v1.6/net?v=latest"
$ curl -fsSo prog/weave-kube/weave-daemonset.yaml "https://localhost:8080/k8s/v1.5/net?v=latest"
```
Annotations then removed.
2017-06-21 14:41:59 +01:00
Bryan Boreham 70f004cc62 Merge pull request #3018 from Shimi/patch-2
modify tolerations to match any taint
2017-06-20 12:02:56 +01:00
Marc CARRE 666631134c Update YAMLs to be what the Launch Generator serves.
See also: weaveworks/launch-generator/pull/117

Command used:
```
$ curl -fsSo prog/weave-kube/weave-daemonset-k8s-1.6.yaml "https://cloud.weave.works/k8s/v1.6/net?v=latest"
$ curl -fsSo prog/weave-kube/weave-daemonset.yaml "https://cloud.weave.works/k8s/v1.5/net?v=latest"
```
Annotations then removed.
2017-06-16 09:06:26 +00:00
Shimi Kiviti 66a569fc40 modify tolerations to match any node
Currently weave doesn't run on nodes with tolerations except the master
The suggested code change it to run on all nodes
2017-06-15 15:32:50 +03:00
Bryan Boreham 7871a0b7a4 Merge branch 1.9 2017-06-01 15:03:59 +00:00
Maxim Filatov cb3b858df6 Weave-NPC should get hostname from K8s Downward API.
It is necessary in case when kubelet's args includes `--hostname_override=`.
Without this change, pods on hostname overriding kubelets will never be local,
and all pod's traffic will be blocked.
2017-05-31 18:06:17 +03:00
Marc CARRE 067ce6de9f Update Weave-Kube daemonset YAML files using Launch Generator.
Note that given the YAML's structure is different from previous version, indentation of `COVERAGE_ARGS` had to change (+4 whitespaces to each line).
Commands used:
```
$ curl -fsS https://frontend.dev.weave.works/k8s/v1.5/net?v=latest > prog/weave-kube/weave-daemonset.yaml
$ curl -fsS https://frontend.dev.weave.works/k8s/v1.6/net?v=latest > prog/weave-kube/weave-daemonset-k8s-1.6.yaml
```
Annotations removed from Launch Generator's files:
```
      annotations:
        cloud.weave.works/launcher-info: |-
          {
            "server-version": "master-050965d",
            "original-request": {
              "url": "/k8s/v1.6/net?v=latest",
              "date": "Thu May 25 2017 13:16:43 GMT+0000 (UTC)"
            },
            "email-address": "support@weave.works"
          }
```
and
```
      annotations:
        cloud.weave.works/launcher-info: |-
          {
            "server-version": "master-050965d",
            "original-request": {
              "url": "/k8s/v1.5/net?v=latest",
              "date": "Thu May 25 2017 13:16:43 GMT+0000 (UTC)"
            },
            "email-address": "support@weave.works"
          }
```
2017-05-26 14:19:26 +01:00
Bryan Boreham bebaa48520 Change weavedb from an emptyDir to a hostPath volume
EmptyDir has the same lifetime as the pod, whereas we want our
persistence to extend past that, e.g. when Weave Net is upgraded.

`weave reset` cleans out the file.

We assume that /var/lib is writable.
2017-05-17 12:38:24 +00:00
Bryan Boreham c85fff8f23 Fix outdated comment 2017-05-09 14:27:24 +00:00
Bryan Boreham ad4fe86285 Merge branch '1.9' 2017-05-09 09:05:38 +00:00
Mike Bryant 9419ad16bf Clean up xt_set_exists components in advance
If this function dies in the middle, the leftover weave-kube-test set
causes this function and weave-npc to break. In particular
this means we can never restart weave with the launch script.

Here we clean up in advance, in case there's leftovers from a previous run

Fixes #2934
2017-05-08 12:15:23 +01:00
Bryan Boreham 9fd1a43769 Put back some weave bridge options
ability to set MTU was lost in an earlier change
2017-04-24 14:55:53 +00:00
Bryan Boreham ef9f95f653 Let weaver create the bridge in weave-kube 2017-04-17 19:41:47 +00:00
Bryan Boreham 9da5cee8d9 Rename Kubernetes daemonset to "1.6"
To emphasise this is the version that adopters of Kubernetes 1.6 need
to use.
2017-03-17 11:45:06 +00:00
Lucas Käldström 2e58e92d48 Add a weave-kube manifest that works for kubeadm v1.6 2017-03-17 11:43:48 +00:00
Marc CARRE 21a0fc30e8 Run weaver without exec.
This:

1. prevents from generating defunct (a.k.a. zombie) processes, and therefore fixes #2836; but also
2. reintroduces #2684/#2688 as shells do not forward signals and we are still running more than one process.

A proper fix would leverage something like tini.
See also:

- github.com/krallin/tini
- github.com/krallin/tini/issues/8
- github.com/docker-library/official-images#init

Sample output:

- during initialisation:

```
$ ps auxf
[...]
root      1380  0.4  1.9 879660 74660 ?        Ssl  Mar13   6:02 /usr/bin/docker daemon -H fd:// -H unix:///
root      1664  0.0  0.4 502116 18528 ?        Ssl  Mar13   0:06  \_ docker-containerd -l /var/run/docker/li
root     15482  0.0  0.0 200496  3364 ?        Sl   15:04   0:00      \_ docker-containerd-shim 3d1f5eb6e090
root     15496  0.0  0.0   1524   996 ?        Ss   15:04   0:00      |   \_ /bin/sh /home/weave/launch.sh
root     15780  0.0  0.0   1524    64 ?        S    15:04   0:00      |       \_ /bin/sh /home/weave/launch.
root     15872  0.0  0.0   1772  1272 ?        S    15:04   0:00      |       |   \_ /bin/sh /home/weave/wea
root     15897  0.0  0.0   1772   324 ?        S    15:04   0:00      |       |       \_ /bin/sh /home/weave
root     15898  0.0  0.0  14656  2948 ?        S    15:04   0:00      |       |           \_ curl -o /tmp/we
root     15781 10.0  1.5 484556 60296 ?        Sl   15:04   0:00      |       \_ /home/weave/weaver --port=6
[...]
```

- after initialisation:

```
$ ps auxf
[...]
root      1380  0.4  1.9 879660 74660 ?        Ssl  Mar13   6:02 /usr/bin/docker daemon -H fd:// -H unix:///
root      1664  0.0  0.4 502116 18592 ?        Ssl  Mar13   0:06  \_ docker-containerd -l /var/run/docker/li
root     15482  0.0  0.0 200496  3364 ?        Sl   15:04   0:00      \_ docker-containerd-shim 3d1f5eb6e090
root     15496  0.0  0.0   1524   996 ?        Ss   15:04   0:00      |   \_ /bin/sh /home/weave/launch.sh
root     15781  2.3  1.6 503064 64320 ?        Sl   15:04   0:00      |       \_ /home/weave/weaver --port=6[...]
```
2017-03-14 15:34:47 +00:00
Martynas Pumputis 3607cf2d7a Add check for xt_set module
The check is based on actually trying to install an iptables rule
which uses the module.
2017-03-01 10:37:43 +00:00
Martynas Pumputis a57cf4bc83 Ignore modprobe errors in weave-kube/launch.sh
weave-kube is run by alpine image which uses busybox. modprobe of
busybox will return the "module not found" error [1], if a module is
built-in, which is an opposite behavior to modprobe distributed in
mainstream distros (e.g. ubuntu).

[1]: https://bugs.busybox.net/show_bug.cgi?id=5270
2017-02-28 11:26:40 +00:00
Bryan Boreham cb73cf3f51 Ensure we have the kernel modules loaded for weave-npc 2017-02-27 12:32:57 +00:00
Bryan Boreham f7c0da3ae1 Merge pull request #2781 from keontang/luanch-update
Add CONN_LIMIT environment var for running container
2017-02-23 12:40:01 +00:00