Damien Lespiau
8d44f49edd
The liveness probe has the not-so-great side effect of killing the weave pods if anything goes wrong, making debugging really difficult. Not only that, we also bumped into the livenessProbe killing pods before they had a chance to start (see below for the full story). The readinessProbe is only used for Kubernetes Services: no traffic will be routed to non-Ready pods when using the Service IP. We can use to to signal the user something in wrong with Weave Net while not having Kubernetes delete the pods behind our backs. I removed the ininitialDelaySeconds field as Kurbernetes won't kill the pod if not ready. Notes from a debugging session: Some nodes may have a vast amount of iptables rules, making each iptables command take a few seconds (~5-10s). At startup, when the weave container creates its chains, it runs quite a few iptables commands and the total time of execution was > 30s. It's important to note that, at this point, the weave process hasn't started as those iptables commands are run by an initial script. Kubernetes was trying to check for the container liveness, was failing, and then proceeded to kill the weave pods before they had a chance to fully start.
101 lines
2.9 KiB
YAML
101 lines
2.9 KiB
YAML
apiVersion: v1
|
|
kind: List
|
|
items:
|
|
- apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
name: weave-net
|
|
labels:
|
|
name: weave-net
|
|
namespace: kube-system
|
|
- apiVersion: extensions/v1beta1
|
|
kind: DaemonSet
|
|
metadata:
|
|
name: weave-net
|
|
labels:
|
|
name: weave-net
|
|
namespace: kube-system
|
|
spec:
|
|
template:
|
|
metadata:
|
|
annotations:
|
|
scheduler.alpha.kubernetes.io/tolerations: >-
|
|
[{"key":"dedicated","operator":"Equal","value":"master","effect":"NoSchedule"}]
|
|
labels:
|
|
name: weave-net
|
|
spec:
|
|
containers:
|
|
- name: weave
|
|
command:
|
|
- /home/weave/launch.sh
|
|
env:
|
|
- name: HOSTNAME
|
|
valueFrom:
|
|
fieldRef:
|
|
apiVersion: v1
|
|
fieldPath: spec.nodeName
|
|
image: 'weaveworks/weave-kube:latest'
|
|
imagePullPolicy: Always
|
|
readinessProbe:
|
|
httpGet:
|
|
host: 127.0.0.1
|
|
path: /status
|
|
port: 6784
|
|
resources:
|
|
requests:
|
|
cpu: 10m
|
|
securityContext:
|
|
privileged: true
|
|
volumeMounts:
|
|
- name: weavedb
|
|
mountPath: /weavedb
|
|
- name: cni-bin
|
|
mountPath: /host/opt
|
|
- name: cni-bin2
|
|
mountPath: /host/home
|
|
- name: cni-conf
|
|
mountPath: /host/etc
|
|
- name: dbus
|
|
mountPath: /host/var/lib/dbus
|
|
- name: lib-modules
|
|
mountPath: /lib/modules
|
|
- name: weave-npc
|
|
env:
|
|
- name: HOSTNAME
|
|
valueFrom:
|
|
fieldRef:
|
|
apiVersion: v1
|
|
fieldPath: spec.nodeName
|
|
image: 'weaveworks/weave-npc:latest'
|
|
imagePullPolicy: Always
|
|
resources:
|
|
requests:
|
|
cpu: 10m
|
|
securityContext:
|
|
privileged: true
|
|
hostNetwork: true
|
|
hostPID: true
|
|
restartPolicy: Always
|
|
securityContext:
|
|
seLinuxOptions: {}
|
|
serviceAccountName: weave-net
|
|
volumes:
|
|
- name: weavedb
|
|
hostPath:
|
|
path: /var/lib/weave
|
|
- name: cni-bin
|
|
hostPath:
|
|
path: /opt
|
|
- name: cni-bin2
|
|
hostPath:
|
|
path: /home
|
|
- name: cni-conf
|
|
hostPath:
|
|
path: /etc
|
|
- name: dbus
|
|
hostPath:
|
|
path: /var/lib/dbus
|
|
- name: lib-modules
|
|
hostPath:
|
|
path: /lib/modules
|