Damien Lespiau
8d44f49edd
The liveness probe has the not-so-great side effect of killing the weave pods if anything goes wrong, making debugging really difficult. Not only that, we also bumped into the livenessProbe killing pods before they had a chance to start (see below for the full story). The readinessProbe is only used for Kubernetes Services: no traffic will be routed to non-Ready pods when using the Service IP. We can use to to signal the user something in wrong with Weave Net while not having Kubernetes delete the pods behind our backs. I removed the ininitialDelaySeconds field as Kurbernetes won't kill the pod if not ready. Notes from a debugging session: Some nodes may have a vast amount of iptables rules, making each iptables command take a few seconds (~5-10s). At startup, when the weave container creates its chains, it runs quite a few iptables commands and the total time of execution was > 30s. It's important to note that, at this point, the weave process hasn't started as those iptables commands are run by an initial script. Kubernetes was trying to check for the container liveness, was failing, and then proceeded to kill the weave pods before they had a chance to fully start.
Weave Net - Weaving Containers into Applications
About Weaveworks
Weaveworks is the company that delivers the most productive way for developers to connect, observe and control Docker containers.
This repository contains Weave Net, the first product developed by Weaveworks, and with over 8 million downloads to date, enables you to get started with Docker clusters and portable apps in a fraction of the time compared with other solutions.
Weave Cloud:
Weave Cloud is software-as-a-service that simplifies deployment, monitoring and management for containers and microservices. It extends and complements popular orchestrators, and enables developers and DevOps to realize faster deployments, insightful monitoring, visualization and networking.
Weave Cloud is built using these Open Source projects: Weave Scope, a powerful container visualization tool that automatically maps Docker containers and their interactions, Weave Cortex, a horizontally-scalable version of Prometheus, and Weave Flux, a continuous deployment tool that works with Kubernetes.
Weave Net
Weave Net creates a virtual network that connects Docker containers across multiple hosts and enables their automatic discovery. With Weave Net, portable microservices-based applications consisting of multiple containers can run anywhere: on one host, multiple hosts or even across cloud providers and data centers. Applications use the network just as if the containers were all plugged into the same network switch, without having to configure port mappings, ambassadors or links.
Services provided by application containers on the Weave network can be exposed to the outside world, regardless of where they are running. Similarly, existing internal systems can be opened to accept connections from application containers irrespective of their location.
Getting help
If you have any questions about, feedback for or a problem with Weave Net:
- Read the Weave Net docs.
- Invite yourself to the #weave-community slack channel.
- Ask a question on the #weave-community slack channel.
- Join the Weave User Group and get invited to online talks, hands-on training and meetups in your area.
- Send an email to weave-users@weave.works
- File an issue.
Your feedback is always welcome!