We want to be compatible with whatever docker the user has; `weave
run` is the only command that passes on arbitrary args to docker
(i.e., that doesn't have a fixed set of docker features that it
needs).
To make this workable we still need to do some things on the remote
host: the IP attachment and figuring out the DNS settings (since those
may depend on the bridge IP).
This is to make a visible break with previous releases, such that a
newly downloaded weave script will definitely work.
Without this change in name, an existing weavetools image that doesn't
include the script might be invoked, resulting in a confusing message
for the user.
This rearrangement means that the `weave` command can be used
remotely; and, as a side-effect, the script that does all the work is
using executables in its own filesystem, rather than invoking them via
docker.
The remote weavetools is given a terminal so it can be interrupted.
With the flag `--local`, the script will assume all its dependencies
are available and run then and there. Without the flag, it will run
via the weavetools image (adding the flag to the invocation).
A choice here is whether to base the tools image on scratch and add
all the deps, or base it on something else. I've used alpine linux,
which itself is based on busybox. This gives us the usual tools (sed
etc.) while keeping the image size small.
Adding the docker binary adds substantially to the image size, but it
cannot always be e.g., bind-mounted. To mitigate this image bloat, we
might build or otherwise get a client-only docker executable, or write
a special-purpose client, or indeed rewrite the driver script and use
a client library.
This also makes the dockerhub user part of the image names
parameterised, to make it easier to test the release process in full.
The most likely cause of failure is the container going away, in which.
case we don't need to clean up. And there are other points of possible
failure in the same category in attach(), so making sure we clean up the
container interface under all conditions is a broader issue.
When an IP address was reused by a new container, other containers
could experience delays of many seconds the first time they tried to
talk to that new container before successful communication would
occur. This delay came from three sources related to ARP cache
behaviour:
- The kernel would take a while to decide that an ARP mapping was
stale due to lack of positive evidence (due to the
base_reachable_time default of 30s)
- Even once an ARP mapping was considered stale, the kernel would wait
a few seconds before sending an ARP request, in the hope that a
higher layer (e.g. TCP) would confirm the ARP entry (due to the
delay_first_probe default of 5s)
- When the kernel sent out an ARP request to update a stale entry,
initial attempts would unicast the request to the old MAC address
before resorting to broadcast (due to the ucast_solicit default of
3)
This change reduces all these values, so that any delays
are a few seconds.
For most workloads, this should not greatly increase the amount
of ARP traffic, because communication via TCP is often sufficient
to confirm ARP entries. And one unicast ARP request is still sent
just in case.
Unfortunately, there's a fixed-but-only-recently bug in the kernel
that means that changing base_reachable_time can take a while to take
effect:
<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=4bf6980dd0328530783fd657c776e3719b421d30>.
This means that a container will continue to use a long reachable_time
delay for the first few minutes of its life.
So we complain with "already running" rather than "Found another
running container" when encountering a 'weave' container running a
different image version. Similarly, we remove stopped 'weave'
containers regardless of version.
Ditto for weavedns.
Which is all how it used to work.
- released weave scripts run their corresponding image versions
- unreleased weave scripts run the 'latest'-tagged image versions
The image version to run can also be specified explicitly by setting
VERSION. Useful for testing, mainly.
and return an error status rather than exit when containers stop just
before an HTTP call, which in turn allows us to make tell_dns properly
ignore weavedns container absences.
Fixes#300
Rather than looking for the host end of the veth pair, we look for the
expected interface inside the container. This has the advantage of
detecting weave-attached-ness even when the container is sharing a
network namespace with others.
Closes#303, #302.
Adam Harrison