As noted in #388 comment 76958961, we need the docker client to be as
old as possible while supporting what weave needs to do. At the minute
that means version 1.3.1.
The `apk` package index does not have old versions of docker, so the
most direct way of getting a docker executable is to download the
specific version from https://get.docker.com.
We want to be compatible with whatever docker the user has; `weave
run` is the only command that passes on arbitrary args to docker
(i.e., that doesn't have a fixed set of docker features that it
needs).
To make this workable we still need to do some things on the remote
host: the IP attachment and figuring out the DNS settings (since those
may depend on the bridge IP).
This is to make a visible break with previous releases, such that a
newly downloaded weave script will definitely work.
Without this change in name, an existing weavetools image that doesn't
include the script might be invoked, resulting in a confusing message
for the user.
`apk-install` is a script that runs apk then tidies up. The problem is
that it doesn't exit with an error if the install fails, which means
that if some packages are unavailable at the time of building, it will
still all go ahead.
This rearrangement means that the `weave` command can be used
remotely; and, as a side-effect, the script that does all the work is
using executables in its own filesystem, rather than invoking them via
docker.
The remote weavetools is given a terminal so it can be interrupted.
With the flag `--local`, the script will assume all its dependencies
are available and run then and there. Without the flag, it will run
via the weavetools image (adding the flag to the invocation).
A choice here is whether to base the tools image on scratch and add
all the deps, or base it on something else. I've used alpine linux,
which itself is based on busybox. This gives us the usual tools (sed
etc.) while keeping the image size small.
Adding the docker binary adds substantially to the image size, but it
cannot always be e.g., bind-mounted. To mitigate this image bloat, we
might build or otherwise get a client-only docker executable, or write
a special-purpose client, or indeed rewrite the driver script and use
a client library.
This also makes the dockerhub user part of the image names
parameterised, to make it easier to test the release process in full.
Tests weren't compiling due to change of ListenHttp() signature. Fixed
for now by reintroducing domain parameter in addition to server, as this
allows the HTTP tests to pass in a nil server parameter. In the long
term DNSServer should probably become an interface so we can mock it
easily in such a scenario.
Weave sends all gossip down new connections. We do this so that new
peers learn about all gossip'ed information quickly, thus enabling
them to process subsequent incremental updates. In particular, for
topology gossip it ensures that the remote peer learns about all other
peers. This ensures that the incremental update, which is sent as a
result of the new connection getting established and which may
reference a number of other peers, can be processed.
So the intent really is to send all gossip to new peers. But what we
actually do is send it down every new connection. Not every new
connection is from a new peer. In fact most connections aren't. Hence
the current approach is wasteful.
To fix this we need a conservative detection of new peers, i.e. it is
preferable to have false positives. This contrasts with existing logic
in the handshake code. The criteria we use is whether our peer has a
unicast route to the other peer. That is only the case when there is a
continuos chain of symmetric, established connections. The
establishment of these connections should have triggered the sending
of all gossip to the other peer. So we don't need to send it ourself.
Fixes#454.
The most likely cause of failure is the container going away, in which.
case we don't need to clean up. And there are other points of possible
failure in the same category in attach(), so making sure we clean up the
container interface under all conditions is a broader issue.
When an IP address was reused by a new container, other containers
could experience delays of many seconds the first time they tried to
talk to that new container before successful communication would
occur. This delay came from three sources related to ARP cache
behaviour:
- The kernel would take a while to decide that an ARP mapping was
stale due to lack of positive evidence (due to the
base_reachable_time default of 30s)
- Even once an ARP mapping was considered stale, the kernel would wait
a few seconds before sending an ARP request, in the hope that a
higher layer (e.g. TCP) would confirm the ARP entry (due to the
delay_first_probe default of 5s)
- When the kernel sent out an ARP request to update a stale entry,
initial attempts would unicast the request to the old MAC address
before resorting to broadcast (due to the ucast_solicit default of
3)
This change reduces all these values, so that any delays
are a few seconds.
For most workloads, this should not greatly increase the amount
of ARP traffic, because communication via TCP is often sufficient
to confirm ARP entries. And one unicast ARP request is still sent
just in case.
Unfortunately, there's a fixed-but-only-recently bug in the kernel
that means that changing base_reachable_time can take a while to take
effect:
<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=4bf6980dd0328530783fd657c776e3719b421d30>.
This means that a container will continue to use a long reachable_time
delay for the first few minutes of its life.
Matthias Radestock