Previously when a peer restarted, information about the new
incarnation (i.e. with a different UID) was not accepted by other
peers (and connections would be dropped) unless all knowledge of the
previous incarnation had been purged. This could result in a lot of
connection churn and hence connectivity disruption, and, in some
pathological cases, very slow convergence and hence acceptance of the
new incarnation into the network.
We now no longer drop connections when encountering different
incarnations of a peer. There are two situations when that can happen:
1) on connection establishment
we simply proceed
2) on receipt of gossip
to ensure convergence we
a) treat the UID as an additional discriminator when deciding whether
we should update our information about a peer with that which was
gossiped. Specifically, we update the information we hold when a) the
gossiped version is greater, or b) is the same and the UID is
greater.
b) include the UID in the information we update
c) move our own version number beyond any we receive for ourselves, if
the received UID differs from ours.
With (a) we establishes a total order of peer information across
several incarnations of the same peer. i.e. we consider information to
be fresher if it has a higher version, or the same version and higher
UID. This may seem somewhat counter intutive, since it will generally
treat information about new incarnations as older than old
incarnations, since incarnations always start life with version 1. But
to do better we'd need to establish a total order of incarnations that
matches their temporal occurrence. Which requires some sort of durable
state.
So instead we have (c). Through that we learn the highest version
number of any old incarnation of ourselves that other peers still
hold, and then make sure that our version is greater than
that. Essentially we continue where the old incarnations left
off. It's as if instead of restarting we had simply changed UIDs. And
due to (a) and (b) the information about the new incarnation of
ourselves, now with a higher version, will supersede that of the old
incarnations.
Fixes#1554.
These are needed to support vxlan encapsulation. In a vxlan packet,
the only place we can straightforwardly hold the source and
destination peer identifiers used by the weave router is in the
24-bit VNI (vxlan network identifier) field. This gives us 12 bits to
indicate a peer. The resulting limit of 4096 peers on a weave network
does not seem like a problem in the near future. But it does mean
that decentralized allocation makes collisions fairly likely even in
weave networks of sizes that are common today. Fortunately, the
problem is much more tractable than IPAM because we can detect when
collisions occur and take action to resolve them.
For fast datapath, the router core will only be invoked upon datapath
misses. And it needs to yield routing decisions in a form that can be
used to configure the in-kernel datapath. So here we make the router
core yield FlowOps, which represent operations which actually do
something with a packet. The router core obtains these in turn from
the Bridge or the Overlay, depending on what it wants to do with
a packet.
For fast datapath, it's critical that the router makes decisions based
only on the required packet header information (as the same header
information will be used when configuring the ODP flow rules). So we
pass only that information into the router core, in the form of
PacketKeys.
The overall effect is to curry the router core: Rather than taking a
packet and doing something with it, it takes packet header information
and returns a function that does something with a packet.
This introduces interfaces to represent the packet forwarding
mechanism, and brings all the scattered UDP-related code together into
a single implementation.
This is not a pure refactoring, although much of the code in sleeve.go
has been copied from pre-existing code, and the UDP protocol remains
the same in spirit. The biggest change is the organization of the
forwarder goroutines (there is now one instead of two, and that
goroutine does a lot of auxiliary work as well as forwarding).
Performance testing between VMs on my laptop suggests that packet
throughput has not gone down, and may even have improved slightly.
When we discover that a connection target address leads to ourself we
set the retry time to the zero time.Time value - the only special
time.Time value. We use that marker to a) suppress subsequent
connection attempts, and b) display the next retry as 'never' in
`weave status|report`.
Closes#1305.
To allow users to forgo compatibility with Weave Net 1.0 in order to
avoid #912, by setting --protocol-min-version=2.
Defaults to --protocol-min-version=1, i.e. compatible but vulnerable.
We did this for symmetry/completeness, but the encoder is actually
"consumed" by handshake() in the creation of the tcp senders, hence it
is in fact dangerous to return it.
We now first exchange a very minimal header, consisting of just the
name of the protocol ("weave") and the min and max version (encoded as
a byte each) we support. This leaves all other details of the
protocol, including the contents of of the handshake, open.
Peers agree to communicate using the maximum mutually supported
version. We record this in the LocalConnection, though atm nothing is
making use of that information.
We set a shorter than normal deadline (10s instead of 60s) for the
header exchange, so we detect version mismatches, garbage, or
non-responsive remotes quickly.
We reset the protocol version to '1' since this is a fresh start; old
weaves will not be able to parse the header.
Closes#871.
This is how fragmentation tests used to happen:
A: conn.stackFrag = false
A->B: StartFragmentationTest
B->A: send large non-DF UDP packet
A->B: FragmentationReceived
B: conn.stackFrag = true
conn.stackFrag = true means "UDP packets sent to the other peer get
correctly fragmented and re-assembled".
Note how A clears its own stackFrag flag, but the test establishes
whether fragmentation works for B->A, and ends up setting stackFrag on
B. Not only is this weird, it also means that if A previously had
stackFrag=true, it will now start fragmenting packets until B
initiates a fragmentation test.
The new sequence is:
A: conn.stackFrag = false
A->B: send large non-DF UDP packet
B->A: FragmentationReceived
A: conn.stackFrag = true
Closes#746.
From the golang spec: 'A "break" statement terminates execution of the
innermost "for", "switch", or "select" statement within the same
function.' But the LocalConnection actorLoop seems to assume it
skips the "select" and terminates the enclosing "for".
As a result, if something did conn.Shutdown(nil), it would not actually
shutdown the connection. It turns out this is never a problem, because
Shutdown is always called with a non-nil error, even though there is code
in a few places to handle the nil case. By cleaning that up, the dubious
"break" is eliminated.
This makes for a more symmetric API - Peers.FetchWithDefault does the
increment and Peers.Dereference does the decrement, whereas previously
the latter was done by Peer.DecrementLocalRefCount.
This does increase lock granularity, i.e. modifying any ref count now
requires a write lock on Peers.
When we want a string representation of a Peer, we nearly always just
want the full name. So make String() return that and introduce Info()
to return expanded info.
Since TCP is reliable, we do not need to transmit the nonce itself;
the recipient can simply construct the nonce from its own message
counter. Hence there is no message envelope and we can thus remove a
whole layer of encoding.
It is wasteful and can trip up crypto at the other end. See comment
for why.
To accomplish this we move the responsibility for shutting down the
new, invalid connection from LocalPeer.AddConnection to its caller,
i.e. LocalConnection.run. So rather than always firing up all the
connection machinery and getting told asynchronously by AddConnection
to shut it all down again, we now only fire it up if AddConnection
succeeded.