These are needed to support vxlan encapsulation. In a vxlan packet,
the only place we can straightforwardly hold the source and
destination peer identifiers used by the weave router is in the
24-bit VNI (vxlan network identifier) field. This gives us 12 bits to
indicate a peer. The resulting limit of 4096 peers on a weave network
does not seem like a problem in the near future. But it does mean
that decentralized allocation makes collisions fairly likely even in
weave networks of sizes that are common today. Fortunately, the
problem is much more tractable than IPAM because we can detect when
collisions occur and take action to resolve them.
For fast datapath, the router core will only be invoked upon datapath
misses. And it needs to yield routing decisions in a form that can be
used to configure the in-kernel datapath. So here we make the router
core yield FlowOps, which represent operations which actually do
something with a packet. The router core obtains these in turn from
the Bridge or the Overlay, depending on what it wants to do with
a packet.
For fast datapath, it's critical that the router makes decisions based
only on the required packet header information (as the same header
information will be used when configuring the ODP flow rules). So we
pass only that information into the router core, in the form of
PacketKeys.
The overall effect is to curry the router core: Rather than taking a
packet and doing something with it, it takes packet header information
and returns a function that does something with a packet.
This introduces interfaces to represent the packet forwarding
mechanism, and brings all the scattered UDP-related code together into
a single implementation.
This is not a pure refactoring, although much of the code in sleeve.go
has been copied from pre-existing code, and the UDP protocol remains
the same in spirit. The biggest change is the organization of the
forwarder goroutines (there is now one instead of two, and that
goroutine does a lot of auxiliary work as well as forwarding).
Performance testing between VMs on my laptop suggests that packet
throughput has not gone down, and may even have improved slightly.
When we discover that a connection target address leads to ourself we
set the retry time to the zero time.Time value - the only special
time.Time value. We use that marker to a) suppress subsequent
connection attempts, and b) display the next retry as 'never' in
`weave status|report`.
Closes#1305.
To allow users to forgo compatibility with Weave Net 1.0 in order to
avoid #912, by setting --protocol-min-version=2.
Defaults to --protocol-min-version=1, i.e. compatible but vulnerable.
We did this for symmetry/completeness, but the encoder is actually
"consumed" by handshake() in the creation of the tcp senders, hence it
is in fact dangerous to return it.
We now first exchange a very minimal header, consisting of just the
name of the protocol ("weave") and the min and max version (encoded as
a byte each) we support. This leaves all other details of the
protocol, including the contents of of the handshake, open.
Peers agree to communicate using the maximum mutually supported
version. We record this in the LocalConnection, though atm nothing is
making use of that information.
We set a shorter than normal deadline (10s instead of 60s) for the
header exchange, so we detect version mismatches, garbage, or
non-responsive remotes quickly.
We reset the protocol version to '1' since this is a fresh start; old
weaves will not be able to parse the header.
Closes#871.
This is how fragmentation tests used to happen:
A: conn.stackFrag = false
A->B: StartFragmentationTest
B->A: send large non-DF UDP packet
A->B: FragmentationReceived
B: conn.stackFrag = true
conn.stackFrag = true means "UDP packets sent to the other peer get
correctly fragmented and re-assembled".
Note how A clears its own stackFrag flag, but the test establishes
whether fragmentation works for B->A, and ends up setting stackFrag on
B. Not only is this weird, it also means that if A previously had
stackFrag=true, it will now start fragmenting packets until B
initiates a fragmentation test.
The new sequence is:
A: conn.stackFrag = false
A->B: send large non-DF UDP packet
B->A: FragmentationReceived
A: conn.stackFrag = true
Closes#746.
From the golang spec: 'A "break" statement terminates execution of the
innermost "for", "switch", or "select" statement within the same
function.' But the LocalConnection actorLoop seems to assume it
skips the "select" and terminates the enclosing "for".
As a result, if something did conn.Shutdown(nil), it would not actually
shutdown the connection. It turns out this is never a problem, because
Shutdown is always called with a non-nil error, even though there is code
in a few places to handle the nil case. By cleaning that up, the dubious
"break" is eliminated.
This makes for a more symmetric API - Peers.FetchWithDefault does the
increment and Peers.Dereference does the decrement, whereas previously
the latter was done by Peer.DecrementLocalRefCount.
This does increase lock granularity, i.e. modifying any ref count now
requires a write lock on Peers.
When we want a string representation of a Peer, we nearly always just
want the full name. So make String() return that and introduce Info()
to return expanded info.
Since TCP is reliable, we do not need to transmit the nonce itself;
the recipient can simply construct the nonce from its own message
counter. Hence there is no message envelope and we can thus remove a
whole layer of encoding.
It is wasteful and can trip up crypto at the other end. See comment
for why.
To accomplish this we move the responsibility for shutting down the
new, invalid connection from LocalPeer.AddConnection to its caller,
i.e. LocalConnection.run. So rather than always firing up all the
connection machinery and getting told asynchronously by AddConnection
to shut it all down again, we now only fire it up if AddConnection
succeeded.
This was a convenient but rather suspect coupling. In particular, it
will break if the period gossip logic changes s.t. it no longer sends
gossip on all connections every interval.
We were invoking AddConnection in a separate goroutine, and this could
result in RemoveConnection getting invoked first, resulting in bogus
entries in our connection map, which would effectively prevent
connectivity to the other peer until the stale entry got removed
through a lost tie break. And if, per chance, the remote end ran into
exactly the same race condition, then connectivity between that peer
and ourself was broken permanently.
Hitting this race condition required a combination of a) no UDP
heartbeat being received for HeartbeatTimeout(30s) - this causes the
actorLoop to terminate and RemoveConnection to be invoked, and b) the
AddConnection invocation getting delayed for at least that long.
The previous scheme set the reference to main channel to nil and then
drained it. That was racy and hence could lead to deadlock in the
following scenario:
1. multiple senders obtain the main channel from the stored reference
2. main channel reference is set to nil
3. main channel is drained
4. multiple senders send to main channel
So the issue here is that the entire forwarder stopping sequence
occurs between senders obtaining of the reference to the main channel,
and then sending on that channel. If more senders sent than there is
capacity in the channel then this leads to deadlock.
David Wragg