The previous scheme set the reference to main channel to nil and then
drained it. That was racy and hence could lead to deadlock in the
following scenario:
1. multiple senders obtain the main channel from the stored reference
2. main channel reference is set to nil
3. main channel is drained
4. multiple senders send to main channel
So the issue here is that the entire forwarder stopping sequence
occurs between senders obtaining of the reference to the main channel,
and then sending on that channel. If more senders sent than there is
capacity in the channel then this leads to deadlock.
The crypto layer doesn't need it, and the cost of an additional
closure allocation for each incoming UDP packet is modest.
Also, make the FrameConsumer the last parameter to IterateFrames.
It seems a more natural ordering.
Treat the no-error case the same as the error case...
- use the standard sendAction to communicate with the actor, which
ensures that we don't block when the actor has terminated
- run in a goroutine to prevent delaying the caller when the actor's
channel is backed up.
The only slight trickiness is Shutdown. Generally connections shut
down when an error is encountered, but we also need to be able to shut
down connections w/o an error. Rather than making all the actor
methods return a (terminated bool, error) tuple, we signal the
non-error shutdown by sending nil as the action (instead of a
closure).
Introduce an intermediary - GossipSender - between GossipChannel and
Connection. This accumulates gossip data (now represented by the new
GossipData interface) from the former until it can be passed onto the
latter, allowing the former to proceed when the latter may be blocked
on i/o.
This
1) prevents deadlocks that arise from cycles in the communication
topology
2) improves performance by allowing GossipChannel and its calling code
to proceed when connection i/o is blocked
3) improves performance by accumulating GossipData - the accumulated
data often is considerably more compact than the sum of all the
accumulated bits, and it is transmitted in a single communication
event.
In order to support accumulation, the GossipData interface has a Merge
method. Furthermore, encoding is deferred until the data can actually
be sent, since accumulation would be hard/impossible to encoded
data. This required changes on some interfaces, most notably Gossiper.
The implementation of GossipData for topology gossip is
TopologyGossipData. It carries a reference to Peers and a set of
PeerNames, indexing into Peers and referencing the Peer entries which
have changed. Previously updates were represented as a list of Peer
entries. The indirection via PeerNames allows Encode to omit entries
which have been removed. The change in representation required changes
to the signature of some methods on Peers.
Note that all the above only applies to the "periodic gossip" portion
of the Gossiper API, which topology gossip in fact uses for *all*
gossip. GossipUnicast and GossipBroadcast are unchanged, but could
conceivably receive the same treatment in the future.
Fixes#445.
Weave sends all gossip down new connections. We do this so that new
peers learn about all gossip'ed information quickly, thus enabling
them to process subsequent incremental updates. In particular, for
topology gossip it ensures that the remote peer learns about all other
peers. This ensures that the incremental update, which is sent as a
result of the new connection getting established and which may
reference a number of other peers, can be processed.
So the intent really is to send all gossip to new peers. But what we
actually do is send it down every new connection. Not every new
connection is from a new peer. In fact most connections aren't. Hence
the current approach is wasteful.
To fix this we need a conservative detection of new peers, i.e. it is
preferable to have false positives. This contrasts with existing logic
in the handshake code. The criteria we use is whether our peer has a
unicast route to the other peer. That is only the case when there is a
continuos chain of symmetric, established connections. The
establishment of these connections should have triggered the sending
of all gossip to the other peer. So we don't need to send it ourself.
Fixes#454.
Matthias Radestock