Compare commits

...
1082 Commits
Author SHA1 Message Date
Adam Harrison af3fc82b77 Add release v1.3.0 2015-11-12 16:49:33 +00:00
Adam Harrison 3bb7036cfd Runner depends on tools 2015-11-12 16:48:56 +00:00
Bryan Boreham d802eb485b Merge pull request #1664 from weaveworks/weave-launch-plugin
Add plugin support to weave script. LGTM.
2015-11-12 16:04:41 +00:00
Adam Harrison b4dd9c2490 Add plugin to features doc 2015-11-12 16:01:01 +00:00
Adam Harrison 04ab198e95 Set restart policy to 'always'
Docker attempts to recreate networks on startup which means the plugin
container has to be started. If it isn't started automatically by Docker
with a restart policy, it's not possible to start it manually because
Docker is blocked.
2015-11-12 14:58:54 +00:00
Adam Harrison c345700963 Add plugin to weave status 2015-11-12 14:58:54 +00:00
Adam Harrison b21afcb78f Implement weave launch-plugin and weave stop-plugin 2015-11-12 14:56:13 +00:00
Adam Harrison 7f65530b26 weave setup fetches plugin 2015-11-12 14:56:13 +00:00
Bryan Boreham b1b03f3a97 Revert "Remove code coverage as temporary work-around to Docker 1.9 issue"
This reverts commit 65fe753a58.
2015-11-12 11:27:48 +00:00
Bryan Boreham 36ef6d8da2 Merge remote-tracking branch 'origin/1.2' 2015-11-12 11:27:01 +00:00
Bryan Boreham 28bb0dd3b0 Merge pull request #1671 from weaveworks/1666-remove-cont-dir
Remove containers directory bind-mount from proxy; fixes #1647 and #1666
2015-11-12 11:25:34 +00:00
Bryan Boreham 79f8a8f909 Merge remote-tracking branch 'origin/1.2' 2015-11-12 10:43:01 +00:00
Bryan Boreham bfe2513540 Merge pull request #1669 from dpw/fdp-fix-npd
Handle the case where Forward returns nil. Fixes #1661.
2015-11-12 10:38:29 +00:00
Bryan Boreham 2536aef737 Proxy doesn't need to mount the containers dir 2015-11-12 10:28:58 +00:00
Adam Harrison 0661ccdea9 Revert "Revert "Disable restart of docker""
This reverts commit 76db732f1b.
2015-11-11 10:10:17 +00:00
Adam Harrison 29f7015e58 Merge pull request #1665 from weaveworks/move-odp-helpers
Move ODP helper functions out to their own package
2015-11-11 09:56:24 +00:00
David Wragg 7dc1b50d3b Handle the case where Forward returns nil
Forward can return nil if the remote peer has HasShortID false.  That
can happen if we learn about a 1.2 peer from a 1.1  peer and attempt to
connect to it.

This change just fixes the panic.  HasShortID will never be set, so we'll
fall back to sleeve.  But if that occurs, if can be fixed by a restart of
the affected routers.
2015-11-10 23:17:01 +00:00
Bryan Boreham 682133a56a Move ODP helper functions out to their own package 2015-11-10 17:09:17 +00:00
Adam Harrison 76db732f1b Revert "Disable restart of docker"
This reverts commit a400a3dd09.
2015-11-10 10:24:58 +00:00
Matthias Radestock 64aa2b5cc3 refactor: remove weave overlay-isms from mesh 2015-11-09 20:56:04 +00:00
Matthias Radestock 1eed5fd271 cosmetic: don't mention UDP in mesh 2015-11-09 19:47:27 +00:00
Matthias Radestock dd7ffa4501 code docs: warn against nonce re-use by overlay connection 2015-11-09 19:46:53 +00:00
Adam Harrison 5e79da0c9c Merge pull request #1658 from weaveworks/tweak-circle
Upgrade CI to Docker 1.9 and work round issues
2015-11-09 17:55:27 +00:00
Bryan Boreham 65fe753a58 Remove code coverage as temporary work-around to Docker 1.9 issue 2015-11-09 17:18:59 +00:00
Bryan Boreham 17c835e672 Compress weave.tar to save upload time 2015-11-09 17:18:58 +00:00
Bryan Boreham fa5bcd0457 Lump some commands together to reduce CircleCI waiting overhead 2015-11-09 17:18:58 +00:00
Bryan Boreham 499d6312b7 Don't need --add-host except in non-FDP case because we put the
mappings in the outer /etc/hosts
2015-11-09 17:18:58 +00:00
Bryan Boreham 20d5903696 Update test template name to pull in new Docker version 2015-11-09 17:18:57 +00:00
Adam Harrison f039221ec9 Merge remote-tracking branch 'origin/create-mesh'
Closes #1659.
2015-11-09 17:16:35 +00:00
Matthias Radestock dc9f8762ca move generic communication substrate into its own 'mesh' package
This is mostly a matter of moving some files, changing package names,
changing some imports, and changing some type references.

We duplicate the odd constant & global var, and the intmac/macint
functions, since neither 'router' or 'mesh' nor, really,' common' are
a natural home for them.
2015-11-09 16:37:39 +00:00
Adam Harrison e5c69801a7 Merge remote-tracking branch 'origin/separate-overlay'
Closes #1648.
2015-11-09 16:17:41 +00:00
Matthias Radestock ab458368de refactor: move NetworkRouterStatus into its own file 2015-11-09 15:51:11 +00:00
Matthias Radestock 8eb680fcc9 refactor: separate out NetworkRouter-specific status 2015-11-09 15:51:11 +00:00
Matthias Radestock 71b05972f4 refactor: move NetworkRouter into its own file 2015-11-09 15:50:55 +00:00
Matthias Radestock 7deeb8dba5 refactor: separate NetworkRouter from Router 2015-11-09 15:47:08 +00:00
Matthias Radestock daad190188 refactor: split overlay.go to extract network overlay stuff 2015-11-09 15:47:08 +00:00
Matthias Radestock 7cc0052451 refactor: separate NullNetworkOverlay from NullOverlay 2015-11-09 15:47:07 +00:00
Matthias Radestock 18f8e29af8 refactor: separate NetworkOverlay from Overlay
...since connections only need to know about the latter.
2015-11-09 15:47:07 +00:00
Matthias Radestock d78f6614cc refactor: separate forwarder from overlay connection
...since the main connections only need to know about the former.

We make LocalConnection.Overlay (formerly '.forwarder') public since
the router needs access to it, and it shouldn't be doing that via a
private field.
2015-11-09 15:45:06 +00:00
Bryan Boreham 53ac43202a Merge pull request #1641 from weaveworks/1554-converge-on-peer-uid-change
better convergence when peers restart; fixes #1554.
2015-11-09 15:09:59 +00:00
Matthias Radestock bfb050c9cc better convergence when peers restart
Previously when a peer restarted, information about the new
incarnation (i.e. with a different UID) was not accepted by other
peers (and connections would be dropped) unless all knowledge of the
previous incarnation had been purged. This could result in a lot of
connection churn and hence connectivity disruption, and, in some
pathological cases, very slow convergence and hence acceptance of the
new incarnation into the network.

We now no longer drop connections when encountering different
incarnations of a peer. There are two situations when that can happen:

1) on connection establishment

we simply proceed

2) on receipt of gossip

to ensure convergence we

a) treat the UID as an additional discriminator when deciding whether
we should update our information about a peer with that which was
gossiped. Specifically, we update the information we hold when a) the
gossiped version is greater, or b) is the same and the UID is
greater.

b) include the UID in the information we update

c) move our own version number beyond any we receive for ourselves, if
the received UID differs from ours.

With (a) we establishes a total order of peer information across
several incarnations of the same peer. i.e. we consider information to
be fresher if it has a higher version, or the same version and higher
UID. This may seem somewhat counter intutive, since it will generally
treat information about new incarnations as older than old
incarnations, since incarnations always start life with version 1. But
to do better we'd need to establish a total order of incarnations that
matches their temporal occurrence. Which requires some sort of durable
state.

So instead we have (c). Through that we learn the highest version
number of any old incarnation of ourselves that other peers still
hold, and then make sure that our version is greater than
that. Essentially we continue where the old incarnations left
off. It's as if instead of restarting we had simply changed UIDs. And
due to (a) and (b) the information about the new incarnation of
ourselves, now with a higher version, will supersede that of the old
incarnations.

Fixes #1554.
2015-11-09 14:58:33 +00:00
Matthias Radestock 97622b961c remove cruft 2015-11-08 13:06:29 +00:00
Matthias Radestock b7e7c11a57 refactor: make Overlay setting more explicit 2015-11-08 12:29:36 +00:00
Matthias Radestock 59e11e4e8a refactor: more flexible Routes API 2015-11-08 11:15:31 +00:00
Matthias Radestock 39b0357517 Merge branch '1.2' 2015-11-08 11:04:48 +00:00
Matthias Radestock 878a12002c set http listen address solely through the weave script
Previously it was set by the script only in fastdp mode; and in the
router's main.go otherwise. Yet every interaction of the script with
weave assumes the port is the script's HTTP_PORT=6784. By always
starting the router with options to listen on that port we ensure that
the router is in fact listening where the script expects it to.

In doing this get rid of router.HTTPPort. We do need to hard-code the
port in the proxy instead, but relying on router.HTTPPort was flawed
anyway, for the above reasons.
2015-11-07 18:01:12 +00:00
Matthias Radestock d3d2863d52 refactor: get rid of const.go 2015-11-07 16:32:23 +00:00
Matthias Radestock f3f23b43b5 refactor: crypto doesn't need to be a generic overlay feature 2015-11-07 16:14:31 +00:00
Matthias Radestock 006262ed8f refactor: Connections don't need to know about overlay crypto 2015-11-07 16:10:00 +00:00
Matthias Radestock 519aef86fa refactor: split crypto code 2015-11-07 15:59:15 +00:00
Matthias Radestock 8bd77c0d47 refactor: get rid of router/utils.go
most functions end up in other files in the router package, but a few
land in common.
2015-11-07 15:45:29 +00:00
Matthias Radestock 7a5c910cd5 tiny refactor: inline a tiny single-use fun and move another 2015-11-07 12:44:13 +00:00
Matthias Radestock 4e34f2001c cosmetic 2015-11-07 11:14:11 +00:00
Matthias Radestock 54b7e2c874 tiny refactor: use checkWarn in a few more places 2015-11-07 11:08:44 +00:00
Matthias Radestock 19f89f7122 remove duplicate test
this is already covered in dns_fallback_test
2015-11-07 07:30:53 +00:00
Matthias Radestock 9f25d72a4a cosmetic 2015-11-07 05:43:20 +00:00
Adam Harrison 3e99b71e0f Merge pull request #1643 from weaveworks/refactor-forwarder-remote-address
Refactor: remove knowledge of overlay connectivity details from Local{Peer,Connection}
2015-11-06 16:02:10 +00:00
Matthias Radestock fb3b690c5f refactor: reduce coupling between LocalConnection and Forwarder 2015-11-05 21:27:14 +00:00
Matthias Radestock 18fe553d23 LocalConnection shouldn't have to know about remote UDP address
...and generally know less about how overlays establish connectivity.
2015-11-05 20:43:52 +00:00
Matthias Radestock e7ff6a2191 LocalPeer shouldn't have to know about remote UDP address
...and LocalConnection shouldn't have to remember it
2015-11-05 19:32:49 +00:00
Matthias Radestock ad15e5a234 cosmetic 2015-11-05 19:01:34 +00:00
Matthias Radestock 094c7aa438 cosmetic 2015-11-05 17:24:38 +00:00
Adam Harrison 472ed5a7d3 Merge pull request #1640 from weaveworks/1588-disable-docker-restart
Disable restart of docker in test 640
2015-11-05 15:40:01 +00:00
Adam Harrison f284300870 Merge pull request #1636 from weaveworks/1635-notation
Drop use of interval notation when printing ranges
2015-11-05 15:38:40 +00:00
Adam Harrison ab5f4410b6 Add release v1.2.1 2015-11-05 14:34:13 +00:00
Bryan Boreham a400a3dd09 Disable restart of docker 2015-11-05 14:20:38 +00:00
Bryan Boreham c4855e8123 Merge branch '1.2' 2015-11-05 12:41:39 +00:00
Adam Harrison 3abb657777 Document port security 2015-11-05 12:23:17 +00:00
Adam Harrison 4dd63d8f72 Bind the HTTP listener to localhost when fastdp enabled
Prevent containers from accessing the weave router control port when
fast datapath is in use.
2015-11-05 12:23:17 +00:00
Bryan Boreham 6c0bdec2c9 Merge pull request #1637 from weaveworks/issues/1632-http-localhost-bind
Bind router HTTP listener to localhost when fastdp enabled
LGTM.  Fixes #1632
2015-11-05 12:20:20 +00:00
Adam Harrison d6dd1f269a Document port security 2015-11-05 12:16:57 +00:00
Bryan Boreham 2bd23f3709 Merge branch '1.2' to pull in latest_release_doc_updates 2015-11-05 12:05:25 +00:00
Bryan Boreham 79dc464b83 Merge branch 'latest_release_doc_updates' into 1.2 2015-11-05 12:04:54 +00:00
Bryan Boreham 3dbd5b11d8 Segregate discussion of range and subnet, and explain subnet a bit 2015-11-05 11:32:06 +00:00
Bryan Boreham 549fd5fe29 Merge branch '1.2' 2015-11-05 11:30:53 +00:00
Bryan Boreham 0ab5c65185 Merge pull request #1630 from weaveworks/issues/1588-improve-proxy-restart-reattach-test
Improve proxy restart reattach test reliability
2015-11-05 11:25:31 +00:00
Bryan Boreham 5f4753f881 Segregate discussion of range and subnet, and explain subnet a bit 2015-11-05 10:13:06 +00:00
Bryan Boreham e03a45f8c5 Update docs to match loss of interval notation 2015-11-05 10:11:43 +00:00
Adam Harrison 67c2d53e59 Bind the HTTP listener to localhost when fastdp enabled
Prevent containers from accessing the weave router control port when
fast datapath is in use.
2015-11-04 17:42:24 +00:00
Bryan Boreham 7b01dd9794 Drop use of interval notation 2015-11-04 16:45:16 +00:00
Bryan Boreham 04e50ff02c Merge branch '1.2' 2015-11-04 16:40:30 +00:00
Bryan Boreham e57ccdc858 Merge branch 'latest_release_doc_updates' into 1.2 2015-11-04 16:39:43 +00:00
Bryan Boreham a73a65b48a Removed obsolete text referring to mixed auto-manual allocation, and clarify Range printout 2015-11-04 16:36:44 +00:00
Adam Harrison 781c805e10 Merge branch '1.2' 2015-11-04 15:32:17 +00:00
Adam Harrison 7c4af88911 Merge pull request #1606 from weaveworks/issues/1578_ufw_ubuntu
Add an iptables rule for letting weaveDNS requests in
2015-11-04 15:30:34 +00:00
Alvaro Saurin 0aca173e96 Add an iptables rule for letting weaveDNS requests in 2015-11-04 14:00:26 +01:00
Matthias Radestock 60db57a41c refactor: remove duplicate isKnownPeer function 2015-11-03 18:11:53 +00:00
Matthias Radestock df68f7dcb9 Merge branch '1.2' 2015-11-03 17:56:36 +00:00
Matthias Radestock e95e3cc29d Merge pull request #1624 from weaveworks/1593-more-peers
Better choice of peers for IPAM to transfer ownership to on `weave reset`

Fixes #1593.
2015-11-03 17:55:40 +00:00
Bryan Boreham 67e9310cc1 Review feedback 2015-11-03 17:01:27 +00:00
Adam Harrison 94984df047 Merge branch '1.2' 2015-11-03 16:57:15 +00:00
Adam Harrison 861bf953bf Merge pull request #1629 from weaveworks/1626-expose-names
Always add 'weave expose -h' argument to DNS
2015-11-03 16:32:13 +00:00
Adam Harrison 06eaac8690 Don't assert ethwe carrier via the proxy
The test script runs commands inside containers via docker exec to
ensure the ethwe carrier is up. We must not run these commands via the
proxy because it will interpose weavewait, which will block forever if
attachment has failed (e.g. due to #1556).
2015-11-03 15:42:37 +00:00
Adam Harrison 6b9e8816d1 Wait for proxy to become available after docker is restarted 2015-11-03 15:33:37 +00:00
Bryan Boreham cf9f715a28 Merge branch '1.2' 2015-11-03 15:31:51 +00:00
Bryan Boreham 08e043fea7 Always add 'weave expose -h' argument to DNS, and add smoke-test for this also 2015-11-03 15:23:28 +00:00
Bryan Boreham 54ba1009d3 Merge pull request #1608 from weaveworks/1600-start-networkmode
LGTM.  Fixes #1600
2015-11-03 15:14:15 +00:00
Adam Harrison 4a6a46fd7b Merge pull request #1627 from tomwilkie/mv-scheduler
Move scheduler to build-tools.
2015-11-03 14:44:52 +00:00
Adam Harrison 0eb7414f72 Merge pull request #1583 from weaveworks/move-runner-to-tools
Remove test runner; it now lives in build-tools.git
2015-11-03 14:44:46 +00:00
Tom Wilkie c02691a70c Move scheduler to build-tools. 2015-11-03 14:38:42 +00:00
Bryan Boreham 8742957f75 Tweak naming to be closer to code changes in nameserver 2015-11-03 14:15:49 +00:00
Bryan Boreham ace372cfc6 More extensive choice of transferee 2015-11-03 12:03:28 +00:00
Bryan Boreham 27c5f5f49f Fix title of test 2015-11-03 12:03:00 +00:00
Paul Bellamy ec5a1a3977 When container is started in host (or container) networkMode, we mount a noop weavewait, so it can continue
review feedback
2015-11-03 10:18:05 +00:00
Adam Harrison abc653c87d Fix merge breakage 2015-11-03 10:13:43 +00:00
Matthias Radestock 30a128c00b Merge branch '1.2' 2015-11-02 20:58:15 +00:00
Matthias Radestock 77c7963bde Merge pull request #1615 from weaveworks/1603-resort-in-encode
Hold DNS entries sorted case-insensitively until just before sending

Fixes #1603 and #1610.
2015-11-02 20:57:40 +00:00
Bryan Boreham b80c1343c3 Take steps to avoid the test timing out 2015-11-02 20:40:17 +00:00
Bryan Boreham e5c5fbcea3 Hold on to the lower-case version of a hostname, rather than re-computing it every time 2015-11-02 20:40:05 +00:00
Bryan Boreham fc7d04ce73 Condense Entries unit tests with helper function 2015-11-02 20:03:10 +00:00
Bryan Boreham 35f57b3ed2 Move decoding code next to encoding code 2015-11-02 20:01:40 +00:00
Bryan Boreham ed9ade05ea Improve merge asserts 2015-11-02 20:01:40 +00:00
Bryan Boreham f485b729bc Hold all entries sorted case-insensitively until just before sending, when we re-sort case-sensitively in Encode() 2015-11-02 20:01:40 +00:00
Bryan Boreham 64f7953423 Add unit test to check outbound mixed-case merging 2015-11-02 20:01:40 +00:00
Bryan Boreham fc1d837501 Extend unit test to cover mixed-case hostnames and daisy-chained gossip;
reduce mock gossip timer to 2 seconds to trigger more often during tests.
2015-11-02 20:01:40 +00:00
Matthias Radestock 6a238818da Merge branch '1.2' 2015-11-02 19:58:37 +00:00
Matthias Radestock 9800c43bfa refactor(ish): don't use reference in method that doesn't modify 2015-11-02 19:57:17 +00:00
Bryan Boreham b3cda39b0b Let IPAM pick any peer we know about to transfer ownership to, when shutting down 2015-11-02 18:34:58 +00:00
Adam Harrison 7e1b7a2fcd Merge branch '1.2' 2015-11-02 17:28:31 +00:00
Adam Harrison ccae8d42fd Merge remote-tracking branch 'dpw/fdp-probe-vxlan' into 1.2
Fixes #1599
2015-11-02 17:24:52 +00:00
David Wragg 38f78286c7 Probe whether the kernel has ODP vxlan support
It's possible to build that kernel with openvswitch, and vxlan, but
without the vport_vxlan module that glues them together.  And at
least one distro has actually done this.
2015-11-02 15:53:16 +00:00
Adam Harrison c3a53a6ec5 Merge remote-tracking branch 'upstream/1.2' 2015-11-02 15:21:06 +00:00
Bryan Boreham 2bbe6a2ad8 Merge pull request #1622 from weaveworks/fix-miekg-dns-truncation-test
LGTM
2015-11-02 15:09:57 +00:00
Adam Harrison f64f63f675 Handle truncated responses from upstream servers 2015-11-02 15:00:02 +00:00
Adam Harrison f0fb44f6d4 Update unit test for miekg DNS API change 2015-11-02 13:24:04 +00:00
Paul Bellamy fa972615e6 Merge pull request #1616 from weaveworks/1549-inspect-wait
Make inspect-rewrite wait for start
2015-11-02 11:00:23 +00:00
Bryan Boreham f7cb2ef25d have waitForStartByIdent() return an error 2015-10-30 13:46:30 +00:00
Bryan Boreham 949f8bb2e7 Move updateContainerNetworkSettings() into proxy 2015-10-30 13:34:16 +00:00
Bryan Boreham 5eaff802d4 Make inspect-rewrite wait if the start operation is still in progress 2015-10-30 13:34:16 +00:00
Matthias Radestock b37d428726 Merge branch '1.2' 2015-10-30 01:31:16 +00:00
Matthias Radestock a6336149f8 guard against chef not running
Provisioning on ubuntu/wily64 was tripping over this.
2015-10-30 01:29:21 +00:00
Matthias Radestock 12f5ec40b5 refactor: reduce coupling of nameserver to router
The nameserver no longer knows anything about router.Peers and
router.Peer. That only leaves PeerName, Gossip and GossipData.
2015-10-29 23:21:07 +00:00
Matthias Radestock 6a672cf8bc tiny refactor: more idiomatic referal to process' /proc 2015-10-28 18:23:15 +00:00
Matthias Radestock 036b7d9f66 Merge branch '1.2' 2015-10-28 09:40:50 +00:00
Matthias Radestock ca6bce3820 Merge branch 'latest_release_doc_updates' into 1.2 2015-10-28 09:20:46 +00:00
Matthias Radestock 9b4f2297b0 drop quotes in eval "$(weave env)"
They are unnecessary and unwieldy.

Fixes #1596
2015-10-28 09:19:10 +00:00
Matthias Radestock 355c4bc58e Merge branch '1.2' 2015-10-28 08:52:01 +00:00
Matthias Radestock a31b2876d0 Merge pull request #1598 from weaveworks/issues/1584-improved-proxy-router-restart
Improved detection of router container restart

Fixes #1584
2015-10-28 08:51:06 +00:00
Matthias Radestock b697b6e57b Merge pull request #1597 from weaveworks/issues/1516-weavedns-ignore-foreign-domains
Check weaveDNS subdomain on REST PUT

Fixes #1516
2015-10-28 08:50:25 +00:00
Adam Harrison 4792fe6391 Improved detection of router container restart
Use the container name in addition to the entrypoint to avoid triggering
reattachment unnecessarily.
2015-10-27 14:41:26 +00:00
Adam Harrison 7408f72de2 Check weaveDNS subdomain on REST PUT 2015-10-27 14:37:57 +00:00
Tom Wilkie 203a3cfdd1 Remove test runner; it now live in build-tools.git 2015-10-23 13:02:00 +00:00
Ilya Dmitrichenko 3a029686e3 Add Pardot script 2015-10-22 15:07:52 -07:00
Matthias Radestock 9fa7b3956d Merge remote-tracking branch 'upstream/master' 2015-10-22 16:17:55 +01:00
Matthias Radestock b888ace508 remove superfluous code in ApplyUpdate
If the local peer's short ID was updated, we broadcastPeerUpdate at
the end (in unlockAndNotify), so there is no need to include the local
peer in the current update.

Also, reading peers.ourself.Version is bad here - we don't take a lock
and it's written by other goroutines. Doesn't actually break anything
though since it's a uint64 and hence atomic.
2015-10-22 16:12:37 +01:00
Bryan Boreham 2e54efe34d Stop the proxy last, so we can stop the router
before being unable to communicate with docker
when $DOCKER_HOST is pointing at the proxy
2015-10-22 15:45:54 +01:00
Bryan Boreham 94d7f33a03 Merge pull request #1541 from weaveworks/weave-1.2-readme-updates
Update README for 1.2 release
2015-10-22 13:22:54 +01:00
Bryan Boreham 11598295cf Add release v1.2.0 2015-10-22 12:53:57 +01:00
Bryan Boreham 383c3769aa Try en even bigger sleep 2015-10-22 12:19:57 +01:00
Adam Harrison caccba18e9 Mention fast datapath port 2015-10-22 11:58:14 +01:00
Paul Bellamy 95ce00e551 moving boot2docker instructions into the readme 2015-10-22 11:58:14 +01:00
David Wragg 85dad58cfe Pass --privileged for the temporary container in rewrite_etc_hosts
Otherwise we run into selinux issues on fedora23 (and probably other
selinux distros).
2015-10-22 11:44:17 +01:00
Bryan Boreham acdc70f72a Merge pull request #1560 from weaveworks/1542-proxy-bug-weave-not-running
LGTM. Fixes #1542
2015-10-22 10:19:32 +01:00
Paul Bellamy bc9db72212 [proxy] Add notifyWaiters to fix anticipated race condition.
* User makes /start request, and blocks on the waiter
* ContainerStarted fires with ID
* Container goes away
* We try to inspect the container to get the entrypoint, which fails
* Previously, this wouldn't call notifyWaiters and the /start would hang (I expect)
2015-10-22 09:50:12 +01:00
Adam Harrison 7905f81d7e Merge pull request #1570 from dpw/mac-invalidate-routes
Do InvalidateRoutes when a forwarded packet updates a MAC cache entry
2015-10-21 23:54:56 +01:00
Matthias Radestock 2ef9cc6a1c start temporary containers without networking when possible
This is more efficient and TRTTD

Closes #1572.
2015-10-21 16:52:19 +01:00
Matthias Radestock 2357b69a5f nicer interface name
and more likely to be ignored by NetworkManager

Closes #1567.
2015-10-21 16:22:16 +01:00
Matthias Radestock 98b47f866b Merge pull request #1569 from weaveworks/issues/1565-docker-1.9.0-workaround
Reorder inspect template to work around docker/docker#17026

Fixes #1565.
2015-10-21 15:29:59 +01:00
David Wragg d2dfb0d15d Do InvalidateRoutes when a forwarded packet updates a MAC cache entry
When we get a forwarded packet, we put its source MAC into the MAC cache,
overriding any existing entry.  But we could have a flow which says
to forward packets destined to that MAC based on the old entry.  Such
a flow could be prolonged indefinitely by attempts to send to the MAC.
This is obviously bad, so do InvalidateRoutes in such cases.
2015-10-21 13:46:18 +01:00
Paul Bellamy 63339c026d [proxy] return errors when attaching new containers
I had implemented this with a promise, but was informed that as
waitForStart is only called once, we are safe to just put the error into
the channel.
2015-10-21 13:05:05 +01:00
Adam Harrison 40922b88f7 Reorder inspect template to work around docker/docker#17026 2015-10-21 12:39:55 +01:00
Bryan Boreham 674c617c16 Merge branch '1.1' 2015-10-21 11:52:49 +01:00
Bryan Boreham 2884b6deca Add release v1.1.2 2015-10-20 17:06:14 +01:00
Adam Harrison 720862dafb Merge pull request #1562 from dpw/fdp-create-bridge
'weave --local create-bridge' is not compatible with fastdp
2015-10-20 16:37:20 +01:00
David Wragg 5120b4b468 'weave --local create-bridge' is not compatible with fastdp 2015-10-20 16:34:50 +01:00
Adam Harrison c5f5d4543e Merge pull request #1518 from weaveworks/401-reattach-weave2
Get proxy to re-attach weave router if it restarts
2015-10-20 15:31:13 +01:00
Paul Bellamy baf92da9f2 Merge pull request #1534 from weaveworks/1531-check-hostconfig
Take more care over HostConfig on start request in proxy
2015-10-20 12:18:49 +01:00
Paul Bellamy 526c9c3d73 Add test for {"HostConfig":{}} overriding previous settings, but still connecting to weave 2015-10-20 11:26:29 +01:00
Bryan Boreham 119d87b795 Extract repeated test into function 2015-10-20 11:26:29 +01:00
Bryan Boreham 17b030cca1 Cope with client sending a HostConfig named HostConfig 2015-10-20 11:26:29 +01:00
Bryan Boreham 81c9b3f327 Check that HostConfig parameters supplied on /start are passed to Docker,
also what happens when you supply a HostConfig named HostConfig
2015-10-20 11:26:29 +01:00
Bryan Boreham 50198a2159 Don't create a HostConfig accidentally if none was passed in 2015-10-20 11:26:29 +01:00
Bryan Boreham 20c8b1eaf6 Reverse the sense of a test, to simplify the next change
and remove unused struct
2015-10-20 11:26:21 +01:00
Bryan Boreham faf94b033d Extend test to check we don't overwrite HostConfig 2015-10-20 11:25:06 +01:00
Adam Harrison 342b3d3d08 Merge pull request #1557 from weaveworks/issues/1497
Add container FQDN to /etc/hosts
2015-10-20 11:22:22 +01:00
Bryan Boreham 26d43f193e Move docker restart test above the place where we stop part of weave 2015-10-20 07:07:04 +00:00
Alvaro Saurin f04ae91c82 Add container FQDN to /etc/hosts 2015-10-19 21:51:38 +02:00
Bryan Boreham bb5515113f Make attach-router work in fastdp+encryption case 2015-10-19 15:49:07 +00:00
Bryan Boreham c09d84efc6 Move setup_router_iface call so we don't kill an already-running router on attach-router 2015-10-19 15:49:07 +00:00
Bryan Boreham 2a99a2a597 Don't fail if the arg we're looking for isn't there 2015-10-19 15:47:44 +00:00
Bryan Boreham 08f6dcf08d Missing format 2015-10-19 15:47:44 +00:00
Bryan Boreham 284d78c6b2 Update fetch_router_args in line with changes elsewhere 2015-10-19 15:47:44 +00:00
Bryan Boreham b8b5fb9eae Review feedback 2015-10-19 15:47:43 +00:00
Bryan Boreham 697ee62d9a Use entrypoint to detect weave router 2015-10-19 15:47:43 +00:00
Bryan Boreham 20ac0edda8 Try a bigger sleep to get Docker restarted 2015-10-19 15:46:48 +00:00
Bryan Boreham a1bbb06e2f Don't need depreciation warnings on internal-only command 2015-10-19 15:46:47 +00:00
Bryan Boreham a87cc5c3b1 Get the proxy to re-attach the weave router if it sees it restart 2015-10-19 15:46:47 +00:00
Matthias Radestock 5ff740cddc cosmetic 2015-10-19 15:43:49 +01:00
Matthias Radestock db9229f485 Merge pull request #1551 from weaveworks/1547-proxy-restart-no-kill-containers
Proxy restart shouldn't kill containers

Fixes #1547.
2015-10-19 13:55:48 +01:00
Paul Bellamy ace6c01b00 Review feedback 2015-10-19 13:53:20 +01:00
Bryan Boreham 2f4d460c04 Merge pull request #1517 from weaveworks/1385-non-container-dns-add
LGTM. Fixes #1385
2015-10-19 12:52:09 +01:00
Matthias Radestock d5401ae3cb Merge pull request #1550 from dpw/weave2-eexist
Don't fail in launch-router if weave2 already exists

Fixes #1548.
2015-10-16 21:36:45 +01:00
David Wragg 3482a7f1f1 Don't fail in launch-router if weave2 already exists
And delete weave2 on stop-router.
2015-10-16 20:59:06 +01:00
David Wragg 23762e0639 Move repeated 'stop' code into functions 2015-10-16 20:55:44 +01:00
Bryan Boreham 50cc15af3b Merge pull request #1477 from weaveworks/weave-1374
LGTM.  Fixes #1374
2015-10-16 18:02:16 +01:00
Paul Bellamy a0f0367e60 [proxy] don't kill containers when starting the proxy 2015-10-16 15:33:43 +01:00
Adam Harrison 3f08ddefe9 Correct DisplayName implementation 2015-10-16 12:24:38 +01:00
Adam Harrison debe656f9f Merge pull request #1522 from dpw/fdp-compat
Fast datapath compatibility with 1.1
2015-10-16 11:26:20 +01:00
Alvaro Saurin 4350b6402f Updated docs 2015-10-16 11:47:07 +02:00
Alvaro Saurin 5d2892f6fc Test we respect --add-host when rewriting /etc/hosts
Simplify some tests
2015-10-16 11:47:07 +02:00
Alvaro Saurin 91f27ddd66 Removed weavehosts 2015-10-16 11:47:07 +02:00
Alvaro Saurin 6a674d0d67 Add hosts when attaching 2015-10-16 11:47:07 +02:00
Alvaro Saurin 0d165487b8 Rewrite /etc/hosts in the containers 2015-10-16 11:47:07 +02:00
Matthias Radestock bacc5836ea log listeners 2015-10-16 09:34:20 +01:00
Matthias Radestock 230931a2c9 prevent accidental var shadowing
Fixes #1544
2015-10-16 01:05:04 +01:00
Matthias Radestock 61d289f414 Merge branch '1.1' 2015-10-15 18:14:20 +01:00
Matthias Radestock 44255862a9 prevent spurious errors from showing up
silly mistake
2015-10-15 18:12:56 +01:00
Matthias Radestock ed9577150b suppress spurious errors 2015-10-15 18:01:06 +01:00
Matthias Radestock 80319e7087 Merge branch '1.1' 2015-10-15 17:41:49 +01:00
Matthias Radestock 379048d8fc make test/setup.sh work under go 1.5
Fixes #1543.
2015-10-15 17:40:51 +01:00
Matthias Radestock 9ac655d774 Merge pull request #1526 from weaveworks/fdp-docs
Document fast data path feature

Fixes #1526.
2015-10-15 17:29:29 +01:00
Adam Harrison 9b17470129 Document fast data path feature 2015-10-15 17:19:13 +01:00
Matthias Radestock 5c114fc6fe Merge pull request #1536 from weaveworks/1515-print-json
Print request and response json when debugging weaveproxy

Fixes #1515
2015-10-15 15:59:35 +01:00
Bryan Boreham efb12f41b2 Also print intercepted responses 2015-10-15 15:07:02 +01:00
Adam Harrison 813edcfe0a Merge branch '1.1' 2015-10-15 14:59:29 +01:00
Adam Harrison dea1405d71 Remove hardcoded docker bridge IP from proxy test 2015-10-15 14:39:56 +01:00
Ilya Dmitrichenko e7f91b7529 Update screencast preview image 2015-10-15 14:26:39 +01:00
Matthias Radestock 5e8a64508e Merge remote-tracking branch 'upstream/master' 2015-10-15 14:19:50 +01:00
Matthias Radestock 21ebb7a585 Merge branch '1.1'
Conflicts:
	Makefile
2015-10-15 14:19:14 +01:00
David Wragg 6edd430538 Use consistent options for all "go get"/"go build" invocations
Without this, building with go-1.5 fails with

    go get ./prog/weavewait
    go install net: open /usr/local/go/pkg/linux_amd64/net.a: permission denied

This makes the options passed too "go get"/"go build" consistent for
all executables. So it should be safe with go-1.4.

Conflicts:
	Makefile
2015-10-15 14:16:33 +01:00
Ilya Dmitrichenko 34606e0f9f Link to the new screencast (close #1402) 2015-10-15 14:07:54 +01:00
Matthias Radestock 5f62f9edc7 Merge branch '1.1' 2015-10-15 13:05:35 +01:00
Matthias Radestock 0d2ff55c07 Merge pull request #1535 from weaveworks/1523-rewrite-inspect-failure
[proxy] only attempt to rewrite successful inspects

Fixes #1523.
2015-10-15 13:04:34 +01:00
Matthias Radestock 207ee8667b Merge pull request #1538 from dpw/detect-no-fastdp-mismatch
signal mismatches between WEAVE_NO_FASTDP and the bridge type

Fixes #1533.
2015-10-14 17:17:35 +01:00
David Wragg 5984ed921e signal mismatches between WEAVE_NO_FASTDP and the bridge type 2015-10-14 16:33:23 +01:00
Matthias Radestock aac5fb8c24 Merge pull request #1537 from weaveworks/1502-flakey-test
Wait until container exits in test to remove race condition

Fixes #1502
2015-10-14 16:19:20 +01:00
Matthias Radestock 8cfc8b3132 Merge pull request #1528 from weaveworks/weave-env-bug
[proxy] Fix bug in weave env/config when passing multiple -H flags

Fixes #1527
2015-10-14 15:50:42 +01:00
Paul Bellamy 0b5ebd7b1d Wait until container exits in test to remove race condition 2015-10-14 14:53:23 +01:00
David Wragg 35141a4aef Forward a packet with sleeve if the source or dest peer lacks a short ID 2015-10-14 13:53:59 +01:00
David Wragg e702234fdc Don't use a nil FlowOp to mean "discard the packet" 2015-10-14 13:53:34 +01:00
Bryan Boreham 05dbe23563 Print request and response json when debugging 2015-10-14 13:45:51 +01:00
Paul Bellamy fd07f59d16 [proxy] Fix bug in weave env/config when passing multiple -H flags
The echo in proxy_addr was not preserving newlines, so the multiple
addrs were all getting returned
2015-10-14 11:58:10 +01:00
Paul Bellamy 36de840c91 [proxy] only attempt to rewrite successful inspects 2015-10-14 11:28:43 +01:00
David Wragg b36ffe1c9d Support peers that don't have short IDs 2015-10-14 11:11:34 +01:00
David Wragg 1f1451e666 Fall back to sleeve when "Overlays" not present in features 2015-10-14 11:11:34 +01:00
David Wragg 1000839309 Merge issues/1505-fast-datapath-weave-status' (PR #1525)
Fast datapath weave status
2015-10-14 11:08:09 +01:00
Adam Harrison ed4ed5fdcd Include short peer ID in report JSON 2015-10-13 17:04:56 +01:00
Adam Harrison eaf1f7cc13 Include overlay diagnostics in report JSON 2015-10-13 17:04:56 +01:00
Adam Harrison ddfc8c2fa4 Handle JSON report marshalling error 2015-10-13 17:04:56 +01:00
Adam Harrison 91afa7f7f4 Include selected overlay type in connection status 2015-10-13 17:04:56 +01:00
Paul Bellamy da49e71eb8 Merge pull request #1529 from weaveworks/expose-proxy-tls
Expose some tls functions from the proxy for use elsewhere.
2015-10-13 16:09:27 +01:00
Tom Wilkie 5b8ce8ebae Expose some tls functions from the proxy for use elsewhere. 2015-10-13 14:56:11 +00:00
Adam Harrison 827bcd898e Merge pull request #1438 from weaveworks/fdp
Fast datapath
2015-10-13 10:34:49 +01:00
Matthias Radestock 54a4add40f cosmetic 2015-10-13 09:02:03 +01:00
David Wragg 0093eb1220 A smoke test for WEAVE_NO_FASTDP
Partly for the side effect of restoring coverage of
router/ethernet_decoder.go
2015-10-12 21:27:11 +01:00
David Wragg b56ff7678c Make code coverage work through the proxy
Previously, the proxy would run the weave script in ways that did
not run the weaver executable.  Now they can.  This is a problem because
of the contortions involved in using golang's code coverage support
in the integration tests, which require the weaver command line args to
be slightly mangled.
2015-10-12 21:27:11 +01:00
David Wragg 431197042d Weave script changes to enable fast datapath
This means:

* The 'weave' netdev is an OVS datapath

* The weave router container runs with --net=host in order to access
  the datapath.

* The MTU is set to a smaller value, to allow vxlan encapsulation.

Setting the WEAVE_NO_FASTDP environment variable disables these
things, effectively running the router without fast datapath.  It is
an environment variable rather than an option to 'weave launch'
because it takes effect in create_bridge, which is called from many
subcommands, and it doesn't seem practical to add option processing to
all of them.
2015-10-12 21:27:11 +01:00
David Wragg 295876e1d1 Fast datapath 2015-10-12 21:27:11 +01:00
David Wragg e8f975e44f Introduce checkFatal in prog/weaver/main.go 2015-10-12 21:27:11 +01:00
David Wragg bc026ab49a Show the overlay name in sleeve log messsages 2015-10-12 21:27:11 +01:00
David Wragg f56457271d OverlaySwitch 2015-10-12 21:27:11 +01:00
David Wragg bc5250c899 Replace OverlayForwarderListener with channels
It's more robust to signal the conditions asynchronously via channels
rather than callbacks.
2015-10-12 21:27:11 +01:00
David Wragg 971511e1c5 Introduce PeerShortIDs
These are needed to support vxlan encapsulation.  In a vxlan packet,
the only place we can straightforwardly hold the source and
destination peer identifiers used by the weave router is in the
24-bit VNI (vxlan network identifier) field.  This gives us 12 bits to
indicate a peer. The resulting limit of 4096 peers on a weave network
does not seem like a problem in the near future.  But it does mean
that decentralized allocation makes collisions fairly likely even in
weave networks of sizes that are common today.  Fortunately, the
problem is much more tractable than IPAM because we can detect when
collisions occur and take action to resolve them.
2015-10-12 21:27:11 +01:00
David Wragg 51317dca70 Introduce PeersPendingNotifications 2015-10-12 21:27:11 +01:00
David Wragg 91dda08165 Rename Peers "table" field to "byName" 2015-10-12 21:27:11 +01:00
David Wragg 67bfa5bc90 applyUpdate can return a PeerNameSet 2015-10-12 21:27:11 +01:00
David Wragg 0841f4a27d Tidy broadcastPeerUpdate 2015-10-12 21:27:11 +01:00
David Wragg c6edf8421f Handle the not-found case implicitly in Broadcast and BroadcastAll 2015-10-12 21:27:11 +01:00
David Wragg 97fb90c37a Always include local peer in Peers 2015-10-12 21:27:11 +01:00
Matthias Radestock 1d00bdd2a4 a better way to restrict coverage
this better fixes #1524.
2015-10-12 18:04:07 +01:00
Matthias Radestock 0ff01871f3 Merge remote-tracking branch 'upstream/master' 2015-10-12 17:43:41 +01:00
Matthias Radestock e00a675e6e update tools to latest version 2015-10-12 17:43:06 +01:00
Paul Bellamy d5e63b875b Merge pull request #1395 from weaveworks/1371-proxy-no-dockerclient
[proxy] all request modification is done through map[string]interface{}
2015-10-12 16:42:44 +01:00
Matthias Radestock a31ccef99b only produce coverage for packages under github.com/weaveworks/weave
Fixes #1524.
2015-10-12 16:06:26 +01:00
Paul Bellamy 375fd9140b review feedback
* Opted to remove json path traversal, as it added more code than it saved.
2015-10-12 15:53:07 +01:00
Paul Bellamy fbb2c213a9 [proxy] all request modification is done through map[string]interface{}
That way we are less likely fall victim to go-dockerclient, or the API changing.
2015-10-12 15:53:07 +01:00
Bryan Boreham d323de3c21 Merge pull request #1520 from weaveworks/1519-rebind-status-sock
LGTM. Fixes #1519
2015-10-09 17:58:33 +01:00
Paul Bellamy 18872cc8ae [proxy] remove status.sock before binding status.sock 2015-10-09 17:44:16 +01:00
Tom Wilkie 33085e3798 Merge pull request #1499 from weaveworks/1463-ipam-seed-mismatch
detect when IPAM has been seeded by different peers
2015-10-09 14:46:55 +01:00
Matthias Radestock e42c811797 docs 2015-10-08 22:43:47 +01:00
Matthias Radestock 6b321e4d02 associate non-container dns entries with 'weave:extern'
...instead of the weave container. This is cleaner.

'weave:extern' because typically these entries will refer to external
(non-weave) IP addresses. And because it's the same length as
'weave:expose', which is desirable for 'weave status dns' output.
2015-10-08 22:21:38 +01:00
Matthias Radestock e6b17c701d dns-{add,remove} can add/remove non-container DNS records
...by omitting the container name/id

Fixes #1385.
2015-10-08 17:37:35 +01:00
Bryan Boreham 60bda7ca25 Merge pull request #1512 from weaveworks/1459-ipam-early-container-death
LGTM. Fixes #1459
2015-10-07 16:01:45 +01:00
Adam Harrison b37d2ec6d1 Merge pull request #1494 from weaveworks/1468-template-on-demand
Build GCE template on demand
2015-10-07 15:42:39 +01:00
Matthias Radestock 751ccd02f6 don't stall in ipam on early container death
When a container died before we got into the allocation code (inside
the allocator run loop) we would not cancel the allocation since the
container death event will have been processed by the allocator before
it had any knowledge of the container.

The fix is to move the container-aliveness check inside the allocation
code. That way we still detect the death even if the event has already
flown past.

We only need to perform the check once, since any subsequent death
will result in an event that is processed by the allocator when it has
recorded the container in its state.

Fixes #1459.
2015-10-07 15:20:22 +01:00
Matthias Radestock 431d04bd40 refactor: move all dataplane code out of LocalPeer 2015-10-06 19:50:41 +01:00
Matthias Radestock e756f07cad Merge branch '1.1' 2015-10-06 16:25:45 +01:00
Matthias Radestock 0ef548b9c9 prevent dropping of large multicast packets
Fixes #1507.
2015-10-06 16:25:02 +01:00
Ilya Dmitrichenko b9e025b50e Merge branch '1.1' 2015-10-06 15:28:54 +01:00
Ilya Dmitrichenko 7743f0e05f Merge branch 'latest_release_doc_updates' into 1.1 2015-10-06 15:26:50 +01:00
Ilya Dmitrichenko a1581ed5b5 Remove stale/invalid flag 2015-10-06 15:26:14 +01:00
Ilya Dmitrichenko 7a83a71a02 Merge branch '1.1' 2015-10-06 15:24:25 +01:00
Ilya Dmitrichenko a6c30088ab Merge branch 'latest_release_doc_updates' into 1.1 2015-10-06 15:24:08 +01:00
Ilya Dmitrichenko fdbe950483 Fix #1509 2015-10-06 15:21:24 +01:00
Matthias Radestock 2f86326405 Merge tag 'v1.1.1' into latest_release_doc_updates
Release v1.1.1
2015-10-06 15:13:50 +01:00
Matthias Radestock 8aaf4cfaf0 include nicknames in error message, if we have them 2015-10-06 14:41:41 +01:00
Ilya Dmitrichenko 8a62d13cee Merge pull request #1500 from weaveworks/1443-dont-assume-docker
allow router exe to be run w/o access to Docker
2015-10-06 14:28:26 +01:00
Matthias Radestock 5a45471c68 more readable errors on ring.Merge
There are three significant changes here:

- rewrite ring.ErrDifferentSeeds errors
- remove the pre-checking of ranges and instead rewrite
  ring.ErrDifferentRange errors
- don't pruneNicknames() and signal ringUpdated() when there was an
  error

We don't rewrite other errors since our current belief is that only
ErrDifferentSeeds and ErrDifferentRange can arise from user error.
2015-10-06 14:20:42 +01:00
Matthias Radestock 6dc21bf433 Merge pull request #1503 from weaveworks/860-detach-hide-no-ipalloc
don't allocate IPs on 'detach' and 'hide'

Fixes #860.
2015-10-06 12:53:51 +01:00
Matthias Radestock 1da24c68ca don't allocate IPs on 'detach' and 'hide'
Instead of POSTing to IPAM in order to obtain IPs, we GET existing
IPs. We ignore 404s, so detaching/hiding remains idempotent.

Fixes #860
2015-10-06 12:35:55 +01:00
Matthias Radestock 3d3b7a5672 Merge branch '1.1'
Conflicts:
	router/ethernet_decoder.go
	router/local_peer.go
2015-10-05 20:02:32 +01:00
Matthias Radestock 6c4ea7a957 unswap src and dst ip in "too big DF broadcast" warning
and some whitespace cosmetics.

Fixes #1508.
2015-10-05 19:56:59 +01:00
Matthias Radestock f221a6d953 Merge branch '1.1' 2015-10-05 12:08:54 +01:00
Matthias Radestock de95859e0d add release v1.1.1 2015-10-05 11:26:24 +01:00
David Wragg bcf71c0fee Merge branch '1.1' 2015-10-04 20:25:31 +01:00
David Wragg 8fe0228d4d Merge 1364-ip-link-kernel-4.2 (PR #1504)
strip off @... from interface names returned by `ip link show`
2015-10-04 20:23:43 +01:00
Matthias Radestock 37ea8a28ad strip off @... from interface names returned by ip link show
Apparently kernel 4.2 adds these, which breaks `weave reset` when we
pass the names to `ip link del'.

Fixes #1364.
2015-10-04 15:43:55 +01:00
Matthias Radestock a689317696 make 'no Docker API access' the default in the exe
...and instead enable it in the script, via the (better named) option.
2015-10-03 17:52:57 +01:00
Matthias Radestock 14a5c12404 allow router to be run w/o access to Docker
Fixes #1443.
2015-10-03 17:52:57 +01:00
Matthias Radestock 35b2968b60 Merge branch '1.1' 2015-10-03 17:20:21 +01:00
Matthias Radestock dac932f553 fix multi-ip attach
This got broken in f7dfe9d
2015-10-03 17:09:25 +01:00
Matthias Radestock 7e0ae6112c detect when IPAM has been seeded by different peers
This is an early detector for a multitude of other symptoms, some of
which cause panics.

When creating the ring we record the paxos-supplied normalised list of
peers used for seeding. This subsequently gets gossipped around as
part of the Ring data structure, and checked in ring.Merge that it is
the same for the recipient.

There are some subtleties due to the desire to maintain backward
compatibility and deal with empty rings

- The gossip gob decoder will ignore the Ring.Seeds field when an old
  weave receives a Ring from a new weave. So everything on the old
  weave will just continue to work as before.

- The gossip gob decoder will set the Ring.Seeds field to nil when a
  new weave receives a Ring from an old weave. We therefore omit the
  seed equivalence check when the received Seeds slice is empty.

- Our ring may not have any seeds - either because it is empty or was
  created from a ring received from an old weave. We therefore a) omit
  the seed equivalence check in that situation, and b) set our seeds
  to those received. Note that this allows us to learn about the seeds
  well after ring creation - a situation which may arise when
  receiving gossip from a mixture of old and new weaves.

Fixes #1463.
2015-10-02 16:33:28 +01:00
Bryan Boreham 9ad1ccf19f Don't listen to SIGUSR1, since we stopped handling it a few months back 2015-10-02 13:52:23 +01:00
Bryan Boreham 5b7599e1b0 Merge pull request #1496 from weaveworks/953-backoff-late-connection-failures
LGTM.  The timing choices are all very arbitrary, so we should wait for more experience in the wild before trying to tune them further.

Fixes #953
2015-10-02 12:44:52 +01:00
Matthias Radestock 19c44b83d2 refactor: better 'try' method names 2015-10-02 11:31:11 +01:00
Matthias Radestock 41691957d2 refactor: simplify target status reporting 2015-10-01 20:05:12 +01:00
Matthias Radestock e3182624fd refactor: more obviously handle all target states in reporting 2015-10-01 20:05:12 +01:00
Matthias Radestock b8b3e6f9cd refactor: better representation of 3-valued target state 2015-10-01 20:05:12 +01:00
Matthias Radestock 520c7286c4 backoff retries for late-failing connections
If a connection gets through all the handshake and subsequent checks -
i.e. it ends up getting added to the LocalPeer's connection list - but
gets terminated within one minute (of the start of the successful
connection attempt), then we apply the existing backoff logic to
schedule a retry, instead of retrying immediately.

In order to do this we retain the 'Target' data structure for such
connections, marking it as 'connected' instead of deleting it (as we
did previously).

Fixes #953.
2015-10-01 20:05:12 +01:00
Matthias Radestock d547829c5f Merge pull request #1495 from weaveworks/1492-proxy-remove-var-run-mount
Remove /var/run mount from exec_remote. Proxy uses /var/run/weave/weave.sock

Fixes #1492.
2015-10-01 16:58:37 +01:00
Matthias Radestock 49459cfc9b Merge pull request #1462 from weaveworks/1461-dns-case-sensitive
DNS lookups should not be case sensitive.

Fixes #1461.
2015-10-01 16:52:00 +01:00
Paul Bellamy 827ba446ab Remove /var/run mount from exec_remote. Proxy uses /var/run/weave/weave.sock
It was left in for backwards compatibility, in 1.1.
2015-10-01 15:52:29 +01:00
Tom Wilkie e5316221bb Review feedback 2015-10-01 14:14:15 +00:00
Tom Wilkie 357174e351 DNS lookups should not be case sensitive.
- On-the-wire gossip messages are sorted case sensitive, but in memory they are sorted case-insensitive.
- Add test for case insensitive match on multiple names.
2015-10-01 12:19:49 +00:00
Tom Wilkie 5fc2f3236e Build GCE template on demand 2015-10-01 11:49:31 +00:00
Matthias Radestock a3484cce43 Merge pull request #1488 from weaveworks/1481-proxy-null-hostconfig
[proxy] Handle null HostConfigs gracefully in start

Fixes #1481.
2015-10-01 12:06:51 +01:00
Matthias Radestock cdd107b06b Merge branch '1.1'
Conflicts:
	weave
2015-10-01 12:04:44 +01:00
Bryan Boreham 6619b584bc Merge pull request #1483 from weaveworks/1478-cm-termination-race
LGTM.  Fixes #1478
2015-10-01 12:01:55 +01:00
Matthias Radestock d798b0cabb Merge pull request #1476 from weaveworks/1455-mount-var-run
unmount /var/run after boot, so the kernel can GC veth pairs

Fixes #1455.
2015-10-01 11:55:24 +01:00
Paul Bellamy 29bfd77855 [proxy] don't mount /var/run into the proxy, so the kernel can GC veth pairs 2015-10-01 11:43:13 +01:00
Alfonso Acosta c24dc6609d [proxy] Handle null HostConfigs gracefully in start 2015-10-01 11:40:50 +01:00
Adam Harrison b5e7e9fc5c Merge pull request #1451 from dpw/sleeve-compat
Restore sleeve compatibility with 1.1
2015-10-01 11:24:31 +01:00
Tom Wilkie 58dec7334b Merge branch '1.1' 2015-10-01 08:23:15 +00:00
Tom Wilkie 0b7af28c86 Typo in Makefile. 2015-10-01 08:21:55 +00:00
Matthias Radestock 725f99ea4c Merge branch '1.1' 2015-10-01 08:23:52 +01:00
Matthias Radestock 526259ef8e Merge pull request #1487 from weaveworks/1480-dns-test-race
Fix races in TestRecursiveCompress

Fixes #1480.
2015-10-01 08:11:48 +01:00
Matthias Radestock 4b43cac093 Merge branch '1.1' 2015-09-30 23:34:57 +01:00
Matthias Radestock 3795c14931 better 2015-09-30 22:57:10 +01:00
Matthias Radestock 3e5cdd8bf2 refactor: remove pointless NXDOMAIN tests 2015-09-30 22:42:27 +01:00
Matthias Radestock 35058a1dde refactor: put negative DNS queries in one test suite 2015-09-30 22:38:48 +01:00
Matthias Radestock 0159e26631 store connections as a set instead of map-of-remote-addresses
...since it is theoretically possible that the lifecycle events from
two consecutive inbound connections from the same remote address
overlap.
2015-09-30 17:29:13 +01:00
Tom Wilkie 11d93b1883 Fix races in TestRecursiveCompress 2015-09-30 15:51:07 +00:00
Tom Wilkie f86f392641 Merge pull request #1486 from weaveworks/1482-disable-160
Disable 160_claim test.
2015-09-30 16:20:34 +01:00
Tom Wilkie 19cb1ae8e1 Merge branch '1.1' 2015-09-30 15:06:26 +00:00
Matthias Radestock cd8befe43e Merge pull request #1485 from weaveworks/1450-check-slice
Sanity-check gossip messages
2015-09-30 15:41:21 +01:00
Tom Wilkie db46204ff9 Disable 160_claim test. 2015-09-30 14:22:24 +00:00
Bryan Boreham d2290cc82c Sanity-check gossip messages 2015-09-30 14:39:27 +01:00
David Wragg f8e06f9dbc Merge 1286-tool (PR #1470): Use test script from build-tools.git 2015-09-30 14:15:20 +01:00
Matthias Radestock e68ca761a9 we no longer need to Refresh() in CM status
since CM now gets an event - ConnectionCreated - for new connections.
2015-09-30 14:00:32 +01:00
Matthias Radestock fc6f3993bd refactor(ish): obtain local connection status from CM alone 2015-09-30 14:00:32 +01:00
Matthias Radestock f8b074f696 a better way to clean up after established connections
...than 3db4e86
2015-09-30 14:00:32 +01:00
Matthias Radestock 66f14099ed prevent ConnectionMaker.ConnectionTerminated race
Get the ConnectionMaker to maintain its own knowledge of the local
connections, based on events emitted in the connection lifecycle.

Fixes #1478.
2015-09-30 14:00:32 +01:00
Tom Wilkie f3dc681f61 Only clean weave packages. 2015-09-30 10:15:50 +00:00
Tom Wilkie 62f62e1fd5 tools.git is now called build-tools.git 2015-09-30 08:21:19 +00:00
Tom Wilkie f3360e61e1 Remove cover tool, this lives in tools.git now. 2015-09-30 08:21:12 +00:00
Tom Wilkie 5208dcd67e Review feedback
- Use git submodule for tools.
- Update Makefile to use submodules.
- Put submodule at tools/
- Tidy up some obsolete references to tools in .gitignore.
2015-09-30 08:21:08 +00:00
Tom Wilkie 6f7d4a7a01 Use lint from tools.git
- Do lint first, on circle.
2015-09-30 08:20:56 +00:00
Tom Wilkie 958a3bbd99 Use test script from tools.git 2015-09-30 07:42:43 +00:00
Matthias Radestock d5c85a3d53 cosmetic: fix typo 2015-09-29 19:40:54 +01:00
Tom Wilkie 5e7b527af2 Make publish from the right place. 2015-09-29 11:02:46 +00:00
Paul Bellamy ddbd33a8e3 Merge pull request #1471 from weaveworks/1295-publish
Publish latest images from CI on successful builds.
2015-09-29 11:37:44 +01:00
Matthias Radestock a7dc368191 Merge pull request #1469 from weaveworks/1204-test-container
Run tests in a separate container.

Fixes #1204.
2015-09-29 11:37:25 +01:00
Matthias Radestock 5b1d0c55a0 Merge pull request #1474 from weaveworks/1472-weave-env-debug
use a container-local unix socket to get the address, instead of logs

Fixes #1453. Fixes #1472.
2015-09-29 11:04:41 +01:00
Matthias Radestock 58515ca36e make weave(proxy) startup death message more generic
...in order to account for `docker logs` not working due to custom
logging drivers.

Partially undoes #958. Partially fixes #1453.
2015-09-29 11:02:05 +01:00
Matthias Radestock 2d753ebf1a cosmetic: reduce confusion w.r.t. meaning of 'connection' 2015-09-29 09:21:39 +01:00
Tom Wilkie 50cd716220 Review feedback. 2015-09-29 07:46:00 +00:00
Matthias Radestock 47c97e8191 refactor: remove some code duplication 2015-09-28 16:10:43 +01:00
Paul Bellamy d704ba175f [proxy] use a container-local unix socket to get the address, instead of logs 2015-09-28 16:08:25 +01:00
Matthias Radestock c1036c5e4f Merge remote-tracking branch 'upstream/master' 2015-09-28 14:53:12 +01:00
Matthias Radestock 356d06915b refactor weave script http calls
- inline http_call into call_weave
- rename http_call_ip to http_call
- make http_call take the IP & port as one single "address" argument
  instead of separate args
2015-09-28 12:43:09 +01:00
Tom Wilkie 2b4fd738da Review feedback 2015-09-28 09:05:10 +00:00
Tom Wilkie f9de6df426 Publish latest images from CI on successful builds.
This time rebuild without insturmentation first.
2015-09-28 08:27:21 +00:00
Tom Wilkie 8ee409d9bd Run tests in a separate container. 2015-09-25 06:19:00 +00:00
Matthias Radestock 0cf3de4541 Merge pull request #1446 from weaveworks/smoke-subdomains
Add smoke-test for subdomains
2015-09-24 15:55:57 +01:00
Bryan Boreham 3a79be7426 Add smoke-test for subdomains 2015-09-24 14:48:39 +01:00
David Wragg 30589bb580 Mention the mailing list 2015-09-24 14:40:20 +01:00
Matthias Radestock e07b18bb18 Merge branch '1.1' 2015-09-24 12:08:13 +01:00
Matthias Radestock 63e3f441d8 cosmetic: link to relevant RFC section for ARP announcements 2015-09-24 12:07:12 +01:00
Tom Wilkie b31eb3db82 Merge remote-tracking branch 'upstream/1.1' 2015-09-24 03:54:09 +00:00
Tom Wilkie 99ae38c9a6 Merge pull request #1467 from weaveworks/docker-machine
Update for docker/machine PR #1729
2015-09-24 11:53:21 +08:00
Tom Wilkie 011f96c6a7 New test image using docker 1.8 2015-09-24 03:42:44 +00:00
Tom Wilkie c7b4136d31 Update for docker/machine PR #1729 2015-09-24 03:25:30 +00:00
Bryan Boreham da2f98fa8f Remove workaround for upstream, no longer required 2015-09-23 10:15:06 +01:00
Tom Wilkie 7ac532a1d0 Merge pull request #1454 from weaveworks/1433-docker-1.8
New test image using docker 1.8
2015-09-23 10:16:06 +08:00
Matthias Radestock b98ab70517 Merge branch '1.1' 2015-09-22 21:18:32 +01:00
Matthias Radestock b1d453a0c2 Merge pull request #1460 from weaveworks/simplify-ipam-cidrs
Simplify ipam_cidrs function in weave script
2015-09-22 21:17:42 +01:00
Bryan Boreham a51f63267a Merge 0-args case in ipam_cidrs() and simplify loop 2015-09-22 21:06:12 +01:00
Matthias Radestock 0bdc84d42a Merge branch '1.1' 2015-09-22 16:28:05 +01:00
Matthias Radestock 7f74ae7912 Merge pull request #1426 from weaveworks/1418-clean-cancel
Handle cancelled allocation in 'weave attach'

Fixes #1418.
2015-09-22 16:27:27 +01:00
Bryan Boreham abf13c7026 Simplify detection of issues 2015-09-22 16:14:14 +01:00
Matthias Radestock 7d7c5ce63e Merge pull request #1449 from weaveworks/1368-start-hostconfig
Override sneaky /start HostConfig params

Fixes #1368.
2015-09-22 15:11:36 +01:00
Paul Bellamy 547c9e7f5a Removing confusing variable from getDNSDomain, just return "" when disabled 2015-09-22 15:09:32 +01:00
Paul Bellamy 3e4d8a3ffe review feedback 2015-09-22 15:09:32 +01:00
Paul Bellamy 74c3b0d079 [proxy] Override sneaky /start HostConfig params 2015-09-22 15:09:32 +01:00
Paul Bellamy 846aa98a9c renaming test so it runs 2015-09-22 14:52:28 +01:00
Matthias Radestock 17b7789531 Merge pull request #1456 from weaveworks/fix-race-condition
[proxy] fix race condition when waiting for containers to start
2015-09-22 14:51:43 +01:00
Paul Bellamy b86d9bf1ab [proxy] fix race condition when waiting for containers to start
If ContainerStarted fired before the response handler, then the "waiter"
chan would be nil, and the response handler would hang.
2015-09-22 12:50:36 +01:00
Tom Wilkie 6e06ce2af1 New test image using docker 1.8 2015-09-22 05:31:26 +00:00
David Wragg dc275d7009 Restore sleeve compatibility with 1.1
Addresses #1396
2015-09-21 08:33:11 +01:00
David Wragg a67b9f566a Use consistent options for all "go get"/"go build" invocations
Without this, building with go-1.5 fails with

    go get ./prog/weavewait
    go install net: open /usr/local/go/pkg/linux_amd64/net.a: permission denied

This makes the options passed too "go get"/"go build" consistent for
all executables. So it should be safe with go-1.4.
2015-09-20 19:58:04 +01:00
Michael Bridgen 9344289262 Merge pull request #1445 from weaveworks/docs-ecs-amis
Document ECS integration
2015-09-17 14:43:38 +01:00
Alfonso Acosta 6713282a43 Link to ECS AMIs from README 2015-09-17 13:27:32 +00:00
Matthias Radestock f834626f00 Merge pull request #1414 from weaveworks/1285-autoconfig-boot2docker-tls
Proxy TLS autoconfiguration

Closes #1285.
2015-09-16 15:49:20 +01:00
Bryan Boreham 98b80c59d9 Detect cancelled allocation more cleanly
to avoid the weave script trying to parse an error message as a CIDR
2015-09-16 14:14:40 +01:00
Bryan Boreham 1beaf79e47 Refactor: extract duplicated code into helper functions 2015-09-16 14:14:30 +01:00
Paul Bellamy 0a475bd4cc docker_tls_args should look for '-d' or 'daemon' to handle docker 1.8 change 2015-09-16 13:25:29 +01:00
Paul Bellamy 0820c484d6 [proxy] Documenting TLS autoconfig 2015-09-16 10:44:18 +01:00
Paul Bellamy dbf39403d3 [proxy] Autoconfigure TLS based on daemon settings
With this on OSX you can:
```sh
$ boot2docker init
$ boot2docker up
$ eval $(boot2docker shellinit)
$ weave launch
$ eval $(weave env)
```
2015-09-16 10:44:17 +01:00
Matthias Radestock 14dde9ec32 Merge branch '1.1' 2015-09-15 16:47:05 +01:00
Matthias Radestock 8e656c1fd7 Merge pull request #1429 from weaveworks/1424-fix-cancel
Use explicit index into operations slice in cancelOpsFor()

Fixes #1424
2015-09-15 16:45:11 +01:00
Bryan Boreham d55c68c248 Review feedback 2015-09-15 16:18:37 +01:00
Matthias Radestock 3c266dce8c Merge branch '1.1' 2015-09-15 15:51:36 +01:00
Matthias Radestock 8ec3ab5e28 Merge pull request #1435 from weaveworks/update-vagrantfile-docker-repos
Update docker repo, keys and package name
2015-09-15 15:49:33 +01:00
Matthias Radestock f7be34f5bb Merge branch '1.1' 2015-09-15 15:44:53 +01:00
Matthias Radestock 58a30f7282 Merge branch 'latest_release_doc_updates' into 1.1 2015-09-15 15:44:15 +01:00
Matthias Radestock 2b5c2874ab remove 'lxc-docker' reference from build-on-ubuntu instructions
...since the Docker docs have been updated to install docker-engine.
2015-09-15 15:34:44 +01:00
Matthias Radestock 64b47ca3c7 Merge branch '1.1' 2015-09-15 13:17:06 +01:00
Matthias Radestock bed11f4214 Merge pull request #1431 from weaveworks/issues/1397-strip-version-prefix
Automatically strip 'v' prefix for mainline and branch releases

Fixes #1397.
2015-09-15 13:15:49 +01:00
Adam Harrison bbf7646751 Update docker repo, keys and package name 2015-09-15 11:33:53 +01:00
Adam Harrison 6dfefb95e8 Advise installation of weaveworks github-release fork 2015-09-15 10:47:43 +01:00
Adam Harrison 44c2861aba Update release process doc 2015-09-15 10:41:41 +01:00
Adam Harrison 3039046a6d Strip v prefix for mainline and branch releases 2015-09-15 10:41:24 +01:00
Paul Bellamy e39974eb07 Merge pull request #1210 from weaveworks/401-proxy-start-events
make proxy attach network on docker start event
2015-09-15 10:18:48 +01:00
Bryan Boreham 76895c76a1 Use Mutex instead of RWMutex, for simplicity 2015-09-14 17:11:56 +01:00
Bryan Boreham a7e0f03647 Use explicit index into operations slice in cancelOpsFor()
to avoid mis-indexing when more than one op is removed
2015-09-14 17:02:35 +01:00
Bryan Boreham 98e8ef4e11 Merge remote-tracking branch 'upstream/1.1' 2015-09-14 16:33:12 +01:00
Bryan Boreham 5635ba1fd0 Update out-dated error message 2015-09-14 16:31:37 +01:00
Adam Harrison 2ffe9b53f7 Merge branch '1.1' 2015-09-14 14:21:36 +01:00
Adam Harrison dd0eee6418 Merge pull request #1411 from weaveworks/1405-arping-dynamic-attach-1
always arp_update new addresses in 'attach'
2015-09-14 14:20:21 +01:00
Paul Bellamy 405c2889f0 Merge pull request #1415 from weaveworks/1341_report_snapshot_versions
correctly report snapshot versions
2015-09-14 10:45:29 +01:00
Matthias Radestock 40cf392504 cosmetic: vertical alignment in version output 2015-09-11 15:56:17 +01:00
Matthias Radestock 42baa03985 report the proxy version instead of weaveexec version
That way we no longer rely on the weavexec image version tag, which
may not always be present. Since the proxy is embedded in the
weaveexec image, this effectively versions the image.

Also report the *local* script version instead of the embedded script
version. There is no point in reporting the latter since it will
always match the weaveexec version.

Closes #1341.
2015-09-11 15:52:50 +01:00
Matthias Radestock 016f20b2d4 Merge branch '1.1' 2015-09-11 14:07:59 +01:00
Matthias Radestock 83ef93f3fb detect running proxy regardless of version
Fixes #1413.
2015-09-11 14:06:44 +01:00
Matthias Radestock d5faf31d43 Merge remote-tracking branch 'upstream/master' 2015-09-11 13:36:59 +01:00
David Wragg 12324e3550 Revert "Make weave build under go1.5; use our build flags for go get, and force stuff to be static."
This reverts commit 5860492a38.

Although the build works under 1.4, this change had an unfortunate
impact (see #1412).
2015-09-11 13:31:15 +01:00
Matthias Radestock ff67499562 cosmetic 2015-09-11 13:30:22 +01:00
Matthias Radestock 2ba5abc2bb rename 'VERSION' to 'WEAVE_VERSION'
to avoid potential accidental clashes with user's environment.

Closes #1389.
2015-09-11 11:02:51 +01:00
Matthias Radestock f7dfe9dd23 always arp_update new addresses in 'attach'
Previously we'd only do that when the container had no ethwe at the
start. Therefore adding a recycled IP address to an already attached
container would not send out an arp update for that IP, which would
break connectivity to that IP for containers which had previously
accessed that IP.

Fixes #1405.
2015-09-10 18:43:49 +01:00
Adam Harrison 3d4a8aaad1 Merge branch '1.1' 2015-09-10 17:24:26 +01:00
Adam Harrison ac145052d6 Correct weave stop advice
Fixes #1408.
2015-09-10 17:23:33 +01:00
Matthias Radestock 4ded1f6193 Merge branch '1.1' 2015-09-09 20:06:34 +01:00
Matthias Radestock 7a41337d17 Merge branch 'latest_release_doc_updates' into 1.1 2015-09-09 20:06:14 +01:00
Matthias Radestock 2d006dffaf add link to latest release to docs homepage
Closes #1400.
2015-09-09 20:04:37 +01:00
Matthias Radestock 940a0a67a5 Merge branch '1.1' 2015-09-09 19:54:46 +01:00
Matthias Radestock 69c935a6e8 Make latest_release description link to real release
Closes #1399.
2015-09-09 19:52:08 +01:00
Adam Harrison be304365a0 Merge branch '1.1' 2015-09-09 14:26:34 +01:00
Adam Harrison 676127893f Escape double curly braces eaten by Jekyll/Liquid 2015-09-09 14:21:31 +01:00
Adam Harrison df62d512cd Correct weave report example in troubleshooting guide 2015-09-09 14:00:53 +01:00
Adam Harrison fc00767cf8 Remove reference to 1.0 from README 2015-09-09 13:56:25 +01:00
David Wragg 2163b39268 Merge weaveworks/go-1.5 (PR #1386)
Make Weave build & work with Go 1.5
2015-09-09 12:23:57 +01:00
Matthias Radestock aa61cbd574 Merge pull request #1388 from weaveworks/alpine
gliderlabs/alpine is now just alpine
2015-09-08 14:01:40 +01:00
Adam Harrison 2a89d53d7d Merge pull request #991 from dpw/fdp-prep
Fast datapath preparations
2015-09-08 13:19:03 +01:00
David Wragg f7ce6a1fdb Injected packets shouldn't happen in handleCapturedPacket
Check for them just in case, but don't take an extra lock to do it.
2015-09-08 10:46:04 +01:00
David Wragg 20b35f3139 Add NewPeerFromSummary
To reduce the number of places that are explicit about which values
go into a new Peer.
2015-09-08 10:46:02 +01:00
David Wragg 46d454730f Notify the Overlay when routes change 2015-09-08 10:44:57 +01:00
David Wragg c2ad4c5633 Flows
For fast datapath, the router core will only be invoked upon datapath
misses.  And it needs to yield routing decisions in a form that can be
used to configure the in-kernel datapath.  So here we make the router
core yield FlowOps, which represent operations which actually do
something with a packet.  The router core obtains these in turn from
the Bridge or the Overlay, depending on what it wants to do with
a packet.

For fast datapath, it's critical that the router makes decisions based
only on the required packet header information (as the same header
information will be used when configuring the ODP flow rules).  So we
pass only that information into the router core, in the form of
PacketKeys.

The overall effect is to curry the router core: Rather than taking a
packet and doing something with it, it takes packet header information
and returns a function that does something with a packet.
2015-09-08 10:44:52 +01:00
David Wragg 6e0795253e Move ICMP frag-needed handling into sleeveForwarder 2015-09-08 10:44:52 +01:00
David Wragg 9956d422f2 Introduce Overlay / sleeve
This introduces interfaces to represent the packet forwarding
mechanism, and brings all the scattered UDP-related code together into
a single implementation.

This is not a pure refactoring, although much of the code in sleeve.go
has been copied from pre-existing code, and the UDP protocol remains
the same in spirit.  The biggest change is the organization of the
forwarder goroutines (there is now one instead of two, and that
goroutine does a lot of auxiliary work as well as forwarding).
Performance testing between VMs on my laptop suggests that packet
throughput has not gone down, and may even have improved slightly.
2015-09-08 10:44:44 +01:00
David Wragg 57e78e2bec Introduce Bridge 2015-09-08 10:44:37 +01:00
David Wragg 5aa58e6043 go 1.5 broke PMTU detection 2015-09-07 17:31:18 +01:00
Adam Harrison 64e3cf04cc Add release v1.1.0 2015-09-07 16:43:03 +01:00
Adam Harrison 4f9185e3fe Merge branch 'weave_1.1_readme_updates' 2015-09-07 16:38:28 +01:00
Adam Harrison 4dbccea5df Add missing quotes 2015-09-07 16:38:02 +01:00
Adam Harrison a412268154 Merge branch '1.0' 2015-09-07 15:59:17 +01:00
Tom Wilkie 5860492a38 Make weave build under go1.5; use our build flags for go get, and force stuff to be static. 2015-09-07 14:45:17 +00:00
Matthias Radestock d79ac3c12c Merge pull request #1387 from weaveworks/issues/1383-switch-release-behaviour-on-type
Only update DockerHub `latest` images on mainline release

Fixes #1383.
2015-09-07 12:53:07 +01:00
Matthias Radestock 85c309eee0 fix broken link 2015-09-05 16:04:21 +01:00
Matthias Radestock f782fb569f Merge pull request #1391 from weaveworks/1390-use-number
Stop marshalling numbers as floats

Fixes #1390.
2015-09-04 20:15:58 +01:00
Bryan Boreham c50968a0a4 Stop marshalling numbers as floats 2015-09-04 20:01:31 +01:00
Matthias Radestock 99b482b363 Merge pull request #1384 from weaveworks/dns-hostname-from-labels
[proxy] Add --hostname-from-label

Closes #1381.
2015-09-04 17:14:44 +01:00
Alfonso Acosta ce5dd847e9 [proxy] Add --hostname-from-label
This allows registering container hostnames based on a label provided when
creating the container.
2015-09-04 17:02:07 +01:00
Adam Harrison 5c0383059e Update documentation 2015-09-04 15:45:27 +01:00
Adam Harrison 6f672f9bc1 Infer release type
Script adjusts its behaviour based on the format of the version tag.
2015-09-04 15:44:48 +01:00
Paul Bellamy e74b6cbcaf gliderlabs/alpine is now just alpine 2015-09-04 14:33:18 +01:00
Matthias Radestock 32a82f2d0e Merge pull request #1376 from weaveworks/1199-proxy-kubernetes
[proxy] add --rewrite-inspect flag

Fixes #117. Fixes #1199.
2015-09-04 14:27:16 +01:00
Matthias Radestock 633df0cdaf refactor: extract docker API endpoint regexp creation 2015-09-04 13:40:07 +01:00
Paul Bellamy 32a12578f3 Don't use go-dockerclient to en/decode inspects in the proxy 2015-09-04 13:40:07 +01:00
Paul Bellamy 11b655154c "Clarifying" proxy --rewrite-inspect documentation 2015-09-04 13:40:07 +01:00
Paul Bellamy 19ceb424c3 [proxy] refactoring out common marshalling logic 2015-09-04 13:40:07 +01:00
Paul Bellamy e21a204eed [proxy] add --rewrite-inspect flag
To return the weave ip/mac for docker inspect calls.
2015-09-04 13:40:06 +01:00
Bryan Boreham 145a527626 Merge pull request #1379 from awh/issues/1369-weave-dns-add-remove-optional-ip
LGTM.  Fixes #1369
2015-09-04 12:13:38 +01:00
Paul Bellamy b33bbdb3f8 Merge commit '704e6a8' 2015-09-04 10:41:27 +01:00
Paul Bellamy 704e6a8676 errcheck says I should handle this case 2015-09-04 10:40:47 +01:00
Matthias Radestock 4c6a8f29c0 Merge pull request #1372 from weaveworks/issues/880-draft-releases
Support draft and pre-releases in `bin/release`

Fixes #880. Fixes #909.
2015-09-03 15:54:07 +01:00
Adam Harrison 5dfe1d28ff Add release 1.0.3 2015-09-03 15:02:57 +01:00
Adam Harrison 5744ed8994 Document release process 2015-09-03 13:01:29 +01:00
Matthias Radestock c71d9b126f Merge pull request #1380 from weaveworks/598-update-docs
Update docs for mixed manual and automatic IP allocation

Missed from #1200.
2015-09-03 12:27:24 +01:00
Bryan Boreham 5b437d2199 Update docs for mixed manual and automatic IP allocation 2015-09-03 12:17:31 +01:00
Adam Harrison 0125e67b35 Implement smoke test 2015-09-03 09:59:37 +01:00
Adam Harrison 1472e35011 Update documentation 2015-09-03 09:59:37 +01:00
Adam Harrison 58ea422ea2 Optional address argument to dns-add/dns-remove 2015-09-02 17:10:48 +01:00
Adam Harrison 3097565664 Merge branch '1.0'
Conflicts:
	site/weavedns.md
2015-09-02 17:09:39 +01:00
Adam Harrison 81134b408d Merge branch 'latest_release_doc_updates' into 1.0 2015-09-02 17:07:48 +01:00
Adam Harrison 26fdd5da03 Remove misplaced sentence 2015-09-02 17:06:23 +01:00
Adam Harrison 6a5df80bda Fix latest_release hash test
Obtain the version and latest_release tag commit hashes before changing
into the cloned repository so that all tagging and pushing happens on
the outer repository.
2015-09-02 14:13:11 +01:00
Bryan Boreham 1e2344341d Wait for container to be attached before returning from {re}start request 2015-09-01 17:48:22 +01:00
Bryan Boreham b712ba1e04 Extend test to cover containers restarting automatically 2015-09-01 17:48:22 +01:00
Bryan Boreham 9ae6435317 Attach existing weave containers at startup 2015-09-01 17:48:22 +01:00
Bryan Boreham 81e8ba223f Do network attach on docker event rather than intercepting start and restart commands.
This means it also works when the docker daemon restarts a container.
2015-09-01 17:48:12 +01:00
Adam Harrison b13baccbd2 Introduce draft step
The publish stage has been split into two pieces - a draft step that
creates a draft version release in Github and a publish step that
publishes the draft, updates `latest_release` and pushes images to
DockerHub.

It is also possible to specify `--pre-release` to draft. Once published
such a release will be marked with a prominent warning in GitHub;
additionally we do not update `latest_release` in GitHub nor push an
update to the `latest` tag in DockerHub.
2015-09-01 17:04:11 +01:00
Adam Harrison 82f2f7b2b9 Check for required tooling at top level
Pull common checks out to top level in preparation for the introduction
of an additional process step.
2015-09-01 17:04:11 +01:00
Adam Harrison f48a4373d1 Allow freeform version tags
To support arbitrarily named pre-releases we no longer impose any
restrictions on the format of the most recent annotated tag - instead we
expect to find a single annotated tag pointing at HEAD of the current
branch and assume this is the version to release.
2015-09-01 17:04:11 +01:00
Adam Harrison ab9d1777a1 Support make UPDATE_LATEST=false publish
When publishing pre-releases via the release script we do not want to
modify the `latest` tag in DockerHub - specifying UPDATE_LATEST=false
causes publish to push to $WEAVE_VERSION only. Omitting the flag
entirely retains the existing behaviour for backwards compatability.
2015-09-01 17:04:11 +01:00
Bryan Boreham 6a34995796 Merge pull request #1370 from weaveworks/docker-py-test-matcher
LGTM. Fixes #1367
2015-09-01 16:47:24 +01:00
Bryan Boreham df25f84011 Merge pull request #1375 from weaveworks/1373-document-no-rewrite-hosts
LGTM.  Fixes #1373
2015-09-01 15:01:51 +01:00
Paul Bellamy cead9579be document --no-rewrite-hosts flag 2015-09-01 11:01:45 +01:00
Paul Bellamy a23c0237c3 using --collect-only so we don't miss tests 2015-09-01 10:50:13 +01:00
Matthias Radestock 5c3efbdca0 Merge pull request #1366 from weaveworks/python-call-changed
Fix docker api proxy smoke-tests after docker-py change
2015-08-28 20:50:35 +01:00
Bryan Boreham 176ee73efe Ensure we have the busybox image 2015-08-28 20:34:43 +01:00
Bryan Boreham 54833195ed Call to docker-py changed upstream 2015-08-28 20:34:43 +01:00
Matthias Radestock 8960a3a898 Merge branch '1.0'
Conflicts:
	proxy/create_exec_interceptor.go
	test/600_proxy_docker_py_test.sh
2015-08-28 20:29:43 +01:00
Matthias Radestock 8184d29d4a Merge pull request #1367 from weaveworks/1.0-python-call-changed
Fix docker api proxy smoke-tests after docker-py change
2015-08-28 20:22:22 +01:00
Matthias Radestock fef91b5339 comment 2015-08-28 18:09:23 +01:00
Matthias Radestock 24de98e188 refactor 2015-08-28 18:08:24 +01:00
Matthias Radestock 6291a89665 Merge pull request #1314 from weaveworks/1209-wait-iface-netlink
Wait for ethwe via netlink, not via polling or signals

Fixes #1209. Fixes #1258. Fixes #1300.
2015-08-28 17:44:55 +01:00
Bryan Boreham a45e5ac152 Test calling 'start' with a HOST_CONFIG not supplied to 'create' 2015-08-28 16:20:31 +01:00
Paul Bellamy 6b5f35e574 [proxy] Only replace the request body if we've modified it.
This is a temporary fix for a failure in docker-py tests:
"test_valid_no_config_specified".
2015-08-28 15:09:33 +01:00
Matthias Radestock 244a679f90 Merge pull request #1349 from weaveworks/issues/1266-detect-adverse-firewall-config
Warn on detection of REJECT rule in FORWARD chain

Fixes #1266.
2015-08-28 12:22:23 +01:00
Matthias Radestock 1a9909e3e2 also warn on weave reset when env's DOCKER_HOST points to proxy
This was missed in #1327.
2015-08-28 09:34:18 +01:00
Matthias Radestock 6655974ef3 cosmetic: tidy up log messages 2015-08-28 09:12:42 +01:00
Matthias Radestock cdf4fb605a Merge branch '1.0' 2015-08-27 20:04:04 +01:00
Michael Bridgen bca31c7335 Merge pull request #1365 from weaveworks/1356_ignore_interface_deletion_errors_in_weave_reset
ignore interface deletion errors during `weave reset`
2015-08-27 15:38:16 +01:00
Bryan Boreham 7652b44c21 Ensure we have the busybox image 2015-08-27 15:14:06 +01:00
Bryan Boreham 6fce798164 Factor out deserializeRoute() func 2015-08-27 15:10:31 +01:00
Bryan Boreham 7801dcd849 Factor out common loop receiving from netlink 2015-08-27 15:10:31 +01:00
Bryan Boreham 2eed101a93 In weavewait, wait for the multicast route to be added before calling container entrypoint.
We have moved the 'route add' command to the end of 'attach', so this
indicates that all network interface set-up is complete.
2015-08-27 15:10:31 +01:00
Bryan Boreham 3d25620d13 In proxy, rewrite /etc/hosts from 'weave attach', via new utility weavehosts 2015-08-27 15:03:41 +01:00
Bryan Boreham 6e89b036e2 Wait for ethwe via netlink, not via polling or signals
('docker kill' breaks restart policy)
2015-08-27 15:03:41 +01:00
Michael Bridgen 8b5992e43f Merge pull request #1290 from weaveworks/labels
Add role label to weave containers so scope can filter
2015-08-27 15:00:15 +01:00
Bryan Boreham 548b4aea40 Call to docker-py changed upstream 2015-08-27 14:54:20 +01:00
Tom Wilkie 42b64f826b Merge pull request #1358 from weaveworks/1357-dns-reverse-tombstone
Make reverse dns lookups honor tombstones.
2015-08-27 14:25:23 +01:00
Tom Wilkie 95379b0c5a Add works.weave.role=system label to weave containers. 2015-08-27 12:22:34 +00:00
Matthias Radestock f5918a3c14 remove dead code
since #1248
2015-08-27 08:42:48 +01:00
Matthias Radestock 1df752cd33 ignore interface deletion errors during weave reset
These can occur when a container dies during `weave reset`.

Fixes #1356.
2015-08-27 07:54:51 +01:00
Tom Wilkie a1f5d90323 Merge pull request #1363 from weaveworks/1362-fix-quoted-params
Fix test 295
2015-08-26 13:46:49 +01:00
Bryan Boreham 5289a380f2 Use $* instead of $@ to get the desired quoting 2015-08-26 12:31:38 +01:00
Tom Wilkie 914c88454d Review feedback 2015-08-25 16:53:21 +00:00
Tom Wilkie 46363f4129 Make reverse dns lookups honor tombstones. 2015-08-25 12:01:09 +00:00
Michael Bridgen bd19b86259 Merge pull request #1352 from weaveworks/doc-format
Small improvements in features doc
2015-08-25 12:56:56 +01:00
Tom Wilkie db2e7e4ba1 Merge pull request #1355 from weaveworks/1343-docker-bridge-in-resolv-conf
Pass docker bridge ip to dns server, and filter it out of upstream dns ips.
2015-08-25 09:48:06 +01:00
Bryan Boreham 760f419e8f Merge pull request #1354 from weaveworks/1347-dns-tombstone-comment
Lgtm.  Closes #1347
2015-08-24 17:05:54 +01:00
Tom Wilkie 58ae0cd113 Pass docker bridge ip to dns server, and filter it out of upstream dns ips. 2015-08-24 16:00:26 +00:00
Tom Wilkie 4809c113b3 Update comment. 2015-08-24 15:43:31 +00:00
Ilya Dmitrichenko 6d0db90f43 Revert "Merge pull request #1337 from weaveworks/1295-push-on-green"
This reverts commit 5054805ac6, reversing
changes made to 396df6bcda.
2015-08-24 14:22:36 +01:00
Ilya Dmitrichenko 75fb8f6ebf Revert "need to be in gopath to run make publish"
This reverts commit 130cf6eb2b.
2015-08-24 14:22:29 +01:00
Bryan Boreham 26739463a4 Escape a minus character to stop it being taken as a bullet, and clarify which example we are talking about 2015-08-24 14:19:14 +01:00
Paul Bellamy 130cf6eb2b need to be in gopath to run make publish 2015-08-24 11:10:59 +01:00
Paul Bellamy 5054805ac6 Merge pull request #1337 from weaveworks/1295-push-on-green
Publish latest images from CI on successful builds.
2015-08-24 10:58:14 +01:00
Adam Harrison ca94cfffe7 Detect REJECT rule in FORWARD chain 2015-08-21 16:25:49 +01:00
Matthias Radestock 396df6bcda Merge pull request #1348 from weaveworks/1345-1332-dns-response-size
Unify logic to tell if DNS response is too big.

Fixes #1345, fixes #1332
2015-08-21 14:03:00 +01:00
Matthias Radestock 58dafbfa9d Merge pull request #1344 from weaveworks/refactor-dns-handlers
Refactor dns handlers
2015-08-21 13:44:57 +01:00
Tom Wilkie 8516ab7c69 Unify logic to tell if DNS response is too big. 2015-08-21 12:39:12 +00:00
Matthias Radestock 2284bc2120 move error response construction to handler
and implement it in terms of existing functions, plus provide a
convenience method for sending the rather common NameError.
2015-08-21 12:09:22 +01:00
Matthias Radestock 31a490bfe6 ensure that make publish publishes up-to-date images
otherwise the git revision based image tag doesn't match what is in
the image.
2015-08-21 09:34:08 +01:00
Matthias Radestock 0b5f5fd74e extract response writing 2015-08-20 20:24:12 +01:00
Matthias Radestock 9a7abe68fd optimise response construction when no truncation is required
no need to do a binary search if all the answers fit
2015-08-20 20:18:56 +01:00
Matthias Radestock 2ea27ec08f simplify response truncation 2015-08-20 19:55:19 +01:00
Matthias Radestock 322441d5c8 inline truncateResponse 2015-08-20 19:55:19 +01:00
Matthias Radestock 7c7c302920 extract response construction 2015-08-20 19:17:03 +01:00
Matthias Radestock 69986e77dd cosmetic 2015-08-20 18:46:55 +01:00
Matthias Radestock 8182022d52 refactor dns handlers
introduce a handler struct, which

- enables lifting of all the handle* closures to methods
- eliminates the passing around of defaultMaxResponseSize
- reduces the number of free-floating functions
2015-08-20 18:27:58 +01:00
Matthias Radestock b075e93319 Merge pull request #1324 from weaveworks/1321_elminate_misleading_flag_parsing_error
eliminate misleading flag parsing error

Fixes #1321.
2015-08-20 17:34:32 +01:00
Matthias Radestock fd6d15e712 eliminate misleading flag parsing error
The standard mflags error reporting produces s.t. like
```
flag provided but not defined: -p
See '/home/weave/weaver --help'.
```
on stderr. This is misleading since the program name is container
local, so not something the user actually invoked.

Fixes #1321.
2015-08-20 17:19:48 +01:00
Matthias Radestock f399c3a888 Merge branch '1.0'
Conflicts:
	ipam/ring/ring_test.go
2015-08-20 16:32:09 +01:00
Matthias Radestock 79d1c404a7 Merge pull request #1342 from weaveworks/1336-fix-ring-fuzztest
Fix ring fuzz-test

Fixes #1336.
2015-08-20 16:25:50 +01:00
Adam Harrison 1e83e7a95c Merge branch '1.0' 2015-08-20 15:33:59 +01:00
Adam Harrison 086ab9d126 Correct MaxUDPPacketSize constant 2015-08-20 15:22:10 +01:00
Tom Wilkie 8a7821a15b Publish latest images from CI on successful builds. 2015-08-20 14:20:32 +00:00
Bryan Boreham 135c34a45a Don't remove peer from table until after we've copied its entries out 2015-08-20 13:11:51 +01:00
Bryan Boreham ca6e54890c Fix Transfer() when removing a peer, so we actually target the other peer. 2015-08-20 13:11:51 +01:00
Tom Wilkie dac219b64a Merge pull request #1339 from weaveworks/revert-1338-1336-check-len
Revert "In ipam ring unit test, check if we have any peers with entri…
2015-08-20 12:23:06 +01:00
Tom Wilkie 790a1757e0 Revert "In ipam ring unit test, check if we have any peers with entries to avoid crash " 2015-08-20 12:18:34 +01:00
Tom Wilkie aafa0f6783 Merge pull request #1338 from weaveworks/1336-check-len
In ipam ring unit test, check if we have any peers with entries to avoid crash
2015-08-20 12:15:27 +01:00
Bryan Boreham df5426dd8b Check if we have any peers with entries, to avoid crash 2015-08-20 11:57:56 +01:00
Adam Harrison 3803d79835 Merge pull request #1335 from weaveworks/1328_handle_invalid_status_sub_command
better error handling for invalid `weave status` sub-command
2015-08-20 11:48:01 +01:00
Matthias Radestock ceff3bed28 better error handling for invalid weave status sub-command
instead of showing '404 page not found'

Fixes #1328.
2015-08-20 11:14:59 +01:00
Matthias Radestock eebdbe226b Merge pull request #1325 from weaveworks/1322_handle_usage_locally
deal with `weave --help|help` locally

Fixes #1322.
2015-08-20 10:13:45 +01:00
Matthias Radestock 8ab639ddaa deal with weave --help|help locally
thus making them unaffected by docker breakages

Fixes #1322.
2015-08-20 10:03:21 +01:00
Matthias Radestock 812579a139 Merge pull request #1326 from weaveworks/576_better_version_check
better docker version check

Fixes #576.
2015-08-19 21:31:43 +01:00
Matthias Radestock 08f31886d6 better docker version check
Fixes #567. Supersedes #1298.
2015-08-19 21:20:14 +01:00
Matthias Radestock f080e5aa5c Merge pull request #1331 from weaveworks/clarify-test-160
clarify test 160
2015-08-19 20:41:59 +01:00
Matthias Radestock d61c719e45 cosmetic 2015-08-19 20:39:16 +01:00
Bryan Boreham 4d49bf3ab9 Add comment to clarify the reason why there is no explicit 'assert' test at the end. 2015-08-19 16:54:24 +01:00
Matthias Radestock f34f837106 Merge branch '1.0'
Conflicts:
	weave
2015-08-19 16:38:55 +01:00
Matthias Radestock 1326994a6d Merge pull request #1319 from weaveworks/dns-tcp
Expose dns server on tcp as well as udp

Fixes #1333.
2015-08-19 16:34:09 +01:00
Matthias Radestock 99ab0d3abc Merge branch '1.0' 2015-08-19 11:29:23 +01:00
Matthias Radestock e61f9afb72 Merge pull request #1330 from weaveworks/1329_plug_gossip_sender_leak
plug GossipSender leak

Fixes #1329.
2015-08-19 11:26:25 +01:00
Matthias Radestock 29c64b1f40 document weave env --restore
leftover from #1327
2015-08-19 11:20:10 +01:00
Bryan Boreham db51b772b4 Merge pull request #1327 from weaveworks/1288_restore_env
Closes #1288
2015-08-19 10:54:43 +01:00
Matthias Radestock 0b764a2a27 plug GossipSender leak
GossipChannel.sendDown is the (only) place which creates
GossipSenders. It has two call sites, Send and SendDown, only the
former of which performed a gc of existing senders. Therefore, in
configurations with

- a lot of connection churn, and
- SendDown invocations for these connections, and
- few or no Send invocations

we can leak GossipSenders.

The most likely scenario for this to occur is a peer that, while
having no other connections, is making/accepting connections which get
torn down post handshake, e.g. because IPAM detected a clash.

Fixes #1329.
2015-08-19 10:22:11 +01:00
Matthias Radestock a93a3e98b9 warn when stopping proxy when the env is using it 2015-08-18 18:08:02 +01:00
Matthias Radestock be94dcc8a3 introduce weave env --restore
`weave env` stashes the existing DOCKER_HOST in ORIG_DOCKER_HOST, and
`weave env --restore` sets DOCKER_HOST to $ORIG_DOCKER_HOST.

Closes #1288.
2015-08-18 17:20:21 +01:00
Matthias Radestock a34b31ad12 refactor: make proxy_addr more single purpose 2015-08-18 15:40:58 +01:00
Matthias Radestock e1d07aa264 cosmetic 2015-08-18 15:34:43 +01:00
Matthias Radestock 0d79a8fc43 cosmetic 2015-08-18 15:08:59 +01:00
Matthias Radestock 5d1411830b Merge branch '1.0' 2015-08-18 13:33:36 +01:00
Matthias Radestock dc91ce9ec6 Merge branch 'latest_release_doc_updates' into 1.0 2015-08-18 13:33:15 +01:00
Matthias Radestock 36a857f343 remove "Contact information" link on docs home page
It's broken, and we have a link in the footer anyway.

Fixes #1260.
2015-08-18 13:31:28 +01:00
Matthias Radestock c1c5450caf more idiomatic logger initialisation, with standard default log level 2015-08-18 12:04:48 +01:00
Bryan Boreham 0b4e8b8754 Merge pull request #1318 from weaveworks/1305_suppress_self_reconnection
lgtm. Fixes #1305.
2015-08-18 11:14:15 +01:00
Matthias Radestock f73dcba306 eliminate spurious warning when --ipalloc-range=""
Fixes #1320.
2015-08-18 09:50:42 +01:00
Ilya Dmitrichenko 147a537b82 We are not using gcloud, but it cause a conflict
https://s3.amazonaws.com/archive.travis-ci.org/jobs/76062893/log.txt
2015-08-18 09:47:37 +01:00
Ilya Dmitrichenko a4844b8ee7 We are not using gcloud, but it cause a conflict
https://s3.amazonaws.com/archive.travis-ci.org/jobs/76062893/log.txt
2015-08-18 09:46:09 +01:00
Ilya Dmitrichenko acde6cb0ee We are not using gcloud, but it cause a conflict
https://s3.amazonaws.com/archive.travis-ci.org/jobs/76062893/log.txt
2015-08-18 09:40:03 +01:00
Paul Bellamy b4e13215c7 expose dns server on tcp as well as udp
issue was discovered when testing scope with master vs 1.0 branches.
2015-08-18 09:35:53 +01:00
Matthias Radestock 4ecaa0f508 don't put blank lines around sub-status
not what I had in mind, and it breaks the tests
2015-08-18 09:00:01 +01:00
Matthias Radestock 86b7e4a6db Merge branch '1.0' 2015-08-18 08:50:41 +01:00
Matthias Radestock aa19028583 Merge branch 'latest_release_doc_updates' into 1.0 2015-08-18 08:50:19 +01:00
Matthias Radestock a6e2296e0a Merge branch '1.0' into latest_release_doc_updates 2015-08-18 08:49:31 +01:00
Matthias Radestock 59859c4aed clarify allocation subnet overlap failure
Closes #1317.
2015-08-18 01:50:30 +01:00
Matthias Radestock 0e85783257 cosmetic: add blank lines around weave status output
Closes #1315.
2015-08-18 01:38:51 +01:00
Matthias Radestock d8e8cd6e51 don't reconnect to ourself
When we discover that a connection target address leads to ourself we
set the retry time to the zero time.Time value - the only special
time.Time value. We use that marker to a) suppress subsequent
connection attempts, and b) display the next retry as 'never' in
`weave status|report`.

Closes #1305.
2015-08-18 00:51:33 +01:00
Matthias Radestock bbc91c1273 Merge pull request #1307 from weaveworks/1306-dns-recurse-compress
Compress recursive answers if they are too big.

Fixes #1306.
2015-08-17 19:23:46 +01:00
Tom Wilkie 67f974923b Test for recursive DNS queries & compression.
Note this doesn't test the golang resolver, where the original error
was found, as I can't find a way to point the golang resolver at a
custom dns server during a test.  It does check that a compressed
response from an upstream server is correctly handled.
2015-08-17 18:25:36 +01:00
Tom Wilkie 4f00dab4c4 Compress recursive answers if they are too big. 2015-08-17 18:25:26 +01:00
Matthias Radestock 74e29c8990 Merge branch '1.0'
Conflicts:
	nameserver/server.go
	nameserver/server_test.go
2015-08-17 18:10:56 +01:00
Paul Bellamy 74883c5da8 Merge pull request #1310 from weaveworks/improve_proxy_ignore_messages
Improve log messages for when the proxy ignores a container
2015-08-17 16:27:45 +01:00
Matthias Radestock 375af0a7b9 Improve log messages for when the proxy ignores a container
- 'ignoring' could be misread as 'dropped request', so change that to
  'leaving container alone'
- more elaborate reasons
2015-08-17 16:15:45 +01:00
Matthias Radestock 8c9679b19c Merge branch '1.0'
Conflicts:
	circle.yml
2015-08-17 15:01:44 +01:00
Alvaro 0a80fdcc1d Merge pull request #1312 from weaveworks/circle
Add sudo apt-get update to circle.yml
2015-08-17 13:52:00 +02:00
Tom Wilkie 5fbcb3f905 Add sudo apt-get update to circle.yml 2015-08-17 08:49:40 +00:00
Matthias Radestock 067f96023a correct option name in log message 2015-08-17 09:37:46 +01:00
Matthias Radestock 9de7c9ed0d Merge pull request #1303 from weaveworks/1302-proxy-net-none
[proxy] attach containers with --net=none to weave

Fixes #1302.
2015-08-14 14:03:36 +01:00
Paul Bellamy 0de292c0d4 [proxy] attach containers with --net=none to weave 2015-08-14 13:48:32 +01:00
Matthias Radestock eaa5385069 refactor: use new proxy_start_container function 2015-08-13 19:54:00 +01:00
Matthias Radestock fa2aa1f791 Merge branch '1.0'
Conflicts:
	test/640_proxy_restart_reattaches_test.sh
2015-08-13 19:46:23 +01:00
Matthias Radestock 78df99e7e7 Merge branch '1.0' 2015-08-13 14:37:31 +01:00
Matthias Radestock 26f4a41900 Merge branch '1.0' 2015-08-13 14:07:22 +01:00
Matthias Radestock 686500b181 Merge branch '1.0' 2015-08-13 14:05:35 +01:00
Matthias Radestock d5de0b41f9 Merge branch '1.0'
Conflicts:
	proxy/proxy.go
2015-08-13 13:57:02 +01:00
Matthias Radestock 2bb84ae28b Merge branch '1.0'
Conflicts:
	weave
2015-08-13 13:18:55 +01:00
Matthias Radestock e319f54d3f Merge branch '1.0'
Conflicts:
	site/proxy.md
2015-08-13 11:36:01 +01:00
Tom Wilkie cdab6a71b7 Merge pull request #1289 from weaveworks/move-listvar
Move ListVar into common, so it can be reused elsewhere.
2015-08-12 12:56:25 +01:00
Tom Wilkie 45293fdcac Move ListVar into common, so it can be reused elsewhere. 2015-08-12 11:35:57 +00:00
Matthias Radestock 7656752d87 unbreak test
caused by faulty merge
2015-08-12 07:40:20 +01:00
Matthias Radestock d0ed225184 Merge branch '1.0'
Conflicts:
	weave
2015-08-11 15:01:54 +01:00
Adam Harrison c2147f80b7 Merge pull request #1283 from weaveworks/1185_report_pcap_stats
add pcap stats to `weave report`
2015-08-11 13:09:34 +01:00
Matthias Radestock f1b9cf4fc0 Merge pull request #1282 from weaveworks/1281-dns-remove-dead-quick
Remove DNS entries for dead containers when adding correctly.

Fixes #1281.
2015-08-11 13:08:46 +01:00
Matthias Radestock c1d799a06c add pcap stats to weave report
Closes #1185.
2015-08-11 12:57:52 +01:00
Tom Wilkie 6a5044fc70 Make nameserver logging more consistent. 2015-08-11 11:32:19 +00:00
Tom Wilkie 7fa568872c Ensure we delete the fqdn is the container fails to start. 2015-08-11 11:32:17 +00:00
Adam Harrison f3ad81d554 Merge pull request #1248 from weaveworks/issues/1025-enhanced-weave-status
Enhanced weave status
2015-08-11 12:29:29 +01:00
Adam Harrison fc9a1c8664 new weave status, weave status ..., and weave report 2015-08-11 12:15:13 +01:00
Adam Harrison 00ecff09f7 Move status HTTP handlers into prog/weaver/http.go 2015-08-11 12:01:27 +01:00
Adam Harrison 278807a199 Move router HTTP handlers into router/http.go 2015-08-11 12:01:27 +01:00
Matthias Radestock 8bce900170 Merge pull request #1247 from weaveworks/test_frag
add test for PMTU discovery and packet fragmentation
2015-08-10 22:02:07 +01:00
Matthias Radestock cf6d6d0cd3 add test for PMTU discovery and packet fragmentation 2015-08-10 17:50:21 +01:00
Matthias Radestock 94af02fbf4 Merge branch '1.0'
Conflicts:
	weave
2015-08-10 17:37:06 +01:00
Matthias Radestock 755a949f6d Merge pull request #1277 from weaveworks/1275-negative-dns
Return NXDOMAIN when we don't have entries

and drop the non-authoritative special case for recursive dns.

Fixes #1275.
2015-08-10 17:19:44 +01:00
Tom Wilkie 42debde746 Merge pull request #1259 from weaveworks/1257-proxy-chunking
Proxy chunking should handle arbitrarily large chunks
2015-08-10 15:57:19 +01:00
Tom Wilkie c2d5b35cd6 Add test for negative DNS queries 2015-08-10 13:13:12 +00:00
Tom Wilkie 073836905a Return NXDOMAIN when we don't have entries, and drop the non-authoritative special case for recursive dns. 2015-08-10 12:19:21 +00:00
Matthias Radestock fd2a59417d cosmetic 2015-08-10 12:39:00 +01:00
Paul Bellamy 6d7e784c25 review feedback 2015-08-10 11:02:51 +01:00
Matthias Radestock b2a24d1cbe Merge branch '1.0'
Conflicts:
	nameserver/zone_lookup.go
2015-08-10 10:27:10 +01:00
Tom Wilkie 0a2701bbbc Merge pull request #1268 from weaveworks/1243_fix_claim_test
add a sleep to improve test reliability
2015-08-10 10:23:48 +01:00
Matthias Radestock 1a11b989e7 simplify test 2015-08-07 14:40:34 +01:00
Matthias Radestock 010792f714 cosmetic: more sensible order of weave commands 2015-08-07 11:44:47 +01:00
Matthias Radestock 536369bc12 Merge pull request #1265 from weaveworks/1252-proxy-non-root-hosts
[proxy] rewrite /etc/hosts from outside the container

Fixes #1252.
2015-08-06 13:30:02 +01:00
Paul Bellamy f55655216f [proxy] rewrite /etc/hosts from outside the container
Avoids problems when container user is non-root. Added a test for this
and the --no-rewrite-hosts flag.
2015-08-06 13:14:18 +01:00
Matthias Radestock 43ef3b8515 give routers some time to establish connectivity 2015-08-06 11:54:58 +01:00
Paul Bellamy 3180c67151 -no-rewrite-hosts -> --no-rewrite-hosts 2015-08-06 11:01:35 +01:00
Adam Harrison 457e162d0c Merge branch '1.0'
Conflicts:
	site/ipam.md
2015-08-06 09:56:50 +01:00
Tom Wilkie f9127e90db Merge pull request #1263 from weaveworks/integration-tests-no-save-file
don't save the python test file when running test/600
2015-08-05 18:50:22 +01:00
Paul Bellamy 94db0beb25 don't save the python test file when running test/600 2015-08-05 18:04:49 +01:00
Adam Harrison 77996158ef Merge branch '1.0' 2015-08-05 14:42:09 +01:00
Matthias Radestock 6cc7749cf2 cosmetic: remove debug logging from test 2015-08-05 12:31:20 +01:00
Matthias Radestock c47e57cc29 cosmetic 2015-08-05 11:55:11 +01:00
Matthias Radestock de6f5a93e8 refactor: extract build flags 2015-08-05 11:11:57 +01:00
Paul Bellamy 157910ec9a adding integration test for proxy handling large http chunks 2015-08-05 10:52:06 +01:00
Paul Bellamy b85b90ef4e Proxy chunking should handle arbitrarily large chunks 2015-08-04 17:47:51 +01:00
Paul Bellamy 1d1ebc103d Merge remote-tracking branch 'upstream/1.0' 2015-08-04 14:29:12 +01:00
Matthias Radestock 9c22066bdb Merge pull request #1256 from weaveworks/1245-dns-loadbalancing
Add note to explain DNS shuffle effects.

Fixes #1245.
2015-08-04 12:22:23 +01:00
Tom Wilkie f241e15ec1 Review feedback 2015-08-04 11:00:32 +00:00
Tom Wilkie 881ece3553 Add note to explain dns shuffle effects. 2015-08-04 10:47:36 +00:00
Michael Bridgen 281bf02871 Merge branch 'pr1254' 2015-08-03 15:26:08 +01:00
Michael Bridgen 778038fdd3 Merge branch 'fix-dotted-imports-for-log' of https://github.com/dpw/weave into pr1254 2015-08-03 14:29:53 +01:00
David Wragg e7ee72836d Log -> log 2015-08-03 14:13:24 +01:00
David Wragg 31c0506b85 Eliminate dotted imports of common for the sake of Log 2015-08-03 14:07:16 +01:00
David Wragg 7d6f85424c Move OnOff into router.go 2015-08-03 14:06:23 +01:00
Tom Wilkie ca89cfee3d Merge pull request #1250 from paulbellamy/testrunner-summarize-fails
Some usability tweaks to the test runner
2015-07-31 12:36:50 +01:00
Paul Bellamy 5a37f652c2 add -v to testrunner, and less output in happy path 2015-07-31 12:25:50 +01:00
Paul Bellamy cc7b781bb3 have testrunner summarize failed tests so they are easier to find 2015-07-31 12:12:33 +01:00
Paul Bellamy 55051f0194 Merge remote-tracking branch 'upstream/1.0' 2015-07-31 11:01:19 +01:00
Paul Bellamy b7e8938ab4 scope the test runner call so scope works 2015-07-31 10:49:21 +01:00
Matthias Radestock f8327c3c6c expand test coverage to unicast relaying
by way of an ipam space request from host3 to host1, via host2.
2015-07-30 20:38:00 +01:00
Paul Bellamy e3851e6e55 Merge pull request #1246 from weaveworks/gce-parameterize
parameterize test/gce.sh so scope can override TEMPLATE_NAME and NUM_HOSTS
2015-07-30 16:18:59 +01:00
Matthias Radestock 6a136eebb2 make parallel test running work locally
@tomwilkie assures me this change won't break CircleCI. We shall see.

This now works for me:

make clean
make COVERAGE=true
COVERDIR=test/coverage SLOW=true ./test/units.sh
cd test
./setup.sh
COVERAGE=true PARALLEL=true ./run_all.sh
./gen_coverage_reports.sh
2015-07-30 15:57:49 +01:00
Paul Bellamy 9396c046cd parameterize test/gce.sh so scope can override TEMPLATE_NAME and NUM_HOSTS 2015-07-30 15:48:22 +01:00
Paul Bellamy dccd199369 Merge pull request #1242 from weaveworks/1239-600-docker-py
Shard 600_proxy_docker_py into 4.
2015-07-30 13:14:54 +01:00
Tom Wilkie a87be89095 Typos. Honestly 2015-07-30 11:07:17 +00:00
Tom Wilkie 5c9a13d343 Shard 600_proxy_docker_py into 4. 2015-07-30 10:38:02 +00:00
Tom Wilkie bfe07fc71a Give each shard 5 VMs 2015-07-30 10:32:16 +00:00
Tom Wilkie fa95b67c92 Add summary to bottom of test runner output.
Also, don't sort tests by #hosts when run sequentially, and fix typo in run_all.sh.

Fixes #1240
2015-07-30 10:29:05 +00:00
Adam Harrison 728203912e .gitignore Vim recovery files 2015-07-30 10:53:59 +01:00
Paul Bellamy 27289e7b7b Merge pull request #1236 from weaveworks/1169-frugal-frugal-frugal
Run integration tests in parallel when possible.
2015-07-30 10:16:39 +01:00
Tom Wilkie 669a0db528 Run integration tests in parallel when possible. 2015-07-29 17:28:58 +00:00
Paul Bellamy 4da999ba86 Merge pull request #1235 from weaveworks/gce
Improve CI run time
2015-07-29 17:38:30 +01:00
Tom Wilkie 3bffd871b8 Merge pull request #1230 from weaveworks/1154-proxy-coverage
Add coverage to proxy for integration tests.
2015-07-29 13:24:21 +01:00
Adam Harrison 2d450ac39d Merge pull request #1225 from weaveworks/21-dns-listen-addr
Allow users to set dns listen address
2015-07-29 13:18:10 +01:00
Tom Wilkie 6fe69590c4 Add coverage to proxy for integration tests. 2015-07-29 12:06:29 +00:00
Tom Wilkie 4df3d7f150 Reduce the amount of parallelism in the ipam tests 2015-07-29 10:13:02 +00:00
Tom Wilkie 51128a6705 Parallelise the copying of /etc/hosts. 2015-07-29 09:58:51 +00:00
Tom Wilkie e7676979ee Parallelise the copying of imaages etc 2015-07-29 09:58:05 +00:00
Tom Wilkie da36c78942 Reduce the number of times gcs.sh ssh's into the VMs 2015-07-29 09:41:57 +00:00
Matthias Radestock 77358140d9 Merge pull request #1219 from squaremo/driveby
Remove misleading comment and lowercase deniedBy
2015-07-28 18:09:01 +01:00
Adam Harrison 1701bc3e92 Merge branch '1.0'
Conflicts:
	weave
2015-07-28 18:06:44 +01:00
Tom Wilkie db6ee9954c Allow users to set dns listen address so we can run the weave router in the host net namespace. 2015-07-28 14:49:09 +00:00
Paul Bellamy 792c4c8de4 Merge pull request #1227 from weaveworks/gce-gc
Extend scheduler to periodically garbage collect stranded VMs
2015-07-28 15:35:28 +01:00
Matthias Radestock 820307dd4e Merge pull request #1228 from weaveworks/1030-150-connect-forget
Don't init the ring if the address is not in range.

Fixes #1030.
2015-07-28 15:29:16 +01:00
Tom Wilkie 88144ac7f8 Review feedback 2015-07-28 14:16:38 +00:00
Tom Wilkie da7ac78694 Don't try and init the ring if the IP address is not in our range. 2015-07-28 14:12:42 +00:00
Tom Wilkie 10dbd9ad4c Merge pull request #1224 from weaveworks/1203_proper_mock_gossip
proper mock periodic gossip
2015-07-28 13:29:20 +01:00
Tom Wilkie 9779e06f66 Extend scheduler to periodically garbage collect stranded VMs 2015-07-28 13:19:38 +01:00
Matthias Radestock f861d22593 do proper mock periodic gossip
instead of abusing broadcast gossip

Fixes #1203.
2015-07-28 10:37:18 +01:00
Paul Bellamy e3f0565517 Merge pull request #1221 from weaveworks/1220_include_test_dir_coverage
include directory of test code in coverage
2015-07-28 10:05:16 +01:00
Matthias Radestock 9a91b3ce9c include directory of test code in coverage
Fixes #1220.
2015-07-27 17:08:31 +01:00
Michael Bridgen c88361473c Remove misleading comment and lowercase deniedBy
* the comment "don't make the caller wait" applies to both branches
   of that if clause, and having it on one branch implies the other
   does make the caller wait
 * there's no reason for DeniedBy to start with a capital: it's a
   method on a private struct and it's not part of an interface.
2015-07-27 16:22:32 +01:00
Michael Bridgen a2e1a2d375 Merge branch 'pr1200' 2015-07-27 15:46:29 +01:00
Matthias Radestock 536ce992fc re-enable test
our switch on CircleCI from Docker 1.7.0 to 1.7.1 should make this
pass now.
2015-07-27 14:47:55 +01:00
Matthias Radestock 8d57b85a3a Merge branch 'master' into weave_1.1_readme_updates
Conflicts:
	README.md
2015-07-27 14:33:05 +01:00
Matthias Radestock ce37b622eb Merge branch '1.0' 2015-07-27 14:25:06 +01:00
Paul Bellamy ed4fb14f52 Merge pull request #1214 from weaveworks/rebuild-gce-templates
[circle] use new gce templates for docker 1.7.1
2015-07-27 13:38:44 +01:00
Matthias Radestock 32cce0c636 Merge branch 'master' into weave_1.1_readme_updates 2015-07-27 13:31:48 +01:00
Paul Bellamy 34bb7ae84a [circle] use new gce templates for docker 1.7.1 2015-07-27 12:51:12 +01:00
Michael Bridgen 5389e31190 Merge branch '687-detect-ipam-clash' of github.com:weaveworks/weave into pr1200 2015-07-27 11:54:31 +01:00
Bryan Boreham f0915221d1 Rename 'reclaim' to 'noErrorUnknown' which describes the behaviour better 2015-07-23 20:16:30 +01:00
Matthias Radestock aa4efd63ef cosmetic: uncomment test the way ordained by emacs 2015-07-22 16:55:43 +01:00
Tom Wilkie a12d63bb12 Make ipam test less stressful, so it doesn't timeout. 2015-07-22 12:53:27 +00:00
Tom Wilkie 062a25a032 Make unit test timeout 8mins. 2015-07-22 12:19:12 +00:00
Tom Wilkie 9e6d08c606 Put 2m timeout on all unit tests. 2015-07-22 11:56:39 +00:00
Alvaro 976641c3b6 Reverted: Fixed typo 2015-07-22 12:17:29 +02:00
Alvaro 2f80f12001 Fixed typo 2015-07-22 12:14:38 +02:00
Bryan Boreham 647b755b64 Check manually-assigned addresses with IPAM to see if they clash 2015-07-21 17:02:52 +01:00
David Wragg 26dc83dbc2 Merge weaveworks/519_remove_outermost_gob (PR #1189) 2015-07-21 12:01:40 +01:00
Matthias Radestock 3c107d675a Merge branch '1.0' 2015-07-21 11:42:49 +01:00
Tom Wilkie 2980586e76 Put timeout on nameserver test to try and expose #1203 2015-07-21 09:25:04 +00:00
Paul Bellamy dba116bae1 Merge remote-tracking branch 'upstream/1.0' 2015-07-21 10:13:24 +01:00
Matthias Radestock 0b7d462adf enforce a message size limit
so we do not allocate loads of memory when receiving (accidental or
deliberate) garbage
2015-07-20 23:51:04 +01:00
Matthias Radestock 6f8622a1a0 replace outermost framing on tcp connection
switch from gob to length-prefix encoding
2015-07-20 23:51:04 +01:00
Matthias Radestock af049efb40 refactor: s/SimpleTCP{Sender,Receiver}/GobTCP{Sender,Receiver} 2015-07-20 23:51:02 +01:00
Matthias Radestock ad4e4ce6b8 refactor: make 'inner' encoder/decoder explicit 2015-07-20 23:50:10 +01:00
Tom Wilkie 4c9b0be117 Merge pull request #1198 from weaveworks/1146-coverage
When running unit tests, output coverage for imported modules.
2015-07-20 20:34:12 +01:00
Tom Wilkie a9dae46110 When running unit tests, output coverage for imported modules. 2015-07-20 15:47:07 +00:00
Bryan Boreham e34093c328 Merge pull request #1197 from weaveworks/1158-flaky
LGTM
2015-07-20 16:02:52 +01:00
Tom Wilkie 9131b46263 Flush gossip messages after each turn, to ensure there isn't a build up of pending messages. 2015-07-20 14:36:40 +00:00
Michael Bridgen 2a9c3fc64a Merge branch 'pr1112' 2015-07-20 15:31:42 +01:00
Michael Bridgen c44c61dc83 Merge branch '1103-proxy-chunking' of github.com:weaveworks/weave into pr1112 2015-07-20 15:09:50 +01:00
Paul Bellamy 11663dd29f [proxy] rewrote chunked response handler
1) We cannot send "Connection: close", because the fsouza docker client
   expects the tcp socket to stay open between requests.

2) Because we cannot force-close the connection, we can't hijack the
   connection (because go's net/http doesn't let use un-hijack it).

3) Because we need to maintain the individual chunking of messages (for
   docker-py), we can't just copy the response body, as Go will remove and
   re-add the chunking willy-nilly.

Therefore, we have to read each chunk one-by-one, and flush the
ResponseWriter after each one.
2015-07-20 14:43:16 +01:00
Bryan Boreham 90b7c3625d Merge pull request #1176 from weaveworks/1156-gossip-timestamp
LGTM
2015-07-20 14:28:01 +01:00
Tom Wilkie 4e29dba5cd Make Gossip DNS reject gossip messages if clockskew > tombstoneTimeout / 2 2015-07-20 13:12:07 +00:00
Matthias Radestock f1b5a00cb3 Merge pull request #1195 from weaveworks/1177-dns-disappear
Don't periodically delete all non-tombstone entries.

Fixes #1177.
2015-07-20 13:07:42 +01:00
Tom Wilkie 1a4b38a10c Don't delete all the non-tombstone entries every 10mins! 2015-07-20 12:03:46 +00:00
Tom Wilkie 9ab2da8deb Add GET /name handler to return all dns entries (including tombstones) for debugging.
Also, make Address marshall into json as an IP string, not a number.
2015-07-20 12:03:33 +00:00
Matthias Radestock 66473e6021 refactor: return slice instead of pointer to slice from tombstone() 2015-07-20 12:37:54 +01:00
Paul Bellamy a9229b9197 weave env should still work with '--log-level=debug' 2015-07-20 11:57:20 +01:00
Paul Bellamy 7b9e59833e test upload_executable helper should mkdir before uploading 2015-07-20 10:58:39 +01:00
Tom Wilkie 5baddb83cb Comment out failing test until we figure out what to do. 2015-07-20 09:45:02 +00:00
Tom Wilkie 4e56247ed9 Merge pull request #1193 from weaveworks/disable_dns_in_multiweave
Disable dns in multiweave
2015-07-20 10:41:56 +01:00
Matthias Radestock ae2348875b disable dns in multiweave
Thus unbreaking it post #1065.
2015-07-19 22:52:42 +01:00
Matthias Radestock fafaeef07d introduce --no-dns option
undocumented, since it is mainly meant for testing
2015-07-19 22:51:40 +01:00
Matthias Radestock 5b62b8c1e3 refactor: inline populate_dns
and ditch the wait_for_status, since we've only just called that
2015-07-19 22:32:33 +01:00
Matthias Radestock 7c17c8aec3 refactor: simplify router arg parsing
using the same "always shift one" approach as in proxy_args
2015-07-19 21:56:04 +01:00
Matthias Radestock 64408b126e weave launch-router wasn't returning the correct container id
since $CONTAINER gets stomped on by various functions, in particular
populate_dns.

Broken by #1065.
2015-07-19 21:50:45 +01:00
Bryan Boreham 2a1f090a88 Merge pull request #1183 from weaveworks/1181-data-race
Correctly stop the test gossip router to prevent complaints of data races in subsequent tests.
2015-07-17 17:14:46 +01:00
Matthias Radestock e2a2a8f18f Merge branch '1.0' 2015-07-17 15:07:45 +01:00
Tom Wilkie a3d32068d2 Correctly stop the test gossip router to prevent complaints of data races in subsequent tests. 2015-07-17 13:48:39 +00:00
Bryan Boreham 68afbe31b0 Shrink image file size 2015-07-17 14:07:23 +01:00
Tom Wilkie ba3451e9ca Merge pull request #1180 from weaveworks/1171-210-failures
Don't swallow errors in the docker client.
2015-07-17 13:38:38 +01:00
Tom Wilkie 25c9180fdf Don't swallow errors in the docker client. 2015-07-17 12:35:12 +00:00
Paul Bellamy e03a4bb176 updating boot2docker docs from 1.0 to master 2015-07-17 12:55:39 +01:00
Paul Bellamy 4d594458d2 Merge branch '1.0' 2015-07-17 12:47:23 +01:00
Paul Bellamy 854d83f294 updating boot2docker docs for proxy 2015-07-17 12:41:04 +01:00
Matthias Radestock 358ae3a936 Merge pull request #1174 from weaveworks/1128-iprange-error
More helpful error message when --ip-alloc-range mismatched

Closes #1128.
2015-07-17 12:35:20 +01:00
Bryan Boreham 2dbf83889b Remove unused error messages 2015-07-17 12:19:54 +01:00
Bryan Boreham 9b2c5b8466 Improve description of merge error 2015-07-17 12:19:33 +01:00
Bryan Boreham 1f4b943454 More helpful error message when --ip-alloc-range mismatched 2015-07-17 11:42:30 +01:00
Matthias Radestock 2ca0fb93c9 Merge pull request #1173 from weaveworks/1168-checkandpanic
Make nameserver.checkAndPanic look for dups, call it more often.

Fixes #1168.
2015-07-17 11:37:34 +01:00
Matthias Radestock 47273d424f Merge pull request #1170 from weaveworks/exec-dockerfile
Re-arrange weaveexec Dockerfile to improve download times
2015-07-17 11:36:15 +01:00
Bryan Boreham f4d6e028d4 Simplify 2015-07-17 11:33:29 +01:00
Tom Wilkie f10e0e4add Merge pull request #1159 from weaveworks/1150-claim-from-peer
Implement claiming addresses from other peers
2015-07-17 11:21:48 +01:00
Tom Wilkie 66e58937ca Make nameserver.checkAndPanic look for dups, call it more often. 2015-07-17 10:19:25 +00:00
Matthias Radestock 62c1cad676 Merge remote-tracking branch 'upstream/master' 2015-07-17 10:58:34 +01:00
Matthias Radestock ca523c716c cosmetic 2015-07-17 10:58:01 +01:00
Tom Wilkie e2deff09a6 Nameserver: log when peer goes away. 2015-07-17 09:36:44 +00:00
Ilya Dmitrichenko 19ddb65036 New screencast URL and image 2015-07-16 18:13:34 +01:00
Bryan Boreham 61c7c034e7 Re-arrange weaveexec Dockerfile to improve download times 2015-07-16 17:02:00 +01:00
Bryan Boreham 4c5920a50a Add fuzz-test for Claim 2015-07-16 16:12:23 +01:00
Bryan Boreham c3c40b15f1 Avoid hanging the caller when we try to claim from a dead peer 2015-07-16 16:12:23 +01:00
Bryan Boreham 2fa65f0a22 Smoke-test for claim operation 2015-07-16 16:12:23 +01:00
Bryan Boreham af7ca99dc4 Extend unit-tests for claim operation 2015-07-16 16:12:23 +01:00
Bryan Boreham 2d224d423d Add SpaceRequestDenied message 2015-07-16 16:12:23 +01:00
Bryan Boreham e7088e5daf Extract decodeRange function 2015-07-16 16:12:23 +01:00
Bryan Boreham d65409e97e Improve comment & log for deferred claim 2015-07-16 16:12:23 +01:00
Matthias Radestock 70110e483c Merge pull request #1166 from weaveworks/dns-tests
Make 210_dns_cross_hosts_test.sh excercise gossip dns tombstones.
2015-07-16 16:11:58 +01:00
Paul Bellamy a0876becbc Merge remote-tracking branch 'upstream/1.0'
Conflicts:
	proxy/create_container_interceptor.go
	proxy/create_exec_interceptor.go
	proxy/proxy.go
	proxy/start_container_interceptor.go
2015-07-16 15:29:29 +01:00
Matthias Radestock aa3f35e38c remove stale reference to weave launch-dns 2015-07-16 14:41:57 +01:00
Tom Wilkie db77570779 Test is too long 2015-07-16 13:40:39 +00:00
Tom Wilkie 6e5995964d Make 210_dns_cross_hosts_test.sh excercise gossip dns tombstones.
Also, make gce.sh less verbose, and do the env dance correctly.
2015-07-16 12:48:00 +00:00
Adam Harrison da13238368 Merge branch '1.0' 2015-07-16 13:45:36 +01:00
Paul Bellamy 0865f4c207 Merge pull request #1099 from weaveworks/1055-proxy-autoconfig
Proxy autoconfig
2015-07-16 13:33:21 +01:00
Paul Bellamy a4a31e632b [proxy] Choose where to listen based on how launching client connected 2015-07-16 12:44:39 +01:00
Matthias Radestock 23ad547db4 Merge pull request #1155 from weaveworks/dns-gossip
Make DNS gossip converge correctly

Fixes #1151
2015-07-15 20:37:43 +01:00
Matthias Radestock 4d887c8a55 Merge branch '1.0'
Conflicts:
	proxy/create_exec_interceptor.go
	test/610_proxy_wait_for_weave_test.sh
2015-07-15 18:07:11 +01:00
Tom Wilkie 73210b6a70 Make sure DNS Gossip converges correctly.
- Abide by gossip interface (return nil, not empty gossip) so gossip converges!
- Make sure we don't say an old entry version is new to us; add some more tests.
- Add dns test for 3 (or N) hosts
- Extend mock gossip router to ensure reconsumption off diffs yield nil
- Don't increment version when re-adding existing entry.
2015-07-15 15:54:52 +00:00
Tom Wilkie 9df7358ded Add pprof handler to weaver. 2015-07-15 15:10:22 +00:00
Matthias Radestock 9507ea93dd rename test since we have two 250_* 2015-07-15 14:46:51 +01:00
Matthias Radestock 52d3bcfe8b Merge pull request #1149 from weaveworks/1118-respond-late-paxos
Respond to paxos messages when we already have a ring

Fixes #1118.
2015-07-15 13:24:40 +01:00
Bryan Boreham b3979c9505 Respond to paxos message if we already have a ring 2015-07-15 13:05:25 +01:00
Bryan Boreham 273a0fba47 Add sender to mock broadcast messages 2015-07-15 13:05:25 +01:00
Alvaro 7e34ec61f6 Merge pull request #1142 from weaveworks/1113-recursive-dns
Closes #1113
2015-07-15 13:42:59 +02:00
Bryan Boreham acf13dc157 Add sender to OnGossipBroadcast() 2015-07-15 12:07:56 +01:00
Tom Wilkie 91395d6728 Use Msg.Copy! 2015-07-15 11:05:36 +00:00
Paul Bellamy ef4a227ae9 Merge pull request #1144 from weaveworks/restart-docker-after-vagrant-tweaks
Restart docker after vagrant tweaking and refactor tweak-code
2015-07-15 11:34:59 +01:00
Alfonso Acosta 25589562cd Restart docker after vagrant tweaking and refactor tweak-code
Docker needs to be restarted and not started, since it already starts running after installation
2015-07-15 09:31:47 +00:00
Matthias Radestock 7063ca51aa remove dead code 2015-07-15 05:35:05 +01:00
Matthias Radestock 97498d26f7 remove dead code 2015-07-15 05:32:35 +01:00
Matthias Radestock 36292216fd rename tests so they actually run
These tests were introduced in #1065 and #1126.
2015-07-14 19:47:15 +01:00
Matthias Radestock 86448a1025 fix breakage
This wasn't spotted because #1065, which removed launch-dns, got
merged just before #1126, which introduced this test.
2015-07-14 19:42:59 +01:00
Ilya Dmitrichenko b4f53edd91 Merge pull request #1138 from weaveworks/vagrantfile-update
update default vagrant box to ubuntu 15
2015-07-14 18:21:02 +01:00
Matthias Radestock 3a22d2e6a1 Merge pull request #1135 from weaveworks/issues/1028-weave-dns-lookup
Implement `weave dns-lookup`

Closes #1028.
2015-07-14 18:19:26 +01:00
Matthias Radestock 9a4bbb55da Merge branch '1.0'
Conflicts:
	proxy/common.go
	weave
2015-07-14 18:09:25 +01:00
Tom Wilkie fa73240812 Copy dns requests and give them a fresh id when doing recursive queries. 2015-07-14 16:44:21 +00:00
Adam Harrison a3dc2d0482 Smoketest dns-lookup 2015-07-14 17:36:30 +01:00
Adam Harrison 41834478d7 Rework documentation to use weave dns-lookup 2015-07-14 17:36:30 +01:00
Adam Harrison 6832ae20b0 Implement weave dns-lookup 2015-07-14 17:36:30 +01:00
Matthias Radestock 7ae06ef00d Merge pull request #1126 from weaveworks/issues/1018-more-flexible-hostname-derivation
[proxy] More flexible container name -> hostname derivation

Closes #1018.
2015-07-14 17:31:53 +01:00
Alvaro a1ac0313e5 Merge pull request #1065 from weaveworks/gossipdns
Gossip-based DNS embedded in the router.
2015-07-14 18:30:16 +02:00
Paul Bellamy d9c5d2f940 update default vagrant box to ubuntu 15 2015-07-14 17:08:26 +01:00
Tom Wilkie 361ed1b4a6 Resolve unqualified domains in the weave.local domain. 2015-07-14 16:05:41 +00:00
Alfonso Acosta dfcc9678f3 Make weave usage fit into 80 columns 2015-07-14 16:02:19 +00:00
Tom Wilkie 8b061fd555 Gossip-based DNS embedded in the router.
- Nameserver holds a sorted list of (hostname, peer, container id, ip) tuples.
- Entries are gossiped using topology gossip / broadcast - only new/updated entries
  are propogated as expected.
- Entries for a given peer are removed when the connection to the peer is lost.
- Entries are tombstoned when containers die.  Tombstones are kept for 10mins,
  as they do not need to survive partition, only gossip propagation delay.
2015-07-14 16:00:38 +00:00
Tom Wilkie 01f7201a9e Remove legacy WeaveDNS code. 2015-07-14 16:00:28 +00:00
Alfonso Acosta ded3d6b40b Address PR comments 2015-07-14 15:31:44 +00:00
Tom Wilkie f7739ef04c Don't fail if docker rmi fails 2015-07-14 14:50:39 +00:00
Alfonso Acosta 4f82dc97fd [proxy] More flexible container name -> hostname derivation
Provides to new two new flags: `--hostname-match <regexp>`
and `--hostname-replacement <replacement>`. When launching a container, its name
matched against regular expression `<regexp>`. Then, based on that match,
`<replacement>` will be used to generate a hostname, which will ultimately be
handed over to weaveDNS for registration.

The substitution is implemented with
https://golang.org/pkg/regexp/#Regexp.ReplaceAllString . Thus, references to
regexp substitution groups should be prepended with a dollar sign.

For instance, if we provide `--hostname-match '^aws-[0-9]+-(.*)$'` and
`--hostname-replacement 'my-app-$1'`, running a container named
`aws-12798186823-foo` through WeaveProxy will lead to WeaveDNS registering
`my-app-foo`.

Closes #1018
2015-07-14 14:41:44 +00:00
Matthias Radestock 16471d0a28 Merge branch '1.0'
Conflicts:
	site/features.md
	test/110_encryption_test.sh
2015-07-14 15:41:10 +01:00
Matthias Radestock 0abd134a0d Merge pull request #1133 from weaveworks/revert-1050-issues/987
Revert "Resolve unqualified names in the local WeaveDNS domain"
2015-07-14 15:06:19 +01:00
Matthias Radestock 18032015b9 Revert "Resolve unqualified names in the local WeaveDNS domain" 2015-07-14 16:06:05 +01:00
Matthias Radestock 39a1f2163f Merge pull request #1050 from weaveworks/issues/987
Resolve unqualified names in the local WeaveDNS domain

Closes #987.
2015-07-14 14:43:29 +01:00
Alvaro Saurin 4c519fc761 Smoke tests 2015-07-14 15:12:40 +02:00
Matthias Radestock bbacaa42b8 cosmetic 2015-07-14 13:23:28 +01:00
Matthias Radestock 2f35ad1d42 Merge pull request #1098 from dpw/encrypt-features
New connection protocol version with encrypted features map

Closes #1029. Closes #912.
2015-07-14 13:20:18 +01:00
Bryan Boreham 6d17fe5df5 Need to restart docker after systemd reconfigure 2015-07-14 12:12:35 +01:00
Bryan Boreham 907de67a40 Merge pull request #1125 from weaveworks/smoke-overlay
Upgrade smoke-tests to use 'overlay' storage and Ubuntu 15.04
2015-07-14 11:51:53 +01:00
Tom Wilkie 499d65e715 Build cover tool by default, and do go get -t for tests, irregardless of $SLOW 2015-07-14 10:50:34 +00:00
Alvaro Saurin efb0ead9ad Resolve unqualified names in the local WeaveDNS domain
Unit tests
2015-07-14 10:46:41 +02:00
Matthias Radestock 85b4a05c9a Merge branch '1.0' 2015-07-13 19:07:47 +01:00
Bryan Boreham 4853a1da05 Tell Docker to use 'overlay' storage driver 2015-07-13 18:28:29 +01:00
Bryan Boreham db8016112e Upgrade to Ubuntu 15.04 2015-07-13 18:28:11 +01:00
Tom Wilkie 4eb6d147b5 Merge pull request #1121 from weaveworks/test_large_packets
test that large packets can be transmitted over weave
2015-07-13 17:06:28 +01:00
Tom Wilkie 1e39b2bdcf Merge pull request #1123 from weaveworks/remove-danglies
Remove Docker dangling images in smoke-tests
2015-07-13 14:38:04 +01:00
Adam Harrison d6443809bd Merge pull request #1120 from weaveworks/998-race
Various improvements to unit test running
2015-07-13 14:09:45 +01:00
Bryan Boreham 80e56d67db Remove Docker dangling images after copying new ones 2015-07-13 14:03:47 +01:00
Tom Wilkie 24a4d6501e Merge pull request #1117 from weaveworks/multi_hop_tests
Multi hop tests
2015-07-13 11:39:06 +01:00
Bryan Boreham 5e43f0fd4c Merge pull request #1079 from weaveworks/68-rewrite-etc-hosts
LGTM.  Closes #68, with remainder to #1122
2015-07-13 11:03:20 +01:00
Tom Wilkie 13f1239a3e Various improvements to unit test running
- units.sh should exit on error.
- Add race detector to unit test runner.
- Use local testing/cover/cover tool in gen_coverage_report.sh and units.sh
- Use the scheduler to parallelise the unit tests.
- Make test run id a string and differentiate between unit runs and integration runs in the scheduler.
- Pipe circle env vars through to build container so we can detect when to use the scheduler
- Add python requests to the weave build container.
- Quote test names in urls, make flask handle url segments with slashes in.
2015-07-13 09:33:52 +00:00
Matthias Radestock 9463ceb873 test that large packets can be transmitted over weave 2015-07-13 10:31:20 +01:00
Matthias Radestock 7aab5cc5b4 refactor: simplify NewPeers 2015-07-11 17:46:57 +01:00
Matthias Radestock 2b61bc91df cosmetic: remove blank log lines 2015-07-11 16:43:33 +01:00
Matthias Radestock 57a4371e9f attempt to cure spurious test failures 2015-07-11 16:31:16 +01:00
Matthias Radestock 1d2e6b4c56 test gossip forwarding 2015-07-11 16:17:36 +01:00
Matthias Radestock 2a0bc91f86 add test for peer discovery and multi-hop routing 2015-07-11 14:57:24 +01:00
Matthias Radestock 210b926064 increase number of test VMs from 2 to 3 2015-07-11 14:57:24 +01:00
Matthias Radestock 9a089a111f don't invoke weave stop-router at end of tests
since that removes the container, rather than just stopping it.
2015-07-11 14:56:11 +01:00
Matthias Radestock 710eabeff3 reduce test vm memory 2015-07-11 12:15:46 +01:00
Matthias Radestock 603f34e944 oops, forgot to commit this 2015-07-11 12:14:43 +01:00
Matthias Radestock a3dff7e1e9 cope with more than two test hosts 2015-07-11 11:59:34 +01:00
Matthias Radestock 25486f7b4d cosmetic: s/weave_stop_router/stop_router_on
Also run `weave stop-router` as part of this, so conntrack cleanup
happens.
2015-07-11 11:58:29 +01:00
Matthias Radestock c22d824db8 .gitignore and clean up coverage reports 2015-07-11 11:57:01 +01:00
Matthias Radestock a44a3a4158 cure "./gen_coverage_reports.sh: line 5: CIRCLECI: unbound variable" 2015-07-11 11:46:49 +01:00
Matthias Radestock 982c9d7650 remove dead code 2015-07-11 11:17:44 +01:00
Matthias Radestock 7042c1506a cosmetic: consistency with other tests 2015-07-11 03:36:21 +01:00
Matthias Radestock 76943f0b17 cosmetic 2015-07-11 02:52:30 +01:00
Matthias Radestock c4f516655d refactor: simplify ipam claim try 2015-07-11 02:28:14 +01:00
Matthias Radestock aaba681db1 remove dead code 2015-07-11 01:54:37 +01:00
Tom Wilkie 56a169b50f Merge pull request #1115 from weaveworks/unit-coverage
Run unit tests in circle and include their coverage in the report.
2015-07-10 19:34:48 +01:00
Tom Wilkie 32b2ec4038 Run unit tests in circle and include their coverage in the report. Report coveralls.io from Circle. 2015-07-10 18:19:50 +00:00
Matthias Radestock ce5052305d fix CircleCI breakage 2015-07-10 18:44:30 +01:00
Matthias Radestock 0e94e3e07b complain even less about absent weave container in coverage collection 2015-07-10 18:30:13 +01:00
Matthias Radestock 3d72670105 don't complain in coverage collection when weave container is absent 2015-07-10 18:24:19 +01:00
Tom Wilkie f8718dcc5d Remove duplication of coverage stanza in weave script. 2015-07-10 16:35:46 +00:00
Paul Bellamy fa6777004f Review feedback 2015-07-10 17:34:06 +01:00
Paul Bellamy e3212d462d Merge pull request #1105 from weaveworks/673-smoke-test-coverage
Code coverage for smoke tests.
2015-07-10 16:53:59 +01:00
Tom Wilkie f5186e74c0 Review feedback 2015-07-10 15:33:00 +00:00
Tom Wilkie bf814739ba Reduce repetition in Makefile. 2015-07-10 15:03:54 +00:00
Tom Wilkie 00a7490a1d Code coverage for smoke tests.
Allow non-circle people to build coverage reports.
2015-07-10 14:45:14 +00:00
Adam Harrison da41604229 Fix IPAM test compilation 2015-07-10 15:43:28 +01:00
Adam Harrison d5a4326b9e Merge branch '1.0'
Conflicts:
	ipam/allocator_test.go
	ipam/claim.go
2015-07-10 14:35:41 +01:00
Paul Bellamy e4458372e8 [proxy] weavewait rewrites /etc/hosts so weave ip is returned first from 'hostname -i'
Adds a --no-rewrite-hosts flag to the proxy, and be default, removes the
docker IP from hosts. We only discard the docker host when it exists, so
--net=none works.
2015-07-10 13:57:20 +01:00
Matthias Radestock a05c65c73b simplify feature extraction 2015-07-09 14:38:08 +01:00
David Wragg 14d346db8b Add a --protocol-min-version option to router
To allow users to forgo compatibility with Weave Net 1.0 in order to
avoid #912, by setting --protocol-min-version=2.
Defaults to --protocol-min-version=1, i.e. compatible but vulnerable.
2015-07-09 14:38:06 +01:00
David Wragg 882f14d50c Version 2 protocol, which encrypts the features map
With V1 compatibility
2015-07-09 14:37:42 +01:00
Bryan Boreham 75646d4786 Merge remote-tracking branch 'origin/1.0'
Conflicts:
	router/connection.go
2015-07-09 12:56:37 +01:00
Bryan Boreham cbcd5da680 Fix log message on peer move 2015-07-09 12:33:39 +01:00
David Wragg 501f40d9c1 Consolidate remote peer checks 2015-07-08 17:21:32 +01:00
Matthias Radestock 08463ea403 Merge branch '1.0' 2015-07-08 16:54:24 +01:00
Tom Wilkie 89ad7e1ba2 Move test gossip router out of ipam code for use in gossip dns code. 2015-07-08 15:03:13 +00:00
Matthias Radestock e6828e1aa5 Merge pull request #1101 from weaveworks/misc-changes
Extend router/peers.go to allow multiple OnGC callbacks
2015-07-08 16:00:22 +01:00
Tom Wilkie c8718d85dc Extend router/peers.go to allow multiple OnGC callbacks, and call the callbacks outside of the peers lock. 2015-07-08 14:47:02 +00:00
Tom Wilkie 56863c3d0e Ensure bin/lint doesn't complain about removed files when used as a git precommit script. 2015-07-08 14:43:21 +00:00
David Wragg e50b1f1fe7 Remove the "found" return value from Peers Fetch method
Callers can simply check whether the returned *Peer is nil.
2015-07-08 09:06:33 +01:00
David Wragg 5e10b901a3 Drop the check for duplicate connections in the handshake
Duplicate connections are also detected and handled in AddConnection,
I have not been able to reproduce #182 without the check, so although
it may once have been necessary, it doesn't seem to be needed now.
2015-07-08 09:03:51 +01:00
David Wragg a6a74001bd Make TCPReceiver more closely mirror TCPSender 2015-07-08 08:53:53 +01:00
Bryan Boreham 0b032386c1 Merge pull request #1092 from weaveworks/1082_usage_to_stderr
Note I have rebased this and merged changes.  Closes #1082.
2015-07-07 15:43:23 +01:00
Matthias Radestock 8eb092c754 display usage on stderr
so that the likes of $(weave proxy-typo-env) don't do weird things

Closes #1082.
2015-07-07 15:39:47 +01:00
Alvaro 33921c2cfe Merge pull request #1073 from weaveworks/971_check_container_alive_in_api
make container aliveness check part of the ipam & dns API
2015-07-07 16:08:40 +02:00
Matthias Radestock 6cc566fd3e Merge pull request #1090 from weaveworks/1076-ignore-iface
Don't complain about weave bridge routes clashing

Fixes #1076.
2015-07-07 13:12:41 +01:00
Matthias Radestock 491a4369d7 Merge pull request #1096 from squaremo/issue1087_password_equals
Handle --option=value as well as --option value

Closes #1087.
2015-07-07 12:57:01 +01:00
Bryan Boreham 6666fd14ac Allow multiple --ignore-iface arguments 2015-07-07 12:52:29 +01:00
Bryan Boreham e621aca77e Added --ignore-iface option to netcheck and use it to ignore weave bridge route 2015-07-07 12:52:29 +01:00
Adam Harrison 3e7c20b4ee Merge branch '1.0' 2015-07-07 12:46:00 +01:00
Michael Bridgen 22446ffc19 Handle --option=value as well as --option value
In any place we munge options, treat the form with an equals the same
as the form without.

The exes ultimately invoked by the script handle both forms, so unless
we need to do something with the option in the script, we can just let
options fall through.
2015-07-07 12:44:37 +01:00
Matthias Radestock f39791616d Merge pull request #1095 from weaveworks/move-ipam-address
Move ipam/address to net/address

Closes #1065.
2015-07-07 11:43:47 +01:00
Paul Bellamy d39aaa86ee Merge pull request #1084 from squaremo/issue602_moar_hyphens
Consistent option naming
2015-07-07 11:37:21 +01:00
Tom Wilkie 4fdb14293b Move ipam/address to net/address 2015-07-07 10:33:20 +00:00
Michael Bridgen 4ede33468e Update docs and smoke-tests with new option names 2015-07-07 11:26:09 +01:00
Michael Bridgen ebbb5d379d Usage and deprecation warnings in script
The usage is rearranged slightly to avoid wrapping at 80 characters.

This will tend to overestimate warnings, since it doesn't try very
hard to ignore values that are not intended to be options; in
particular, arguments that are intended for `docker run`. (Using `--`
to delimit these arguments would make this easier!)

It does ignore values that follow known options though, for instance
in

        weave -nickname "-password"

(-nickname results in a deprecation warning, -password does not)

and

        weave --password "-iprange"

(no deprecation warning, since -iprange is ignored)
2015-07-07 11:26:09 +01:00
Michael Bridgen 379b8c4e6b Standardise on options --like-this
Hyphenate words, and always expect double dashes for long options. A
small number of options have entirely new names.

In general I have added aliases for the unhyphenated and single-dashed
varieties, and used mflag's deprecation syntax to make it print a
warning when it sees them. I left some abbreviated options
unhyphenated (e.g., `bufsz`), and the TLS options as-is since they
correspond with Docker options.

NB the deprecation wanrings will mostly get printed to container logs,
since mostly the executables are run in containers. The next step is
to emit warnings from the script for the options mentioned in the
script usage message.
2015-07-07 11:13:30 +01:00
Matthias Radestock 41b0c2a03f Merge pull request #1070 from weaveworks/1036-change-iprange
Change default IP allocation range from 10.128/10 to 10.32/12

Closes #1036.
2015-07-07 02:24:13 +01:00
Bryan Boreham 646cf84b4a Merge pull request #1072 from weaveworks/1043_common_launch_args
LGTM
2015-07-06 17:15:51 +01:00
Bryan Boreham f593c2fbf2 Change default IP allocation range from 10.128/10 to 10.32/12 2015-07-06 17:09:05 +01:00
Bryan Boreham bd24307c2b Merge pull request #1069 from 2opremio/fons/fix-tests-OSX
LGTM
2015-07-06 16:43:26 +01:00
Michael Bridgen e0c7054f9b Merge pull request #1075 from weaveworks/fons/test-vagrant-creation
Explain how to create the test VMs from vagrant
2015-07-06 16:31:22 +01:00
Michael Bridgen 59550e0fcb Merge branch 'pr1077' 2015-07-06 16:27:45 +01:00
Michael Bridgen af64c8e2af Merge branch '1063_status_json_accept_header' of github.com:weaveworks/weave into pr1077 2015-07-06 16:25:58 +01:00
Matthias Radestock 03ac7a86c6 make bin/multiweave work again in 'launch launches everything' world 2015-07-06 15:13:21 +01:00
Matthias Radestock 80a232f44c copy latest random_mac() from weave script to bin/multiweave 2015-07-06 15:09:23 +01:00
Matthias Radestock d1041c17d3 Merge pull request #1081 from weaveworks/weave_1.1_readme_updates_proxy_env
Remove weave proxy-(env|config) for just (env|config). Readme changes
2015-07-06 15:05:05 +01:00
Matthias Radestock cf7fff3da2 Merge pull request #1080 from weaveworks/1014-weave-env
rename proxy-(env|config) to (env|config)

Closes #1014.
2015-07-06 15:04:40 +01:00
Alfonso Acosta 373d663b5e Fix integration tests for OSX 2015-07-06 14:48:53 +01:00
Alfonso Acosta e86d255a47 Add comment about running integration tests from the Vagrant build VM 2015-07-06 14:07:16 +01:00
Paul Bellamy 5c44f74475 rename proxy-(env|config) to (env|config)
Keep error message to stderr, as the proxy should be launched with
'weave launch'. 'proxy-(env|config)' are kept with a deprecation
warning, for a while.
2015-07-06 13:41:23 +01:00
Paul Bellamy d649bdddc7 Remove weave proxy-(env|config) for just (env|config). Readme changes 2015-07-06 12:59:07 +01:00
Matthias Radestock 5c7763b075 Merge branch '1.0' 2015-07-06 11:00:15 +01:00
Matthias Radestock c8bd47e948 set correct content-type header when returning json status 2015-07-05 19:25:28 +01:00
Matthias Radestock 098fc81825 output json status based on 'Accept' header
...instead if having a separate URL.

Closes #1063.
2015-07-05 19:19:02 +01:00
Alfonso Acosta cab4a7e1ee Explain how to create the test VMs from vagrant 2015-07-05 18:59:53 +01:00
Matthias Radestock 6596c8b1a7 make container aliveness check part of the ipam & dns API
...rather than conducting it implicitly when an ID "looks like" a
container ID.

Closes #971
2015-07-04 19:13:13 +01:00
Matthias Radestock 5559f1df2e cosmetic: remove some unnecessary quoting
for consistency, since we don't quote the same term anywhere else
2015-07-04 18:14:13 +01:00
Matthias Radestock 156597a965 small simplifying refactor: use shortcircuit boolean expressions 2015-07-04 18:04:25 +01:00
Matthias Radestock d0641062c0 pass common launch args to all three components
ATM that's just --loglevel=...

Closes #1043.
2015-07-04 04:38:18 +01:00
Matthias Radestock cf4fffa394 Merge pull request #1057 from weaveworks/329-log-level-choice
Give the user more choice of log levels

Closes #329
2015-07-04 03:46:59 +01:00
Matthias Radestock b30de63f18 Merge pull request #1067 from paulbellamy/proxy-stderr
print all stderr output from callWeave to the proxy logs, even if successful

Closes #1056
2015-07-04 03:44:50 +01:00
Bryan Boreham ff2d7451e2 Remove -debug option, superceded by --log-level 2015-07-04 03:27:07 +01:00
Matthias Radestock 910b57731d remove stray -debug in tests 2015-07-04 03:19:27 +01:00
Matthias Radestock b8aa1d473b remove dead code 2015-07-04 02:53:36 +01:00
Matthias Radestock ce26f88ee2 Merge branch '1.0'
Conflicts:
	router/crypto.go
	router/ethernet_decoder.go
	router/handshake_test.go
	router/mocks_test.go
	router/peers_test.go
	router/router.go
	router/udp_sender.go
2015-07-03 20:47:29 +01:00
Matthias Radestock 1524796408 Merge pull request #1068 from weaveworks/1004-seed-random
Seed random-number generator from the host time

Fixes #1004.
2015-07-03 16:01:16 +01:00
Paul Bellamy 7821412bf7 [proxy] print all stderr from callWeave to the proxy logs, even if successful 2015-07-03 15:45:09 +01:00
Matthias Radestock b72b7a9ed5 Merge branch '1.0' 2015-07-03 14:28:42 +01:00
Matthias Radestock 8b0089565f Merge branch '1.0' 2015-07-03 14:18:46 +01:00
Bryan Boreham 8c6c692926 Seed the math/rand generator from the time 2015-07-03 13:39:47 +01:00
Matthias Radestock 7554bd3913 Merge branch 'master' into weave_1.1_readme_updates
Conflicts:
	README.md
2015-07-03 11:55:32 +01:00
Matthias Radestock b81fce32f6 one more s/proxy/Weave Docker API proxy 2015-07-03 11:51:12 +01:00
Matthias Radestock 3ea1448c52 cosmetic 2015-07-03 11:49:20 +01:00
Matthias Radestock f31a3c1261 Merge branch '1.0' 2015-07-03 11:48:07 +01:00
Matthias Radestock fcea6fcbf8 Merge branch '1.0' 2015-07-03 10:03:43 +01:00
Matthias Radestock 07ee2df822 cosmetic(ish): use dotted import. like everywhere else 2015-07-02 21:07:51 +01:00
Paul Bellamy 5b79e274e8 fixup logging changes, so it compiles 2015-07-02 17:04:21 +01:00
Matthias Radestock e25d1b8365 Merge branch '1.0' 2015-07-02 16:35:40 +01:00
Paul Bellamy 3fd146557a Merge pull request #1060 from weaveworks/issues/1059-remove-google-code-refs
Remove all the imports from "code.google.com"
2015-07-02 15:59:57 +01:00
Alvaro Saurin f6ffd604dd Removed all the imports from "code.google.com" 2015-07-02 13:32:52 +02:00
Matthias Radestock 18a4c14670 clarify network/address route overlap error/warning message
Closes #1034.
2015-07-02 11:51:22 +01:00
Matthias Radestock 0f67dbd8bd Merge branch '1.0' 2015-07-02 11:48:22 +01:00
Matthias Radestock ecfc5d091d Merge pull request #1053 from weaveworks/437-check-address
Check addresses supplied by users for conflict with existing routes.

Fixes #437. Fixes #480.
2015-07-01 15:03:28 +01:00
Bryan Boreham 84238a0262 Amend error message to reflect usage situation 2015-07-01 14:58:42 +01:00
Matthias Radestock edf3ec9f09 Merge pull request #1044 from weaveworks/664-use-logrus
Use off-the-shelf logger logrus

Closes #664.
2015-07-01 14:47:52 +01:00
Matthias Radestock 2fee0f0184 Merge branch '1.0' 2015-07-01 14:02:57 +01:00
Bryan Boreham a5ddb63fca Remove Info.Print in favour of Log.Info 2015-07-01 14:01:45 +01:00
Bryan Boreham 013380c7e7 Update router code to use common logger 2015-07-01 12:26:23 +01:00
Bryan Boreham 13c0b62f96 When looking at proxy logs, use both stderr and stdout 2015-07-01 12:26:23 +01:00
Bryan Boreham 7276fc1d4b Wire up selection of logging levels to match old behaviour
- note that all output is now going to stderr.
2015-07-01 12:26:23 +01:00
Bryan Boreham 67a6961b4b Adjust logrus feature of padding all lines less than 44 chars with spaces at the end 2015-07-01 12:26:23 +01:00
Bryan Boreham d82578d895 Replace logging calls with new style 2015-07-01 12:26:23 +01:00
Bryan Boreham c786e7eb5a Switch from basic logger to logrus 2015-07-01 12:26:23 +01:00
Bryan Boreham 91099694c9 Send error message to stderr 2015-07-01 12:17:02 +01:00
Bryan Boreham c9a20ae10c Warn user if they are trying to use a specific address that clashes with an existing route 2015-07-01 12:17:02 +01:00
Bryan Boreham f05225da22 Check specific addresses differently to networks
For a network, as used by IPAM, we care if *any* address in the range
could possibly conflict with an existing address routed by the host.
When given a single address, we only care if *that specific address*
conflicts with an existing route.
2015-07-01 12:15:04 +01:00
Matthias Radestock badfdee5e4 Merge branch '1.0' 2015-07-01 11:34:50 +01:00
Bryan Boreham 6403e7a11c Add missing rule to rebuild netcheck when its source changes 2015-07-01 11:13:40 +01:00
Matthias Radestock 312122e1be fix typo 2015-07-01 10:30:12 +01:00
Matthias Radestock bb1da63bfc Merge pull request #1048 from squaremo/issue1040_eliminate_ipam_mentions
Remove references to 'IPAM' in docs and UI

Closes #1040.
2015-06-29 23:15:58 +01:00
Matthias Radestock d1b2e74985 Merge pull request #1045 from squaremo/issue1041_usage_for_initpeercount
Add -initpeercount to usage

Closes #1041.
2015-06-29 19:48:52 +01:00
Michael Bridgen c0b2fe3e3b Remove references to 'IPAM' in docs and UI
The option `--no-default-ipam` is replaced with
`--no-default-ipalloc`; the former will still work (in a 'last one
wins' way), but its usage message says it's deprecated.
2015-06-29 19:20:26 +01:00
Michael Bridgen 864461918f Add -initpeercount to usage 2015-06-29 17:06:14 +01:00
Matthias Radestock 9127eb8acf cosmetic: make weave run usage fit in 80 columns 2015-06-29 16:56:57 +01:00
Paul Bellamy 8e629d9471 Merge pull request #999 from paulbellamy/scope-tests
Let Scope use these integration test helpers
2015-06-29 13:51:26 +01:00
Matthias Radestock ab943d938a refactor: don't return encoder from handshake()
We did this for symmetry/completeness, but the encoder is actually
"consumed" by handshake() in the creation of the tcp senders, hence it
is in fact dangerous to return it.
2015-06-29 12:38:21 +01:00
David Wragg d6fedc1e6e Remove error return value from DecodeLayers
It is never checked, so add a comment explaining why.
2015-06-29 10:47:56 +01:00
David Wragg 3222478449 Remove the LogFrame when fragmenting
Subsequent commits make this LogFrame call a little tricky to retain,
and it's not clear that it is really needed.
2015-06-29 10:47:26 +01:00
Paul Bellamy 26cb65c357 making test helpers more robust so scope can use them 2015-06-29 10:22:10 +01:00
Paul Bellamy 1ef3bd3a8b add docker version and weave version to test sanity checks 2015-06-29 10:20:55 +01:00
Matthias Radestock 51ecf05091 cosmetic: fix typo in comment 2015-06-27 09:35:14 +01:00
Bryan Boreham 565454d3cb fixed Logf format specifier 2015-06-26 17:22:35 +01:00
Matthias Radestock 906ff210fe Merge pull request #1026 from weaveworks/1003-proxy-unix-socket
Make proxy listen on unix:///var/run/weave.sock

Closes #1003
2015-06-26 16:44:41 +01:00
Tom Wilkie c4565f6ed3 Make proxy listen on /var/run/weave.sock 2015-06-26 15:08:21 +00:00
Matthias Radestock 8dd9020803 Merge branch '1.0'
Conflicts:
	ipam/allocator_test.go
2015-06-26 14:08:09 +01:00
Matthias Radestock 55fbffef75 Merge pull request #1009 from awh/weave_launch_all_the_things
Weave launch all the things

Closes #962.
2015-06-26 13:21:44 +01:00
Matthias Radestock 039a2030db refactor: make DF flag passing logic more obvious
...and provide a better abstraction for sending special frames.
2015-06-26 11:11:17 +01:00
Matthias Radestock 33d85aaee5 Merge pull request #1016 from dpw/defer-df
Don't inspect the DF flag earlier than we need to
2015-06-26 11:10:42 +01:00
Adam Harrison d4c1766532 Fix smoke tests 2015-06-26 11:10:08 +01:00
Matthias Radestock 39a6d1c87e refactor: simplify a test 2015-06-25 20:13:03 +01:00
Matthias Radestock ee2a66323e Merge branch '1.0'
Conflicts:
	ipam/ring/ring_test.go
2015-06-25 19:57:09 +01:00
Matthias Radestock 635a3e5b26 cosmetic: remove attempt at pretty shell fragment
there is just no way to format this so it looks good everywhere, due
to different tab widths.
2015-06-25 19:51:39 +01:00
Matthias Radestock 82da45a638 go get test dependencies
The previous 'fix' wasn't working since it `go get -t`'ed the dirs
where our executables live. Which don't contain any tests.
2015-06-25 19:31:38 +01:00
David Wragg 64ac7745fe Don't inspect the DF flag earlier than we need to 2015-06-25 17:11:52 +01:00
David Wragg 5690902173 Move DF indication into LogFrame 2015-06-25 17:11:20 +01:00
David Wragg 4162dcafd3 Make the full EthernetDecoder available to LogFrame
So that LogFrame can include information about the decoded frame beyond
the ethernet layer.
2015-06-25 17:11:20 +01:00
Matthias Radestock ea59c6c065 Merge pull request #1015 from weaveworks/go-get-tests
Extend 'go get' to tests
2015-06-25 17:01:26 +01:00
Bryan Boreham 7c741d6f7e Extend 'go get' to tests
The `-t` flag asks Go to fetch anything required for tests, which is more necessary now we have added testify.
2015-06-25 16:42:53 +01:00
Adam Harrison 99dc70ea7a Eliminate references to proxy and weaveDNS 2015-06-25 15:42:06 +01:00
Adam Harrison b78d4f05d0 Use new weave launch command introduced by #962 2015-06-25 14:59:43 +01:00
Adam Harrison a9e3320e7c Update documentation 2015-06-25 14:43:29 +01:00
Adam Harrison 08f3d5b8e8 Implement new weave stop behaviour 2015-06-25 12:47:18 +01:00
Adam Harrison 0d5aad3e90 Rename stop to stop-router 2015-06-25 12:47:18 +01:00
Adam Harrison fcd79a78af Implement new weave launch behaviour 2015-06-25 12:47:14 +01:00
Matthias Radestock 657bbbffbc tiny refactor: raise abstraction level 2015-06-24 18:17:15 +01:00
Tom Wilkie 7780ac1078 Fix gce.sh destory timeouts. 2015-06-24 17:13:37 +00:00
Matthias Radestock 76a12edf57 Merge pull request #997 from dpw/eliminate-starts
Eliminate most Start methods in router package
2015-06-24 18:04:57 +01:00
David Wragg 7387caa2eb Eliminate forwarder Start methods 2015-06-24 17:37:07 +01:00
David Wragg 576dd69b0b Add a comment to Router's Start method 2015-06-24 17:37:02 +01:00
David Wragg 5e6b107d01 Eliminate the GossipSender Start method 2015-06-24 17:35:26 +01:00
David Wragg f444ae9224 Eliminate ConnectionMaker Start method 2015-06-24 17:35:26 +01:00
David Wragg 8f5d1eec11 Eliminate the Routes Start method 2015-06-24 17:35:17 +01:00
Matthias Radestock 9a6afcd236 Merge branch '1.0' 2015-06-24 17:26:19 +01:00
Matthias Radestock 94a8338be5 fix typo in comment 2015-06-24 17:02:35 +01:00
David Wragg 12c17d39cc Merge weaveworks/separate_image_build_and_export (PR #994)
separate the image build and export phases
2015-06-24 15:26:11 +01:00
Adam Harrison 409961fff3 Rename launch to launch-router 2015-06-24 14:30:35 +01:00
David Wragg 159d47a934 Merge rade/992_fractional_sleep (PR #993)
Sleep for 0.1s instead of 1s in the wait_for_status loop.

Closes #992.
2015-06-24 12:44:52 +01:00
Matthias Radestock 4b11874390 sleep for 0.1s instead of 1s in the wait_for_status loop
This reduces `weave launch|launch-dns` execution time.

Closes #992.
2015-06-24 12:16:52 +01:00
Matthias Radestock 3c7cff1c9c Merge pull request #996 from weaveworks/rm_dockercon_pointer
Remove Link to Dockercon from README
2015-06-24 10:47:00 +01:00
Fintan Ryan e3574f243f Remove Link to Dockercon from README 2015-06-24 10:35:12 +01:00
Matthias Radestock bdda9d03ae bump docker version in weave-build docker image
to 1.3.1, the same version we use everywhere else, e.g. in weavexec.
2015-06-24 03:24:58 +01:00
Matthias Radestock ae1805de76 separate the image build and export phases
That way the export can be just a single tar, which is more
convenient. And without introducing unnecessary image builds.
2015-06-24 02:54:45 +01:00
Matthias Radestock 0ce123e63a Merge branch '1.0' 2015-06-24 02:53:06 +01:00
Matthias Radestock 215d96fdc3 Merge branch '1.0' 2015-06-23 23:18:09 +01:00
Matthias Radestock eb9e450cf1 cosmetic 2015-06-23 19:10:10 +01:00
David Wragg 72039bbe5a Eliminate the LocalPeer Start method 2015-06-23 16:08:43 +01:00
David Wragg d6bceb139e Eliminate MacCache Start method 2015-06-23 16:08:43 +01:00
David Wragg a311623bb9 Eliminate the Start method on LocalConnection 2015-06-23 16:08:43 +01:00
Paul Bellamy 95d72a57e8 Merge pull request #989 from rade/prefetch_test_images
prefetch test images
2015-06-23 15:14:59 +01:00
Adam Harrison aa8d9eab53 Merge branch '1.0' 2015-06-23 15:05:32 +01:00
Matthias Radestock 43b8840dd2 Merge branch '1.0' 2015-06-23 14:58:55 +01:00
Adam Harrison 59e667801d Merge GCE fix from branch '1.0' 2015-06-23 14:50:58 +01:00
Matthias Radestock ae8172ec97 simplify docker image removal 2015-06-23 14:39:29 +01:00
Adam Harrison d6ab78ff03 Merge documentation from branch '1.0' 2015-06-23 14:36:04 +01:00
Adam Harrison dde5782625 Merge documentation from branch '1.0' 2015-06-23 14:21:05 +01:00
Matthias Radestock 369bd0f4dc prefetch test images
This gives more consistent test execution times, and also make image
fetch failures more apparent and decoupled from the tests.

We could *always* just run pull, but it's an expensive operation, so
we check for the presence of the image first. The downside is that we
could be running old images. That is not a problem on our CI
infrastructure though since it spins up fresh VMs per for every tun.
2015-06-23 14:16:52 +01:00
Tom Wilkie 3b4a47a961 Update gce.sh make_template command, and build a fresh template. 2015-06-23 13:07:09 +00:00
Paul Bellamy 0a287217b6 Merge pull request #955 from paulbellamy/testing
Replacing custom testing helpers with testify, where possible
2015-06-23 12:21:24 +01:00
Matthias Radestock 2c4cebe9d9 Merge branch '1.0' 2015-06-23 10:31:40 +01:00
Matthias Radestock cedf9bb82d Merge branch '1.0' 2015-06-23 09:54:24 +01:00
Matthias Radestock 2a9f8a3d75 Merge branch '1.0' 2015-06-22 23:00:29 +01:00
Paul Bellamy acd5b91eb8 Merge remote-tracking branch 'upstream/1.0' 2015-06-22 17:28:50 +01:00
Adam Harrison bda245bfdb Merge pull request #968 from rade/954_show_attach_expose_addresses
show IP addresses in attach/expose/detach/hide
2015-06-22 14:49:41 +01:00
Bryan Boreham d1eb2917e8 Merge documentation from branch 1.0 into master 2015-06-22 14:32:12 +01:00
Matthias Radestock 9f0cd93859 refactor(ish): get the more fundamental setup out of the way first 2015-06-22 12:50:50 +01:00
Matthias Radestock 7cf0d59003 update docs 2015-06-22 12:34:32 +01:00
Adam Harrison 74f8c07674 Merge documentation from branch '1.0' 2015-06-22 12:05:40 +01:00
Paul Bellamy 6d89867a74 Merge pull request #978 from rade/972_remove_proxy_d_option
remove proxy -D option
2015-06-22 11:36:37 +01:00
Matthias Radestock ebf725d869 add some tests 2015-06-22 11:26:33 +01:00
Matthias Radestock 257774a093 make weave attach|expose|detach|hide show IP addresses
The output contains one IP address for each addr (i.e. [ip:]<cidr> |
net:<cidr> | net:default) supplied on the command line; or, if none
was specified, a single address from the default IPAM subnet.

We don't want `weave run` to output the addresses, since it is meant
to (only) output the container id. Hence we redirect the `attach`
output to stderr in that case.

Closes #954.
2015-06-22 11:26:33 +01:00
Matthias Radestock 5fbda6b094 Merge remote-tracking branch 'upstream/master' 2015-06-22 11:15:58 +01:00
Matthias Radestock b483be8c4b remove the -D proxy option
It doesn't do anything useful since the weave script needs to be able
to talk to docker on the standard unix socket.

Fixes #972.
2015-06-22 11:13:53 +01:00
Adam Harrison a751237164 Merge documentation from branch '1.0' 2015-06-22 11:00:25 +01:00
Matthias Radestock 0c87dcd656 cosmetic 2015-06-22 11:00:12 +01:00
Matthias Radestock e276f3e026 cosmetic 2015-06-22 10:38:59 +01:00
Matthias Radestock 84e5659090 Merge branch '1.0' 2015-06-22 10:31:50 +01:00
Matthias Radestock bbeb7966b4 Merge remote-tracking branch 'upstream/1.0' 2015-06-21 10:42:28 +01:00
Matthias Radestock edb4911bf6 Merge branch '1.0' 2015-06-21 09:39:45 +01:00
Michael Bridgen e3917121fa Merge remote-tracking branch 'upstream/1.0.0' 2015-06-20 16:45:34 -07:00
Michael Bridgen ede288ad38 Merge pull request #969 from rade/967_include_image_name_in_inspect_error
include image name in errors
2015-06-20 16:41:05 -07:00
Michael Bridgen 58f14551dc Merge branch 'pr969' 2015-06-20 16:28:44 -07:00
Michael Bridgen 97816d6c54 Merge branch '967_include_image_name_in_inspect_error' of https://github.com/rade/weave into pr969 2015-06-20 16:21:20 -07:00
Matthias Radestock 939a095bf4 Merge pull request #966 from weaveworks/readme_website_sync
README update to point at weave.works
2015-06-19 20:38:37 +01:00
Fintan Ryan ad0d53f7fe Update proposed text 2015-06-19 19:17:43 +01:00
Fintan Ryan e748ddc20d README update to point at weave.works 2015-06-19 13:00:01 +01:00
Matthias Radestock e51ecd73b3 refactor: extract helper 2015-06-18 19:01:10 +01:00
Matthias Radestock 5479d0a54d improve test performance - only fetch status once per check 2015-06-18 18:53:06 +01:00
Matthias Radestock 6830b27317 cosmetic: correct some comments, tweak some arg parsing 2015-06-18 18:13:27 +01:00
Matthias Radestock 4399be7ad2 correct comments 2015-06-18 17:55:42 +01:00
Matthias Radestock 42d2a860eb test more things, more obviously 2015-06-18 17:48:21 +01:00
Matthias Radestock 3ec0200067 cosmetic oops 2015-06-18 16:57:54 +01:00
Matthias Radestock d1e4a8ec8a cosmetic: vertical alignment and consistency for better readability 2015-06-18 16:26:16 +01:00
Paul Bellamy 54b3061c50 Merge gofmt and goimports run 2015-06-18 14:05:51 +01:00
Bryan Boreham 19fe808dff Update links in changelog to point at latest_release 2015-06-18 11:59:44 +01:00
Paul Bellamy aea1a2b08c have travis fetch any test-only dependencies before building tests 2015-06-17 17:00:36 +01:00
Paul Bellamy cdee7a5d6f Replacing custom testing helpers with testify, where possible 2015-06-17 16:58:03 +01:00
Paul Bellamy 21e07c0db9 Running gofmt -s 2015-06-17 15:32:08 +01:00
Paul Bellamy 06101d134b Running goimports 2015-06-17 15:32:08 +01:00
230 changed files with 15031 additions and 11527 deletions

No files matched your search

+17 -5
View File
@@ -29,24 +29,36 @@ coverage.html
# Emacs backup files
*~
# Vim backup files
*.swp
# Project specific
prog/weaver/weaver
prog/weavedns/weavedns
prog/weavehosts/weavehosts
prog/weaveproxy/weaveproxy
prog/sigproxy/sigproxy
prog/weavewait/weavewait
prog/weavewait/weavewait_noop
prog/netcheck/netcheck
tools/bin
tools/build
prog/docker_tls_args/docker_tls_args
testing/cover/cover
testing/runner/runner
releases
weave.tar
weavedns.tar
weaveexec.tar
.weaver.uptodate
.weavedns.uptodate
.weaveexec.uptodate
weave.tar.gz
prog/weaveexec/weave
prog/weaveexec/weavehosts
prog/weaveexec/weaveproxy
prog/weaveexec/weavewait
prog/weaveexec/weavewait_noop
prog/weaveexec/sigproxy
prog/weaveexec/netcheck
prog/weaveexec/docker_tls_args
prog/weaveexec/docker*.tgz
Vagrantfile.local
test/tls/*.pem
test/coverage
test/coverage.*
+3
View File
@@ -0,0 +1,3 @@
[submodule "tools"]
path = tools
url = https://github.com/weaveworks/build-tools
+2 -3
View File
@@ -15,15 +15,14 @@ install:
- go clean -i net
- go install -tags netgo std
- go get github.com/golang/lint/golint
- pip install --user --upgrade gcloud gsutil
- pip install --user --upgrade gsutil
- bin/setup-ci-secrets $encrypted_5ba036e89377_key $encrypted_5ba036e89377_iv
- make travis
- go get ./...
script:
- bin/lint .
- make tests
after_success:
- go get github.com/mattn/goveralls
- $HOME/gopath/bin/goveralls -coverprofile=profile.cov -service=travis-ci
- test "${TRAVIS_PULL_REQUEST}" = "false" && site/build-and-publish.sh
+95 -3
View File
@@ -1,3 +1,95 @@
## Release 1.3.0
**Highlights**
This release includes a Docker Plugin, so you have the option to use
Weave Net that way. More details in the
[change log](https://github.com/weaveworks/weave/issues?q=milestone%3A1.3.0).
## Release 1.2.1
This release contains a number of bug fixes and minor
enhancements. More details in the
[change log](https://github.com/weaveworks/weave/issues?q=milestone%3A1.2.1).
The release is fully compatible with 1.2.0 versions, so existing
clusters can be upgraded incrementally.
## Release 1.2.0
**Highlights**
This release introduces the
[Fast Data Path](http://docs.weave.works/weave/master/head/features.html#fast-data-path),
which allows Weave networks to operate at near wire-level speeds. This
new feature is enabled by default.
Other highlights:
- auto-configuration of TLS for the Weave Docker API proxy, making it
easier to run Weave on Macs and in conjunction with tools like
Docker Swarm
- support for restart policies on application containers and weave
infrastructure containers
- better compatibility with recent and future Docker versions
More details in the
[change log](https://github.com/weaveworks/weave/issues?q=milestone%3A1.2.0).
## Release 1.1.2
This release contains a small number of bug fixes. More details in the
[change log](https://github.com/weaveworks/weave/issues?q=milestone%3A1.1.2).
The release is fully compatible with 1.1.0 versions, so existing
clusters can be upgraded incrementally. When upgrading from 1.0.x,
note the compatibility information in the
[Installation & Upgrading instructions for Weave 1.1.0](https://github.com/weaveworks/weave/releases/tag/v1.1.0).
## Release 1.1.1
This release contains a number of bug fixes and minor
enhancements. More details in the
[change log](https://github.com/weaveworks/weave/issues?q=milestone%3A1.1.1).
The release is fully compatible with 1.1.0 versions, so existing
clusters can be upgraded incrementally. When upgrading from 1.0.x,
note the compatibility information in the
[Installation & Upgrading instructions for Weave 1.1.0](https://github.com/weaveworks/weave/releases/tag/v1.1.0).
## Release v1.1.0
**Highlights**:
- `weave launch` now launches all weave components, simplifying
startup.
- `weave status` has been completely revamped, with a much improved
presentation of the information, and the option to select and output
data in JSON.
- weaveDNS has been rewritten and embedded in the router. The new
implementation simplifies configuration, improves performance, and
provides fault resilience for services.
- the weave Docker API proxy now provides an even more seamless user
experience, and enables easier integration of weave with other
systems such as kubernetes.
- many usability improvements
- a few minor bug fixes, including a couple of security
vulnerabilities
More details in the
[change log](https://github.com/weaveworks/weave/issues?q=milestone%3A1.1.0).
## Release 1.0.3
This release contains a weaveDNS feature enhancement as well as minor fixes for
improved stability and robustness.
More details in the
[change log](https://github.com/weaveworks/weave/issues?q=milestone%3A1.0.3).
The release is fully compatible with other 1.0.x versions, so existing
clusters can be upgraded incrementally.
## Release 1.0.2
This release fixes a number of bugs, including some security
@@ -34,14 +126,14 @@ This is a bug fix release, addressing the following issue:
root@d11e9287f65b:/# ping pingme
````
- Containers can now be
[load-balanced](http://docs.weave.works/weave/master/head/weavedns.html#load-balancing)
[load-balanced](http://docs.weave.works/weave/latest_release/weavedns.html#load-balancing)
easily.
- IP address allocation is now available across [multiple
subnets](http://docs.weave.works/weave/master/head/ipam.html#range),
subnets](http://docs.weave.works/weave/latest_release/ipam.html#range),
and hence can be employed when running multiple, isolated
applications.
- The proxy now supports [TLS
connections](http://docs.weave.works/weave/master/head/proxy.html#tls),
connections](http://docs.weave.works/weave/latest_release/proxy.html#tls),
enabling its deployment when the communication between docker
clients and the server must be secured.
+80 -56
View File
@@ -1,7 +1,7 @@
PUBLISH=publish_weave publish_weavedns publish_weaveexec
PUBLISH=publish_weave publish_weaveexec
.DEFAULT: all
.PHONY: all update tests publish $(PUBLISH) clean prerequisites build travis run-smoketests
.PHONY: all update tests lint publish $(PUBLISH) clean clean-bin prerequisites build travis run-smoketests
# If you can use docker without being root, you can do "make SUDO="
SUDO=sudo
@@ -10,110 +10,134 @@ DOCKERHUB_USER=weaveworks
WEAVE_VERSION=git-$(shell git rev-parse --short=12 HEAD)
WEAVER_EXE=prog/weaver/weaver
WEAVEDNS_EXE=prog/weavedns/weavedns
WEAVEPROXY_EXE=prog/weaveproxy/weaveproxy
SIGPROXY_EXE=prog/sigproxy/sigproxy
WEAVEWAIT_EXE=prog/weavewait/weavewait
WEAVEWAIT_NOOP_EXE=prog/weavewait/weavewait_noop
NETCHECK_EXE=prog/netcheck/netcheck
DOCKERTLSARGS_EXE=prog/docker_tls_args/docker_tls_args
RUNNER_EXE=tools/runner/runner
EXES=$(WEAVER_EXE) $(WEAVEDNS_EXE) $(SIGPROXY_EXE) $(WEAVEPROXY_EXE) $(WEAVEWAIT_EXE) $(NETCHECK_EXE)
EXES=$(WEAVER_EXE) $(SIGPROXY_EXE) $(WEAVEPROXY_EXE) $(WEAVEWAIT_EXE) $(WEAVEWAIT_NOOP_EXE) $(NETCHECK_EXE) $(DOCKERTLSARGS_EXE)
WEAVER_UPTODATE=.weaver.uptodate
WEAVEEXEC_UPTODATE=.weaveexec.uptodate
IMAGES_UPTODATE=$(WEAVER_UPTODATE) $(WEAVEEXEC_UPTODATE)
WEAVER_IMAGE=$(DOCKERHUB_USER)/weave
WEAVEDNS_IMAGE=$(DOCKERHUB_USER)/weavedns
WEAVEEXEC_IMAGE=$(DOCKERHUB_USER)/weaveexec
IMAGES=$(WEAVER_IMAGE) $(WEAVEDNS_IMAGE) $(WEAVEEXEC_IMAGE)
IMAGES=$(WEAVER_IMAGE) $(WEAVEEXEC_IMAGE)
WEAVER_EXPORT=weave.tar
WEAVEDNS_EXPORT=weavedns.tar
WEAVEEXEC_EXPORT=weaveexec.tar
EXPORTS=$(WEAVER_EXPORT) $(WEAVEDNS_EXPORT) $(WEAVEEXEC_EXPORT)
WEAVE_EXPORT=weave.tar.gz
WEAVEEXEC_DOCKER_VERSION=1.3.1
DOCKER_DISTRIB=prog/weaveexec/docker-$(WEAVEEXEC_DOCKER_VERSION).tgz
DOCKER_DISTRIB_URL=https://get.docker.com/builds/Linux/x86_64/docker-$(WEAVEEXEC_DOCKER_VERSION).tgz
NETGO_CHECK=@strings $@ | grep cgo_stub\\\.go >/dev/null || { \
rm $@; \
echo "\nYour go standard library was built without the 'netgo' build tag."; \
echo "To fix that, run"; \
echo " sudo go clean -i net"; \
echo " sudo go install -tags netgo std"; \
false; \
}
BUILD_FLAGS=-ldflags "-extldflags \"-static\" -X main.version $(WEAVE_VERSION)" -tags netgo
all: $(EXPORTS)
PACKAGE_BASE=$(shell go list -e ./)
all: $(WEAVE_EXPORT) $(RUNNER_EXE)
travis: $(EXES)
update:
go get -u -f -v -tags -netgo $(addprefix ./,$(dir $(EXES)))
go get -u -f -v -tags netgo $(addprefix ./,$(dir $(EXES)))
$(WEAVER_EXE) $(WEAVEDNS_EXE) $(WEAVEPROXY_EXE) $(NETCHECK_EXE): common/*.go common/*/*.go net/*.go
$(WEAVER_EXE) $(WEAVEPROXY_EXE): common/*.go common/*/*.go net/*.go
ifeq ($(COVERAGE),true)
$(eval COVERAGE_MODULES := $(shell (go list ./$(@D); go list -f '{{join .Deps "\n"}}' ./$(@D) | grep "^$(PACKAGE_BASE)/") | paste -s -d,))
go get -t -tags netgo ./$(@D)
go test -c -o ./$@ $(BUILD_FLAGS) -v -covermode=atomic -coverpkg $(COVERAGE_MODULES) ./$(@D)/
else
go get -tags netgo ./$(@D)
go build -ldflags "-extldflags \"-static\" -X main.version $(WEAVE_VERSION)" -tags netgo -o $@ ./$(@D)
@strings $@ | grep cgo_stub\\\.go >/dev/null || { \
rm $@; \
echo "\nYour go standard library was built without the 'netgo' build tag."; \
echo "To fix that, run"; \
echo " sudo go clean -i net"; \
echo " sudo go install -tags netgo std"; \
false; \
}
go build $(BUILD_FLAGS) -o $@ ./$(@D)
endif
$(NETGO_CHECK)
$(WEAVER_EXE): router/*.go ipam/*.go ipam/*/*.go prog/weaver/main.go
$(WEAVEDNS_EXE): nameserver/*.go prog/weavedns/main.go
$(NETCHECK_EXE): common/*.go common/*/*.go net/*.go
go get -tags netgo ./$(@D)
go build $(BUILD_FLAGS) -o $@ ./$(@D)
$(NETGO_CHECK)
$(WEAVER_EXE): router/*.go mesh/*.go ipam/*.go ipam/*/*.go nameserver/*.go prog/weaver/*.go
$(WEAVEPROXY_EXE): proxy/*.go prog/weaveproxy/main.go
$(NETCHECK_EXE): prog/netcheck/netcheck.go
# Sigproxy and weavewait need separate rules as they fail the netgo check in
# the main build stanza due to not importing net package
$(SIGPROXY_EXE): prog/sigproxy/main.go
go build -o $@ ./$(@D)
$(WEAVEWAIT_EXE): prog/weavewait/*.go net/*.go
$(DOCKERTLSARGS_EXE): prog/docker_tls_args/*.go
$(WEAVEWAIT_EXE): prog/weavewait/main.go
go build -o $@ ./$(@D)
$(WEAVEWAIT_EXE) $(SIGPROXY_EXE) $(DOCKERTLSARGS_EXE):
go get -tags netgo ./$(@D)
go build $(BUILD_FLAGS) -o $@ ./$(@D)
$(WEAVER_EXPORT): prog/weaver/Dockerfile $(WEAVER_EXE)
$(WEAVEWAIT_NOOP_EXE): prog/weavewait/*.go
go get -tags netgo ./$(@D)
go build $(BUILD_FLAGS) -tags noop -o $@ ./$(@D)
$(WEAVER_UPTODATE): prog/weaver/Dockerfile $(WEAVER_EXE)
$(SUDO) docker build -t $(WEAVER_IMAGE) prog/weaver
$(SUDO) docker save $(WEAVER_IMAGE):latest > $@
touch $@
$(WEAVEDNS_EXPORT): prog/weavedns/Dockerfile $(WEAVEDNS_EXE)
$(SUDO) docker build -t $(WEAVEDNS_IMAGE) prog/weavedns
$(SUDO) docker save $(WEAVEDNS_IMAGE):latest > $@
$(WEAVEEXEC_EXPORT): prog/weaveexec/Dockerfile $(DOCKER_DISTRIB) weave $(SIGPROXY_EXE) $(WEAVEPROXY_EXE) $(WEAVEWAIT_EXE) $(NETCHECK_EXE)
$(WEAVEEXEC_UPTODATE): prog/weaveexec/Dockerfile $(DOCKER_DISTRIB) weave $(SIGPROXY_EXE) $(WEAVEPROXY_EXE) $(WEAVEWAIT_EXE) $(WEAVEWAIT_NOOP_EXE) $(NETCHECK_EXE) $(DOCKERTLSARGS_EXE)
cp weave prog/weaveexec/weave
cp $(SIGPROXY_EXE) prog/weaveexec/sigproxy
cp $(WEAVEPROXY_EXE) prog/weaveexec/weaveproxy
cp $(WEAVEWAIT_EXE) prog/weaveexec/weavewait
cp $(WEAVEWAIT_NOOP_EXE) prog/weaveexec/weavewait_noop
cp $(NETCHECK_EXE) prog/weaveexec/netcheck
cp $(DOCKERTLSARGS_EXE) prog/weaveexec/docker_tls_args
cp $(DOCKER_DISTRIB) prog/weaveexec/docker.tgz
$(SUDO) docker build -t $(WEAVEEXEC_IMAGE) prog/weaveexec
$(SUDO) docker save $(WEAVEEXEC_IMAGE):latest > $@
touch $@
$(WEAVE_EXPORT): $(IMAGES_UPTODATE)
$(SUDO) docker save $(addsuffix :latest,$(IMAGES)) | gzip > $@
$(DOCKER_DISTRIB):
curl -o $(DOCKER_DISTRIB) $(DOCKER_DISTRIB_URL)
tests:
echo "mode: count" > profile.cov
fail=0 ; \
for dir in $$(find . -type f -name '*_test.go' | xargs -n1 dirname | sort -u); do \
output=$$(mktemp cover.XXXXXXXXXX) ; \
if ! go test -cpu 4 -tags netgo -covermode=count -coverprofile=$$output $$dir ; then \
fail=1 ; \
fi ; \
if [ -f $$output ]; then \
tail -n +2 <$$output >>profile.cov; \
rm $$output; \
fi \
done ; \
exit $$fail
go tool cover -html=profile.cov -o=coverage.html
tests: tools/.git
tools/test
$(PUBLISH): publish_%:
lint: tools/.git
tools/lint -nocomment -notestpackage .
tools/.git:
git submodule update --init
$(RUNNER_EXE): tools/.git
make -C tools/runner
$(PUBLISH): publish_%: $(IMAGES_UPTODATE)
$(SUDO) docker tag -f $(DOCKERHUB_USER)/$* $(DOCKERHUB_USER)/$*:$(WEAVE_VERSION)
$(SUDO) docker push $(DOCKERHUB_USER)/$*:$(WEAVE_VERSION)
ifneq ($(UPDATE_LATEST),false)
$(SUDO) docker push $(DOCKERHUB_USER)/$*:latest
endif
publish: $(PUBLISH)
clean:
clean-bin:
-$(SUDO) docker rmi $(IMAGES)
rm -f $(EXES) $(EXPORTS)
rm -f test/tls/*.pem
go clean -r $(addprefix ./,$(dir $(EXES)))
rm -f $(EXES) $(IMAGES_UPTODATE) $(WEAVE_EXPORT)
clean: clean-bin
rm -rf test/tls/*.pem test/coverage.* test/coverage
build:
$(SUDO) go clean -i net
+47 -27
View File
@@ -2,6 +2,17 @@
[![Build Status](https://travis-ci.org/weaveworks/weave.svg?branch=master)](https://travis-ci.org/weaveworks/weave) [![Integration Tests](https://circleci.com/gh/weaveworks/weave/tree/master.svg?style=shield&circle-token=4933c7dabb3d0383e62117565cb9d16df7b1a811)](https://circleci.com/gh/weaveworks/weave) [![Coverage Status](https://coveralls.io/repos/weaveworks/weave/badge.svg)](https://coveralls.io/r/weaveworks/weave)
# About Weaveworks
[Weaveworks](http://weave.works) is the company that develops Weave -
the most productive way for developers to connect, observe and control
Docker containers. To learn about our products, including getting
started tutorials, visit our [website](http://weave.works) and
[documentation](http://docs.weave.works) or continue to read about
some of the more technical aspects of Weave in this readme.
# Weave
Weave creates a virtual network that connects Docker containers
deployed across multiple hosts and enables their automatic discovery.
@@ -35,14 +46,24 @@ Ensure you are running Linux (kernel 3.8 or later) and have Docker
sudo curl -L git.io/weave -o /usr/local/bin/weave
sudo chmod a+x /usr/local/bin/weave
For usage on OSX (with Docker Machine) you first need to make sure
that a VM is running and configured, as shown in [the Docker Machine
documentation](https://docs.docker.com/installation/mac/#from-your-shell).
Then you can launch weave directly from the OSX host.
CoreOS users see [here](https://github.com/fintanr/weave-gs/blob/master/coreos-simple/user-data) for an example of installing weave using cloud-config.
Amazon ECS users see
[here](https://github.com/weaveworks/guides/blob/master/aws-ecs/LATESTAMIs.md)
for the latest Weave AMIs and
[here](http://weave.works/guides/service-discovery-with-weave-aws-ecs.html) to get started with Weave on ECS.
Weave respects the environment variable `DOCKER_HOST`, so you can run
it locally to control a weave network on a remote host.
## Quick Start Screencast
<a href="http://youtu.be/k6r7yuSr0hE" alt="Click to watch the screencast" target="_blank">
<a href="https://youtu.be/kihQCCT1ykE" alt="Click to watch the screencast" target="_blank">
<img src="/docs/hello-screencast.png" />
</a>
@@ -54,8 +75,8 @@ two containers, one on each host.
On $HOST1 we run:
host1$ weave launch && weave launch-dns && weave launch-proxy
host1$ eval $(weave proxy-env)
host1$ weave launch
host1$ eval $(weave env)
host1$ docker run --name a1 -ti ubuntu
> NB: If the first command results in an error like
@@ -68,30 +89,26 @@ On $HOST1 we run:
> `sudo`, since some commands modify environment entries and hence
> they all need to be executed from the same shell.
The first line runs the weave router, DNS and Docker API proxy, each
in their own container. The second line sets the `DOCKER_HOST`
environment variable to point to the proxy, so that containers
launched via the docker command line are automatically attached to the
weave network. Finally, we run our application container; this happens
via the proxy so it is automatically allocated an IP address and
registered in DNS.
The first line runs weave. The second line configures our environment
so that containers launched via the docker command line are
automatically attached to the weave network. Finally, we run our
application container.
That's it! If our application consists of more than one container on
this host we simply launch them with `docker run` as appropriate.
Next we repeat similar steps on `$HOST2`...
host2$ weave launch $HOST1 && weave launch-dns && weave launch-proxy
host2$ eval $(weave proxy-env)
host2$ weave launch $HOST1
host2$ eval $(weave env)
host2$ docker run --name a2 -ti ubuntu
The only difference, apart from the name of the application container,
is that we tell our weave that it should peer with the weave on
`$HOST1` (specified as the IP address or hostname, and optional
`:port`, by which `$HOST2` can reach it). NB: if there is a firewall
between `$HOST1` and `$HOST2`, you must open the weave port (6783 by
default; this can be overriden by setting `WEAVE_PORT`) for TCP and
UDP.
between `$HOST1` and `$HOST2`, you must permit traffic to the weave
control port (TCP 6783) and data ports (UDP 6783/6784).
Note that we could instead have told the weave on `$HOST1` to connect to
`$HOST2`, or told both about each other. Order does not matter here;
@@ -100,11 +117,10 @@ available. Also, we can tell weave to connect to multiple peers by
supplying multiple addresses, separated by spaces. And we can
[add peers dynamically](http://docs.weave.works/weave/latest_release/features.html#dynamic-topologies).
The router, DNS and Docker API proxy need to be started once per
host. The relevant container images are pulled down on demand, but if
you wish you can preload them by running `weave setup` - this is
particularly useful for automated deployments, and ensures that there
are no delays during later operations.
Weave must be started once per host. The relevant container images are
pulled down on demand, but if you wish you can preload them by running
`weave setup` - this is particularly useful for automated deployments,
and ensures that there are no delays during later operations.
Now that we've got everything set up, let's see whether our containers
can talk to each other...
@@ -112,7 +128,7 @@ can talk to each other...
In the container started on `$HOST1`...
root@a1:/# ping -c 1 -q a2
PING a2.weave.local (10.160.0.2) 56(84) bytes of data.
PING a2.weave.local (10.40.0.2) 56(84) bytes of data.
--- a2.weave.local ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.341/0.341/0.341/0.000 ms
@@ -120,7 +136,7 @@ In the container started on `$HOST1`...
Similarly, in the container started on `$HOST2`...
root@a2:/# ping -c 1 -q a1
PING a1.weave.local (10.128.0.2) 56(84) bytes of data.
PING a1.weave.local (10.32.0.2) 56(84) bytes of data.
--- a1.weave.local ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.366/0.366/0.366/0.000 ms
@@ -139,12 +155,16 @@ to each other.
## Contact Us
Found a bug, want to suggest a feature, or have a question?
[File an issue](https://github.com/weaveworks/weave/issues), or email
help@weave.works. When reporting a bug, please include which version of
weave you are running, as shown by `weave version`.
Found a bug, want to suggest a feature, or have a question? You can
[File an issue](https://github.com/weaveworks/weave/issues). And
there is a [Weave Users Google
Group](https://groups.google.com/a/weave.works/forum/#!forum/weave-users),
which you can post to by email at weave-users@weave.works. Or you can
contact Weaveworks directly at help@weave.works. When reporting a
bug, please include which version of weave you are running, as shown
by `weave version`.
Follow weave on Twitter:
Follow on Twitter:
[@weaveworks](https://twitter.com/weaveworks).
Read the Weave blog:
Vendored
+3 -3
View File
@@ -5,7 +5,7 @@ require './vagrant-common.rb'
vm_ip = "172.16.0.3" # arbitrary private IP
pkgs = %w(
lxc-docker
docker-engine
aufs-tools
build-essential
ethtool
@@ -18,8 +18,8 @@ pkgs = %w(
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "ubuntu/ubuntu-14.10-amd64"
config.vm.box_url = "https://cloud-images.ubuntu.com/vagrant/utopic/current/utopic-server-cloudimg-amd64-vagrant-disk1.box"
config.vm.box = "ubuntu/ubuntu-15.04-amd64"
config.vm.box_url = "https://cloud-images.ubuntu.com/vagrant/vivid/current/vivid-server-cloudimg-amd64-vagrant-disk1.box"
config.vm.network "private_network", ip: vm_ip
config.vm.provider :virtualbox do |vb|
-81
View File
@@ -1,81 +0,0 @@
#!/bin/bash
# This scipt lints go files for common errors.
#
# Its runs gofmt and go vet, and optionally golint and
# gocyclo, if they are installed.
#
# With no arguments, it lints the current files staged
# for git commit. Or you can pass it explicit filenames
# (or directories) and it will lint them.
#
# To use this script automatically, run:
# ln -s ../../bin/lint .git/hooks/pre-commit
set -eu
function lint_go {
filename="$1"
result=0
if [ -n "$(gofmt -l "${filename}")" ]; then
result=1
echo "${filename}: run gofmt -w ${filename}!"
fi
go tool vet "${filename}" || result=$?
# golint is completely optional. If you don't like it
# don't have it installed.
if type golint >/dev/null 2>&1; then
# golint doesn't set an exit code it seems
lintoutput=$(golint "${filename}" | grep -vE 'comment|dot imports')
if [ "$lintoutput" != "" ]; then
result=1
echo "$lintoutput"
fi
fi
# gocyclo is completely optional. If you don't like it
# don't have it installed. Also never blocks a commit,
# it just warns.
if type gocyclo >/dev/null 2>&1; then
gocyclo -over 25 "${filename}" | while read line; do
echo "${filename}": higher than 25 cyclomatic complexity - "${line}"
done
fi
return $result
}
function lint {
filename="$1"
ext="${filename##*\.}"
# Don't lint this script
if [ "${filename}" = "bin/lint" ]; then
return
fi
case "$ext" in
go) lint_go "${filename}"
;;
esac
}
function lint_files {
lint_result=0
while read filename; do
lint "${filename}" || lint_result=1
done
exit $lint_result
}
function list_files {
if [ $# -gt 0 ]; then
find "$@" -type f
else
git diff --cached --name-only
fi
}
list_files "$@" | lint_files
+3 -3
View File
@@ -23,14 +23,14 @@ random_mac() {
# In the first byte of the MAC, the 'multicast' bit should be
# clear and 'locally administered' bit should be set. All other
# bits should be random.
od -txC -An -N6 /dev/urandom | sed 's|^ ||;s| |:|;s|[0-3]:|2:|;s|[4-7]:|6:|;s|[89ab]:|a:|;s|[c-f]:|e:|;s| |:|g'
od -txC -An -N6 /dev/urandom | ( read a b c d e f && printf "%02x:$b:$c:$d:$e:$f" $((0x$a & ~1 | 2)) )
}
case "$1" in
launch)
shift 1
for i in $(seq $START $FINISH); do
WEAVE_CONTAINER_NAME=weave$i WEAVE_PORT=$((5000+i)) $weavedir/weave launch -iface '' -name $(random_mac) -nickname weave$i -port $PORT "$@"
WEAVE_CONTAINER_NAME=weave$i WEAVE_PORT=$((5000+i)) $weavedir/weave launch-router --iface '' --name $(random_mac) --nickname weave$i --port $PORT --no-dns "$@"
done
if [ $START -ne 1 ] ; then
status=$(docker inspect --format='{{.State.Running}} {{.NetworkSettings.IPAddress}}' weave$((START-1)))
@@ -48,7 +48,7 @@ case "$1" in
stop)
for i in $(seq $START $FINISH); do
echo Stopping weave$i
WEAVE_CONTAINER_NAME=weave$i $weavedir/weave stop
WEAVE_CONTAINER_NAME=weave$i $weavedir/weave stop-router
done
;;
esac
+156 -65
View File
@@ -10,15 +10,53 @@ RELEASE_DESCRIPTION=${RELEASE_DESCRIPTION:-"Weaving Containers into Applications
PWD=`pwd`
setup() {
## Get the new version number from the most recent tag
if ! LATEST_TAG=$(git describe --abbrev=0 --match='v*' 2>/dev/null) ; then
echo "Could not find an annotated 'v*' version tag." >&2
exit 1
infer_release_type() {
if echo $1 | grep -qE '^v[0-9]+\.[0-9]+\.0+$' ; then
echo MAINLINE
elif echo $1 | grep -qE '^v[0-9]+\.[0-9]+\.[0-9]+$' ; then
echo BRANCH
else
echo PRERELEASE
fi
}
setup() {
# Ensure we have exactly one annotated tag pointing at HEAD
HEAD_TAGS=$(git tag --points-at HEAD)
case $(echo $HEAD_TAGS | wc -w) in
1)
if [ $HEAD_TAGS != "latest_release" ] ; then
LATEST_TAG=$HEAD_TAGS
else
echo "Cannot determine version - latest_release points at HEAD" >&2
exit 1
fi
;;
0)
echo "Cannot determine version - no tags point at HEAD" >&2
exit 1
;;
*)
echo "Cannot determine version - multiple tags point at HEAD:" >&2
for TAG in $HEAD_TAGS ; do
echo -e "\t$TAG" >&2
done
exit 1
;;
esac
RELEASE_TYPE=$(infer_release_type $LATEST_TAG)
echo "== Inferred release type $RELEASE_TYPE from tag $LATEST_TAG"
LATEST_TAG_SHA=$(git rev-parse $LATEST_TAG)
LATEST_TAG_COMMIT_SHA=$(git rev-list -1 $LATEST_TAG)
LATEST_RELEASE_SHA=$(git rev-parse latest_release)
VERSION=${LATEST_TAG#v}
LATEST_RELEASE_COMMIT_SHA=$(git rev-list -1 latest_release)
if [ "$RELEASE_TYPE" != 'PRERELEASE' ] ; then
VERSION=${LATEST_TAG#v}
else
VERSION=${LATEST_TAG}
fi
# NB does not check that this tag is on master
RELEASE_DIR=./releases/$LATEST_TAG
}
@@ -39,8 +77,8 @@ build() {
cd $RELEASE_DIR
## Check that the top changelog entry is this version
if ! latest_changelog=$(grep -o "## Release [0-9].*" -m1 ./CHANGELOG.md) || \
! [ `echo "$latest_changelog" | grep -o '[0-9][^ ]*'` == "$VERSION" ]; then
if ! latest_changelog=$(grep -oP '(?<=^## Release ).*' -m1 ./CHANGELOG.md) || \
! [ `echo "$latest_changelog" = "$VERSION"` ]; then
echo -e "\u2757 Latest changelog entry \"$latest_changelog\" does not match the release version $VERSION" >&2
exit 1
fi
@@ -61,7 +99,7 @@ build() {
fi
## Run tests with the distributables, including version check
v=$(./prog/weaver/weaver --version | grep -o '[0-9].*')
v=$(./prog/weaver/weaver --version | grep -oP '(?<=^weave router ).*')
if ! [ "$v" == "$VERSION" ]; then
echo "Version of distributable "$v" does not match release version $VERSION" >&2
exit 1
@@ -71,24 +109,13 @@ build() {
echo '** Release artefacts in' $RELEASE_DIR
}
publish() {
draft() {
setup
cd $PWD/$RELEASE_DIR
echo "== Sanity checks"
if ! which github-release >/dev/null; then
echo "Please install git-release:" >&2
echo -e "\tgo get github.com/aktau/github-release" >&2
echo "and create a git token per https://github.com/aktau/github-release" >&2
exit 1
fi
if ! [ $(git rev-list -1 $LATEST_TAG) == $(git rev-list -1 latest_release) ]; then
echo -e "\u2757 The tag latest_release does not point to the same commit as $LATEST_TAG"
exit 1
fi
## Check that the tag exists by looking at github
if ! curl -sSf https://api.github.com/repos/$GITHUB_USER/weave/git/tags/$LATEST_TAG_SHA >/dev/null 2>&1; then
echo -e "\u2757 Tag $LATEST_TAG is not on GitHub, or is not the same as the local tag" >&2
@@ -98,19 +125,9 @@ publish() {
exit 1
fi
## Check that the 'latest_release' tag exists by looking at github
if ! curl -sSf https://api.github.com/repos/$GITHUB_USER/weave/git/tags/$LATEST_RELEASE_SHA >/dev/null 2>&1; then
echo -e "\u2757 Tag latest_release is not on GitHub, or is not the same as the local tag" >&2
echo -e "\thttps://github.com/$GITHUB_USER/weave/tags" >&2
echo "You may need to" >&2
echo -e "\tgit tag -af latest_release"
echo -e "\tgit push -f git@github.com:$GITHUB_USER/weave latest_release"
exit 1
fi
echo -e "\u2713 Tag $LATEST_TAG exists in GitHub repo $GITHUB_USER/weave"
## Check that the version doesn't already exist by looking at github
## Check that the version does not already exist by looking at github
## releases
if github-release info --user $GITHUB_USER --repo weave --tag $LATEST_TAG >/dev/null 2>&1; then
echo -e "\u2757 Release $LATEST_TAG already exists on GitHub" >&2
@@ -120,13 +137,13 @@ publish() {
echo '** Sanity checks OK for publishing tag' $LATEST_TAG as $DOCKERHUB_USER/weave:$VERSION
echo "== Tagging and publishing images on docker hub as user $DOCKERHUB_USER"
make SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER publish
RELEASE_ARGS="--draft"
if [ "$RELEASE_TYPE" = 'PRERELEASE' ] ; then
RELEASE_ARGS="$RELEASE_ARGS --pre-release"
fi
echo "== Creating GitHub release $RELEASE_NAME $VERSION"
# This cannot be done as a draft because of a bug in
# github-release: https://github.com/aktau/github-release/issues/7
github-release release \
echo "== Creating GitHub release $RELEASE_ARGS $RELEASE_NAME $VERSION"
github-release release $RELEASE_ARGS \
--user $GITHUB_USER \
--repo weave \
--tag $LATEST_TAG \
@@ -140,52 +157,126 @@ publish() {
--name "weave" \
--file "./weave"
echo "** Published release $RELEASE_NAME $VERSION"
echo "** Draft $TYPE $RELEASE_NAME $VERSION created at"
echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/$LATEST_TAG"
}
if github-release info --user $GITHUB_USER --repo weave \
--tag latest_release >/dev/null 2>&1; then
github-release delete \
--user $GITHUB_USER \
--repo weave \
--tag latest_release
publish() {
setup
cd $PWD/$RELEASE_DIR
UPDATE_LATEST=false
if [ "$RELEASE_TYPE" = 'MAINLINE' ] ; then
UPDATE_LATEST=true
fi
github-release release \
--user $GITHUB_USER \
--repo weave \
--tag latest_release \
--name "$RELEASE_NAME latest ($VERSION)" \
--description "$RELEASE_DESCRIPTION"
if [ "$RELEASE_TYPE" = 'PRERELEASE' ] ; then
echo "== Tagging and pushing images on docker hub as user $DOCKERHUB_USER"
make UPDATE_LATEST=$UPDATE_LATEST SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER publish
echo "** Docker images tagged and pushed"
github-release upload \
--user $GITHUB_USER \
--repo weave \
--tag latest_release \
--name "weave" \
--file "./weave"
echo "== Publishing pre-release on GitHub"
echo "** Published $RELEASE_NAME as latest_release at"
echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/latest_release"
github-release publish \
--user $GITHUB_USER \
--repo weave \
--tag $LATEST_TAG
echo "** Pre-release $RELEASE_NAME $VERSION published at"
echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/$LATEST_TAG"
else
echo "== Sanity checks"
if ! [ "$LATEST_TAG_COMMIT_SHA" == "$LATEST_RELEASE_COMMIT_SHA" ] ; then
echo -e "\u2757 The tag latest_release does not point to the same commit as $LATEST_TAG" >&2
echo "You may need to" >&2
echo -e "\tgit tag -af latest_release $LATEST_TAG" >&2
exit 1
fi
## Check that the 'latest_release' tag exists by looking at github
if ! curl -sSf https://api.github.com/repos/$GITHUB_USER/weave/git/tags/$LATEST_RELEASE_SHA >/dev/null 2>&1; then
echo -e "\u2757 Tag latest_release is not on GitHub, or is not the same as the local tag" >&2
echo -e "\thttps://github.com/$GITHUB_USER/weave/tags" >&2
echo "You may need to" >&2
echo -e "\tgit push -f git@github.com:$GITHUB_USER/weave latest_release" >&2
exit 1
fi
echo '** Sanity checks OK for publishing tag' $LATEST_TAG as $DOCKERHUB_USER/weave:$VERSION
echo "== Tagging and pushing images on docker hub as user $DOCKERHUB_USER"
make UPDATE_LATEST=$UPDATE_LATEST SUDO=$SUDO WEAVE_VERSION=$VERSION DOCKERHUB_USER=$DOCKERHUB_USER publish
echo "** Docker images tagged and pushed"
echo "== Publishing release on GitHub"
github-release publish \
--user $GITHUB_USER \
--repo weave \
--tag $LATEST_TAG
if github-release info --user $GITHUB_USER --repo weave \
--tag latest_release >/dev/null 2>&1; then
github-release delete \
--user $GITHUB_USER \
--repo weave \
--tag latest_release
fi
github-release release \
--user $GITHUB_USER \
--repo weave \
--tag latest_release \
--name "$RELEASE_NAME latest ($VERSION)" \
--description "[Release Notes](https://github.com/$GITHUB_USER/weave/releases/$LATEST_TAG)"
github-release upload \
--user $GITHUB_USER \
--repo weave \
--tag latest_release \
--name "weave" \
--file "./weave"
echo "** Release $RELEASE_NAME $VERSION published at"
echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/$LATEST_TAG"
echo -e "\thttps://github.com/$GITHUB_USER/weave/releases/latest_release"
fi
}
usage() {
echo "Usage:"
echo -e "\t./bin/release build"
echo "-- Build artefacts for the latest version tag"
echo -e "\t./bin/release draft"
echo "-- Create draft release with artefacts in GitHub"
echo -e "\t./bin/release publish"
echo "-- Publish the artefacts for the latest tag to GitHub and DockerHub"
echo "-- Publish the GitHub release and update DockerHub"
exit 1
}
case "$1" in
# Ensure required tooling is installed
if ! which github-release >/dev/null; then
echo "Please install git-release:" >&2
echo -e "\tgo get github.com/weaveworks/github-release" >&2
echo "and create a git token per https://github.com/weaveworks/github-release" >&2
exit 1
fi
[ $# -eq 0 ] && usage
COMMAND=$1
shift
case "$COMMAND" in
build)
build
build "$@"
;;
draft)
draft "$@"
;;
publish)
publish
publish "$@"
;;
*)
echo "Unknown command \"$1\""
echo "Unknown command \"$COMMAND\""
usage
;;
esac
+4 -1
View File
@@ -1,6 +1,9 @@
FROM ubuntu
RUN apt-get -y update && apt-get -y install --no-install-recommends build-essential git ca-certificates docker.io mercurial libpcap-dev curl make pkg-config gcc bison flex
RUN apt-get -y update && apt-get -y install --no-install-recommends ca-certificates apt-transport-https
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
RUN echo deb https://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list
RUN apt-get -y update && apt-get -y install --no-install-recommends build-essential git lxc-docker-1.3.1 mercurial libpcap-dev curl make pkg-config gcc bison flex python-requests
# When doing a build in a container, "apt-get update" happens twice,
# which can be a very significant overhead for incremental builds.
+2 -1
View File
@@ -6,7 +6,7 @@ export GOPATH
WEAVE_SRC=$GOPATH/src/github.com/weaveworks/weave
if [ $# -eq 0 ] ; then
if [ $# -eq 0 -o "$1" = "tests" ] ; then
# No arguments. Expect that the weave repo will be bind-mounted
# into $GOPATH
if ! [ -e $WEAVE_SRC ] ; then
@@ -35,6 +35,7 @@ EOF
echo "weave:x:$uid:$gid::$WEAVE_SRC:/bin/sh" >>/etc/passwd
echo "weave:*:::::::" >>/etc/shadow
echo "weave ALL=(ALL) NOPASSWD: ALL" >>/etc/sudoers
su weave -c "PATH=$PATH make -C $WEAVE_SRC build"
else
# There are arguments to pass to git-clone
+37 -14
View File
@@ -12,39 +12,62 @@ machine:
SRCDIR: /home/ubuntu/src/github.com/weaveworks/weave
PATH: $PATH:$HOME/.local/bin:$HOME/google-cloud-sdk/bin/
CLOUDSDK_CORE_DISABLE_PROMPTS: 1
WEAVE_BUILD: $HOME/docker/weave-build.tar
WEAVE_TEST: $HOME/docker/weave-test.tar
dependencies:
cache_directories:
- "~/docker"
override:
- if [[ -e "$WEAVE_BUILD" ]]; then
docker load -i $WEAVE_BUILD;
- if [[ -e "$WEAVE_TEST" ]]; then
docker load -i $WEAVE_TEST;
else
docker build -t weaveworks/weave-build build;
mkdir -p $(dirname "$WEAVE_BUILD");
docker save weaveworks/weave-build >$WEAVE_BUILD;
docker build -t weaveworks/weave-test test/image;
mkdir -p $(dirname "$WEAVE_TEST");
docker save weaveworks/weave-test >$WEAVE_TEST;
fi
post:
- sudo apt-get update
- sudo apt-get install bc
- pip install requests
- sudo apt-get update && sudo apt-get install bc jq && pip install requests
- curl https://sdk.cloud.google.com | bash
- bin/setup-circleci-secrets "$SECRET_PASSWORD"
- mkdir -p $(dirname $SRCDIR)
- cp -r $(pwd)/ $SRCDIR
- mkdir -p $(dirname $SRCDIR) && cp -r $(pwd)/ $SRCDIR
- cd $SRCDIR/test; ./gce.sh make_template:
parallel: false
test:
override:
- docker run -v /var/run/docker.sock:/run/docker.sock -v /home/ubuntu:/home/go weaveworks/weave-build:
- cd $SRCDIR; make lint:
parallel: true
- cd $SRCDIR/test; ./gce.sh setup:
- docker run -v /home/ubuntu:/home/go -e COVERDIR=test/coverage -e SLOW=true -e CIRCLECI -e CIRCLE_BUILD_NUM -e CIRCLE_NODE_TOTAL -e CIRCLE_NODE_INDEX weaveworks/weave-test:
parallel: true
- cd $SRCDIR/test; . ./gce.sh hosts; ./setup.sh:
- docker run -v /var/run/docker.sock:/run/docker.sock -v /home/ubuntu:/home/go -e COVERAGE=true weaveworks/weave-build:
parallel: true
- cd $SRCDIR/test; . ./gce.sh hosts; ./run_all.sh:
- cd $SRCDIR/test; ./gce.sh setup && eval $(./gce.sh hosts) && ./setup.sh:
parallel: true
- cd $SRCDIR/test; eval $(./gce.sh hosts); export COVERAGE=true; ./run_all.sh:
parallel: true
timeout: 300
- cd $SRCDIR; make clean-bin:
parallel: true
- docker run -v /var/run/docker.sock:/run/docker.sock -v /home/ubuntu:/home/go weaveworks/weave-build:
parallel: true
post:
- cd $SRCDIR/test; ./gce.sh destroy:
parallel: true
teardown:
pre:
- test "$CIRCLE_NODE_INDEX" != "0" || (cd $SRCDIR/test; ./gen_coverage_reports.sh):
parallel: true
- test "$CIRCLE_NODE_INDEX" != "0" || (go get github.com/mattn/goveralls && goveralls -repotoken $COVERALLS_REPO_TOKEN -coverprofile=$SRCDIR/test/profile.cov -service=circleci || true):
parallel: true
- test "$CIRCLE_NODE_INDEX" != "0" || (cd $SRCDIR/test; cp coverage.* $CIRCLE_ARTIFACTS):
parallel: true
deployment:
hub:
branch: master
owner: weaveworks
commands:
- docker login -e $DOCKER_EMAIL -u $DOCKER_USER -p $DOCKER_PASS
- cd $SRCDIR; make publish
+24 -22
View File
@@ -1,23 +1,14 @@
package docker
import (
"regexp"
"github.com/fsouza/go-dockerclient"
. "github.com/weaveworks/weave/common"
)
// Regexp that matches container IDs
var containerIDregexp = regexp.MustCompile("^[a-f0-9]+$")
// IsContainerID returns True if the string provided is a valid container id
func IsContainerID(idStr string) bool {
return containerIDregexp.MatchString(idStr)
}
// An observer for container events
type ContainerObserver interface {
ContainerDied(ident string) error
ContainerStarted(ident string)
ContainerDied(ident string)
}
type Client struct {
@@ -32,25 +23,42 @@ func NewClient(apiPath string) (*Client, error) {
}
client := &Client{dc}
env, err := client.Version()
return client, client.checkWorking(apiPath)
}
func NewVersionedClient(apiPath string, apiVersionString string) (*Client, error) {
dc, err := docker.NewVersionedClient(apiPath, apiVersionString)
if err != nil {
return nil, err
}
Info.Printf("[docker] Using Docker API on %s: %v", apiPath, env)
return client, nil
client := &Client{dc}
return client, client.checkWorking(apiPath)
}
func (c *Client) checkWorking(apiPath string) error {
env, err := c.Version()
if err != nil {
return err
}
Log.Infof("[docker] Using Docker API on %s: %v", apiPath, env)
return nil
}
// AddObserver adds an observer for docker events
func (c *Client) AddObserver(ob ContainerObserver) error {
events := make(chan *docker.APIEvents)
if err := c.AddEventListener(events); err != nil {
Error.Printf("[docker] Unable to add listener to Docker API: %s", err)
Log.Errorf("[docker] Unable to add listener to Docker API: %s", err)
return err
}
go func() {
for event := range events {
switch event.Status {
case "start":
id := event.ID
ob.ContainerStarted(id)
case "die":
id := event.ID
ob.ContainerDied(id)
@@ -62,12 +70,6 @@ func (c *Client) AddObserver(ob ContainerObserver) error {
// IsContainerNotRunning returns true if we have checked with Docker that the ID is not running
func (c *Client) IsContainerNotRunning(idStr string) bool {
if !IsContainerID(idStr) {
Debug.Printf("[docker] '%s' does not seem to be a container id", idStr)
return false
}
// check with Docker whether the container is really running
container, err := c.InspectContainer(idStr)
if err == nil {
return !container.State.Running
@@ -75,6 +77,6 @@ func (c *Client) IsContainerNotRunning(idStr string) bool {
if _, notThere := err.(*docker.NoSuchContainer); notThere {
return true
}
Error.Printf("[docker] Could not check container status: %s", err)
Log.Errorf("[docker] Could not check container status: %s", err)
return false
}
-20
View File
@@ -1,20 +0,0 @@
package common
func CheckFatal(e error) {
if e != nil {
Error.Fatal(e)
}
}
func CheckWarn(e error) {
if e != nil {
Warning.Println(e)
}
}
// Assert test is true, panic otherwise
func Assert(test bool) {
if !test {
panic("Assertion failure")
}
}
-10
View File
@@ -1,10 +0,0 @@
package common
import (
"io/ioutil"
"os"
)
func init() {
InitLogging(ioutil.Discard, os.Stdout, os.Stdout, os.Stderr)
}
+51 -31
View File
@@ -1,46 +1,66 @@
package common
import (
"io"
"io/ioutil"
"log"
"os"
"bytes"
"fmt"
"strings"
"github.com/Sirupsen/logrus"
)
const (
standardLogFlags = log.Ldate | log.Ltime | log.Lmicroseconds
)
type textFormatter struct {
}
// Largely taken from
// http://www.goinggo.net/2013/11/using-log-package-in-go.html
// Based off logrus.TextFormatter, which behaves completely
// differently when you don't want colored output
func (f *textFormatter) Format(entry *logrus.Entry) ([]byte, error) {
b := &bytes.Buffer{}
levelText := strings.ToUpper(entry.Level.String())[0:4]
timeStamp := entry.Time.Format("2006/01/02 15:04:05.000000")
if len(entry.Data) > 0 {
fmt.Fprintf(b, "%s: %s %-44s ", levelText, timeStamp, entry.Message)
for k, v := range entry.Data {
fmt.Fprintf(b, " %s=%v", k, v)
}
} else {
// No padding when there's no fields
fmt.Fprintf(b, "%s: %s %s", levelText, timeStamp, entry.Message)
}
b.WriteByte('\n')
return b.Bytes(), nil
}
var (
Debug *log.Logger
Info *log.Logger
Warning *log.Logger
Error *log.Logger
debugF bool
standardTextFormatter = &textFormatter{}
)
func InitLogging(debugHandle io.Writer,
infoHandle io.Writer,
warningHandle io.Writer,
errorHandle io.Writer) {
var (
Log *logrus.Logger
)
Debug = log.New(debugHandle, "DEBUG: ", standardLogFlags)
Info = log.New(infoHandle, "INFO: ", standardLogFlags)
Warning = log.New(warningHandle, "WARNING: ", standardLogFlags)
Error = log.New(errorHandle, "ERROR: ", standardLogFlags)
func init() {
Log = logrus.New()
Log.Formatter = standardTextFormatter
}
func InitDefaultLogging(debug bool) {
if debug == debugF {
return
func SetLogLevel(levelname string) {
level, err := logrus.ParseLevel(levelname)
if err != nil {
Log.Fatal(err)
}
Log.Level = level
}
func CheckFatal(e error) {
if e != nil {
Log.Fatal(e)
}
}
func CheckWarn(e error) {
if e != nil {
Log.Warnln(e)
}
debugF = debug
debugOut := ioutil.Discard
if debug {
debugOut = os.Stderr
}
InitLogging(debugOut, os.Stdout, os.Stdout, os.Stderr)
}
+31
View File
@@ -0,0 +1,31 @@
package mflagext
import (
"fmt"
"github.com/docker/docker/pkg/mflag"
)
type listOpts struct {
value *[]string
hasBeenSet bool
}
func ListVar(p *[]string, names []string, value []string, usage string) {
*p = value
mflag.Var(&listOpts{p, false}, names, usage)
}
func (opts *listOpts) Set(value string) error {
if opts.hasBeenSet {
(*opts.value) = append((*opts.value), value)
} else {
(*opts.value) = []string{value}
opts.hasBeenSet = true
}
return nil
}
func (opts *listOpts) String() string {
return fmt.Sprintf("%v", []string(*opts.value))
}
+93
View File
@@ -0,0 +1,93 @@
package odp
import (
"fmt"
"net"
"syscall"
"github.com/weaveworks/go-odp/odp"
)
// ODP admin functionality
func CreateDatapath(dpname string) (err error, supported bool) {
dpif, err := odp.NewDpif()
if err != nil {
if odp.IsKernelLacksODPError(err) {
return nil, false
}
return err, true
}
defer dpif.Close()
dp, err := dpif.CreateDatapath(dpname)
if err != nil && !odp.IsDatapathNameAlreadyExistsError(err) {
return err, true
}
// Pick an ephemeral port number to use in probing for vxlan
// support.
udpconn, err := net.ListenUDP("udp4", nil)
if err != nil {
return err, true
}
// we leave the UDP socket open, so creating a vxlan vport on
// the same port number should fail. But that's fine: It's
// still sufficient to probe for support.
portno := uint16(udpconn.LocalAddr().(*net.UDPAddr).Port)
vpid, err := dp.CreateVport(odp.NewVxlanVportSpec(
fmt.Sprintf("vxlan-%d", portno), portno))
if nlerr, ok := err.(odp.NetlinkError); ok {
if syscall.Errno(nlerr) == syscall.EAFNOSUPPORT {
dp.Delete()
return fmt.Errorf("kernel does not have Open vSwitch VXLAN support"), false
}
}
if err == nil {
dp.DeleteVport(vpid)
}
udpconn.Close()
return nil, true
}
func DeleteDatapath(dpname string) error {
dpif, err := odp.NewDpif()
if err != nil {
return err
}
defer dpif.Close()
dp, err := dpif.LookupDatapath(dpname)
if err != nil {
if odp.IsNoSuchDatapathError(err) {
return nil
}
return err
}
return dp.Delete()
}
func AddDatapathInterface(dpname string, ifname string) error {
dpif, err := odp.NewDpif()
if err != nil {
return err
}
defer dpif.Close()
dp, err := dpif.LookupDatapath(dpname)
if err != nil {
return err
}
_, err = dp.CreateVport(odp.NewNetdevVportSpec(ifname))
return err
}
-155
View File
@@ -1,155 +0,0 @@
package common
import (
"container/heap"
"math"
"time"
"sync/atomic"
"github.com/benbjohnson/clock"
)
// A callable is a function that is called and return an (optional) next scheduled time
type callable func() time.Time
type schedCall struct {
t time.Time
c callable
}
func (sc *schedCall) FromNow(now time.Time) time.Duration {
if sc.t.After(now) {
return time.Duration(sc.t.Sub(now).Nanoseconds())
}
return time.Duration(0)
}
////////////////////////////////////////////////////////////////////////////////
// A min-heap of scheduled callables
type schedCallsHeap []*schedCall
func (ch *schedCallsHeap) Len() int { return len(*ch) }
func (ch *schedCallsHeap) Less(i, j int) bool { return (*ch)[i].t.Before((*ch)[j].t) }
func (ch *schedCallsHeap) Swap(i, j int) {
(*ch)[i], (*ch)[j] = (*ch)[j], (*ch)[i]
}
func (ch *schedCallsHeap) Push(x interface{}) {
// Push and Pop use pointer receivers because they modify the slice's length,
// not just its contents.
entry := x.(*schedCall)
*ch = append(*ch, entry)
}
func (ch *schedCallsHeap) Pop() interface{} {
old := *ch
n := len(old)
item := old[n-1]
*ch = old[0 : n-1]
return item
}
////////////////////////////////////////////////////////////////////////////////
// SchedQueue is queue of scheduled callables
type SchedQueue struct {
clock clock.Clock
callablesH schedCallsHeap
schedChan chan *schedCall
closeChan chan bool
counter uint64 // number of calls invoked so far (used for stats). Note: it will wrap.
}
// NewSchedQueue creates a new scheduled queue
func NewSchedQueue(clock clock.Clock) *SchedQueue {
cq := SchedQueue{
clock: clock,
schedChan: make(chan *schedCall),
closeChan: make(chan bool),
}
heap.Init(&cq.callablesH)
return &cq
}
// Start starts the scheduled queue
func (cq *SchedQueue) Start() {
go func() {
defer func() { Debug.Printf("[sched-q] Quitting (%d calls pending)", len(cq.callablesH)) }()
var now time.Time
timer := cq.clock.Timer(time.Duration(math.MaxInt64))
for {
// Wait until an insertion, until the next callable or until we are Stop()ed
select {
case ce := <-cq.schedChan:
heap.Push(&cq.callablesH, ce)
now = cq.clock.Now()
case <-cq.closeChan:
return
case now = <-timer.C:
}
timer.Stop()
durationUntilNext := cq.durationUntilNext(now)
for durationUntilNext == 0 {
sched := heap.Pop(&cq.callablesH).(*schedCall)
atomic.AddUint64(&cq.counter, 1)
schedNextTime := sched.c()
if !schedNextTime.IsZero() {
sched.t = schedNextTime
heap.Push(&cq.callablesH, sched) // TODO: use a Fix() instead of Pop()&Push()
}
now = cq.clock.Now()
durationUntilNext = cq.durationUntilNext(now)
}
timer = cq.clock.Timer(durationUntilNext)
}
}()
}
// Stop stops the scheduled queue
func (cq *SchedQueue) Stop() {
Debug.Printf("[sched-q] Stopping...")
cq.closeChan <- true
}
// Add schedules a call.
// The callable should not modify the scheduled queue in any way.
func (cq *SchedQueue) Add(c callable, t time.Time) {
Debug.Printf("[sched-q] Adding call at %s", t)
ce := schedCall{c: c, t: t}
cq.schedChan <- &ce
}
// Counter returns the number of calls invoked in the queued
// Note: the result will wrap over time.
func (cq *SchedQueue) Count() uint64 {
return atomic.LoadUint64(&cq.counter)
}
func (cq *SchedQueue) durationUntilNext(now time.Time) time.Duration {
if len(cq.callablesH) > 0 {
firstSched := cq.callablesH[0]
return firstSched.FromNow(now)
}
return time.Duration(math.MaxInt64)
}
// ensure that the background goroutine has executed any pending callbacks
func (cq *SchedQueue) Flush() {
ch := make(chan struct{})
flushFunc := func() time.Time {
atomic.AddUint64(&cq.counter, ^uint64(0)) // decrement so nobody sees this
close(ch)
return time.Time{}
}
cq.schedChan <- &schedCall{c: flushFunc, t: time.Time{}}
<-ch
}
-92
View File
@@ -1,92 +0,0 @@
package common
import (
"testing"
"time"
"github.com/benbjohnson/clock"
wt "github.com/weaveworks/weave/testing"
)
// Ensure we can add new calls while forwarding the clock
func TestSchedCallsBasic(t *testing.T) {
InitDefaultLogging(testing.Verbose())
Info.Println("TestSchedCallsBasic starting")
const testSecs = 1000
clk := clock.NewMock()
schedQueue := NewSchedQueue(clk)
schedQueue.Start()
defer schedQueue.Stop()
c := func() time.Time {
return clk.Now().Add(time.Second)
}
schedQueue.Add(c, clk.Now().Add(time.Second))
schedQueue.Flush()
for i := 0; i < testSecs; i++ {
clk.Add(time.Second)
schedQueue.Flush()
}
t.Logf("Now: %s - calls: %d", clk.Now(), schedQueue.Count())
wt.AssertEqualInt(t, (int)(schedQueue.Count()), testSecs, "Number of calls")
}
// Ensure we can create a 100 seconds gap in the middle of the time travel
func TestSchedCallsGap(t *testing.T) {
InitDefaultLogging(testing.Verbose())
Info.Println("TestSchedCallsGap starting")
const testSecs = 1000
clk := clock.NewMock()
schedQueue := NewSchedQueue(clk)
schedQueue.Start()
defer schedQueue.Stop()
c2 := func() time.Time {
if schedQueue.Count() == testSecs/2 {
return clk.Now().Add(time.Duration(100) * time.Second)
}
return clk.Now().Add(time.Second)
}
schedQueue.Add(c2, clk.Now().Add(time.Second))
schedQueue.Flush()
for i := 0; i < testSecs; i++ {
clk.Add(time.Second)
schedQueue.Flush()
}
t.Logf("Now: %s - calls: %d", clk.Now(), schedQueue.Count())
wt.AssertEqualInt(t, (int)(schedQueue.Count()), testSecs-100+1, "Number of calls")
}
func TestSchedCallsStop(t *testing.T) {
InitDefaultLogging(testing.Verbose())
Info.Println("TestSchedCallsStop starting")
const testSecs = 1000
clk := clock.NewMock()
schedQueue := NewSchedQueue(clk)
schedQueue.Start()
defer schedQueue.Stop()
c2 := func() time.Time {
if schedQueue.Count() == testSecs/2 {
return time.Time{}
}
return clk.Now().Add(time.Second)
}
schedQueue.Add(c2, clk.Now().Add(time.Second))
schedQueue.Flush()
for i := 0; i < testSecs; i++ {
clk.Add(time.Second)
schedQueue.Flush()
}
t.Logf("Now: %s - calls: %d", clk.Now(), schedQueue.Count())
wt.AssertEqualInt(t, (int)(schedQueue.Count()), testSecs/2, "Number of calls")
}
+4 -9
View File
@@ -10,28 +10,23 @@ import (
// A subsystem/server/... that can be stopped or queried about the status with a signal
type SignalReceiver interface {
Stop() error
Status() string
}
func SignalHandlerLoop(ss ...SignalReceiver) {
sigs := make(chan os.Signal, 1)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGQUIT, syscall.SIGUSR1)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGQUIT, syscall.SIGTERM)
buf := make([]byte, 1<<20)
for {
switch <-sigs {
case syscall.SIGINT:
Info.Printf("=== received SIGINT ===\n*** exiting\n")
case syscall.SIGINT, syscall.SIGTERM:
Log.Infof("=== received SIGINT/SIGTERM ===\n*** exiting")
for _, subsystem := range ss {
subsystem.Stop()
}
return
case syscall.SIGQUIT:
stacklen := runtime.Stack(buf, true)
Info.Printf("=== received SIGQUIT ===\n*** goroutine dump...\n%s\n*** end\n", buf[:stacklen])
case syscall.SIGUSR1:
for _, subsystem := range ss {
Info.Printf("=== received SIGUSR1 ===\n*** status...\n%s\n*** end\n", subsystem.Status())
}
Log.Infof("=== received SIGQUIT ===\n*** goroutine dump...\n%s\n*** end", buf[:stacklen])
}
}
}
+16 -4
View File
@@ -1,8 +1,20 @@
package common
func OnOff(b bool) string {
if b {
return "on"
import (
"strings"
)
// Assert test is true, panic otherwise
func Assert(test bool) {
if !test {
panic("Assertion failure")
}
return "off"
}
func ErrorMessages(errors []error) string {
var result []string
for _, err := range errors {
result = append(result, err.Error())
}
return strings.Join(result, "\n")
}
+13 -13
View File
@@ -66,23 +66,23 @@ Router:
simply responds to traffic received from the remote peer via TCP
5. We register this connection with the local peer
6a. If we initiated this connection then we now start sending fast
heartbeats to the remote peer so that the remote peer can determine what
address/port it should use to send UDP back to us. To do this, we
spawn off two further threads which are the forwarder
loops. These receive frames which are to be sent to the remote
peer. One of them sends frames without the DF flag set. To do this
it just sends the packets out of the UDP Listener socket. The
other needs to send frames with the DF flag set and needs its own
socket so that it can do PMTU discovery easily. To do this, it
uses a Raw IP socket (IP has no ports, so there's no collision
issue with the UDP Listener socket) and so it must add UDP headers
itself.
6b. If we did not initiate this connection then the UDP Listener
heartbeats to the remote peer so that the remote peer can
determine what address/port it should use to send UDP back to
us. To do this, we spawn off a "forwarder" thread to send
heartbeats, monitor incoming heartbeats, and some other auxiliary
duties. It also consumes frames to be encapsulated and send via
UDP from two channels, for DF and non-DF cases. In the non-DF
case, it can just send the packets out of the UDP Listener
socket. In the DF case, it needs its own socket so that it can do
PMTU discovery easily. To do this, it uses a Raw IP socket (IP has
no ports, so there's no collision issue with the UDP Listener
socket) and so it must add UDP headers itself.
6b. If we did not initiate this connection then the UDP Listener
should start receiving fast heartbeats from the remote peer. From
those it should be able to identify the local connection via the
local peer. It will tell the local connection (communicating to
the actor thread) about the UDP address of the remote peer. The
local connection will then start its forwarder threads as
local connection will then start its forwarder thread as
described in 6a, and start sending fast heartbeats. We send to the
remote peer via TCP a ConnectionEstablished message. The remote
peer receives this (on the TCP receiver process), tells the
+2 -2
View File
@@ -125,8 +125,8 @@ VM2> nc 192.168.122.2 6783
If all is well, you should now be able to launch weave on VM1
then on VM2:
VM1> weaver -iface veth6 -name vm1
VM2> weaver -iface veth6 -name vm2 192.168.122.2
VM1> weaver --iface veth6 --name vm1
VM2> weaver --iface veth6 --name vm2 192.168.122.2
Once the connection is up and running, you should be able to connect
from VM1/ns2 to VM2/ns2 and vice versa:
Binary file not shown.

Before

Width:  |  Height:  |  Size: 276 KiB

After

Width:  |  Height:  |  Size: 43 KiB

+145
View File
@@ -0,0 +1,145 @@
# Release Process
## Prerequisites
* Up-to-date versions of `git`, `go` etc
* Install GitHub release tool `go get github.com/weaveworks/github-release`
* Create a [github token for
github-release](https://help.github.com/articles/creating-an-access-token-for-command-line-use/);
set and export `$GITHUB_TOKEN` with this value
* Update all dependencies with `make update`
## Release Types
The release script behaves differently depending on the kind of
release you are doing. There are three types:
* **Mainline** - a release (typically from master) with version tag
`vX.Y.Z` where Z is zero (e.g. `v2.1.0`)
* **Branch** - a bugfix release (typically from a branch) with version tag
`vX.Y.Z` where Z is non-zero (e.g `v2.1.1`)
* **Prerelease** - a release from an arbitrary branch with an arbitrary
version tag (e.g. `feature-preview-20150904`)
N.B. the release script _infers the release type from the format of the
version tag_. Ensure your tag is in the correct format and the desired
behaviours for that type of release will be obtained from the script.
## Build Phase
### Update CHANGELOG.md
* Checkout the branch from which you wish to release
* Choose a version tag (see above) henceforth referred to as `$TAG`.
* Add a changelog entry for the new tag at the top of `CHANGELOG.md`.
The first line must be a markdown header of the form `## Release
$TAG` for **Prerelease** builds, `## Release ${TAG#v}` otherwise.
Commit the changelog update:
git commit -m "Add release $TAG" CHANGELOG.md
### Create Version Tag
Next you must tag the changelog commit with `$TAG`
git tag -a -m "Release $TAG" $TAG
### Execute Build
You are now ready to perform the build. If you have skipped the
previous steps (e.g. because you're doing a rebuild), you must ensure
that `HEAD` points to the tagged commit. You may then execute
bin/release build
This has the following effects:
* `git tag --points-at HEAD` is used to determine `$TAG` (hence the
`HEAD` requirement)
* Your *local* repository is cloned into `releases/$TAG`
* `CHANGELOG.md` is checked to ensure it has an entry for `$TAG`
* Distributables injected with `$TAG` are built
* Tests are executed
## Draft Phase
### Push Version Tag Upstream
First you must push your version tag upstream, so that an associated
GitHub release may be created:
git push git@github.com:weaveworks/weave $TAG
N.B. if you're testing the release process, push to your fork
instead!
### Create Draft Release
You're now ready to draft your release notes:
bin/release draft
This has the following effects:
* A [release](https://help.github.com/articles/about-releases) is
created in GitHub for `$TAG`. This release is in the draft state, so
it is only visible to contributors; for **Prerelease** builds the
pre-release attribute will also be set
* The `weave` script is uploaded as an attachment to the release
Navigate to https://github.com/weaveworks/weave/releases, 'Edit' the
draft and input the release notes. When you are done make sure you
'Save draft' (and not 'Publish release'!).
Once the release notes have passed review, proceed to the publish
phase.
## Publish Phase
### Move/Force Push `latest_release` Tag
This step must only be performed for **Mainline** and **Branch**
releases:
git tag -af -m "Release $TAG" latest_release $TAG
git push -f git@github.com:weaveworks/weave latest_release
The `latest_release` tag *must* point at `$TAG`, *not* at `HEAD` -
the build script will complain otherwise.
N.B. if you're testing the release process, push to your fork
instead!
### Publish Release & Distributable Artefacts
You can now publish the release and upload the remaining
distributables to DockerHub:
bin/release publish
The effects of this step depend on the inferred release type. The
following occurs for all types:
* Docker images are tagged `$TAG` and pushed to DockerHub
* GitHub release moves from draft to published state
Additionally, for **Mainline** and **Branch** types:
* Release named `latest_release` is updated on GitHub
Finally, for **Mainline** releases only:
* Images tagged `latest` are updated on DockerHub
## Troubleshooting
There's a few things that can go wrong.
* If the build is wonky, e.g., the tests don't pass, you can delete
the directory in `./releases/`, fix whatever it is, move the
version tag (which should still be only local) and have another go.
* If the DockerHub pushes fail (which sadly seems to happen a lot),
you can just run `./bin/release publish` again.
* If you need to overwrite a release you can do so by manually
deleting the GitHub version release and re-running the process.
Please note that the DockerHub `latest` images, GitHub
`latest_release` and download links may be in an inconsistent state
until the overwrite is completed.
+4 -6
View File
@@ -3,7 +3,7 @@ package ipam
import (
"fmt"
"github.com/weaveworks/weave/ipam/address"
"github.com/weaveworks/weave/net/address"
)
type allocateResult struct {
@@ -35,6 +35,8 @@ func (g *allocate) Try(alloc *Allocator) bool {
return true
}
alloc.establishRing()
if ok, addr := alloc.space.Allocate(g.r); ok {
alloc.debugln("Allocated", addr, "for", g.ident, "in", g.r)
alloc.addOwned(g.ident, addr)
@@ -57,11 +59,7 @@ func (g *allocate) Try(alloc *Allocator) bool {
}
func (g *allocate) Cancel() {
g.resultChan <- allocateResult{0, fmt.Errorf("Allocate request for %s cancelled", g.ident)}
}
func (g *allocate) String() string {
return fmt.Sprintf("Allocate for %s", g.ident)
g.resultChan <- allocateResult{0, &errorCancelled{"Allocate", g.ident}}
}
func (g *allocate) ForContainer(ident string) bool {
+165 -114
View File
@@ -8,17 +8,18 @@ import (
"time"
"github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/ipam/address"
"github.com/weaveworks/weave/ipam/paxos"
"github.com/weaveworks/weave/ipam/ring"
"github.com/weaveworks/weave/ipam/space"
"github.com/weaveworks/weave/router"
"github.com/weaveworks/weave/mesh"
"github.com/weaveworks/weave/net/address"
)
// Kinds of message we can unicast to other peers
const (
msgSpaceRequest = iota
msgRingUpdate
msgSpaceRequestDenied
paxosInterval = time.Second * 5
MinSubnetSize = 4 // first and last addresses are excluded, so 2 would be too small
@@ -32,8 +33,6 @@ type operation interface {
Cancel()
String() string
// Does this operation pertain to the given container id?
// Used for tidying up pending operations when containers die.
ForContainer(ident string) bool
@@ -44,37 +43,39 @@ type operation interface {
// are used around data structures.
type Allocator struct {
actionChan chan<- func()
ourName router.PeerName
ourName mesh.PeerName
universe address.Range // superset of all ranges
ring *ring.Ring // information on ranges owned by all peers
space space.Space // more detail on ranges owned by us
owned map[string][]address.Address // who owns what addresses, indexed by container-ID
nicknames map[router.PeerName]string // so we can map nicknames for rmpeer
nicknames map[mesh.PeerName]string // so we can map nicknames for rmpeer
pendingAllocates []operation // held until we get some free space
pendingClaims []operation // held until we know who owns the space
gossip router.Gossip // our link to the outside world for sending messages
gossip mesh.Gossip // our link to the outside world for sending messages
paxos *paxos.Node
paxosTicker *time.Ticker
shuttingDown bool // to avoid doing any requests while trying to shut down
isKnownPeer func(mesh.PeerName) bool
now func() time.Time
}
// NewAllocator creates and initialises a new Allocator
func NewAllocator(ourName router.PeerName, ourUID router.PeerUID, ourNickname string, universe address.Range, quorum uint) *Allocator {
func NewAllocator(ourName mesh.PeerName, ourUID mesh.PeerUID, ourNickname string, universe address.Range, quorum uint, isKnownPeer func(name mesh.PeerName) bool) *Allocator {
return &Allocator{
ourName: ourName,
universe: universe,
ring: ring.New(universe.Start, address.Add(universe.Start, universe.Size()), ourName),
owned: make(map[string][]address.Address),
paxos: paxos.NewNode(ourName, ourUID, quorum),
nicknames: map[router.PeerName]string{ourName: ourNickname},
now: time.Now,
ourName: ourName,
universe: universe,
ring: ring.New(universe.Start, universe.End, ourName),
owned: make(map[string][]address.Address),
paxos: paxos.NewNode(ourName, ourUID, quorum),
nicknames: map[mesh.PeerName]string{ourName: ourNickname},
isKnownPeer: isKnownPeer,
now: time.Now,
}
}
// Start runs the allocator goroutine
func (alloc *Allocator) Start() {
actionChan := make(chan func(), router.ChannelSize)
actionChan := make(chan func(), mesh.ChannelSize)
alloc.actionChan = actionChan
go alloc.actorLoop(actionChan)
}
@@ -94,7 +95,6 @@ func (alloc *Allocator) doOperation(op operation, ops *[]operation) {
op.Cancel()
return
}
alloc.establishRing()
if !op.Try(alloc) {
*ops = append(*ops, op)
}
@@ -128,11 +128,13 @@ func (alloc *Allocator) cancelOps(ops *[]operation) {
// if we found any.
func (alloc *Allocator) cancelOpsFor(ops *[]operation, ident string) bool {
var found bool
for i, op := range *ops {
if op.ForContainer(ident) {
for i := 0; i < len(*ops); {
if op := (*ops)[i]; op.ForContainer(ident) {
found = true
op.Cancel()
*ops = append((*ops)[:i], (*ops)[i+1:]...)
} else {
i++
}
}
return found
@@ -163,25 +165,34 @@ func (alloc *Allocator) tryPendingOps() {
}
}
func hasBeenCancelled(cancelChan <-chan bool) func() bool {
return func() bool {
select {
case <-cancelChan:
return true
default:
return false
func (alloc *Allocator) spaceRequestDenied(sender mesh.PeerName, r address.Range) {
for i := 0; i < len(alloc.pendingClaims); {
claim := alloc.pendingClaims[i].(*claim)
if r.Contains(claim.addr) {
claim.deniedBy(alloc, sender)
alloc.pendingClaims = append(alloc.pendingClaims[:i], alloc.pendingClaims[i+1:]...)
continue
}
i++
}
}
type errorCancelled struct {
kind string
ident string
}
func (e *errorCancelled) Error() string {
return fmt.Sprintf("%s request for %s cancelled", e.kind, e.ident)
}
// Actor client API
// Allocate (Sync) - get new IP address for container with given name in range
// if there isn't any space in that range we block indefinitely
func (alloc *Allocator) Allocate(ident string, r address.Range, cancelChan <-chan bool) (address.Address, error) {
func (alloc *Allocator) Allocate(ident string, r address.Range, hasBeenCancelled func() bool) (address.Address, error) {
resultChan := make(chan allocateResult)
op := &allocate{resultChan: resultChan, ident: ident, r: r,
hasBeenCancelled: hasBeenCancelled(cancelChan)}
op := &allocate{resultChan: resultChan, ident: ident, r: r, hasBeenCancelled: hasBeenCancelled}
alloc.doOperation(op, &alloc.pendingAllocates)
result := <-resultChan
return result.addr, result.err
@@ -202,22 +213,22 @@ func (alloc *Allocator) Lookup(ident string, r address.Range) (address.Address,
}
// Claim an address that we think we should own (Sync)
func (alloc *Allocator) Claim(ident string, addr address.Address) error {
func (alloc *Allocator) Claim(ident string, addr address.Address, noErrorOnUnknown bool) error {
resultChan := make(chan error)
op := &claim{resultChan: resultChan, ident: ident, addr: addr}
op := &claim{resultChan: resultChan, ident: ident, addr: addr, noErrorOnUnknown: noErrorOnUnknown}
alloc.doOperation(op, &alloc.pendingClaims)
return <-resultChan
}
// ContainerDied is provided to satisfy the updater interface; does a 'Delete' underneath. Sync.
func (alloc *Allocator) ContainerDied(ident string) error {
err := alloc.Delete(ident)
if err == nil {
func (alloc *Allocator) ContainerDied(ident string) {
if err := alloc.Delete(ident); err == nil {
alloc.debugln("Container", ident, "died; released addresses")
}
return err
}
func (alloc *Allocator) ContainerStarted(ident string) {}
// Delete (Sync) - release all IP addresses for container with given name
func (alloc *Allocator) Delete(ident string) error {
errChan := make(chan error)
@@ -265,13 +276,31 @@ func (alloc *Allocator) Free(ident string, addrToFree address.Address) error {
return <-errChan
}
// Sync.
func (alloc *Allocator) String() string {
resultChan := make(chan string)
alloc.actionChan <- func() {
resultChan <- alloc.string()
func (alloc *Allocator) pickPeerFromNicknames(isValid func(mesh.PeerName) bool) mesh.PeerName {
for name := range alloc.nicknames {
if name != alloc.ourName && isValid(name) {
return name
}
}
return <-resultChan
return mesh.UnknownPeerName
}
func (alloc *Allocator) pickPeerForTransfer() mesh.PeerName {
// first try alive peers that actively participate in IPAM (i.e. have entries)
if heir := alloc.ring.PickPeerForTransfer(alloc.isKnownPeer); heir != mesh.UnknownPeerName {
return heir
}
// next try alive peers that have IPAM enabled but have no entries
if heir := alloc.pickPeerFromNicknames(alloc.isKnownPeer); heir != mesh.UnknownPeerName {
return heir
}
// next try disappeared peers that still have entries
t := func(mesh.PeerName) bool { return true }
if heir := alloc.ring.PickPeerForTransfer(t); heir != mesh.UnknownPeerName {
return heir
}
// finally, disappeared peers that that passively participated in IPAM
return alloc.pickPeerFromNicknames(t)
}
// Shutdown (Sync)
@@ -282,7 +311,7 @@ func (alloc *Allocator) Shutdown() {
alloc.shuttingDown = true
alloc.cancelOps(&alloc.pendingClaims)
alloc.cancelOps(&alloc.pendingAllocates)
if heir := alloc.ring.PickPeerForTransfer(); heir != router.UnknownPeerName {
if heir := alloc.pickPeerForTransfer(); heir != mesh.UnknownPeerName {
alloc.ring.Transfer(alloc.ourName, heir)
alloc.space.Clear()
alloc.gossip.GossipBroadcast(alloc.Gossip())
@@ -310,7 +339,6 @@ func (alloc *Allocator) AdminTakeoverRanges(peerNameOrNickname string) error {
return
}
delete(alloc.nicknames, peername)
newRanges, err := alloc.ring.Transfer(peername, alloc.ourName)
alloc.space.AddRanges(newRanges)
resultChan <- err
@@ -322,55 +350,75 @@ func (alloc *Allocator) AdminTakeoverRanges(peerNameOrNickname string) error {
// call into the router for this because we are interested in peers
// that have gone away but are still in the ring, which is why we
// maintain our own nicknames map.
func (alloc *Allocator) lookupPeername(name string) (router.PeerName, error) {
func (alloc *Allocator) lookupPeername(name string) (mesh.PeerName, error) {
for peername, nickname := range alloc.nicknames {
if nickname == name {
return peername, nil
}
}
return router.PeerNameFromString(name)
return mesh.PeerNameFromString(name)
}
// Restrict the peers in "nicknames" to those in the ring and our own
// Restrict the peers in "nicknames" to those in the ring plus peers known to the router
func (alloc *Allocator) pruneNicknames() {
ringPeers := alloc.ring.PeerNames()
for name := range alloc.nicknames {
if _, ok := ringPeers[name]; !ok && name != alloc.ourName {
if _, ok := ringPeers[name]; !ok && !alloc.isKnownPeer(name) {
delete(alloc.nicknames, name)
}
}
}
func (alloc *Allocator) annotatePeernames(names []mesh.PeerName) []string {
var res []string
for _, name := range names {
if nickname, found := alloc.nicknames[name]; found {
res = append(res, fmt.Sprint(name, "(", nickname, ")"))
} else {
res = append(res, name.String())
}
}
return res
}
func decodeRange(msg []byte) (r address.Range, err error) {
decoder := gob.NewDecoder(bytes.NewReader(msg))
return r, decoder.Decode(&r)
}
// OnGossipUnicast (Sync)
func (alloc *Allocator) OnGossipUnicast(sender router.PeerName, msg []byte) error {
func (alloc *Allocator) OnGossipUnicast(sender mesh.PeerName, msg []byte) error {
alloc.debugln("OnGossipUnicast from", sender, ": ", len(msg), "bytes")
resultChan := make(chan error)
alloc.actionChan <- func() {
switch msg[0] {
case msgSpaceRequest:
// some other peer asked us for space
decoder := gob.NewDecoder(bytes.NewReader(msg[1:]))
var r address.Range
if err := decoder.Decode(&r); err != nil {
resultChan <- err
return
r, err := decodeRange(msg[1:])
if err == nil {
alloc.donateSpace(r, sender)
}
alloc.donateSpace(r, sender)
resultChan <- nil
resultChan <- err
case msgSpaceRequestDenied:
r, err := decodeRange(msg[1:])
if err == nil {
alloc.spaceRequestDenied(sender, r)
}
resultChan <- err
case msgRingUpdate:
resultChan <- alloc.update(msg[1:])
resultChan <- alloc.update(sender, msg[1:])
}
}
return <-resultChan
}
// OnGossipBroadcast (Sync)
func (alloc *Allocator) OnGossipBroadcast(msg []byte) (router.GossipData, error) {
alloc.debugln("OnGossipBroadcast:", len(msg), "bytes")
func (alloc *Allocator) OnGossipBroadcast(sender mesh.PeerName, msg []byte) (mesh.GossipData, error) {
alloc.debugln("OnGossipBroadcast from", sender, ":", len(msg), "bytes")
resultChan := make(chan error)
alloc.actionChan <- func() {
resultChan <- alloc.update(msg)
resultChan <- alloc.update(sender, msg)
}
return alloc.Gossip(), <-resultChan
}
@@ -379,7 +427,7 @@ type gossipState struct {
// We send a timstamp along with the information to be
// gossipped in order to detect skewed clocks
Now int64
Nicknames map[router.PeerName]string
Nicknames map[mesh.PeerName]string
Paxos paxos.GossipState
Ring *ring.Ring
@@ -415,11 +463,11 @@ func (alloc *Allocator) Encode() []byte {
}
// OnGossip (Sync)
func (alloc *Allocator) OnGossip(msg []byte) (router.GossipData, error) {
func (alloc *Allocator) OnGossip(msg []byte) (mesh.GossipData, error) {
alloc.debugln("Allocator.OnGossip:", len(msg), "bytes")
resultChan := make(chan error)
alloc.actionChan <- func() {
resultChan <- alloc.update(msg)
resultChan <- alloc.update(mesh.UnknownPeerName, msg)
}
return nil, <-resultChan // for now, we never propagate updates. TBD
}
@@ -430,7 +478,7 @@ type ipamGossipData struct {
alloc *Allocator
}
func (d *ipamGossipData) Merge(other router.GossipData) {
func (d *ipamGossipData) Merge(other mesh.GossipData) {
// no-op
}
@@ -440,12 +488,12 @@ func (d *ipamGossipData) Encode() [][]byte {
// Gossip returns a GossipData implementation, which in this case always
// returns the latest ring state (and does nothing on merge)
func (alloc *Allocator) Gossip() router.GossipData {
func (alloc *Allocator) Gossip() mesh.GossipData {
return &ipamGossipData{alloc}
}
// SetInterfaces gives the allocator two interfaces for talking to the outside world
func (alloc *Allocator) SetInterfaces(gossip router.Gossip) {
func (alloc *Allocator) SetInterfaces(gossip mesh.Gossip) {
alloc.gossip = gossip
}
@@ -475,30 +523,6 @@ func (alloc *Allocator) actorLoop(actionChan <-chan func()) {
// Helper functions
func (alloc *Allocator) string() string {
var buf bytes.Buffer
fmt.Fprintf(&buf, "Allocator range %s", alloc.universe)
if alloc.ring.Empty() {
if alloc.paxosTicker != nil {
fmt.Fprintf(&buf, " awaiting consensus: %s", alloc.paxos.String())
}
} else {
fmt.Fprint(&buf, "\nOwned Ranges:")
alloc.ring.FprintWithNicknames(&buf, alloc.nicknames)
}
if len(alloc.pendingAllocates)+len(alloc.pendingClaims) > 0 {
fmt.Fprintf(&buf, "\nPending requests:")
for _, op := range alloc.pendingAllocates {
fmt.Fprintf(&buf, "\n %s", op.String())
}
for _, op := range alloc.pendingClaims {
fmt.Fprintf(&buf, "\n %s", op.String())
}
}
return buf.String()
}
// Ensure we are making progress towards an established ring
func (alloc *Allocator) establishRing() {
if !alloc.ring.Empty() || alloc.paxosTicker != nil {
@@ -516,7 +540,7 @@ func (alloc *Allocator) establishRing() {
}
}
func (alloc *Allocator) createRing(peers []router.PeerName) {
func (alloc *Allocator) createRing(peers []mesh.PeerName) {
alloc.debugln("Paxos consensus:", peers)
alloc.ring.ClaimForPeers(normalizeConsensus(peers))
alloc.gossip.GossipBroadcast(alloc.Gossip())
@@ -539,7 +563,7 @@ func (alloc *Allocator) ringUpdated() {
}
// For compatibility with sort.Interface
type peerNames []router.PeerName
type peerNames []mesh.PeerName
func (a peerNames) Len() int { return len(a) }
func (a peerNames) Less(i, j int) bool { return a[i] < a[j] }
@@ -548,7 +572,7 @@ func (a peerNames) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
// When we get a consensus from Paxos, the peer names are not in a
// defined order and may contain duplicates. This function sorts them
// and de-dupes.
func normalizeConsensus(consensus []router.PeerName) []router.PeerName {
func normalizeConsensus(consensus []mesh.PeerName) []mesh.PeerName {
if len(consensus) == 0 {
return nil
}
@@ -574,22 +598,31 @@ func (alloc *Allocator) propose() {
alloc.gossip.GossipBroadcast(alloc.Gossip())
}
func (alloc *Allocator) sendSpaceRequest(dest router.PeerName, r address.Range) error {
func encodeRange(r address.Range) []byte {
buf := new(bytes.Buffer)
enc := gob.NewEncoder(buf)
if err := enc.Encode(r); err != nil {
panic(err)
}
msg := router.Concat([]byte{msgSpaceRequest}, buf.Bytes())
return buf.Bytes()
}
func (alloc *Allocator) sendSpaceRequest(dest mesh.PeerName, r address.Range) error {
msg := append([]byte{msgSpaceRequest}, encodeRange(r)...)
return alloc.gossip.GossipUnicast(dest, msg)
}
func (alloc *Allocator) sendRingUpdate(dest router.PeerName) {
msg := router.Concat([]byte{msgRingUpdate}, alloc.encode())
func (alloc *Allocator) sendSpaceRequestDenied(dest mesh.PeerName, r address.Range) error {
msg := append([]byte{msgSpaceRequestDenied}, encodeRange(r)...)
return alloc.gossip.GossipUnicast(dest, msg)
}
func (alloc *Allocator) sendRingUpdate(dest mesh.PeerName) {
msg := append([]byte{msgRingUpdate}, alloc.encode()...)
alloc.gossip.GossipUnicast(dest, msg)
}
func (alloc *Allocator) update(msg []byte) error {
func (alloc *Allocator) update(sender mesh.PeerName, msg []byte) error {
reader := bytes.NewReader(msg)
decoder := gob.NewDecoder(reader)
var data gossipState
@@ -613,31 +646,45 @@ func (alloc *Allocator) update(msg []byte) error {
// shouldn't get updates for a empty Ring. But tolerate
// them just in case.
if data.Ring != nil {
err = alloc.ring.Merge(*data.Ring)
if !alloc.ring.Empty() {
alloc.pruneNicknames()
alloc.ringUpdated()
switch err = alloc.ring.Merge(*data.Ring); err {
case ring.ErrDifferentSeeds:
return fmt.Errorf("IP allocation was seeded by different peers (received: %v, ours: %v)",
alloc.annotatePeernames(data.Ring.Seeds), alloc.annotatePeernames(alloc.ring.Seeds))
case ring.ErrDifferentRange:
return fmt.Errorf("Incompatible IP allocation ranges (received: %s, ours: %s)",
data.Ring.Range().AsCIDRString(), alloc.ring.Range().AsCIDRString())
default:
if err == nil && !alloc.ring.Empty() {
alloc.pruneNicknames()
alloc.ringUpdated()
}
return err
}
return err
}
if data.Paxos != nil && alloc.ring.Empty() {
if alloc.paxos.Update(data.Paxos) {
if alloc.paxos.Think() {
// If something important changed, broadcast
alloc.gossip.GossipBroadcast(alloc.Gossip())
}
if data.Paxos != nil {
if alloc.ring.Empty() {
if alloc.paxos.Update(data.Paxos) {
if alloc.paxos.Think() {
// If something important changed, broadcast
alloc.gossip.GossipBroadcast(alloc.Gossip())
}
if ok, cons := alloc.paxos.Consensus(); ok {
alloc.createRing(cons.Value)
if ok, cons := alloc.paxos.Consensus(); ok {
alloc.createRing(cons.Value)
}
}
} else if sender != mesh.UnknownPeerName {
// Sender is trying to initialize a ring, but we have one
// already - send it straight back
alloc.sendRingUpdate(sender)
}
}
return nil
}
func (alloc *Allocator) donateSpace(r address.Range, to router.PeerName) {
func (alloc *Allocator) donateSpace(r address.Range, to mesh.PeerName) {
// No matter what we do, we'll send a unicast gossip
// of our ring back to tha chap who asked for space.
// This serves to both tell him of any space we might
@@ -651,10 +698,14 @@ func (alloc *Allocator) donateSpace(r address.Range, to router.PeerName) {
free := alloc.space.NumFreeAddressesInRange(r)
common.Assert(free == 0)
alloc.debugln("No space to give to peer", to)
// separate message maintains backwards-compatibility:
// down-level peers will ignore this and still get the ring update.
alloc.sendSpaceRequestDenied(to, r)
return
}
alloc.debugln("Giving range", chunk, "to", to)
alloc.ring.GrantRangeToHost(chunk.Start, chunk.End, to)
alloc.sendRingUpdate(to)
}
func (alloc *Allocator) assertInvariants() {
@@ -717,11 +768,11 @@ func (alloc *Allocator) findOwner(addr address.Address) string {
// Logging
func (alloc *Allocator) infof(fmt string, args ...interface{}) {
common.Info.Printf("[allocator %s] "+fmt, append([]interface{}{alloc.ourName}, args...)...)
common.Log.Infof("[allocator %s] "+fmt, append([]interface{}{alloc.ourName}, args...)...)
}
func (alloc *Allocator) debugln(args ...interface{}) {
common.Debug.Println(append([]interface{}{fmt.Sprintf("[allocator %s]:", alloc.ourName)}, args...)...)
common.Log.Debugln(append([]interface{}{fmt.Sprintf("[allocator %s]:", alloc.ourName)}, args...)...)
}
func (alloc *Allocator) debugf(fmt string, args ...interface{}) {
common.Debug.Printf("[allocator %s] "+fmt, append([]interface{}{alloc.ourName}, args...)...)
common.Log.Debugf("[allocator %s] "+fmt, append([]interface{}{alloc.ourName}, args...)...)
}
+190 -99
View File
@@ -7,10 +7,11 @@ import (
"testing"
"time"
"github.com/stretchr/testify/require"
"github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/ipam/address"
"github.com/weaveworks/weave/router"
wt "github.com/weaveworks/weave/testing"
"github.com/weaveworks/weave/net/address"
"github.com/weaveworks/weave/testing/gossip"
)
const (
@@ -19,6 +20,8 @@ const (
testStart3 = "10.0.3.0"
)
func returnFalse() bool { return false }
func TestAllocFree(t *testing.T) {
const (
container1 = "abcdef"
@@ -38,40 +41,39 @@ func TestAllocFree(t *testing.T) {
_, cidr2, _ := address.ParseCIDR(subnet2)
alloc.claimRingForTesting()
addr1, err := alloc.Allocate(container1, cidr1.HostRange(), nil)
wt.AssertNoErr(t, err)
wt.AssertEqualString(t, addr1.String(), testAddr1, "address")
addr1, err := alloc.Allocate(container1, cidr1.HostRange(), returnFalse)
require.NoError(t, err)
require.Equal(t, testAddr1, addr1.String(), "address")
addr2, err := alloc.Allocate(container1, cidr2.HostRange(), nil)
wt.AssertNoErr(t, err)
wt.AssertEqualString(t, addr2.String(), testAddr2, "address")
addr2, err := alloc.Allocate(container1, cidr2.HostRange(), returnFalse)
require.NoError(t, err)
require.Equal(t, testAddr2, addr2.String(), "address")
// Ask for another address for a different container and check it's different
addr1b, _ := alloc.Allocate(container2, cidr1.HostRange(), nil)
addr1b, _ := alloc.Allocate(container2, cidr1.HostRange(), returnFalse)
if addr1b.String() == testAddr1 {
t.Fatalf("Expected different address but got %s", addr1b.String())
}
// Ask for the first container again and we should get the same addresses again
addr1a, _ := alloc.Allocate(container1, cidr1.HostRange(), nil)
wt.AssertEqualString(t, addr1a.String(), testAddr1, "address")
addr2a, _ := alloc.Allocate(container1, cidr2.HostRange(), nil)
wt.AssertEqualString(t, addr2a.String(), testAddr2, "address")
addr1a, _ := alloc.Allocate(container1, cidr1.HostRange(), returnFalse)
require.Equal(t, testAddr1, addr1a.String(), "address")
addr2a, _ := alloc.Allocate(container1, cidr2.HostRange(), returnFalse)
require.Equal(t, testAddr2, addr2a.String(), "address")
// Now delete the first container, and we should get its addresses back
wt.AssertSuccess(t, alloc.Delete(container1))
addr3, _ := alloc.Allocate(container3, cidr1.HostRange(), nil)
wt.AssertEqualString(t, addr3.String(), testAddr1, "address")
addr4, _ := alloc.Allocate(container3, cidr2.HostRange(), nil)
wt.AssertEqualString(t, addr4.String(), testAddr2, "address")
require.NoError(t, alloc.Delete(container1))
addr3, _ := alloc.Allocate(container3, cidr1.HostRange(), returnFalse)
require.Equal(t, testAddr1, addr3.String(), "address")
addr4, _ := alloc.Allocate(container3, cidr2.HostRange(), returnFalse)
require.Equal(t, testAddr2, addr4.String(), "address")
alloc.ContainerDied(container2)
alloc.ContainerDied(container3)
wt.AssertEquals(t, alloc.NumFreeAddresses(subnet), address.Offset(spaceSize))
require.Equal(t, address.Offset(spaceSize), alloc.NumFreeAddresses(subnet))
}
func TestBootstrap(t *testing.T) {
common.InitDefaultLogging(false)
const (
donateSize = 5
donateStart = "10.0.1.7"
@@ -86,14 +88,14 @@ func TestBootstrap(t *testing.T) {
alloc2, _ := makeAllocatorWithMockGossip(t, peerNameString, testStart1+"/22", 2)
defer alloc2.Stop()
alloc1.OnGossipBroadcast(alloc2.Encode())
alloc1.OnGossipBroadcast(alloc2.ourName, alloc2.Encode())
alloc1.actionChan <- func() { alloc1.tryPendingOps() }
ExpectBroadcastMessage(alloc1, nil) // alloc1 will try to form consensus
done := make(chan bool)
go func() {
alloc1.Allocate("somecontainer", subnet, nil)
alloc1.Allocate("somecontainer", subnet, returnFalse)
done <- true
}()
time.Sleep(100 * time.Millisecond)
@@ -108,19 +110,19 @@ func TestBootstrap(t *testing.T) {
// alloc2 receives paxos update and broadcasts its reply
ExpectBroadcastMessage(alloc2, nil)
alloc2.OnGossipBroadcast(alloc1.Encode())
alloc2.OnGossipBroadcast(alloc1.ourName, alloc1.Encode())
ExpectBroadcastMessage(alloc1, nil)
alloc1.OnGossipBroadcast(alloc2.Encode())
alloc1.OnGossipBroadcast(alloc2.ourName, alloc2.Encode())
// both nodes will get consensus now so initialize the ring
ExpectBroadcastMessage(alloc2, nil)
ExpectBroadcastMessage(alloc2, nil)
alloc2.OnGossipBroadcast(alloc1.Encode())
alloc2.OnGossipBroadcast(alloc1.ourName, alloc1.Encode())
CheckAllExpectedMessagesSent(alloc1, alloc2)
alloc1.OnGossipBroadcast(alloc2.Encode())
alloc1.OnGossipBroadcast(alloc2.ourName, alloc2.Encode())
// now alloc1 should have space
AssertSent(t, done)
@@ -131,23 +133,45 @@ func TestBootstrap(t *testing.T) {
func TestAllocatorClaim(t *testing.T) {
const (
container1 = "abcdef"
container2 = "baddf00d"
container3 = "b01df00d"
universe = "10.0.3.0/30"
testAddr1 = "10.0.3.1" // first address allocated should be .1 because .0 is network addr
universe = "10.0.3.0/24"
testAddr1 = "10.0.3.2"
testAddr2 = "10.0.4.2"
)
alloc, subnet := makeAllocatorWithMockGossip(t, "01:00:00:01:00:00", universe, 1)
allocs, _, subnet := makeNetworkOfAllocators(2, universe)
alloc := allocs[1]
defer alloc.Stop()
alloc.claimRingForTesting()
addr1, _ := address.ParseIP(testAddr1)
err := alloc.Claim(container3, addr1)
wt.AssertNoErr(t, err)
// First claim should trigger "dunno, I'm going to wait"
err := alloc.Claim(container3, addr1, true)
require.NoError(t, err)
// Do one allocate to ensure paxos is all done
alloc.Allocate("unused", subnet, returnFalse)
// Do an allocate on the other peer, which we will try to claim later
addrx, err := allocs[0].Allocate(container1, subnet, returnFalse)
// Now try the claim again
err = alloc.Claim(container3, addr1, true)
require.NoError(t, err)
// Check we get this address back if we try an allocate
addr3, _ := alloc.Allocate(container3, subnet, nil)
wt.AssertEqualString(t, addr3.String(), testAddr1, "address")
addr3, _ := alloc.Allocate(container3, subnet, returnFalse)
require.Equal(t, testAddr1, addr3.String(), "address")
// one more claim should still work
err = alloc.Claim(container3, addr1, true)
require.NoError(t, err)
// claim for a different container should fail
err = alloc.Claim(container1, addr1, true)
require.Error(t, err)
// claiming the address allocated on the other peer should fail
err = alloc.Claim(container1, addrx, true)
require.Error(t, err, "claiming address allocated on other peer should fail")
// Check an address outside of our universe
addr2, _ := address.ParseIP(testAddr2)
err = alloc.Claim(container1, addr2, true)
require.NoError(t, err)
}
func (alloc *Allocator) pause() func() {
@@ -163,18 +187,17 @@ func (alloc *Allocator) pause() func() {
}
func TestCancel(t *testing.T) {
common.InitDefaultLogging(false)
const (
CIDR = "10.0.1.7/26"
)
router := TestGossipRouter{make(map[router.PeerName]chan interface{}), 0.0}
router := gossip.NewTestRouter(0.0)
alloc1, subnet := makeAllocator("01:00:00:02:00:00", CIDR, 2)
alloc1.SetInterfaces(router.connect(alloc1.ourName, alloc1))
alloc1.SetInterfaces(router.Connect(alloc1.ourName, alloc1))
alloc2, _ := makeAllocator("02:00:00:02:00:00", CIDR, 2)
alloc2.SetInterfaces(router.connect(alloc2.ourName, alloc2))
alloc2.SetInterfaces(router.Connect(alloc2.ourName, alloc2))
alloc1.claimRingForTesting(alloc1, alloc2)
alloc2.claimRingForTesting(alloc1, alloc2)
@@ -182,15 +205,15 @@ func TestCancel(t *testing.T) {
alloc2.Start()
// tell peers about each other
alloc1.OnGossipBroadcast(alloc2.Encode())
alloc1.OnGossipBroadcast(alloc2.ourName, alloc2.Encode())
// Get some IPs, so each allocator has some space
res1, _ := alloc1.Allocate("foo", subnet, nil)
common.Debug.Printf("res1 = %s", res1.String())
res2, _ := alloc2.Allocate("bar", subnet, nil)
common.Debug.Printf("res2 = %s", res2.String())
res1, _ := alloc1.Allocate("foo", subnet, returnFalse)
common.Log.Debugf("res1 = %s", res1.String())
res2, _ := alloc2.Allocate("bar", subnet, returnFalse)
common.Log.Debugf("res2 = %s", res2.String())
if res1 == res2 {
wt.Fatalf(t, "Error: got same ips!")
require.FailNow(t, "Error: got same ips!")
}
// Now we're going to pause alloc2 and ask alloc1
@@ -199,23 +222,63 @@ func TestCancel(t *testing.T) {
// Use up all the IPs that alloc1 owns, so the allocation after this will prompt a request to alloc2
for i := 0; alloc1.NumFreeAddresses(subnet) > 0; i++ {
alloc1.Allocate(fmt.Sprintf("tmp%d", i), subnet, nil)
alloc1.Allocate(fmt.Sprintf("tmp%d", i), subnet, returnFalse)
}
cancelChan := make(chan bool, 1)
doneChan := make(chan bool)
go func() {
_, ok := alloc1.Allocate("baz", subnet, cancelChan)
_, ok := alloc1.Allocate("baz", subnet,
func() bool {
select {
case <-cancelChan:
return true
default:
return false
}
})
doneChan <- ok == nil
}()
AssertNothingSent(t, doneChan)
time.Sleep(100 * time.Millisecond)
AssertNothingSent(t, doneChan)
cancelChan <- true
unpause()
if <-doneChan {
wt.Fatalf(t, "Error: got result from Allocate")
require.FailNow(t, "Error: got result from Allocate")
}
}
func TestCancelOnDied(t *testing.T) {
const (
CIDR = "10.0.1.7/26"
container1 = "abcdef"
)
router := gossip.NewTestRouter(0.0)
alloc1, subnet := makeAllocator("01:00:00:02:00:00", CIDR, 2)
alloc1.SetInterfaces(router.Connect(alloc1.ourName, alloc1))
alloc1.Start()
doneChan := make(chan bool)
f := func() {
_, ok := alloc1.Allocate(container1, subnet, returnFalse)
doneChan <- ok == nil
}
// Attempt two allocations in parallel, to check that this is handled correctly
go f()
go f()
// Nothing should happen, because we declared the quorum as 2
time.Sleep(100 * time.Millisecond)
AssertNothingSent(t, doneChan)
alloc1.ContainerDied(container1)
// Check that the two allocations both exit with an error
if <-doneChan || <-doneChan {
require.FailNow(t, "Error: got result from Allocate")
}
}
@@ -230,12 +293,12 @@ func TestGossipShutdown(t *testing.T) {
defer alloc.Stop()
alloc.claimRingForTesting()
alloc.Allocate(container1, subnet, nil)
alloc.Allocate(container1, subnet, returnFalse)
alloc.Shutdown()
_, err := alloc.Allocate(container2, subnet, nil) // trying to allocate after shutdown should fail
wt.AssertFalse(t, err == nil, "no address")
_, err := alloc.Allocate(container2, subnet, returnFalse) // trying to allocate after shutdown should fail
require.False(t, err == nil, "no address")
CheckAllExpectedMessagesSent(alloc)
}
@@ -249,34 +312,33 @@ func TestTransfer(t *testing.T) {
alloc2 := allocs[1]
alloc3 := allocs[2] // This will be 'master' and get the first range
_, err := alloc2.Allocate("foo", subnet, nil)
wt.AssertTrue(t, err == nil, "Failed to get address")
_, err := alloc2.Allocate("foo", subnet, returnFalse)
require.True(t, err == nil, "Failed to get address")
_, err = alloc3.Allocate("bar", subnet, nil)
wt.AssertTrue(t, err == nil, "Failed to get address")
_, err = alloc3.Allocate("bar", subnet, returnFalse)
require.True(t, err == nil, "Failed to get address")
router.GossipBroadcast(alloc2.Gossip())
router.flush()
router.GossipBroadcast(alloc3.Gossip())
router.flush()
router.removePeer(alloc2.ourName)
router.removePeer(alloc3.ourName)
alloc2.gossip.GossipBroadcast(alloc2.Gossip())
router.Flush()
alloc2.gossip.GossipBroadcast(alloc3.Gossip())
router.Flush()
router.RemovePeer(alloc2.ourName)
router.RemovePeer(alloc3.ourName)
alloc2.Stop()
alloc3.Stop()
router.flush()
wt.AssertSuccess(t, alloc1.AdminTakeoverRanges(alloc2.ourName.String()))
wt.AssertSuccess(t, alloc1.AdminTakeoverRanges(alloc3.ourName.String()))
router.flush()
router.Flush()
require.NoError(t, alloc1.AdminTakeoverRanges(alloc2.ourName.String()))
require.NoError(t, alloc1.AdminTakeoverRanges(alloc3.ourName.String()))
router.Flush()
wt.AssertEquals(t, alloc1.NumFreeAddresses(subnet), address.Offset(1022))
require.Equal(t, address.Offset(1022), alloc1.NumFreeAddresses(subnet))
_, err = alloc1.Allocate("foo", subnet, nil)
wt.AssertTrue(t, err == nil, "Failed to get address")
_, err = alloc1.Allocate("foo", subnet, returnFalse)
require.True(t, err == nil, "Failed to get address")
alloc1.Stop()
}
func TestFakeRouterSimple(t *testing.T) {
common.InitDefaultLogging(false)
const (
cidr = "10.0.1.7/22"
)
@@ -286,18 +348,17 @@ func TestFakeRouterSimple(t *testing.T) {
alloc1 := allocs[0]
//alloc2 := allocs[1]
_, err := alloc1.Allocate("foo", subnet, nil)
wt.AssertTrue(t, err == nil, "Failed to get address")
_, err := alloc1.Allocate("foo", subnet, returnFalse)
require.NoError(t, err, "Failed to get address")
}
func TestAllocatorFuzz(t *testing.T) {
common.InitDefaultLogging(false)
const (
firstpass = 1000
secondpass = 20000
nodes = 10
secondpass = 10000
nodes = 5
maxAddresses = 1000
concurrency = 30
concurrency = 5
cidr = "10.0.1.7/22"
)
allocs, _, subnet := makeNetworkOfAllocators(nodes, cidr)
@@ -329,28 +390,19 @@ func TestAllocatorFuzz(t *testing.T) {
return xs[:ls]
}
// Do a Allocate and check the address
// is unique. Needs a unique container
// name.
allocate := func(name string) {
bumpPending := func() bool {
stateLock.Lock()
if len(addrs)+numPending >= maxAddresses {
stateLock.Unlock()
return
return false
}
numPending++
stateLock.Unlock()
return true
}
allocIndex := rand.Int31n(nodes)
alloc := allocs[allocIndex]
//common.Info.Printf("Allocate: asking allocator %d", allocIndex)
addr, err := alloc.Allocate(name, subnet, nil)
if err != nil {
panic(fmt.Sprintf("Could not allocate addr"))
}
//common.Info.Printf("Allocate: got address %s for name %s", addr, name)
noteAllocation := func(allocIndex int32, name string, addr address.Address) {
//common.Log.Infof("Allocate: got address %s for name %s", addr, name)
addrStr := addr.String()
stateLock.Lock()
@@ -366,6 +418,26 @@ func TestAllocatorFuzz(t *testing.T) {
numPending--
}
// Do a Allocate and check the address
// is unique. Needs a unique container
// name.
allocate := func(name string) {
if !bumpPending() {
return
}
allocIndex := rand.Int31n(nodes)
alloc := allocs[allocIndex]
//common.Log.Infof("Allocate: asking allocator %d", allocIndex)
addr, err := alloc.Allocate(name, subnet, returnFalse)
if err != nil {
panic(fmt.Sprintf("Could not allocate addr"))
}
noteAllocation(allocIndex, name, addr)
}
// Free a random address.
free := func() {
stateLock.Lock()
@@ -387,13 +459,13 @@ func TestAllocatorFuzz(t *testing.T) {
stateLock.Unlock()
alloc := allocs[res.alloc]
//common.Info.Printf("Freeing %s (%s) on allocator %d", res.name, addr, res.alloc)
//common.Log.Infof("Freeing %s (%s) on allocator %d", res.name, addr, res.alloc)
oldAddr, err := address.ParseIP(addr)
if err != nil {
panic(err)
}
wt.AssertSuccess(t, alloc.Free(res.name, oldAddr))
require.NoError(t, alloc.Free(res.name, oldAddr))
}
// Do a Allocate on an existing container & allocator
@@ -412,9 +484,9 @@ func TestAllocatorFuzz(t *testing.T) {
stateLock.Unlock()
alloc := allocs[res.alloc]
//common.Info.Printf("Asking for %s (%s) on allocator %d again", res.name, addr, res.alloc)
//common.Log.Infof("Asking for %s (%s) on allocator %d again", res.name, addr, res.alloc)
newAddr, _ := alloc.Allocate(res.name, subnet, nil)
newAddr, _ := alloc.Allocate(res.name, subnet, returnFalse)
oldAddr, _ := address.ParseIP(addr)
if newAddr != oldAddr {
panic(fmt.Sprintf("Got different address for repeat request for %s: %s != %s", res.name, newAddr, oldAddr))
@@ -426,6 +498,21 @@ func TestAllocatorFuzz(t *testing.T) {
stateLock.Unlock()
}
// Claim a random address for a unique container name - may not succeed
claim := func(name string) {
if !bumpPending() {
return
}
allocIndex := rand.Int31n(nodes)
addressIndex := rand.Int31n(int32(subnet.Size()))
alloc := allocs[allocIndex]
addr := address.Add(subnet.Start, address.Offset(addressIndex))
err := alloc.Claim(name, addr, true)
if err == nil {
noteAllocation(allocIndex, name, addr)
}
}
// Run function _f_ _iterations_ times, in _concurrency_
// number of goroutines
doConcurrentIterations := func(iterations int, f func(int)) {
@@ -465,9 +552,13 @@ func TestAllocatorFuzz(t *testing.T) {
// free a random addr
free()
case 0.8 <= r && r < 1.0:
case 0.8 <= r && r < 0.95:
// ask for an existing name again, check we get same ip
allocateAgain()
case 0.95 <= r && r < 1.0:
name := fmt.Sprintf("second%d", iteration)
claim(name)
}
})
}
@@ -479,7 +570,7 @@ func TestGossipSkew(t *testing.T) {
alloc2.now = func() time.Time { return time.Now().Add(time.Hour * 2) }
defer alloc2.Stop()
if _, err := alloc1.OnGossipBroadcast(alloc2.Encode()); err == nil {
if _, err := alloc1.OnGossipBroadcast(alloc2.ourName, alloc2.Encode()); err == nil {
t.Fail()
}
}
+57 -42
View File
@@ -4,25 +4,28 @@ import (
"fmt"
"github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/ipam/address"
"github.com/weaveworks/weave/router"
"github.com/weaveworks/weave/mesh"
"github.com/weaveworks/weave/net/address"
)
type claim struct {
resultChan chan<- error
ident string
addr address.Address
resultChan chan<- error
ident string
addr address.Address
noErrorOnUnknown bool
}
// Send an error (or nil for success) back to caller listening on resultChan
func (c *claim) sendResult(result error) {
// Make sure we only send a result once, since listener stops listening after that
if c.resultChan != nil {
c.resultChan <- result
close(c.resultChan)
c.resultChan = nil
return
}
if result != nil {
common.Error.Println("[allocator] " + result.Error())
common.Log.Errorln("[allocator] " + result.Error())
}
}
@@ -30,57 +33,69 @@ func (c *claim) sendResult(result error) {
func (c *claim) Try(alloc *Allocator) bool {
if !alloc.ring.Contains(c.addr) {
// Address not within our universe; assume user knows what they are doing
alloc.infof("Ignored address %s claimed by %s - not in our universe\n", c.addr, c.ident)
alloc.infof("Ignored address %s claimed by %s - not in our universe", c.addr, c.ident)
c.sendResult(nil)
return true
}
// If our ring doesn't know, it must be empty. We will have initiated the
// bootstrap of the ring, so wait until we find some owner for this
// range (might be us).
owner := alloc.ring.Owner(c.addr)
if owner == router.UnknownPeerName {
alloc.infof("Ring is empty; will try later.\n", c.addr, owner)
c.sendResult(nil) // don't make the caller wait
alloc.establishRing()
switch owner := alloc.ring.Owner(c.addr); owner {
case alloc.ourName:
// success
case mesh.UnknownPeerName:
// If our ring doesn't know, it must be empty.
if c.noErrorOnUnknown {
alloc.infof("Claim %s for %s: address allocator still initializing; will try later.", c.addr, c.ident)
c.sendResult(nil)
} else {
c.sendResult(fmt.Errorf("%s is in the range %s, but the allocator is not initialized yet", c.addr, alloc.universe.AsCIDRString()))
}
return false
default:
alloc.debugf("requesting address %s from other peer %s", c.addr, owner)
err := alloc.sendSpaceRequest(owner, address.NewRange(c.addr, 1))
if err != nil { // can't speak to owner right now
if c.noErrorOnUnknown {
alloc.infof("Claim %s for %s: %s; will try later.", c.addr, c.ident, err)
c.sendResult(nil)
} else { // just tell the user they can't do this.
c.deniedBy(alloc, owner)
}
}
return false
}
if owner != alloc.ourName {
name, found := alloc.nicknames[owner]
if found {
name = " (" + name + ")"
}
c.sendResult(fmt.Errorf("address %s is owned by other peer %s%s", c.addr.String(), owner, name))
return true
}
// We are the owner, check we haven't given it to another container
existingIdent := alloc.findOwner(c.addr)
if existingIdent == c.ident {
switch existingIdent := alloc.findOwner(c.addr); existingIdent {
case "":
if err := alloc.space.Claim(c.addr); err == nil {
alloc.debugln("Claimed", c.addr, "for", c.ident)
alloc.addOwned(c.ident, c.addr)
c.sendResult(nil)
} else {
c.sendResult(err)
}
case c.ident:
// same identifier is claiming same address; that's OK
c.sendResult(nil)
return true
default:
// Addr already owned by container on this machine
c.sendResult(fmt.Errorf("address %s is already owned by %s", c.addr.String(), existingIdent))
}
if existingIdent == "" {
err := alloc.space.Claim(c.addr)
if err != nil {
c.sendResult(err)
return true
}
alloc.debugln("Claimed", c.addr, "for", c.ident)
alloc.addOwned(c.ident, c.addr)
c.sendResult(nil)
return true
}
// Addr already owned by container on this machine
c.sendResult(fmt.Errorf("address %s is already owned by %s", c.addr.String(), existingIdent))
return true
}
func (c *claim) Cancel() {
c.sendResult(fmt.Errorf("Operation cancelled."))
func (c *claim) deniedBy(alloc *Allocator, owner mesh.PeerName) {
name, found := alloc.nicknames[owner]
if found {
name = " (" + name + ")"
}
c.sendResult(fmt.Errorf("address %s is owned by other peer %s%s", c.addr.String(), owner, name))
}
func (c *claim) String() string {
return fmt.Sprintf("Claim %s -> %s", c.ident, c.addr.String())
func (c *claim) Cancel() {
c.sendResult(&errorCancelled{"Claim", c.ident})
}
func (c *claim) ForContainer(ident string) bool {
+54 -51
View File
@@ -8,12 +8,51 @@ import (
"github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/common/docker"
"github.com/weaveworks/weave/ipam/address"
"github.com/weaveworks/weave/net/address"
)
func badRequest(w http.ResponseWriter, err error) {
http.Error(w, err.Error(), http.StatusBadRequest)
common.Warning.Println("[allocator]:", err.Error())
common.Log.Warningln("[allocator]:", err.Error())
}
func parseCIDR(w http.ResponseWriter, cidrStr string) (address.CIDR, bool) {
subnetAddr, cidr, err := address.ParseCIDR(cidrStr)
if err != nil {
badRequest(w, err)
return address.CIDR{}, false
}
if cidr.Start != subnetAddr {
badRequest(w, fmt.Errorf("Invalid subnet %s - bits after network prefix are not all zero", cidrStr))
return address.CIDR{}, false
}
return cidr, true
}
func (alloc *Allocator) handleHTTPAllocate(dockerCli *docker.Client, w http.ResponseWriter, ident string, checkAlive bool, subnet address.CIDR) {
closedChan := w.(http.CloseNotifier).CloseNotify()
addr, err := alloc.Allocate(ident, subnet.HostRange(),
func() bool {
select {
case <-closedChan:
return true
default:
res := checkAlive && dockerCli != nil && dockerCli.IsContainerNotRunning(ident)
checkAlive = false // we check only once; if the container dies later we learn about that through events
return res
}
})
if err != nil {
if _, ok := err.(*errorCancelled); ok { // cancellation is not really an error
common.Log.Infoln("[allocator]:", err.Error())
fmt.Fprint(w, "cancelled")
return
}
badRequest(w, err)
return
}
fmt.Fprintf(w, "%s/%d", addr, subnet.PrefixLen)
}
// HandleHTTP wires up ipams HTTP endpoints to the provided mux.
@@ -22,10 +61,11 @@ func (alloc *Allocator) HandleHTTP(router *mux.Router, defaultSubnet address.CID
vars := mux.Vars(r)
ident := vars["id"]
ipStr := vars["ip"]
noErrorOnUnknown := r.FormValue("noErrorOnUnknown") == "true"
if ip, err := address.ParseIP(ipStr); err != nil {
badRequest(w, err)
return
} else if err := alloc.Claim(ident, ip); err != nil {
} else if err := alloc.Claim(ident, ip, noErrorOnUnknown); err != nil {
badRequest(w, fmt.Errorf("Unable to claim: %s", err))
return
}
@@ -35,18 +75,14 @@ func (alloc *Allocator) HandleHTTP(router *mux.Router, defaultSubnet address.CID
router.Methods("GET").Path("/ip/{id}/{ip}/{prefixlen}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
cidr := vars["ip"] + "/" + vars["prefixlen"]
_, subnet, err := address.ParseCIDR(cidr)
if err != nil {
badRequest(w, err)
return
if subnet, ok := parseCIDR(w, vars["ip"]+"/"+vars["prefixlen"]); ok {
addr, err := alloc.Lookup(vars["id"], subnet.HostRange())
if err != nil {
http.NotFound(w, r)
return
}
fmt.Fprintf(w, "%s/%d", addr, subnet.PrefixLen)
}
addr, err := alloc.Lookup(vars["id"], subnet.HostRange())
if err != nil {
http.NotFound(w, r)
return
}
fmt.Fprintf(w, "%s/%d", addr, subnet.PrefixLen)
})
router.Methods("GET").Path("/ip/{id}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -59,48 +95,15 @@ func (alloc *Allocator) HandleHTTP(router *mux.Router, defaultSubnet address.CID
})
router.Methods("POST").Path("/ip/{id}/{ip}/{prefixlen}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
closedChan := w.(http.CloseNotifier).CloseNotify()
vars := mux.Vars(r)
ident := vars["id"]
cidrStr := vars["ip"] + "/" + vars["prefixlen"]
subnetAddr, cidr, err := address.ParseCIDR(cidrStr)
if err != nil {
badRequest(w, err)
return
if subnet, ok := parseCIDR(w, vars["ip"]+"/"+vars["prefixlen"]); ok {
alloc.handleHTTPAllocate(dockerCli, w, vars["id"], r.FormValue("check-alive") == "true", subnet)
}
if cidr.Start != subnetAddr {
badRequest(w, fmt.Errorf("Invalid subnet %s - bits after network prefix are not all zero", cidrStr))
return
}
addr, err := alloc.Allocate(ident, cidr.HostRange(), closedChan)
if err != nil {
badRequest(w, fmt.Errorf("Unable to allocate: %s", err))
return
}
if dockerCli != nil && dockerCli.IsContainerNotRunning(ident) {
common.Info.Printf("[allocator] '%s' is not running: freeing %s", ident, addr)
alloc.Free(ident, addr)
return
}
fmt.Fprintf(w, "%s/%d", addr, cidr.PrefixLen)
})
router.Methods("POST").Path("/ip/{id}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
closedChan := w.(http.CloseNotifier).CloseNotify()
ident := mux.Vars(r)["id"]
newAddr, err := alloc.Allocate(ident, defaultSubnet.HostRange(), closedChan)
if err != nil {
badRequest(w, err)
return
}
if dockerCli != nil && dockerCli.IsContainerNotRunning(ident) {
common.Info.Printf("[allocator] '%s' is not running: freeing %s", ident, newAddr)
alloc.Free(ident, newAddr)
return
}
fmt.Fprintf(w, "%s/%d", newAddr, defaultSubnet.PrefixLen)
vars := mux.Vars(r)
alloc.handleHTTPAllocate(dockerCli, w, vars["id"], r.FormValue("check-alive") == "true", defaultSubnet)
})
router.Methods("DELETE").Path("/ip/{id}/{ip}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+22 -23
View File
@@ -11,16 +11,16 @@ import (
"time"
"github.com/gorilla/mux"
"github.com/stretchr/testify/require"
"github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/ipam/address"
"github.com/weaveworks/weave/net/address"
wt "github.com/weaveworks/weave/testing"
)
func HTTPPost(t *testing.T, url string) string {
resp, err := http.Post(url, "", nil)
wt.AssertNoErr(t, err)
wt.AssertStatus(t, resp.StatusCode, http.StatusOK, "http response")
require.NoError(t, err)
require.Equal(t, http.StatusOK, resp.StatusCode, "http response")
defer resp.Body.Close()
body, _ := ioutil.ReadAll(resp.Body)
return string(body)
@@ -28,7 +28,7 @@ func HTTPPost(t *testing.T, url string) string {
func HTTPGet(t *testing.T, url string) string {
resp, err := http.Get(url)
wt.AssertNoErr(t, err)
require.NoError(t, err)
defer resp.Body.Close()
body, _ := ioutil.ReadAll(resp.Body)
return string(body)
@@ -48,13 +48,13 @@ func listenHTTP(alloc *Allocator, subnet address.CIDR) int {
httpListener, err := net.Listen("tcp", ":0")
if err != nil {
common.Error.Fatal("Unable to create http listener: ", err)
common.Log.Fatal("Unable to create http listener: ", err)
}
go func() {
srv := &http.Server{Handler: router}
if err := srv.Serve(httpListener); err != nil {
common.Error.Fatal("Unable to serve http: ", err)
common.Log.Fatal("Unable to serve http: ", err)
}
}()
return httpListener.Addr().(*net.TCPAddr).Port
@@ -87,28 +87,28 @@ func TestHttp(t *testing.T) {
// Allocate an address in each subnet, and check we got what we expected
cidr1a := HTTPPost(t, allocURL(port, testCIDR1, containerID))
wt.AssertEqualString(t, cidr1a, testAddr1, "address")
require.Equal(t, testAddr1, cidr1a, "address")
cidr2a := HTTPPost(t, allocURL(port, testCIDR2, containerID))
wt.AssertEqualString(t, cidr2a, testAddr2, "address")
require.Equal(t, testAddr2, cidr2a, "address")
// Now, make the same requests again to check the operation is idempotent
check := HTTPGet(t, allocURL(port, testCIDR1, containerID))
wt.AssertEqualString(t, check, cidr1a, "address")
require.Equal(t, cidr1a, check, "address")
check = HTTPGet(t, allocURL(port, testCIDR2, containerID))
wt.AssertEqualString(t, check, cidr2a, "address")
require.Equal(t, cidr2a, check, "address")
// Ask the http server for a pair of addresses for another container and check they're different
cidr1b := HTTPPost(t, allocURL(port, testCIDR1, container2))
wt.AssertFalse(t, cidr1b == testAddr1, "address")
require.False(t, cidr1b == testAddr1, "address")
cidr2b := HTTPPost(t, allocURL(port, testCIDR2, container2))
wt.AssertFalse(t, cidr2b == testAddr2, "address")
require.False(t, cidr2b == testAddr2, "address")
// Now free the first container, and we should get its addresses back when we ask
doHTTP("DELETE", identURL(port, containerID))
cidr1c := HTTPPost(t, allocURL(port, testCIDR1, container3))
wt.AssertEqualString(t, cidr1c, testAddr1, "address")
require.Equal(t, testAddr1, cidr1c, "address")
cidr2c := HTTPPost(t, allocURL(port, testCIDR2, container3))
wt.AssertEqualString(t, cidr2c, testAddr2, "address")
require.Equal(t, testAddr2, cidr2c, "address")
// Would like to shut down the http server at the end of this test
// but it's complicated.
@@ -132,16 +132,16 @@ func TestBadHttp(t *testing.T) {
testAddr1 := parts[0]
// Verb that's not handled
resp, err := doHTTP("HEAD", fmt.Sprintf("http://localhost:%d/ip/%s/%s", port, containerID, testAddr1))
wt.AssertNoErr(t, err)
wt.AssertStatus(t, resp.StatusCode, http.StatusNotFound, "http response")
require.NoError(t, err)
require.Equal(t, http.StatusNotFound, resp.StatusCode, "http response")
// Mis-spelled URL
resp, err = doHTTP("POST", fmt.Sprintf("http://localhost:%d/xip/%s/", port, containerID))
wt.AssertNoErr(t, err)
wt.AssertStatus(t, resp.StatusCode, http.StatusNotFound, "http response")
require.NoError(t, err)
require.Equal(t, http.StatusNotFound, resp.StatusCode, "http response")
// Malformed URL
resp, err = doHTTP("POST", fmt.Sprintf("http://localhost:%d/ip/%s/foo/bar/baz", port, containerID))
wt.AssertNoErr(t, err)
wt.AssertStatus(t, resp.StatusCode, http.StatusNotFound, "http response")
require.NoError(t, err)
require.Equal(t, http.StatusNotFound, resp.StatusCode, "http response")
}
func TestHTTPCancel(t *testing.T) {
@@ -151,7 +151,6 @@ func TestHTTPCancel(t *testing.T) {
}
func impTestHTTPCancel(t *testing.T) {
common.InitDefaultLogging(true)
var (
containerID = "deadbeef"
testCIDR1 = "10.0.3.0/29"
@@ -180,6 +179,6 @@ func impTestHTTPCancel(t *testing.T) {
unpause()
res := <-done
if res != nil {
wt.Fatalf(t, "Error: Allocate returned non-nil")
require.FailNow(t, "Error: Allocate returned non-nil")
}
}
+15 -12
View File
@@ -1,9 +1,7 @@
package paxos
import (
"fmt"
"github.com/weaveworks/weave/router"
"github.com/weaveworks/weave/mesh"
)
// The node identifier. The use of the UID here is important: Paxos
@@ -11,8 +9,8 @@ import (
// node does not restart and lose its Paxos state but claim to have
// the same ID.
type NodeID struct {
Name router.PeerName
UID router.PeerUID
Name mesh.PeerName
UID mesh.PeerUID
}
// note all fields exported in structs so we can Gob them
@@ -41,7 +39,7 @@ func (a ProposalID) valid() bool {
}
// For seeding IPAM, the value we want consensus on is a set of peer names
type Value []router.PeerName
type Value []mesh.PeerName
// An AcceptedValue is a Value plus the proposal which originated that
// Value. The origin is not essential, but makes comparing
@@ -74,7 +72,7 @@ type Node struct {
knows GossipState
}
func NewNode(name router.PeerName, uid router.PeerUID, quorum uint) *Node {
func NewNode(name mesh.PeerName, uid mesh.PeerUID, quorum uint) *Node {
return &Node{
id: NodeID{name, uid},
quorum: quorum,
@@ -225,7 +223,7 @@ func (node *Node) Think() bool {
// about. This is not necessarily all peer names, but it is at least
// a quorum, and so good enough for seeding the ring.
func (node *Node) pickValue() Value {
val := make([]router.PeerName, len(node.knows))
val := make([]mesh.PeerName, len(node.knows))
i := 0
for id := range node.knows {
val[i] = id.Name
@@ -252,9 +250,14 @@ func (node *Node) Consensus() (bool, AcceptedValue) {
return false, AcceptedValue{}
}
func (node *Node) String() string {
if ok, val := node.Consensus(); ok {
return fmt.Sprintf("Consensus reached with size %d", len(val.Value))
type Status struct {
KnownNodes int
Quorum uint
}
func NewStatus(node *Node) *Status {
if node == nil {
return nil
}
return fmt.Sprintf("Nodes known: %d, Quorum size: %d", len(node.knows), node.quorum)
return &Status{len(node.knows), node.quorum}
}
+5 -5
View File
@@ -6,7 +6,7 @@ import (
"testing"
"time"
"github.com/weaveworks/weave/router"
"github.com/weaveworks/weave/mesh"
)
type TestNode struct {
@@ -97,8 +97,8 @@ func makeRandomModel(params *TestParams, r *rand.Rand, t *testing.T) *Model {
}
for i := range m.nodes {
m.nodes[i].Node = NewNode(router.PeerName(i/2+1),
router.PeerUID(r.Int63()), m.quorum)
m.nodes[i].Node = NewNode(mesh.PeerName(i/2+1),
mesh.PeerUID(r.Int63()), m.quorum)
m.nodes[i].Propose()
}
@@ -172,8 +172,8 @@ func (m *Model) isolateNode(node *TestNode) {
// Restart a node
func (m *Model) restart(node *TestNode) {
node.Node = NewNode(router.PeerName(m.nextID),
router.PeerUID(m.r.Int63()), m.quorum)
node.Node = NewNode(mesh.PeerName(m.nextID),
mesh.PeerUID(m.r.Int63()), m.quorum)
m.nextID++
node.Propose()
+4 -8
View File
@@ -4,14 +4,14 @@ import (
"sort"
"github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/ipam/address"
"github.com/weaveworks/weave/router"
"github.com/weaveworks/weave/mesh"
"github.com/weaveworks/weave/net/address"
)
// Entry represents entries around the ring
type entry struct {
Token address.Address // The start of this range
Peer router.PeerName // Who owns this range
Peer mesh.PeerName // Who owns this range
Version uint32 // Version of this range
Free address.Offset // Number of free IPs in this range
}
@@ -21,7 +21,7 @@ func (e *entry) Equal(e2 *entry) bool {
e.Version == e2.Version
}
func (e *entry) update(peername router.PeerName, free address.Offset) {
func (e *entry) update(peername mesh.PeerName, free address.Offset) {
e.Peer = peername
e.Version++
e.Free = free
@@ -68,10 +68,6 @@ func (es entries) get(token address.Address) (*entry, bool) {
return nil, false
}
func (es *entries) remove(i int) {
*es = (*es)[:i+copy((*es)[i:], (*es)[i+1:])]
}
// Is token between entries at i and j?
// NB i and j can overflow and will wrap
// NBB this function does not work very well if there is only one
+58 -33
View File
@@ -10,17 +10,19 @@ import (
"io"
"math/rand"
"sort"
"time"
"github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/ipam/address"
"github.com/weaveworks/weave/router"
"github.com/weaveworks/weave/mesh"
"github.com/weaveworks/weave/net/address"
)
// Ring represents the ring itself
type Ring struct {
Start, End address.Address // [min, max) tokens in this ring. Due to wrapping, min == max (effectively)
Peer router.PeerName // name of peer owning this ring instance
Peer mesh.PeerName // name of peer owning this ring instance
Entries entries // list of entries sorted by token
Seeds []mesh.PeerName // peers with which the ring was seeded
}
func (r *Ring) assertInvariants() {
@@ -32,16 +34,15 @@ func (r *Ring) assertInvariants() {
// Errors returned by Merge
var (
ErrNotSorted = errors.New("Ring not sorted")
ErrTokenRepeated = errors.New("Token appears twice in ring")
ErrTokenOutOfRange = errors.New("Token is out of range")
ErrDifferentSubnets = errors.New("IP Allocator with different subnet detected")
ErrNewerVersion = errors.New("Received new version for entry I own!")
ErrInvalidEntry = errors.New("Received invalid state update!")
ErrEntryInMyRange = errors.New("Received new entry in my range!")
ErrNoFreeSpace = errors.New("No free space found!")
ErrInvalidTimeout = errors.New("dt must be greater than 0")
ErrNotFound = errors.New("No entries for peer found")
ErrNotSorted = errors.New("Ring not sorted")
ErrTokenRepeated = errors.New("Token appears twice in ring")
ErrTokenOutOfRange = errors.New("Token is out of range")
ErrDifferentSeeds = errors.New("Received ring was seeded differently from ours!")
ErrDifferentRange = errors.New("Received range differs from ours!")
ErrNewerVersion = errors.New("Received new version for entry I own!")
ErrInvalidEntry = errors.New("Received invalid state update!")
ErrEntryInMyRange = errors.New("Received new entry in my range!")
ErrNotFound = errors.New("No entries for peer found")
)
func (r *Ring) checkInvariants() error {
@@ -83,7 +84,7 @@ func (r *Ring) checkInvariants() error {
}
// New creates an empty ring belonging to peer.
func New(start, end address.Address, peer router.PeerName) *Ring {
func New(start, end address.Address, peer mesh.PeerName) *Ring {
common.Assert(start < end)
ring := &Ring{Start: start, End: end, Peer: peer, Entries: make([]*entry, 0)}
@@ -91,6 +92,10 @@ func New(start, end address.Address, peer router.PeerName) *Ring {
return ring
}
func (r *Ring) Range() address.Range {
return address.Range{Start: r.Start, End: r.End}
}
// Returns the distance between two tokens on this ring, dealing
// with ranges which cross the origin
func (r *Ring) distance(start, end address.Address) address.Offset {
@@ -106,7 +111,7 @@ func (r *Ring) distance(start, end address.Address) address.Offset {
// Preconditions:
// - start < end
// - [start, end) must be owned by the calling peer
func (r *Ring) GrantRangeToHost(start, end address.Address, peer router.PeerName) {
func (r *Ring) GrantRangeToHost(start, end address.Address, peer mesh.PeerName) {
//fmt.Printf("%s GrantRangeToHost [%v,%v) -> %s\n", r.Peer, start, end, peer)
r.assertInvariants()
@@ -185,8 +190,19 @@ func (r *Ring) Merge(gossip Ring) error {
return err
}
if len(gossip.Seeds) > 0 && len(r.Seeds) > 0 {
if len(gossip.Seeds) != len(r.Seeds) {
return ErrDifferentSeeds
}
for i, seed := range gossip.Seeds {
if seed != r.Seeds[i] {
return ErrDifferentSeeds
}
}
}
if r.Start != gossip.Start || r.End != gossip.End {
return ErrDifferentSubnets
return ErrDifferentRange
}
// Now merge their ring with yours, in a temporary ring.
@@ -194,7 +210,7 @@ func (r *Ring) Merge(gossip Ring) error {
addToResult := func(e entry) { result = append(result, &e) }
var mine, theirs *entry
var previousOwner *router.PeerName
var previousOwner *mesh.PeerName
// i is index into r.Entries; j is index into gossip.Entries
var i, j int
for i < len(r.Entries) && j < len(gossip.Entries) {
@@ -217,7 +233,7 @@ func (r *Ring) Merge(gossip Ring) error {
switch {
case mine.Version >= theirs.Version:
if mine.Version == theirs.Version && !mine.Equal(theirs) {
common.Debug.Printf("Error merging entries at %s - %v != %v\n", mine.Token, mine, theirs)
common.Log.Debugf("Error merging entries at %s - %v != %v", mine.Token, mine, theirs)
return ErrInvalidEntry
}
addToResult(*mine)
@@ -251,6 +267,9 @@ func (r *Ring) Merge(gossip Ring) error {
previousOwner = nil
}
if len(r.Seeds) == 0 {
r.Seeds = gossip.Seeds
}
r.Entries = result
return nil
}
@@ -299,7 +318,7 @@ func (r *Ring) OwnedRanges() (result []address.Range) {
// ClaimForPeers claims the entire ring for the array of peers passed
// in. Only works for empty rings.
func (r *Ring) ClaimForPeers(peers []router.PeerName) {
func (r *Ring) ClaimForPeers(peers []mesh.PeerName) {
common.Assert(r.Empty())
defer r.assertInvariants()
defer r.updateExportedVariables()
@@ -327,9 +346,11 @@ func (r *Ring) ClaimForPeers(peers []router.PeerName) {
}
common.Assert(pos == r.End)
r.Seeds = peers
}
func (r *Ring) FprintWithNicknames(w io.Writer, m map[router.PeerName]string) {
func (r *Ring) FprintWithNicknames(w io.Writer, m map[mesh.PeerName]string) {
for _, entry := range r.Entries {
nickname, found := m[entry.Peer]
if found {
@@ -344,7 +365,7 @@ func (r *Ring) FprintWithNicknames(w io.Writer, m map[router.PeerName]string) {
func (r *Ring) String() string {
var buffer bytes.Buffer
fmt.Fprintf(&buffer, "Ring [%s, %s)", r.Start, r.End)
r.FprintWithNicknames(&buffer, make(map[router.PeerName]string))
r.FprintWithNicknames(&buffer, make(map[mesh.PeerName]string))
return buffer.String()
}
@@ -394,7 +415,7 @@ func (r *Ring) ReportFree(freespace map[address.Address]address.Offset) {
type weightedPeer struct {
weight float64
peername router.PeerName
peername mesh.PeerName
}
type weightedPeers []weightedPeer
@@ -405,10 +426,10 @@ func (ws weightedPeers) Swap(i, j int) { ws[i], ws[j] = ws[j], ws[i] }
// ChoosePeersToAskForSpace returns all peers we can ask for space in
// the range [start, end), in weighted-random order. Assumes start<end.
func (r *Ring) ChoosePeersToAskForSpace(start, end address.Address) []router.PeerName {
func (r *Ring) ChoosePeersToAskForSpace(start, end address.Address) []mesh.PeerName {
var (
sum address.Offset
totalSpacePerPeer = make(map[router.PeerName]address.Offset) // Compute total free space per peer
totalSpacePerPeer = make(map[mesh.PeerName]address.Offset) // Compute total free space per peer
)
// iterate through tokens
@@ -442,25 +463,25 @@ func (r *Ring) ChoosePeersToAskForSpace(start, end address.Address) []router.Pee
ws = append(ws, weightedPeer{weight: float64(space) * rand.Float64(), peername: peername})
}
sort.Sort(ws)
result := make([]router.PeerName, len(ws))
result := make([]mesh.PeerName, len(ws))
for i, wp := range ws {
result[i] = wp.peername
}
return result
}
func (r *Ring) PickPeerForTransfer() router.PeerName {
func (r *Ring) PickPeerForTransfer(isValid func(mesh.PeerName) bool) mesh.PeerName {
for _, entry := range r.Entries {
if entry.Peer != r.Peer {
if entry.Peer != r.Peer && isValid(entry.Peer) {
return entry.Peer
}
}
return router.UnknownPeerName
return mesh.UnknownPeerName
}
// Transfer will mark all entries associated with 'from' peer as owned by 'to' peer
// and return ranges indicating the new space we picked up
func (r *Ring) Transfer(from, to router.PeerName) ([]address.Range, error) {
func (r *Ring) Transfer(from, to mesh.PeerName) ([]address.Range, error) {
r.assertInvariants()
defer r.assertInvariants()
defer r.updateExportedVariables()
@@ -490,13 +511,13 @@ func (r *Ring) Contains(addr address.Address) bool {
}
// Owner returns the peername which owns the range containing addr
func (r *Ring) Owner(token address.Address) router.PeerName {
func (r *Ring) Owner(token address.Address) mesh.PeerName {
common.Assert(r.Start <= token && token < r.End)
r.assertInvariants()
// There can be no owners on an empty ring
if r.Empty() {
return router.UnknownPeerName
return mesh.UnknownPeerName
}
// Look for the right-most entry, less than or equal to token
@@ -509,8 +530,8 @@ func (r *Ring) Owner(token address.Address) router.PeerName {
}
// Get the set of PeerNames mentioned in the ring
func (r *Ring) PeerNames() map[router.PeerName]struct{} {
res := make(map[router.PeerName]struct{})
func (r *Ring) PeerNames() map[mesh.PeerName]struct{} {
res := make(map[mesh.PeerName]struct{})
for _, entry := range r.Entries {
res[entry.Peer] = struct{}{}
@@ -518,3 +539,7 @@ func (r *Ring) PeerNames() map[router.PeerName]struct{} {
return res
}
func init() {
rand.Seed(time.Now().UTC().UnixNano())
}
+145 -136
View File
@@ -7,16 +7,16 @@ import (
"sort"
"testing"
"github.com/stretchr/testify/require"
"github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/ipam/address"
"github.com/weaveworks/weave/router"
wt "github.com/weaveworks/weave/testing"
"github.com/weaveworks/weave/mesh"
"github.com/weaveworks/weave/net/address"
)
var (
peer1name, _ = router.PeerNameFromString("01:00:00:00:02:00")
peer2name, _ = router.PeerNameFromString("02:00:00:00:02:00")
peer3name, _ = router.PeerNameFromString("03:00:00:00:02:00")
peer1name, _ = mesh.PeerNameFromString("01:00:00:00:02:00")
peer2name, _ = mesh.PeerNameFromString("02:00:00:00:02:00")
peer3name, _ = mesh.PeerNameFromString("03:00:00:00:02:00")
start, end = ParseIP("10.0.0.0"), ParseIP("10.0.0.255")
dot10, dot245 = ParseIP("10.0.0.10"), ParseIP("10.0.0.245")
@@ -34,26 +34,26 @@ func TestInvariants(t *testing.T) {
// Check ring is sorted
ring.Entries = []*entry{{Token: dot245, Peer: peer1name}, {Token: dot10, Peer: peer2name}}
wt.AssertTrue(t, ring.checkInvariants() == ErrNotSorted, "Expected error")
require.True(t, ring.checkInvariants() == ErrNotSorted, "Expected error")
// Check tokens don't appear twice
ring.Entries = []*entry{{Token: dot245, Peer: peer1name}, {Token: dot245, Peer: peer2name}}
wt.AssertTrue(t, ring.checkInvariants() == ErrTokenRepeated, "Expected error")
require.True(t, ring.checkInvariants() == ErrTokenRepeated, "Expected error")
// Check tokens are in bounds
ring = New(dot10, dot245, peer1name)
ring.Entries = []*entry{{Token: start, Peer: peer1name}}
wt.AssertTrue(t, ring.checkInvariants() == ErrTokenOutOfRange, "Expected error")
require.True(t, ring.checkInvariants() == ErrTokenOutOfRange, "Expected error")
ring.Entries = []*entry{{Token: end, Peer: peer1name}}
wt.AssertTrue(t, ring.checkInvariants() == ErrTokenOutOfRange, "Expected error")
require.True(t, ring.checkInvariants() == ErrTokenOutOfRange, "Expected error")
}
func TestInsert(t *testing.T) {
ring := New(start, end, peer1name)
ring.Entries = []*entry{{Token: start, Peer: peer1name, Free: 255}}
wt.AssertPanic(t, func() {
require.Panics(t, func() {
ring.Entries.insert(entry{Token: start, Peer: peer1name})
})
@@ -61,11 +61,11 @@ func TestInsert(t *testing.T) {
ring.Entries.insert(entry{Token: dot245, Peer: peer1name})
ring2 := New(start, end, peer1name)
ring2.Entries = []*entry{{Token: start, Peer: peer1name, Free: 0}, {Token: dot245, Peer: peer1name}}
wt.AssertEquals(t, ring, ring2)
require.Equal(t, ring2, ring)
ring.Entries.insert(entry{Token: dot10, Peer: peer1name})
ring2.Entries = []*entry{{Token: start, Peer: peer1name, Free: 0}, {Token: dot10, Peer: peer1name}, {Token: dot245, Peer: peer1name}}
wt.AssertEquals(t, ring, ring2)
require.Equal(t, ring2, ring)
}
func TestBetween(t *testing.T) {
@@ -76,7 +76,7 @@ func TestBetween(t *testing.T) {
// between should return true for everything
for i := 1; i <= 255; i++ {
ip := ParseIP(fmt.Sprintf("10.0.0.%d", i))
wt.AssertTrue(t, ring1.Entries.between(ip, 0, 1), "between should be true!")
require.True(t, ring1.Entries.between(ip, 0, 1), "between should be true!")
}
// Now, construct a ring with entries at +10 and -10
@@ -87,25 +87,25 @@ func TestBetween(t *testing.T) {
for i := 10; i <= 244; i++ {
ipStr := fmt.Sprintf("10.0.0.%d", i)
ip := ParseIP(ipStr)
wt.AssertTrue(t, ring1.Entries.between(ip, 0, 1),
require.True(t, ring1.Entries.between(ip, 0, 1),
fmt.Sprintf("Between should be true for %s!", ipStr))
wt.AssertFalse(t, ring1.Entries.between(ip, 1, 2),
require.False(t, ring1.Entries.between(ip, 1, 2),
fmt.Sprintf("Between should be false for %s!", ipStr))
}
for i := 0; i <= 9; i++ {
ipStr := fmt.Sprintf("10.0.0.%d", i)
ip := ParseIP(ipStr)
wt.AssertFalse(t, ring1.Entries.between(ip, 0, 1),
require.False(t, ring1.Entries.between(ip, 0, 1),
fmt.Sprintf("Between should be false for %s!", ipStr))
wt.AssertTrue(t, ring1.Entries.between(ip, 1, 2),
require.True(t, ring1.Entries.between(ip, 1, 2),
fmt.Sprintf("Between should be true for %s!", ipStr))
}
for i := 245; i <= 255; i++ {
ipStr := fmt.Sprintf("10.0.0.%d", i)
ip := ParseIP(ipStr)
wt.AssertFalse(t, ring1.Entries.between(ip, 0, 1),
require.False(t, ring1.Entries.between(ip, 0, 1),
fmt.Sprintf("Between should be false for %s!", ipStr))
wt.AssertTrue(t, ring1.Entries.between(ip, 1, 2),
require.True(t, ring1.Entries.between(ip, 1, 2),
fmt.Sprintf("Between should be true for %s!", ipStr))
}
}
@@ -116,32 +116,33 @@ func TestGrantSimple(t *testing.T) {
// Claim everything for peer1
ring1.ClaimItAll()
wt.AssertEquals(t, ring1.Entries, entries{{Token: start, Peer: peer1name, Free: 255}})
require.Equal(t, entries{{Token: start, Peer: peer1name, Free: 255}}, ring1.Entries)
// Now grant everything to peer2
ring1.GrantRangeToHost(start, end, peer2name)
ring2.Entries = []*entry{{Token: start, Peer: peer2name, Free: 255, Version: 1}}
wt.AssertEquals(t, ring1.Entries, ring2.Entries)
require.Equal(t, ring2.Entries, ring1.Entries)
// Now spint back to peer 1
ring2.GrantRangeToHost(dot10, end, peer1name)
ring1.Entries = []*entry{{Token: start, Peer: peer2name, Free: 10, Version: 2},
{Token: dot10, Peer: peer1name, Free: 245}}
wt.AssertEquals(t, ring1.Entries, ring2.Entries)
require.Equal(t, ring2.Entries, ring1.Entries)
// And spint back to peer 2 again
ring1.GrantRangeToHost(dot245, end, peer2name)
wt.AssertEquals(t, ring1.Entries, entries{{Token: start, Peer: peer2name, Free: 10, Version: 2},
require.Equal(t, entries{{Token: start, Peer: peer2name, Free: 10, Version: 2},
{Token: dot10, Peer: peer1name, Free: 235, Version: 1},
{Token: dot245, Peer: peer2name, Free: 10}})
{Token: dot245, Peer: peer2name, Free: 10}}, ring1.Entries)
// Grant range spanning a live token
ring1.Entries = []*entry{{Token: start, Peer: peer1name, Free: 10, Version: 2},
{Token: dot10, Peer: peer1name, Free: 235}, {Token: dot245, Peer: peer1name, Free: 10}}
ring1.GrantRangeToHost(dot10, end, peer2name)
wt.AssertEquals(t, ring1.Entries, entries{{Token: start, Peer: peer1name, Free: 10, Version: 2},
require.Equal(t, entries{{Token: start, Peer: peer1name, Free: 10, Version: 2},
{Token: dot10, Peer: peer2name, Free: 235, Version: 1},
{Token: dot245, Peer: peer2name, Free: 10, Version: 1}})
{Token: dot245, Peer: peer2name, Free: 10, Version: 1}}, ring1.Entries)
}
func TestGrantSplit(t *testing.T) {
@@ -151,23 +152,25 @@ func TestGrantSplit(t *testing.T) {
// Claim everything for peer1
ring1.Entries = []*entry{{Token: start, Peer: peer1name, Free: 255}}
ring2.Merge(*ring1)
wt.AssertEquals(t, ring1.Entries, ring2.Entries)
require.Equal(t, ring2.Entries, ring1.Entries)
// Now grant a split range to peer2
ring1.GrantRangeToHost(dot10, dot245, peer2name)
wt.AssertEquals(t, ring1.Entries, entries{{Token: start, Peer: peer1name, Free: 10, Version: 1},
require.Equal(t, entries{{Token: start, Peer: peer1name, Free: 10, Version: 1},
{Token: dot10, Peer: peer2name, Free: 235},
{Token: dot245, Peer: peer1name, Free: 10}})
{Token: dot245, Peer: peer1name, Free: 10}}, ring1.Entries)
ring1.assertInvariants()
// Grant range spanning a live token, and inserting a new token
ring1.Entries = []*entry{{Token: start, Peer: peer1name, Free: 10, Version: 2},
{Token: dot10, Peer: peer1name, Free: 118}, {Token: middle, Peer: peer1name, Free: 127}}
ring1.GrantRangeToHost(dot10, dot245, peer2name)
wt.AssertEquals(t, ring1.Entries, entries{{Token: start, Peer: peer1name, Free: 10, Version: 2},
require.Equal(t, entries{{Token: start, Peer: peer1name, Free: 10, Version: 2},
{Token: dot10, Peer: peer2name, Free: 118, Version: 1},
{Token: middle, Peer: peer2name, Free: 117, Version: 1},
{Token: dot245, Peer: peer1name, Free: 10, Version: 0}})
{Token: dot245, Peer: peer1name, Free: 10, Version: 0}}, ring1.Entries)
ring1.assertInvariants()
}
@@ -178,23 +181,25 @@ func TestMergeSimple(t *testing.T) {
// Claim everything for peer1
ring1.ClaimItAll()
ring1.GrantRangeToHost(middle, end, peer2name)
wt.AssertSuccess(t, ring2.Merge(*ring1))
require.NoError(t, ring2.Merge(*ring1))
wt.AssertEquals(t, ring1.Entries, entries{{Token: start, Peer: peer1name, Free: 128, Version: 1},
{Token: middle, Peer: peer2name, Free: 127}})
wt.AssertEquals(t, ring1.Entries, ring2.Entries)
require.Equal(t, entries{{Token: start, Peer: peer1name, Free: 128, Version: 1},
{Token: middle, Peer: peer2name, Free: 127}}, ring1.Entries)
require.Equal(t, ring2.Entries, ring1.Entries)
// Now to two different operations on either side,
// check we can Merge again
ring1.GrantRangeToHost(start, middle, peer2name)
ring2.GrantRangeToHost(middle, end, peer1name)
wt.AssertSuccess(t, ring2.Merge(*ring1))
wt.AssertSuccess(t, ring1.Merge(*ring2))
require.NoError(t, ring2.Merge(*ring1))
require.NoError(t, ring1.Merge(*ring2))
wt.AssertEquals(t, ring1.Entries, entries{{Token: start, Peer: peer2name, Free: 128, Version: 2},
{Token: middle, Peer: peer1name, Free: 127, Version: 1}})
wt.AssertEquals(t, ring1.Entries, ring2.Entries)
require.Equal(t, entries{{Token: start, Peer: peer2name, Free: 128, Version: 2},
{Token: middle, Peer: peer1name, Free: 127, Version: 1}}, ring1.Entries)
require.Equal(t, ring2.Entries, ring1.Entries)
}
func TestMergeErrors(t *testing.T) {
@@ -202,28 +207,28 @@ func TestMergeErrors(t *testing.T) {
ring1 := New(start, end, peer1name)
ring2 := New(start, end, peer2name)
ring2.Entries = []*entry{{Token: middle, Peer: peer2name}, {Token: start, Peer: peer2name}}
wt.AssertTrue(t, ring1.Merge(*ring2) == ErrNotSorted, "Expected ErrNotSorted")
require.True(t, ring1.Merge(*ring2) == ErrNotSorted, "Expected ErrNotSorted")
// Should Merge two rings for different ranges
ring2 = New(start, middle, peer2name)
ring2.Entries = []*entry{}
wt.AssertTrue(t, ring1.Merge(*ring2) == ErrDifferentSubnets, "Expected ErrDifferentSubnets")
require.True(t, ring1.Merge(*ring2) == ErrDifferentRange, "Expected ErrDifferentRange")
// Cannot Merge newer version of entry I own
ring2 = New(start, end, peer2name)
ring1.Entries = []*entry{{Token: start, Peer: peer1name}}
ring2.Entries = []*entry{{Token: start, Peer: peer1name, Version: 1}}
wt.AssertTrue(t, ring1.Merge(*ring2) == ErrNewerVersion, "Expected ErrNewerVersion")
require.True(t, ring1.Merge(*ring2) == ErrNewerVersion, "Expected ErrNewerVersion")
// Cannot Merge two entries with same version but different hosts
ring1.Entries = []*entry{{Token: start, Peer: peer1name}}
ring2.Entries = []*entry{{Token: start, Peer: peer2name}}
wt.AssertTrue(t, ring1.Merge(*ring2) == ErrInvalidEntry, "Expected ErrInvalidEntry")
require.True(t, ring1.Merge(*ring2) == ErrInvalidEntry, "Expected ErrInvalidEntry")
// Cannot Merge an entry into a range I own
ring1.Entries = []*entry{{Token: start, Peer: peer1name}}
ring2.Entries = []*entry{{Token: middle, Peer: peer2name}}
wt.AssertTrue(t, ring1.Merge(*ring2) == ErrEntryInMyRange, "Expected ErrEntryInMyRange")
require.True(t, ring1.Merge(*ring2) == ErrEntryInMyRange, "Expected ErrEntryInMyRange")
}
func TestMergeMore(t *testing.T) {
@@ -231,7 +236,7 @@ func TestMergeMore(t *testing.T) {
ring2 := New(start, end, peer2name)
assertRing := func(ring *Ring, entries entries) {
wt.AssertEquals(t, ring.Entries, entries)
require.Equal(t, entries, ring.Entries)
}
assertRing(ring1, []*entry{})
@@ -243,7 +248,7 @@ func TestMergeMore(t *testing.T) {
assertRing(ring2, []*entry{})
// Check the Merge sends it to the other ring
wt.AssertSuccess(t, ring2.Merge(*ring1))
require.NoError(t, ring2.Merge(*ring1))
assertRing(ring1, []*entry{{Token: start, Peer: peer1name, Free: 255}})
assertRing(ring2, []*entry{{Token: start, Peer: peer1name, Free: 255}})
@@ -252,7 +257,7 @@ func TestMergeMore(t *testing.T) {
assertRing(ring1, []*entry{{Token: start, Peer: peer2name, Free: 255, Version: 1}})
assertRing(ring2, []*entry{{Token: start, Peer: peer1name, Free: 255}})
wt.AssertSuccess(t, ring2.Merge(*ring1))
require.NoError(t, ring2.Merge(*ring1))
assertRing(ring1, []*entry{{Token: start, Peer: peer2name, Free: 255, Version: 1}})
assertRing(ring2, []*entry{{Token: start, Peer: peer2name, Free: 255, Version: 1}})
@@ -263,14 +268,14 @@ func TestMergeMore(t *testing.T) {
assertRing(ring1, []*entry{{Token: start, Peer: peer2name, Free: 255, Version: 1}})
// And Merge back
wt.AssertSuccess(t, ring1.Merge(*ring2))
require.NoError(t, ring1.Merge(*ring2))
assertRing(ring1, []*entry{{Token: start, Peer: peer2name, Free: 128, Version: 2},
{Token: middle, Peer: peer1name, Free: 127}})
assertRing(ring2, []*entry{{Token: start, Peer: peer2name, Free: 128, Version: 2},
{Token: middle, Peer: peer1name, Free: 127}})
// This should be a no-op
wt.AssertSuccess(t, ring2.Merge(*ring1))
require.NoError(t, ring2.Merge(*ring1))
assertRing(ring1, []*entry{{Token: start, Peer: peer2name, Free: 128, Version: 2},
{Token: middle, Peer: peer1name, Free: 127}})
assertRing(ring2, []*entry{{Token: start, Peer: peer2name, Free: 128, Version: 2},
@@ -283,19 +288,21 @@ func TestMergeSplit(t *testing.T) {
// Claim everything for peer2
ring1.Entries = []*entry{{Token: start, Peer: peer2name, Free: 255}}
wt.AssertSuccess(t, ring2.Merge(*ring1))
wt.AssertEquals(t, ring1.Entries, ring2.Entries)
require.NoError(t, ring2.Merge(*ring1))
require.Equal(t, ring2.Entries, ring1.Entries)
// Now grant a split range to peer1
ring2.GrantRangeToHost(dot10, dot245, peer1name)
wt.AssertEquals(t, ring2.Entries, entries{{Token: start, Peer: peer2name, Free: 10, Version: 1},
require.Equal(t, entries{{Token: start, Peer: peer2name, Free: 10, Version: 1},
{Token: dot10, Peer: peer1name, Free: 235},
{Token: dot245, Peer: peer2name, Free: 10}})
wt.AssertSuccess(t, ring1.Merge(*ring2))
wt.AssertEquals(t, ring1.Entries, entries{{Token: start, Peer: peer2name, Free: 10, Version: 1},
{Token: dot245, Peer: peer2name, Free: 10}}, ring2.Entries)
require.NoError(t, ring1.Merge(*ring2))
require.Equal(t, entries{{Token: start, Peer: peer2name, Free: 10, Version: 1},
{Token: dot10, Peer: peer1name, Free: 235},
{Token: dot245, Peer: peer2name, Free: 10}})
wt.AssertEquals(t, ring1.Entries, ring2.Entries)
{Token: dot245, Peer: peer2name, Free: 10}}, ring1.Entries)
require.Equal(t, ring2.Entries, ring1.Entries)
}
func TestMergeSplit2(t *testing.T) {
@@ -304,19 +311,21 @@ func TestMergeSplit2(t *testing.T) {
// Claim everything for peer2
ring1.Entries = []*entry{{Token: start, Peer: peer2name, Free: 250}, {Token: dot250, Peer: peer2name, Free: 5}}
wt.AssertSuccess(t, ring2.Merge(*ring1))
wt.AssertEquals(t, ring1.Entries, ring2.Entries)
require.NoError(t, ring2.Merge(*ring1))
require.Equal(t, ring2.Entries, ring1.Entries)
// Now grant a split range to peer1
ring2.GrantRangeToHost(dot10, dot245, peer1name)
wt.AssertEquals(t, ring2.Entries, entries{{Token: start, Peer: peer2name, Free: 10, Version: 1},
require.Equal(t, entries{{Token: start, Peer: peer2name, Free: 10, Version: 1},
{Token: dot10, Peer: peer1name, Free: 235},
{Token: dot245, Peer: peer2name, Free: 5}, {Token: dot250, Peer: peer2name, Free: 5}})
wt.AssertSuccess(t, ring1.Merge(*ring2))
wt.AssertEquals(t, ring1.Entries, entries{{Token: start, Peer: peer2name, Free: 10, Version: 1},
{Token: dot245, Peer: peer2name, Free: 5}, {Token: dot250, Peer: peer2name, Free: 5}}, ring2.Entries)
require.NoError(t, ring1.Merge(*ring2))
require.Equal(t, entries{{Token: start, Peer: peer2name, Free: 10, Version: 1},
{Token: dot10, Peer: peer1name, Free: 235},
{Token: dot245, Peer: peer2name, Free: 5}, {Token: dot250, Peer: peer2name, Free: 5}})
wt.AssertEquals(t, ring1.Entries, ring2.Entries)
{Token: dot245, Peer: peer2name, Free: 5}, {Token: dot250, Peer: peer2name, Free: 5}}, ring1.Entries)
require.Equal(t, ring2.Entries, ring1.Entries)
}
// A simple test, very similar to above, but using the marshalling to byte[]s
@@ -325,7 +334,7 @@ func TestGossip(t *testing.T) {
ring2 := New(start, end, peer2name)
assertRing := func(ring *Ring, entries entries) {
wt.AssertEquals(t, ring.Entries, entries)
require.Equal(t, entries, ring.Entries)
}
assertRing(ring1, []*entry{})
@@ -337,50 +346,50 @@ func TestGossip(t *testing.T) {
assertRing(ring2, []*entry{})
// Check the Merge sends it to the other ring
wt.AssertSuccess(t, ring2.Merge(*ring1))
require.NoError(t, ring2.Merge(*ring1))
assertRing(ring1, []*entry{{Token: start, Peer: peer1name, Free: 255}})
assertRing(ring2, []*entry{{Token: start, Peer: peer1name, Free: 255}})
}
func assertPeersWithSpace(t *testing.T, ring *Ring, start, end address.Address, expected int) []mesh.PeerName {
peers := ring.ChoosePeersToAskForSpace(start, end)
require.Equal(t, expected, len(peers))
return peers
}
func TestFindFree(t *testing.T) {
ring1 := New(start, end, peer1name)
peers := ring1.ChoosePeersToAskForSpace(start, end)
wt.AssertTrue(t, len(peers) == 0, "Expected no peers")
assertPeersWithSpace(t, ring1, start, end, 0)
ring1.Entries = []*entry{{Token: start, Peer: peer1name}}
peers = ring1.ChoosePeersToAskForSpace(start, end)
wt.AssertTrue(t, len(peers) == 0, "Expected no peers")
assertPeersWithSpace(t, ring1, start, end, 0)
// We shouldn't return outselves
ring1.ReportFree(map[address.Address]address.Offset{start: 10})
peers = ring1.ChoosePeersToAskForSpace(start, end)
wt.AssertTrue(t, len(peers) == 0, "Expected no peers")
assertPeersWithSpace(t, ring1, start, end, 0)
ring1.Entries = []*entry{{Token: start, Peer: peer1name, Free: 1},
{Token: middle, Peer: peer1name, Free: 1}}
peers = ring1.ChoosePeersToAskForSpace(start, end)
wt.AssertTrue(t, len(peers) == 0, "Expected no peers")
assertPeersWithSpace(t, ring1, start, end, 0)
ring1.assertInvariants()
// We should return others
var peers []mesh.PeerName
ring1.Entries = []*entry{{Token: start, Peer: peer2name, Free: 1}}
peers = ring1.ChoosePeersToAskForSpace(start, end)
wt.AssertTrue(t, len(peers) == 1, "Expected one peer")
wt.AssertEquals(t, peers[0], peer2name)
peers = assertPeersWithSpace(t, ring1, start, end, 1)
require.Equal(t, peer2name, peers[0])
ring1.Entries = []*entry{{Token: start, Peer: peer2name, Free: 1},
{Token: middle, Peer: peer3name, Free: 1}}
peers = ring1.ChoosePeersToAskForSpace(start, middle)
wt.AssertTrue(t, len(peers) == 1, "Expected one peer")
wt.AssertEquals(t, peers[0], peer2name)
peers = assertPeersWithSpace(t, ring1, start, middle, 1)
require.Equal(t, peer2name, peers[0])
peers = ring1.ChoosePeersToAskForSpace(middle, end)
wt.AssertTrue(t, len(peers) == 1, "Expected one peer")
wt.AssertEquals(t, peers[0], peer3name)
peers = assertPeersWithSpace(t, ring1, middle, end, 1)
require.Equal(t, peer3name, peers[0])
peers = ring1.ChoosePeersToAskForSpace(start, end)
wt.AssertTrue(t, len(peers) == 2, "Expected two peers")
assertPeersWithSpace(t, ring1, start, end, 2)
ring1.assertInvariants()
}
@@ -390,7 +399,7 @@ func TestReportFree(t *testing.T) {
ring1.ClaimItAll()
ring1.GrantRangeToHost(middle, end, peer2name)
wt.AssertSuccess(t, ring2.Merge(*ring1))
require.NoError(t, ring2.Merge(*ring1))
freespace := make(map[address.Address]address.Offset)
for _, r := range ring2.OwnedRanges() {
@@ -402,7 +411,7 @@ func TestReportFree(t *testing.T) {
func TestMisc(t *testing.T) {
ring := New(start, end, peer1name)
wt.AssertTrue(t, ring.Empty(), "empty")
require.True(t, ring.Empty(), "empty")
ring.ClaimItAll()
println(ring.String())
@@ -414,7 +423,7 @@ func TestEmptyGossip(t *testing.T) {
ring1.ClaimItAll()
// This used to panic, and it shouldn't
wt.AssertSuccess(t, ring1.Merge(*ring2))
require.NoError(t, ring1.Merge(*ring2))
}
func TestMergeOldMessage(t *testing.T) {
@@ -422,10 +431,10 @@ func TestMergeOldMessage(t *testing.T) {
ring2 := New(start, end, peer2name)
ring1.ClaimItAll()
wt.AssertSuccess(t, ring2.Merge(*ring1))
require.NoError(t, ring2.Merge(*ring1))
ring1.GrantRangeToHost(middle, end, peer1name)
wt.AssertSuccess(t, ring1.Merge(*ring2))
require.NoError(t, ring1.Merge(*ring2))
}
func TestSplitRangeAtBeginning(t *testing.T) {
@@ -433,33 +442,32 @@ func TestSplitRangeAtBeginning(t *testing.T) {
ring2 := New(start, end, peer2name)
ring1.ClaimItAll()
wt.AssertSuccess(t, ring2.Merge(*ring1))
require.NoError(t, ring2.Merge(*ring1))
ring1.GrantRangeToHost(start, middle, peer2name)
wt.AssertSuccess(t, ring2.Merge(*ring1))
require.NoError(t, ring2.Merge(*ring1))
}
func TestOwnedRange(t *testing.T) {
ring1 := New(start, end, peer1name)
ring1.ClaimItAll()
wt.AssertEquals(t, ring1.OwnedRanges(), []address.Range{{Start: start, End: end}})
require.Equal(t, []address.Range{{Start: start, End: end}}, ring1.OwnedRanges())
ring1.GrantRangeToHost(middle, end, peer2name)
wt.AssertEquals(t, ring1.OwnedRanges(), []address.Range{{Start: start, End: middle}})
require.Equal(t, []address.Range{{Start: start, End: middle}}, ring1.OwnedRanges())
ring2 := New(start, end, peer2name)
ring2.Merge(*ring1)
wt.AssertEquals(t, ring2.OwnedRanges(), []address.Range{{Start: middle, End: end}})
require.Equal(t, []address.Range{{Start: middle, End: end}}, ring2.OwnedRanges())
ring2.Entries = []*entry{{Token: middle, Peer: peer2name}}
wt.AssertEquals(t, ring2.OwnedRanges(),
[]address.Range{{Start: start, End: middle}, {Start: middle, End: end}})
require.Equal(t, []address.Range{{Start: start, End: middle}, {Start: middle, End: end}}, ring2.OwnedRanges())
ring2.Entries = []*entry{{Token: dot10, Peer: peer2name}, {Token: middle, Peer: peer2name}}
wt.AssertEquals(t, ring2.OwnedRanges(),
[]address.Range{{Start: start, End: dot10}, {Start: dot10, End: middle},
{Start: middle, End: end}})
require.Equal(t, []address.Range{{Start: start, End: dot10}, {Start: dot10, End: middle},
{Start: middle, End: end}}, ring2.OwnedRanges())
}
func TestTransfer(t *testing.T) {
@@ -468,7 +476,7 @@ func TestTransfer(t *testing.T) {
ring1.ClaimItAll()
ring1.GrantRangeToHost(middle, end, peer2name)
ring1.Transfer(peer2name, peer1name)
wt.AssertEquals(t, ring1.OwnedRanges(), []address.Range{{start, middle}, {middle, end}})
require.Equal(t, []address.Range{{start, middle}, {middle, end}}, ring1.OwnedRanges())
// Second test is what happens when a token exists at the end of a range but is transferred
// - does it get resurrected correctly?
@@ -477,23 +485,24 @@ func TestTransfer(t *testing.T) {
ring1.GrantRangeToHost(middle, end, peer2name)
ring1.Transfer(peer2name, peer1name)
ring1.GrantRangeToHost(dot10, middle, peer2name)
wt.AssertEquals(t, ring1.OwnedRanges(), []address.Range{{start, dot10}, {middle, end}})
require.Equal(t, []address.Range{{start, dot10}, {middle, end}}, ring1.OwnedRanges())
}
func TestOwner(t *testing.T) {
ring1 := New(start, end, peer1name)
wt.AssertTrue(t, ring1.Contains(start), "start should be in ring")
wt.AssertFalse(t, ring1.Contains(end), "end should not be in ring")
require.True(t, ring1.Contains(start), "start should be in ring")
require.False(t, ring1.Contains(end), "end should not be in ring")
wt.AssertEquals(t, ring1.Owner(start), router.UnknownPeerName)
require.Equal(t, mesh.UnknownPeerName, ring1.Owner(start))
ring1.ClaimItAll()
ring1.GrantRangeToHost(middle, end, peer2name)
wt.AssertEquals(t, ring1.Owner(start), peer1name)
wt.AssertEquals(t, ring1.Owner(middle), peer2name)
wt.AssertPanic(t, func() {
require.Equal(t, peer1name, ring1.Owner(start))
require.Equal(t, peer2name, ring1.Owner(middle))
require.Panics(t, func() {
ring1.Owner(end)
})
}
type addressSlice []address.Address
@@ -508,9 +517,9 @@ func TestFuzzRing(t *testing.T) {
iterations = 1000
)
peers := make([]router.PeerName, numPeers)
peers := make([]mesh.PeerName, numPeers)
for i := 0; i < numPeers; i++ {
peer, _ := router.PeerNameFromString(fmt.Sprintf("%02d:00:00:00:02:00", i))
peer, _ := mesh.PeerNameFromString(fmt.Sprintf("%02d:00:00:00:02:00", i))
peers[i] = peer
}
@@ -588,18 +597,18 @@ func TestFuzzRing(t *testing.T) {
}
func TestFuzzRingHard(t *testing.T) {
//common.InitDefaultLogging(true)
//common.SetLogLevel("debug")
var (
numPeers = 100
iterations = 3000
peers []router.PeerName
peers []mesh.PeerName
rings []*Ring
nextPeerID = 0
)
addPeer := func() {
peer, _ := router.PeerNameFromString(fmt.Sprintf("%02d:%02d:00:00:00:00", nextPeerID/10, nextPeerID%10))
common.Debug.Printf("%s: Adding peer", peer)
peer, _ := mesh.PeerNameFromString(fmt.Sprintf("%02d:%02d:00:00:00:00", nextPeerID/10, nextPeerID%10))
common.Log.Debugf("%s: Adding peer", peer)
nextPeerID++
peers = append(peers, peer)
rings = append(rings, New(start, end, peer))
@@ -611,7 +620,7 @@ func TestFuzzRingHard(t *testing.T) {
rings[0].ClaimItAll()
randomPeer := func(exclude int) (int, router.PeerName, *Ring) {
randomPeer := func(exclude int) (int, mesh.PeerName, *Ring) {
var peerIndex int
if exclude >= 0 {
peerIndex = rand.Intn(len(peers) - 1)
@@ -635,27 +644,27 @@ func TestFuzzRingHard(t *testing.T) {
return
}
// Pick one peer to remove, and a different one to transfer to
peerIndex, peername, _ := randomPeer(-1)
_, otherPeername, otherRing := randomPeer(peerIndex)
// We need to be in a ~converged ring to rmpeer
for _, ring := range rings {
require.NoError(t, otherRing.Merge(*ring))
}
common.Log.Debugf("%s: transferring from peer %s", otherPeername, peername)
otherRing.Transfer(peername, otherPeername)
// Remove peer from our state
peers = append(peers[:peerIndex], peers[peerIndex+1:]...)
rings = append(rings[:peerIndex], rings[peerIndex+1:]...)
theRanges = make(map[int][]address.Range)
// Transfer the space for this peer on another peer, but not this one
_, otherPeername, otherRing := randomPeer(peerIndex)
// We need to be in a ~converged ring to rmpeer
for _, ring := range rings {
wt.AssertSuccess(t, otherRing.Merge(*ring))
}
common.Debug.Printf("%s: transferring from peer %s", otherPeername, peername)
otherRing.Transfer(peername, peername)
// And now tell everyone about the transfer - rmpeer is
// not partition safe
for i, ring := range rings {
wt.AssertSuccess(t, ring.Merge(*otherRing))
require.NoError(t, ring.Merge(*otherRing))
theRanges[i] = ring.OwnedRanges()
}
}
@@ -678,13 +687,13 @@ func TestFuzzRingHard(t *testing.T) {
size := address.Subtract(rangeToSplit.End, rangeToSplit.Start)
ipInRange := address.Add(rangeToSplit.Start, address.Offset(rand.Intn(int(size))))
_, peerToGiveTo, _ := randomPeer(-1)
common.Debug.Printf("%s: Granting [%v, %v) to %s", ring.Peer, ipInRange, rangeToSplit.End, peerToGiveTo)
common.Log.Debugf("%s: Granting [%v, %v) to %s", ring.Peer, ipInRange, rangeToSplit.End, peerToGiveTo)
ring.GrantRangeToHost(ipInRange, rangeToSplit.End, peerToGiveTo)
// Now 'gossip' this to a random host (note, note could be same host as above)
otherIndex, _, otherRing := randomPeer(-1)
common.Debug.Printf("%s: 'Gossiping' to %s", ring.Peer, otherRing.Peer)
wt.AssertSuccess(t, otherRing.Merge(*ring))
common.Log.Debugf("%s: 'Gossiping' to %s", ring.Peer, otherRing.Peer)
require.NoError(t, otherRing.Merge(*ring))
theRanges[indexWithRanges] = ring.OwnedRanges()
theRanges[otherIndex] = otherRing.OwnedRanges()
@@ -702,8 +711,8 @@ func TestFuzzRingHard(t *testing.T) {
}
ring1 := ringsWithEntries[rand.Intn(len(ringsWithEntries))]
ring2index, _, ring2 := randomPeer(-1)
common.Debug.Printf("%s: 'Gossiping' to %s", ring1.Peer, ring2.Peer)
wt.AssertSuccess(t, ring2.Merge(*ring1))
common.Log.Debugf("%s: 'Gossiping' to %s", ring1.Peer, ring2.Peer)
require.NoError(t, ring2.Merge(*ring1))
theRanges[ring2index] = ring2.OwnedRanges()
}
@@ -720,7 +729,7 @@ func TestFuzzRingHard(t *testing.T) {
}
func (r *Ring) ClaimItAll() {
r.ClaimForPeers([]router.PeerName{r.Peer})
r.ClaimForPeers([]mesh.PeerName{r.Peer})
}
func (es entries) String() string {
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"sort"
"github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/ipam/address"
"github.com/weaveworks/weave/net/address"
)
type Space struct {
+77 -76
View File
@@ -3,7 +3,8 @@ package space
import (
"testing"
"github.com/weaveworks/weave/ipam/address"
"github.com/stretchr/testify/require"
"github.com/weaveworks/weave/net/address"
wt "github.com/weaveworks/weave/testing"
)
@@ -31,65 +32,65 @@ func (s *Space) NumFreeAddresses() address.Offset {
func TestLowlevel(t *testing.T) {
a := []address.Address{}
a = add(a, 100, 200)
wt.AssertEquals(t, a, []address.Address{100, 200})
wt.AssertTrue(t, !contains(a, 99), "")
wt.AssertTrue(t, contains(a, 100), "")
wt.AssertTrue(t, contains(a, 199), "")
wt.AssertTrue(t, !contains(a, 200), "")
require.Equal(t, []address.Address{100, 200}, a)
require.True(t, !contains(a, 99), "")
require.True(t, contains(a, 100), "")
require.True(t, contains(a, 199), "")
require.True(t, !contains(a, 200), "")
a = add(a, 700, 800)
wt.AssertEquals(t, a, []address.Address{100, 200, 700, 800})
require.Equal(t, []address.Address{100, 200, 700, 800}, a)
a = add(a, 300, 400)
wt.AssertEquals(t, a, []address.Address{100, 200, 300, 400, 700, 800})
require.Equal(t, []address.Address{100, 200, 300, 400, 700, 800}, a)
a = add(a, 400, 500)
wt.AssertEquals(t, a, []address.Address{100, 200, 300, 500, 700, 800})
require.Equal(t, []address.Address{100, 200, 300, 500, 700, 800}, a)
a = add(a, 600, 700)
wt.AssertEquals(t, a, []address.Address{100, 200, 300, 500, 600, 800})
require.Equal(t, []address.Address{100, 200, 300, 500, 600, 800}, a)
a = add(a, 500, 600)
wt.AssertEquals(t, a, []address.Address{100, 200, 300, 800})
require.Equal(t, []address.Address{100, 200, 300, 800}, a)
a = subtract(a, 500, 600)
wt.AssertEquals(t, a, []address.Address{100, 200, 300, 500, 600, 800})
require.Equal(t, []address.Address{100, 200, 300, 500, 600, 800}, a)
a = subtract(a, 600, 700)
wt.AssertEquals(t, a, []address.Address{100, 200, 300, 500, 700, 800})
require.Equal(t, []address.Address{100, 200, 300, 500, 700, 800}, a)
a = subtract(a, 400, 500)
wt.AssertEquals(t, a, []address.Address{100, 200, 300, 400, 700, 800})
require.Equal(t, []address.Address{100, 200, 300, 400, 700, 800}, a)
a = subtract(a, 300, 400)
wt.AssertEquals(t, a, []address.Address{100, 200, 700, 800})
require.Equal(t, []address.Address{100, 200, 700, 800}, a)
a = subtract(a, 700, 800)
wt.AssertEquals(t, a, []address.Address{100, 200})
require.Equal(t, []address.Address{100, 200}, a)
a = subtract(a, 100, 200)
wt.AssertEquals(t, a, []address.Address{})
require.Equal(t, []address.Address{}, a)
s := New()
wt.AssertEquals(t, s.NumFreeAddresses(), address.Offset(0))
require.Equal(t, address.Offset(0), s.NumFreeAddresses())
ok, got := s.Allocate(address.NewRange(0, 1000))
wt.AssertFalse(t, ok, "allocate in empty space should fail")
require.False(t, ok, "allocate in empty space should fail")
s.Add(100, 100)
wt.AssertEquals(t, s.NumFreeAddresses(), address.Offset(100))
require.Equal(t, address.Offset(100), s.NumFreeAddresses())
ok, got = s.Allocate(address.NewRange(0, 1000))
wt.AssertTrue(t, ok && got == 100, "allocate")
wt.AssertEquals(t, s.NumFreeAddresses(), address.Offset(99))
wt.AssertNoErr(t, s.Claim(150))
wt.AssertEquals(t, s.NumFreeAddresses(), address.Offset(98))
wt.AssertNoErr(t, s.Free(100))
wt.AssertEquals(t, s.NumFreeAddresses(), address.Offset(99))
wt.AssertErrorInterface(t, s.Free(0), (*error)(nil), "free not allocated")
wt.AssertErrorInterface(t, s.Free(100), (*error)(nil), "double free")
require.True(t, ok && got == 100, "allocate")
require.Equal(t, address.Offset(99), s.NumFreeAddresses())
require.NoError(t, s.Claim(150))
require.Equal(t, address.Offset(98), s.NumFreeAddresses())
require.NoError(t, s.Free(100))
require.Equal(t, address.Offset(99), s.NumFreeAddresses())
wt.AssertErrorInterface(t, (*error)(nil), s.Free(0), "free not allocated")
wt.AssertErrorInterface(t, (*error)(nil), s.Free(100), "double free")
r, ok := s.Donate(address.NewRange(0, 1000))
wt.AssertTrue(t, ok && r.Start == 125 && r.Size() == 25, "donate")
require.True(t, ok && r.Start == 125 && r.Size() == 25, "donate")
// test Donate when addresses are scarce
s = New()
r, ok = s.Donate(address.NewRange(0, 1000))
wt.AssertTrue(t, !ok, "donate on empty space should fail")
require.True(t, !ok, "donate on empty space should fail")
s.Add(0, 3)
wt.AssertNoErr(t, s.Claim(0))
wt.AssertNoErr(t, s.Claim(2))
require.NoError(t, s.Claim(0))
require.NoError(t, s.Claim(2))
r, ok = s.Donate(address.NewRange(0, 1000))
wt.AssertTrue(t, ok && r.Start == 1 && r.End == 2, "donate")
require.True(t, ok && r.Start == 1 && r.End == 2, "donate")
r, ok = s.Donate(address.NewRange(0, 1000))
wt.AssertTrue(t, !ok, "donate should fail")
require.True(t, !ok, "donate should fail")
}
func TestSpaceAllocate(t *testing.T) {
@@ -106,27 +107,27 @@ func TestSpaceAllocate(t *testing.T) {
)
space1 := makeSpace(start, size)
wt.AssertEquals(t, space1.NumFreeAddresses(), address.Offset(20))
require.Equal(t, address.Offset(20), space1.NumFreeAddresses())
space1.assertInvariants()
_, addr1 := space1.Allocate(address.NewRange(start, size))
wt.AssertEqualString(t, addr1.String(), testAddr1, "address")
wt.AssertEquals(t, space1.NumFreeAddresses(), address.Offset(19))
require.Equal(t, testAddr1, addr1.String(), "address")
require.Equal(t, address.Offset(19), space1.NumFreeAddresses())
space1.assertInvariants()
_, addr2 := space1.Allocate(address.NewRange(start, size))
wt.AssertFalse(t, addr2.String() == testAddr1, "address")
wt.AssertEquals(t, space1.NumFreeAddresses(), address.Offset(18))
wt.AssertEquals(t, space1.NumFreeAddressesInRange(address.Range{Start: ip(testAddr1), End: ip(testAddrx)}), address.Offset(13))
wt.AssertEquals(t, space1.NumFreeAddressesInRange(address.Range{Start: ip(testAddr1), End: ip(testAddry)}), address.Offset(18))
require.False(t, addr2.String() == testAddr1, "address")
require.Equal(t, address.Offset(18), space1.NumFreeAddresses())
require.Equal(t, address.Offset(13), space1.NumFreeAddressesInRange(address.Range{Start: ip(testAddr1), End: ip(testAddrx)}))
require.Equal(t, address.Offset(18), space1.NumFreeAddressesInRange(address.Range{Start: ip(testAddr1), End: ip(testAddry)}))
space1.assertInvariants()
space1.Free(addr2)
space1.assertInvariants()
wt.AssertErrorInterface(t, space1.Free(addr2), (*error)(nil), "double free")
wt.AssertErrorInterface(t, space1.Free(ip(testAddrx)), (*error)(nil), "address not allocated")
wt.AssertErrorInterface(t, space1.Free(ip(testAddry)), (*error)(nil), "wrong out of range")
wt.AssertErrorInterface(t, (*error)(nil), space1.Free(addr2), "double free")
wt.AssertErrorInterface(t, (*error)(nil), space1.Free(ip(testAddrx)), "address not allocated")
wt.AssertErrorInterface(t, (*error)(nil), space1.Free(ip(testAddry)), "wrong out of range")
space1.assertInvariants()
}
@@ -144,51 +145,51 @@ func TestSpaceFree(t *testing.T) {
// Check we are prepared to give up the entire space
r := space.biggestFreeRange(entireRange)
wt.AssertTrue(t, r.Start == ip(testAddr1) && r.Size() == 20, "Wrong space")
require.True(t, r.Start == ip(testAddr1) && r.Size() == 20, "Wrong space")
for i := 0; i < 20; i++ {
ok, _ := space.Allocate(entireRange)
wt.AssertTrue(t, ok, "Failed to get address")
require.True(t, ok, "Failed to get address")
}
// Check we are full
ok, _ := space.Allocate(entireRange)
wt.AssertTrue(t, !ok, "Should have failed to get address")
require.True(t, !ok, "Should have failed to get address")
r, ok = space.Donate(entireRange)
wt.AssertTrue(t, r.Size() == 0, "Wrong space")
require.True(t, r.Size() == 0, "Wrong space")
// Free in the middle
wt.AssertSuccess(t, space.Free(ip("10.0.3.13")))
require.NoError(t, space.Free(ip("10.0.3.13")))
r = space.biggestFreeRange(entireRange)
wt.AssertTrue(t, r.Start == ip("10.0.3.13") && r.Size() == 1, "Wrong space")
require.True(t, r.Start == ip("10.0.3.13") && r.Size() == 1, "Wrong space")
// Free one at the end
wt.AssertSuccess(t, space.Free(ip("10.0.3.23")))
require.NoError(t, space.Free(ip("10.0.3.23")))
r = space.biggestFreeRange(entireRange)
wt.AssertTrue(t, r.Start == ip("10.0.3.23") && r.Size() == 1, "Wrong space")
require.True(t, r.Start == ip("10.0.3.23") && r.Size() == 1, "Wrong space")
// Now free a few at the end
wt.AssertSuccess(t, space.Free(ip("10.0.3.22")))
wt.AssertSuccess(t, space.Free(ip("10.0.3.21")))
require.NoError(t, space.Free(ip("10.0.3.22")))
require.NoError(t, space.Free(ip("10.0.3.21")))
wt.AssertEquals(t, space.NumFreeAddresses(), address.Offset(4))
require.Equal(t, address.Offset(4), space.NumFreeAddresses())
// Now get the biggest free space; should be 3.21
r = space.biggestFreeRange(entireRange)
wt.AssertTrue(t, r.Start == ip("10.0.3.21") && r.Size() == 3, "Wrong space")
require.True(t, r.Start == ip("10.0.3.21") && r.Size() == 3, "Wrong space")
// Now free a few in the middle
wt.AssertSuccess(t, space.Free(ip("10.0.3.12")))
wt.AssertSuccess(t, space.Free(ip("10.0.3.11")))
wt.AssertSuccess(t, space.Free(ip("10.0.3.10")))
require.NoError(t, space.Free(ip("10.0.3.12")))
require.NoError(t, space.Free(ip("10.0.3.11")))
require.NoError(t, space.Free(ip("10.0.3.10")))
wt.AssertEquals(t, space.NumFreeAddresses(), address.Offset(7))
require.Equal(t, address.Offset(7), space.NumFreeAddresses())
// Now get the biggest free space; should be 3.21
r = space.biggestFreeRange(entireRange)
wt.AssertTrue(t, r.Start == ip("10.0.3.10") && r.Size() == 4, "Wrong space")
require.True(t, r.Start == ip("10.0.3.10") && r.Size() == 4, "Wrong space")
wt.AssertEquals(t, space.OwnedRanges(), []address.Range{address.Range{Start: ip("10.0.3.4"), End: ip("10.0.3.24")}})
require.Equal(t, []address.Range{{Start: ip("10.0.3.4"), End: ip("10.0.3.24")}}, space.OwnedRanges())
}
func TestDonateSimple(t *testing.T) {
@@ -207,10 +208,10 @@ func TestDonateSimple(t *testing.T) {
// Empty space set should split in two and give me the second half
r, ok := ps1.Donate(address.NewRange(ip(testAddr1), size))
numGivenUp := r.Size()
wt.AssertTrue(t, ok, "Donate result")
wt.AssertEqualString(t, r.Start.String(), "10.0.1.24", "Invalid start")
wt.AssertEquals(t, numGivenUp, address.Offset(size/2))
wt.AssertEquals(t, ps1.NumFreeAddresses(), address.Offset(size/2))
require.True(t, ok, "Donate result")
require.Equal(t, "10.0.1.24", r.Start.String(), "Invalid start")
require.Equal(t, address.Offset(size/2), numGivenUp)
require.Equal(t, address.Offset(size/2), ps1.NumFreeAddresses())
// Now check we can give the rest up.
count := 0 // count to avoid infinite loop
@@ -221,8 +222,8 @@ func TestDonateSimple(t *testing.T) {
}
numGivenUp += r.Size()
}
wt.AssertEquals(t, ps1.NumFreeAddresses(), address.Offset(0))
wt.AssertEquals(t, numGivenUp, address.Offset(size))
require.Equal(t, address.Offset(0), ps1.NumFreeAddresses())
require.Equal(t, address.Offset(size), numGivenUp)
}
func TestDonateHard(t *testing.T) {
@@ -236,27 +237,27 @@ func TestDonateHard(t *testing.T) {
spaceset := makeSpace(start, size)
for i := address.Offset(0); i < size; i++ {
ok, _ := spaceset.Allocate(address.NewRange(start, size))
wt.AssertTrue(t, ok, "Failed to get IP!")
require.True(t, ok, "Failed to get IP!")
}
wt.AssertEquals(t, spaceset.NumFreeAddresses(), address.Offset(0))
require.Equal(t, address.Offset(0), spaceset.NumFreeAddresses())
// Now free all but the last address
// this will force us to split the free list
for i := address.Offset(0); i < size-1; i++ {
wt.AssertSuccess(t, spaceset.Free(address.Add(start, i)))
require.NoError(t, spaceset.Free(address.Add(start, i)))
}
// Now split
newRange, ok := spaceset.Donate(address.NewRange(start, size))
wt.AssertTrue(t, ok, "GiveUpSpace result")
wt.AssertEquals(t, newRange.Start, ip("10.0.1.23"))
wt.AssertEquals(t, newRange.Size(), address.Offset(24))
wt.AssertEquals(t, spaceset.NumFreeAddresses(), address.Offset(23))
require.True(t, ok, "GiveUpSpace result")
require.Equal(t, ip("10.0.1.23"), newRange.Start)
require.Equal(t, address.Offset(24), newRange.Size())
require.Equal(t, address.Offset(23), spaceset.NumFreeAddresses())
//Space set should now have 2 spaces
expected := New()
expected.Add(start, 23)
expected.ours = add(nil, ip("10.0.1.47"), ip("10.0.1.48"))
wt.AssertEquals(t, spaceset, expected)
require.Equal(t, expected, spaceset)
}
+82
View File
@@ -0,0 +1,82 @@
package ipam
import (
"github.com/weaveworks/weave/ipam/paxos"
"github.com/weaveworks/weave/net/address"
)
type Status struct {
Paxos *paxos.Status
Range string
DefaultSubnet string
Entries []EntryStatus
PendingClaims []ClaimStatus
PendingAllocates []string
}
type EntryStatus struct {
Token string
Peer string
Version uint32
}
type ClaimStatus struct {
Ident string
Address address.Address
}
func NewStatus(allocator *Allocator, defaultSubnet address.CIDR) *Status {
if allocator == nil {
return nil
}
var paxosStatus *paxos.Status
if allocator.paxosTicker != nil {
paxosStatus = paxos.NewStatus(allocator.paxos)
}
resultChan := make(chan *Status)
allocator.actionChan <- func() {
resultChan <- &Status{
paxosStatus,
allocator.universe.String(),
defaultSubnet.String(),
newEntryStatusSlice(allocator),
newClaimStatusSlice(allocator),
newAllocateIdentSlice(allocator)}
}
return <-resultChan
}
func newEntryStatusSlice(allocator *Allocator) []EntryStatus {
var slice []EntryStatus
if allocator.ring.Empty() {
return slice
}
for _, entry := range allocator.ring.Entries {
slice = append(slice, EntryStatus{entry.Token.String(), entry.Peer.String(), entry.Version})
}
return slice
}
func newClaimStatusSlice(allocator *Allocator) []ClaimStatus {
var slice []ClaimStatus
for _, op := range allocator.pendingClaims {
claim := op.(*claim)
slice = append(slice, ClaimStatus{claim.ident, claim.addr})
}
return slice
}
func newAllocateIdentSlice(allocator *Allocator) []string {
var slice []string
for _, op := range allocator.pendingAllocates {
allocate := op.(*allocate)
slice = append(slice, allocate.ident)
}
return slice
}
+34 -150
View File
@@ -7,14 +7,14 @@ import (
"testing"
"time"
"github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/ipam/address"
"github.com/weaveworks/weave/router"
wt "github.com/weaveworks/weave/testing"
"github.com/stretchr/testify/require"
"github.com/weaveworks/weave/mesh"
"github.com/weaveworks/weave/net/address"
"github.com/weaveworks/weave/testing/gossip"
)
type mockMessage struct {
dst router.PeerName
dst mesh.PeerName
msgType byte
buf []byte
}
@@ -33,7 +33,7 @@ func toStringArray(messages []mockMessage) []string {
type mockGossipComms struct {
sync.RWMutex
t *testing.T
*testing.T
name string
messages []mockMessage
}
@@ -48,16 +48,16 @@ func (m *mockGossipComms) String() string {
// that the contents of messages are never re-ordered. Which, for instance,
// requires they are not based off iterating through a map.
func (m *mockGossipComms) GossipBroadcast(update router.GossipData) error {
func (m *mockGossipComms) GossipBroadcast(update mesh.GossipData) error {
m.Lock()
defer m.Unlock()
buf := []byte{}
if len(m.messages) == 0 {
m.Fatalf("%s: Gossip broadcast message unexpected: \n%x", m.name, buf)
} else if msg := m.messages[0]; msg.dst != router.UnknownPeerName {
m.Fatalf("%s: Expected Gossip message to %s but got broadcast", m.name, msg.dst)
require.FailNow(m, fmt.Sprintf("%s: Gossip broadcast message unexpected: \n%x", m.name, buf))
} else if msg := m.messages[0]; msg.dst != mesh.UnknownPeerName {
require.FailNow(m, fmt.Sprintf("%s: Expected Gossip message to %s but got broadcast", m.name, msg.dst))
} else if msg.buf != nil && !equalByteBuffer(msg.buf, buf) {
m.Fatalf("%s: Gossip message not sent as expected: \nwant: %x\ngot : %x", m.name, msg.buf, buf)
require.FailNow(m, fmt.Sprintf("%s: Gossip message not sent as expected: \nwant: %x\ngot : %x", m.name, msg.buf, buf))
} else {
// Swallow this message
m.messages = m.messages[1:]
@@ -77,24 +77,19 @@ func equalByteBuffer(a, b []byte) bool {
return true
}
func (m *mockGossipComms) Fatalf(format string, args ...interface{}) {
// this sometimes hangs: wt.Fatalf(m.t, args...)
panic(fmt.Sprintf(format, args...))
}
func (m *mockGossipComms) GossipUnicast(dstPeerName router.PeerName, buf []byte) error {
func (m *mockGossipComms) GossipUnicast(dstPeerName mesh.PeerName, buf []byte) error {
m.Lock()
defer m.Unlock()
if len(m.messages) == 0 {
m.Fatalf("%s: Gossip message to %s unexpected: \n%s", m.name, dstPeerName, buf)
} else if msg := m.messages[0]; msg.dst == router.UnknownPeerName {
m.Fatalf("%s: Expected Gossip broadcast message but got dest %s", m.name, dstPeerName)
require.FailNow(m, fmt.Sprintf("%s: Gossip message to %s unexpected: \n%s", m.name, dstPeerName, buf))
} else if msg := m.messages[0]; msg.dst == mesh.UnknownPeerName {
require.FailNow(m, fmt.Sprintf("%s: Expected Gossip broadcast message but got dest %s", m.name, dstPeerName))
} else if msg.dst != dstPeerName {
m.Fatalf("%s: Expected Gossip message to %s but got dest %s", m.name, msg.dst, dstPeerName)
require.FailNow(m, fmt.Sprintf("%s: Expected Gossip message to %s but got dest %s", m.name, msg.dst, dstPeerName))
} else if buf[0] != msg.msgType {
m.Fatalf("%s: Expected Gossip message of type %d but got type %d", m.name, msg.msgType, buf[0])
require.FailNow(m, fmt.Sprintf("%s: Expected Gossip message of type %d but got type %d", m.name, msg.msgType, buf[0]))
} else if msg.buf != nil && !equalByteBuffer(msg.buf, buf[1:]) {
m.Fatalf("%s: Gossip message not sent as expected: \nwant: %x\ngot : %x", m.name, msg.buf, buf[1:])
require.FailNow(m, fmt.Sprintf("%s: Gossip message not sent as expected: \nwant: %x\ngot : %x", m.name, msg.buf, buf[1:]))
} else {
// Swallow this message
m.messages = m.messages[1:]
@@ -104,7 +99,7 @@ func (m *mockGossipComms) GossipUnicast(dstPeerName router.PeerName, buf []byte)
func ExpectMessage(alloc *Allocator, dst string, msgType byte, buf []byte) {
m := alloc.gossip.(*mockGossipComms)
dstPeerName, _ := router.PeerNameFromString(dst)
dstPeerName, _ := mesh.PeerNameFromString(dst)
m.Lock()
m.messages = append(m.messages, mockMessage{dstPeerName, msgType, buf})
m.Unlock()
@@ -113,7 +108,7 @@ func ExpectMessage(alloc *Allocator, dst string, msgType byte, buf []byte) {
func ExpectBroadcastMessage(alloc *Allocator, buf []byte) {
m := alloc.gossip.(*mockGossipComms)
m.Lock()
m.messages = append(m.messages, mockMessage{router.UnknownPeerName, 0, buf})
m.messages = append(m.messages, mockMessage{mesh.UnknownPeerName, 0, buf})
m.Unlock()
}
@@ -122,14 +117,14 @@ func CheckAllExpectedMessagesSent(allocs ...*Allocator) {
m := alloc.gossip.(*mockGossipComms)
m.RLock()
if len(m.messages) > 0 {
wt.Fatalf(m.t, "%s: Gossip message(s) not sent as expected: \n%x", m.name, m.messages)
require.FailNow(m, fmt.Sprintf("%s: Gossip message(s) not sent as expected: \n%x", m.name, m.messages))
}
m.RUnlock()
}
}
func makeAllocator(name string, cidrStr string, quorum uint) (*Allocator, address.Range) {
peername, err := router.PeerNameFromString(name)
peername, err := mesh.PeerNameFromString(name)
if err != nil {
panic(err)
}
@@ -139,22 +134,22 @@ func makeAllocator(name string, cidrStr string, quorum uint) (*Allocator, addres
panic(err)
}
alloc := NewAllocator(peername, router.PeerUID(rand.Int63()),
"nick-"+name, cidr.Range(), quorum)
alloc := NewAllocator(peername, mesh.PeerUID(rand.Int63()),
"nick-"+name, cidr.Range(), quorum, func(mesh.PeerName) bool { return true })
return alloc, cidr.HostRange()
}
func makeAllocatorWithMockGossip(t *testing.T, name string, universeCIDR string, quorum uint) (*Allocator, address.Range) {
alloc, subnet := makeAllocator(name, universeCIDR, quorum)
gossip := &mockGossipComms{t: t, name: name}
gossip := &mockGossipComms{T: t, name: name}
alloc.SetInterfaces(gossip)
alloc.Start()
return alloc, subnet
}
func (alloc *Allocator) claimRingForTesting(allocs ...*Allocator) {
peers := []router.PeerName{alloc.ourName}
peers := []mesh.PeerName{alloc.ourName}
for _, alloc2 := range allocs {
peers = append(peers, alloc2.ourName)
}
@@ -177,14 +172,14 @@ func AssertSent(t *testing.T, ch <-chan bool) {
case <-ch:
// This case is ok
case <-timeout:
wt.Fatalf(t, "Nothing sent on channel")
require.FailNow(t, "Nothing sent on channel")
}
}
func AssertNothingSent(t *testing.T, ch <-chan bool) {
select {
case val := <-ch:
wt.Fatalf(t, "Unexpected value on channel: %t", val)
require.FailNow(t, fmt.Sprintf("Unexpected value on channel: %v", val))
default:
// no message received
}
@@ -193,125 +188,14 @@ func AssertNothingSent(t *testing.T, ch <-chan bool) {
func AssertNothingSentErr(t *testing.T, ch <-chan error) {
select {
case val := <-ch:
wt.Fatalf(t, "Unexpected value on channel: %t", val)
require.FailNow(t, fmt.Sprintf("Unexpected value on channel: %v", val))
default:
// no message received
}
}
// Router to convey gossip from one gossiper to another, for testing
type unicastMessage struct {
sender router.PeerName
buf []byte
}
type broadcastMessage struct {
data router.GossipData
}
type exitMessage struct {
exitChan chan struct{}
}
type flushMessage struct {
flushChan chan struct{}
}
type TestGossipRouter struct {
gossipChans map[router.PeerName]chan interface{}
loss float32 // 0.0 means no loss
}
func (grouter *TestGossipRouter) GossipBroadcast(update router.GossipData) error {
for _, gossipChan := range grouter.gossipChans {
select {
case gossipChan <- broadcastMessage{data: update}:
default: // drop the message if we cannot send it
}
}
return nil
}
func (grouter *TestGossipRouter) flush() {
for _, gossipChan := range grouter.gossipChans {
flushChan := make(chan struct{})
gossipChan <- flushMessage{flushChan: flushChan}
<-flushChan
}
}
func (grouter *TestGossipRouter) removePeer(peer router.PeerName) {
gossipChan := grouter.gossipChans[peer]
resultChan := make(chan struct{})
gossipChan <- exitMessage{exitChan: resultChan}
<-resultChan
delete(grouter.gossipChans, peer)
}
type TestGossipRouterClient struct {
router *TestGossipRouter
sender router.PeerName
}
func (grouter *TestGossipRouter) run(gossiper router.Gossiper, gossipChan chan interface{}) {
gossipTimer := time.Tick(10 * time.Second)
for {
select {
case gossip := <-gossipChan:
switch message := gossip.(type) {
case exitMessage:
close(message.exitChan)
return
case flushMessage:
close(message.flushChan)
case unicastMessage:
if rand.Float32() > (1.0 - grouter.loss) {
continue
}
if err := gossiper.OnGossipUnicast(message.sender, message.buf); err != nil {
panic(fmt.Sprintf("Error doing gossip unicast to %s: %s", message.sender, err))
}
case broadcastMessage:
if rand.Float32() > (1.0 - grouter.loss) {
continue
}
for _, msg := range message.data.Encode() {
if _, err := gossiper.OnGossipBroadcast(msg); err != nil {
panic(fmt.Sprintf("Error doing gossip broadcast: %s", err))
}
}
}
case <-gossipTimer:
grouter.GossipBroadcast(gossiper.Gossip())
}
}
}
func (grouter *TestGossipRouter) connect(sender router.PeerName, gossiper router.Gossiper) router.Gossip {
gossipChan := make(chan interface{}, 100)
go grouter.run(gossiper, gossipChan)
grouter.gossipChans[sender] = gossipChan
return TestGossipRouterClient{grouter, sender}
}
func (client TestGossipRouterClient) GossipUnicast(dstPeerName router.PeerName, buf []byte) error {
select {
case client.router.gossipChans[dstPeerName] <- unicastMessage{sender: client.sender, buf: buf}:
default: // drop the message if we cannot send it
common.Error.Printf("Dropping message")
}
return nil
}
func (client TestGossipRouterClient) GossipBroadcast(update router.GossipData) error {
return client.router.GossipBroadcast(update)
}
func makeNetworkOfAllocators(size int, cidr string) ([]*Allocator, TestGossipRouter, address.Range) {
gossipRouter := TestGossipRouter{make(map[router.PeerName]chan interface{}), 0.0}
func makeNetworkOfAllocators(size int, cidr string) ([]*Allocator, *gossip.TestRouter, address.Range) {
gossipRouter := gossip.NewTestRouter(0.0)
allocs := make([]*Allocator, size)
var subnet address.Range
@@ -319,13 +203,13 @@ func makeNetworkOfAllocators(size int, cidr string) ([]*Allocator, TestGossipRou
var alloc *Allocator
alloc, subnet = makeAllocator(fmt.Sprintf("%02d:00:00:02:00:00", i),
cidr, uint(size/2+1))
alloc.SetInterfaces(gossipRouter.connect(alloc.ourName, alloc))
alloc.SetInterfaces(gossipRouter.Connect(alloc.ourName, alloc))
alloc.Start()
allocs[i] = alloc
}
gossipRouter.GossipBroadcast(allocs[size-1].Gossip())
gossipRouter.flush()
allocs[size-1].gossip.GossipBroadcast(allocs[size-1].Gossip())
gossipRouter.Flush()
return allocs, gossipRouter, subnet
}
+437
View File
@@ -0,0 +1,437 @@
package mesh
import (
"fmt"
"net"
"strconv"
"sync"
"time"
)
type Connection interface {
Local() *Peer
Remote() *Peer
RemoteTCPAddr() string
Outbound() bool
Established() bool
BreakTie(Connection) ConnectionTieBreak
Shutdown(error)
Log(args ...interface{})
}
type ConnectionTieBreak int
const (
TieBreakWon ConnectionTieBreak = iota
TieBreakLost
TieBreakTied
)
var ErrConnectToSelf = fmt.Errorf("Cannot connect to ourself")
type RemoteConnection struct {
local *Peer
remote *Peer
remoteTCPAddr string
outbound bool
established bool
}
type LocalConnection struct {
sync.RWMutex
RemoteConnection
TCPConn *net.TCPConn
version byte
tcpSender TCPSender
SessionKey *[32]byte
heartbeatTCP *time.Ticker
Router *Router
uid uint64
actionChan chan<- ConnectionAction
finished <-chan struct{} // closed to signal that actorLoop has finished
OverlayConn OverlayConnection
}
type ConnectionAction func() error
func NewRemoteConnection(from, to *Peer, tcpAddr string, outbound bool, established bool) *RemoteConnection {
return &RemoteConnection{
local: from,
remote: to,
remoteTCPAddr: tcpAddr,
outbound: outbound,
established: established,
}
}
func (conn *RemoteConnection) Local() *Peer { return conn.local }
func (conn *RemoteConnection) Remote() *Peer { return conn.remote }
func (conn *RemoteConnection) RemoteTCPAddr() string { return conn.remoteTCPAddr }
func (conn *RemoteConnection) Outbound() bool { return conn.outbound }
func (conn *RemoteConnection) Established() bool { return conn.established }
func (conn *RemoteConnection) BreakTie(Connection) ConnectionTieBreak { return TieBreakTied }
func (conn *RemoteConnection) Shutdown(error) {}
func (conn *RemoteConnection) Log(args ...interface{}) {
log.Println(append(append([]interface{}{}, fmt.Sprintf("->[%s|%s]:", conn.remoteTCPAddr, conn.remote)), args...)...)
}
func (conn *RemoteConnection) ErrorLog(args ...interface{}) {
log.Errorln(append(append([]interface{}{}, fmt.Sprintf("->[%s|%s]:", conn.remoteTCPAddr, conn.remote)), args...)...)
}
// Does not return anything. If the connection is successful, it will
// end up in the local peer's connections map.
func StartLocalConnection(connRemote *RemoteConnection, tcpConn *net.TCPConn, router *Router, acceptNewPeer bool) {
if connRemote.local != router.Ourself.Peer {
log.Fatal("Attempt to create local connection from a peer which is not ourself")
}
// NB, we're taking a copy of connRemote here.
actionChan := make(chan ConnectionAction, ChannelSize)
finished := make(chan struct{})
conn := &LocalConnection{
RemoteConnection: *connRemote,
Router: router,
TCPConn: tcpConn,
uid: randUint64(),
actionChan: actionChan,
finished: finished}
go conn.run(actionChan, finished, acceptNewPeer)
}
func (conn *LocalConnection) BreakTie(dupConn Connection) ConnectionTieBreak {
dupConnLocal := dupConn.(*LocalConnection)
// conn.uid is used as the tie breaker here, in the knowledge that
// both sides will make the same decision.
if conn.uid < dupConnLocal.uid {
return TieBreakWon
} else if dupConnLocal.uid < conn.uid {
return TieBreakLost
} else {
return TieBreakTied
}
}
func (conn *LocalConnection) Established() bool {
return conn.established
}
// Send directly, not via the Actor. If it goes via the Actor we can
// get a deadlock where LocalConnection is blocked talking to
// LocalPeer and LocalPeer is blocked trying send a ProtocolMsg via
// LocalConnection, and the channels are full in both directions so
// nothing can proceed.
func (conn *LocalConnection) SendProtocolMsg(m ProtocolMsg) {
if err := conn.sendProtocolMsg(m); err != nil {
conn.Shutdown(err)
}
}
// ACTOR methods
// NB: The conn.* fields are only written by the connection actor
// process, which is the caller of the ConnectionAction funs. Hence we
// do not need locks for reading, and only need write locks for fields
// read by other processes.
// Async
func (conn *LocalConnection) Shutdown(err error) {
// err should always be a real error, even if only io.EOF
if err == nil {
panic("nil error")
}
// Run on its own goroutine in case the channel is backed up
go func() { conn.sendAction(func() error { return err }) }()
}
// Send an actor request to the actorLoop, but don't block if
// actorLoop has exited - see http://blog.golang.org/pipelines for
// pattern
func (conn *LocalConnection) sendAction(action ConnectionAction) {
select {
case conn.actionChan <- action:
case <-conn.finished:
}
}
// ACTOR server
func (conn *LocalConnection) run(actionChan <-chan ConnectionAction, finished chan<- struct{}, acceptNewPeer bool) {
var err error // important to use this var and not create another one with 'err :='
defer func() { conn.shutdown(err) }()
defer close(finished)
conn.TCPConn.SetLinger(0)
intro, err := ProtocolIntroParams{
MinVersion: conn.Router.ProtocolMinVersion,
MaxVersion: ProtocolMaxVersion,
Features: conn.makeFeatures(),
Conn: conn.TCPConn,
Password: conn.Router.Password,
Outbound: conn.outbound,
}.DoIntro()
if err != nil {
return
}
conn.SessionKey = intro.SessionKey
conn.tcpSender = intro.Sender
conn.version = intro.Version
remote, err := conn.parseFeatures(intro.Features)
if err != nil {
return
}
if err = conn.registerRemote(remote, acceptNewPeer); err != nil {
return
}
conn.Log("connection ready; using protocol version", conn.version)
params := OverlayConnectionParams{
RemotePeer: conn.remote,
LocalAddr: conn.TCPConn.LocalAddr().(*net.TCPAddr),
RemoteAddr: conn.TCPConn.RemoteAddr().(*net.TCPAddr),
Outbound: conn.outbound,
ConnUID: conn.uid,
SessionKey: conn.SessionKey,
SendControlMessage: conn.sendOverlayControlMessage,
Features: intro.Features,
}
if conn.OverlayConn, err = conn.Router.Overlay.PrepareConnection(params); err != nil {
return
}
// As soon as we do AddConnection, the new connection becomes
// visible to the packet routing logic. So AddConnection must
// come after PrepareConnection
if err = conn.Router.Ourself.AddConnection(conn); err != nil {
return
}
conn.Router.ConnectionMaker.ConnectionCreated(conn)
// OverlayConnection confirmation comes after AddConnection,
// because only after that completes do we know the connection is
// valid: in particular that it is not a duplicate connection to
// the same peer. Overlay communication on a duplicate connection
// can cause problems such as tripping up overlay crypto at the
// other end due to data being decoded by the other connection. It
// is also generally wasteful to engage in any interaction with
// the remote on a connection that turns out to be invalid.
conn.OverlayConn.Confirm()
// receiveTCP must follow also AddConnection. In the absence
// of any indirect connectivity to the remote peer, the first
// we hear about it (and any peers reachable from it) is
// through topology gossip it sends us on the connection. We
// must ensure that the connection has been added to Ourself
// prior to processing any such gossip, otherwise we risk
// immediately gc'ing part of that newly received portion of
// the topology (though not the remote peer itself, since that
// will have a positive ref count), leaving behind dangling
// references to peers. Hence we must invoke AddConnection,
// which is *synchronous*, first.
conn.heartbeatTCP = time.NewTicker(TCPHeartbeat)
go conn.receiveTCP(intro.Receiver)
// AddConnection must precede actorLoop. More precisely, it
// must precede shutdown, since that invokes DeleteConnection
// and is invoked on termination of this entire
// function. Essentially this boils down to a prohibition on
// running AddConnection in a separate goroutine, at least not
// without some synchronisation. Which in turn requires the
// launching of the receiveTCP goroutine to precede actorLoop.
err = conn.actorLoop(actionChan)
}
func (conn *LocalConnection) makeFeatures() map[string]string {
features := map[string]string{
"PeerNameFlavour": PeerNameFlavour,
"Name": conn.local.Name.String(),
"NickName": conn.local.NickName,
"ShortID": fmt.Sprint(conn.local.ShortID),
"UID": fmt.Sprint(conn.local.UID),
"ConnID": fmt.Sprint(conn.uid),
}
conn.Router.Overlay.AddFeaturesTo(features)
return features
}
type features map[string]string
func (features features) MustHave(keys []string) error {
for _, key := range keys {
if _, ok := features[key]; !ok {
return fmt.Errorf("Field %s is missing", key)
}
}
return nil
}
func (features features) Get(key string) string {
return features[key]
}
func (conn *LocalConnection) parseFeatures(features features) (*Peer, error) {
if err := features.MustHave([]string{"PeerNameFlavour", "Name", "NickName", "UID", "ConnID"}); err != nil {
return nil, err
}
remotePeerNameFlavour := features.Get("PeerNameFlavour")
if remotePeerNameFlavour != PeerNameFlavour {
return nil, fmt.Errorf("Peer name flavour mismatch (ours: '%s', theirs: '%s')", PeerNameFlavour, remotePeerNameFlavour)
}
name, err := PeerNameFromString(features.Get("Name"))
if err != nil {
return nil, err
}
nickName := features.Get("NickName")
var shortID uint64
var hasShortID bool
if shortIDStr, present := features["ShortID"]; present {
hasShortID = true
shortID, err = strconv.ParseUint(shortIDStr, 10,
PeerShortIDBits)
if err != nil {
return nil, err
}
}
uid, err := ParsePeerUID(features.Get("UID"))
if err != nil {
return nil, err
}
remoteConnID, err := strconv.ParseUint(features.Get("ConnID"), 10, 64)
if err != nil {
return nil, err
}
conn.uid ^= remoteConnID
peer := NewPeer(name, nickName, uid, 0, PeerShortID(shortID))
peer.HasShortID = hasShortID
return peer, nil
}
func (conn *LocalConnection) registerRemote(remote *Peer, acceptNewPeer bool) error {
if acceptNewPeer {
conn.remote = conn.Router.Peers.FetchWithDefault(remote)
} else {
conn.remote = conn.Router.Peers.FetchAndAddRef(remote.Name)
if conn.remote == nil {
return fmt.Errorf("Found unknown remote name: %s at %s", remote.Name, conn.remoteTCPAddr)
}
}
if conn.remote == conn.local {
return ErrConnectToSelf
}
return nil
}
func (conn *LocalConnection) actorLoop(actionChan <-chan ConnectionAction) (err error) {
fwdErrorChan := conn.OverlayConn.ErrorChannel()
fwdEstablishedChan := conn.OverlayConn.EstablishedChannel()
for err == nil {
select {
case action := <-actionChan:
err = action()
case <-conn.heartbeatTCP.C:
err = conn.sendSimpleProtocolMsg(ProtocolHeartbeat)
case <-fwdEstablishedChan:
conn.established = true
fwdEstablishedChan = nil
conn.Router.Ourself.ConnectionEstablished(conn)
case err = <-fwdErrorChan:
}
}
return
}
func (conn *LocalConnection) shutdown(err error) {
if conn.remote == nil {
log.Errorf("->[%s] connection shutting down due to error during handshake: %v", conn.remoteTCPAddr, err)
} else {
conn.ErrorLog("connection shutting down due to error:", err)
}
if conn.TCPConn != nil {
checkWarn(conn.TCPConn.Close())
}
if conn.remote != nil {
conn.Router.Peers.Dereference(conn.remote)
conn.Router.Ourself.DeleteConnection(conn)
}
if conn.heartbeatTCP != nil {
conn.heartbeatTCP.Stop()
}
if conn.OverlayConn != nil {
conn.OverlayConn.Stop()
}
conn.Router.ConnectionMaker.ConnectionTerminated(conn, err)
}
func (conn *LocalConnection) sendOverlayControlMessage(tag byte, msg []byte) error {
return conn.sendProtocolMsg(ProtocolMsg{ProtocolTag(tag), msg})
}
// Helpers
func (conn *LocalConnection) sendSimpleProtocolMsg(tag ProtocolTag) error {
return conn.sendProtocolMsg(ProtocolMsg{tag: tag})
}
func (conn *LocalConnection) sendProtocolMsg(m ProtocolMsg) error {
return conn.tcpSender.Send(append([]byte{byte(m.tag)}, m.msg...))
}
func (conn *LocalConnection) receiveTCP(receiver TCPReceiver) {
var err error
for {
conn.extendReadDeadline()
var msg []byte
if msg, err = receiver.Receive(); err != nil {
break
}
if len(msg) < 1 {
conn.Log("ignoring blank msg")
continue
}
if err = conn.handleProtocolMsg(ProtocolTag(msg[0]), msg[1:]); err != nil {
break
}
}
conn.Shutdown(err)
}
func (conn *LocalConnection) handleProtocolMsg(tag ProtocolTag, payload []byte) error {
switch tag {
case ProtocolHeartbeat:
case ProtocolReserved1, ProtocolReserved2, ProtocolReserved3, ProtocolOverlayControlMsg:
conn.OverlayConn.ControlMessage(byte(tag), payload)
case ProtocolGossipUnicast, ProtocolGossipBroadcast, ProtocolGossip:
return conn.Router.handleGossip(tag, payload)
default:
conn.Log("ignoring unknown protocol tag:", tag)
}
return nil
}
func (conn *LocalConnection) extendReadDeadline() {
conn.TCPConn.SetReadDeadline(time.Now().Add(TCPHeartbeat * 2))
}
@@ -1,9 +1,7 @@
package router
package mesh
import (
"bytes"
"fmt"
"log"
"math/rand"
"net"
"time"
@@ -12,6 +10,7 @@ import (
const (
InitialInterval = 2 * time.Second
MaxInterval = 6 * time.Minute
ResetAfter = 1 * time.Minute
)
type peerAddrs map[string]*net.TCPAddr
@@ -22,13 +21,22 @@ type ConnectionMaker struct {
port int
discovery bool
targets map[string]*Target
connections map[Connection]struct{}
directPeers peerAddrs
actionChan chan<- ConnectionMakerAction
}
type TargetState int
const (
TargetWaiting TargetState = iota // we are waiting to connect there
TargetAttempting // we are attempting to connect there
TargetConnected // we are connected to there
)
// Information about an address where we may find a peer
type Target struct {
attempting bool // are we currently attempting to connect there?
state TargetState
lastError error // reason for disconnection last time
tryAfter time.Time // next time to try this address
tryInterval time.Duration // retry delay on next failure
@@ -37,19 +45,18 @@ type Target struct {
type ConnectionMakerAction func() bool
func NewConnectionMaker(ourself *LocalPeer, peers *Peers, port int, discovery bool) *ConnectionMaker {
return &ConnectionMaker{
actionChan := make(chan ConnectionMakerAction, ChannelSize)
cm := &ConnectionMaker{
ourself: ourself,
peers: peers,
port: port,
discovery: discovery,
directPeers: peerAddrs{},
targets: make(map[string]*Target)}
}
func (cm *ConnectionMaker) Start() {
actionChan := make(chan ConnectionMakerAction, ChannelSize)
cm.actionChan = actionChan
targets: make(map[string]*Target),
connections: make(map[Connection]struct{}),
actionChan: actionChan}
go cm.queryLoop(actionChan)
return cm
}
func (cm *ConnectionMaker) InitiateConnections(peers []string, replace bool) []error {
@@ -75,7 +82,7 @@ func (cm *ConnectionMaker) InitiateConnections(peers []string, replace bool) []e
cm.directPeers[peer] = addr
// curtail any existing reconnect interval
if target, found := cm.targets[addr.String()]; found {
target.tryNow()
target.nextTryNow()
}
}
return true
@@ -92,12 +99,42 @@ func (cm *ConnectionMaker) ForgetConnections(peers []string) {
}
}
func (cm *ConnectionMaker) ConnectionTerminated(address string, err error) {
func (cm *ConnectionMaker) ConnectionAborted(address string, err error) {
cm.actionChan <- func() bool {
if target, found := cm.targets[address]; found {
target.attempting = false
target := cm.targets[address]
target.state = TargetWaiting
target.lastError = err
target.nextTryLater()
return true
}
}
func (cm *ConnectionMaker) ConnectionCreated(conn Connection) {
cm.actionChan <- func() bool {
cm.connections[conn] = void
if conn.Outbound() {
target := cm.targets[conn.RemoteTCPAddr()]
target.state = TargetConnected
}
return false
}
}
func (cm *ConnectionMaker) ConnectionTerminated(conn Connection, err error) {
cm.actionChan <- func() bool {
delete(cm.connections, conn)
if conn.Outbound() {
target := cm.targets[conn.RemoteTCPAddr()]
target.state = TargetWaiting
target.lastError = err
target.retry()
switch {
case err == ErrConnectToSelf:
target.nextTryNever()
case time.Now().After(target.tryAfter.Add(ResetAfter)):
target.nextTryNow()
default:
target.nextTryLater()
}
}
return true
}
@@ -107,58 +144,6 @@ func (cm *ConnectionMaker) Refresh() {
cm.actionChan <- func() bool { return true }
}
func (cm *ConnectionMaker) Status() ConnectionMakerStatus {
// We need to Refresh first in order to clear out any 'attempting'
// connections from cm.targets that have been established since
// the last run of cm.checkStateAndAttemptConnections. These
// entries are harmless but do represent stale state that we do
// not want to report.
cm.Refresh()
resultChan := make(chan ConnectionMakerStatus, 0)
cm.actionChan <- func() bool {
status := ConnectionMakerStatus{
DirectPeers: []string{},
Reconnects: make(map[string]Target),
}
for peer := range cm.directPeers {
status.DirectPeers = append(status.DirectPeers, peer)
}
for address, target := range cm.targets {
status.Reconnects[address] = *target
}
resultChan <- status
return false
}
return <-resultChan
}
type ConnectionMakerStatus struct {
DirectPeers []string
Reconnects map[string]Target
}
func (status ConnectionMakerStatus) String() string {
var buf bytes.Buffer
fmt.Fprint(&buf, "Direct Peers:")
for _, peer := range status.DirectPeers {
fmt.Fprintf(&buf, " %s", peer)
}
fmt.Fprintln(&buf, "\nReconnects:")
for address, target := range status.Reconnects {
fmt.Fprintf(&buf, "->[%s]", address)
if target.lastError != nil {
fmt.Fprintf(&buf, " (%s)", target.lastError)
}
if target.attempting {
fmt.Fprintf(&buf, " trying since %v\n", target.tryAfter)
} else {
fmt.Fprintf(&buf, " next try at %v\n", target.tryAfter)
}
}
return buf.String()
}
func (cm *ConnectionMaker) queryLoop(actionChan <-chan ConnectionMakerAction) {
timer := time.NewTimer(MaxDuration)
run := func() { timer.Reset(cm.checkStateAndAttemptConnections()) }
@@ -179,9 +164,6 @@ func (cm *ConnectionMaker) checkStateAndAttemptConnections() time.Duration {
validTarget = make(map[string]struct{})
directTarget = make(map[string]struct{})
)
// Copy the set of things we are connected to, so we can access
// them without locking. Also clear out any entries in cm.targets
// for existing connections.
ourConnectedPeers, ourConnectedTargets, ourInboundIPs := cm.ourConnections()
addTarget := func(address string) {
@@ -192,8 +174,8 @@ func (cm *ConnectionMaker) checkStateAndAttemptConnections() time.Duration {
if _, found := cm.targets[address]; found {
return
}
target := &Target{}
target.tryNow()
target := &Target{state: TargetWaiting}
target.nextTryNow()
cm.targets[address] = target
}
@@ -232,9 +214,8 @@ func (cm *ConnectionMaker) ourConnections() (PeerNameSet, map[string]struct{}, m
ourConnectedTargets = make(map[string]struct{})
ourInboundIPs = make(map[string]struct{})
)
for conn := range cm.ourself.Connections() {
for conn := range cm.connections {
address := conn.RemoteTCPAddr()
delete(cm.targets, address)
ourConnectedPeers[conn.Remote().Name] = void
ourConnectedTargets[address] = void
if conn.Outbound() {
@@ -268,7 +249,7 @@ func (cm *ConnectionMaker) addPeerTargets(ourConnectedPeers PeerNameSet, addTarg
} else if ip, _, err := net.SplitHostPort(address); err == nil {
// There is no point connecting to the (likely
// ephemeral) remote port of an inbound connection
// that some peer has. Let's try to connect to on the
// that some peer has. Let's try to connect on the
// weave port instead.
addTarget(fmt.Sprintf("%s:%d", ip, cm.port))
}
@@ -280,16 +261,19 @@ func (cm *ConnectionMaker) connectToTargets(validTarget map[string]struct{}, dir
now := time.Now() // make sure we catch items just added
after := MaxDuration
for address, target := range cm.targets {
if target.attempting {
if target.state != TargetWaiting {
continue
}
if _, valid := validTarget[address]; !valid {
delete(cm.targets, address)
continue
}
if target.tryAfter.IsZero() {
continue
}
switch duration := target.tryAfter.Sub(now); {
case duration <= 0:
target.attempting = true
target.state = TargetAttempting
_, isCmdLineTarget := directTarget[address]
go cm.attemptConnection(address, isCmdLineTarget)
case duration < after:
@@ -300,21 +284,26 @@ func (cm *ConnectionMaker) connectToTargets(validTarget map[string]struct{}, dir
}
func (cm *ConnectionMaker) attemptConnection(address string, acceptNewPeer bool) {
log.Printf("->[%s] attempting connection\n", address)
log.Printf("->[%s] attempting connection", address)
if err := cm.ourself.CreateConnection(address, acceptNewPeer); err != nil {
log.Printf("->[%s] error during connection attempt: %v\n", address, err)
cm.ConnectionTerminated(address, err)
log.Errorf("->[%s] error during connection attempt: %v", address, err)
cm.ConnectionAborted(address, err)
}
}
func (t *Target) tryNow() {
func (t *Target) nextTryNever() {
t.tryAfter = time.Time{}
t.tryInterval = MaxInterval
}
func (t *Target) nextTryNow() {
t.tryAfter = time.Now()
t.tryInterval = InitialInterval
}
// The delay at the nth retry is a random value in the range
// [i-i/2,i+i/2], where i = InitialInterval * 1.5^(n-1).
func (t *Target) retry() {
func (t *Target) nextTryLater() {
t.tryAfter = time.Now().Add(t.tryInterval/2 + time.Duration(rand.Int63n(int64(t.tryInterval))))
t.tryInterval = t.tryInterval * 3 / 2
if t.tryInterval > MaxInterval {
+10 -8
View File
@@ -1,4 +1,4 @@
package router
package mesh
import (
"fmt"
@@ -21,7 +21,7 @@ type Gossiper interface {
OnGossipUnicast(sender PeerName, msg []byte) error
// merge received data into state and return a representation of
// the received data, for further propagation
OnGossipBroadcast(update []byte) (GossipData, error)
OnGossipBroadcast(sender PeerName, update []byte) (GossipData, error)
// return state of everything we know; gets called periodically
Gossip() GossipData
// merge received data into state and return "everything new I've
@@ -40,13 +40,15 @@ type GossipSender struct {
}
func NewGossipSender(send func(GossipData)) *GossipSender {
return &GossipSender{send: send}
}
func (sender *GossipSender) Start() {
sender.cell = make(chan GossipData, 1)
sender.flushch = make(chan chan bool)
cell := make(chan GossipData, 1)
flushch := make(chan chan bool)
sender := &GossipSender{
send: send,
cell: cell,
flushch: flushch,
}
go sender.run()
return sender
}
func (sender *GossipSender) run() {
@@ -1,10 +1,9 @@
package router
package mesh
import (
"bytes"
"encoding/gob"
"fmt"
"log"
"sync"
)
@@ -78,7 +77,7 @@ func (c *GossipChannel) deliverBroadcast(srcName PeerName, _ []byte, dec *gob.De
if err := dec.Decode(&payload); err != nil {
return err
}
data, err := c.gossiper.OnGossipBroadcast(payload)
data, err := c.gossiper.OnGossipBroadcast(srcName, payload)
if err != nil || data == nil {
return err
}
@@ -113,17 +112,32 @@ func (c *GossipChannel) Send(srcName PeerName, data GossipData) {
connections := c.ourself.Connections()
c.Lock()
defer c.Unlock()
// GC - randomly (courtesy of go's map iterator) pick some
// existing entries and stop&remove them if the associated
// connection is no longer active. We stop as soon as we
// encounter a valid entry; the idea being that when there is
// little or no garbage then this executes close to O(1)[1],
// whereas when there is lots of garbage we remove it quickly.
//
// [1] TODO Unfortunately, due to the desire to avoid nested
// locks, instead of simply invoking Peer.ConnectionTo(name)
// below, we have that Peer.Connections() invocation above. That
// is O(n_our_connections) at best.
c.gcSenders(connections)
for conn := range selectedConnections {
c.sendDown(conn, data)
}
}
func (c *GossipChannel) SendDown(conn Connection, data GossipData) {
connections := c.ourself.Connections()
c.Lock()
defer c.Unlock()
c.gcSenders(connections)
c.sendDown(conn, data)
}
// GC - randomly (courtesy of go's map iterator) pick some existing
// senders and stop&remove them if the associated connection is no
// longer active. We stop as soon as we encounter a valid entry; the
// idea being that when there is little or no garbage then this
// executes close to O(1)[1], whereas when there is lots of garbage we
// remove it quickly.
//
// [1] TODO Unfortunately, due to the desire to avoid nested locks,
// instead of simply invoking LocalPeer.ConnectionTo(name), we pass in
// the result of LocalPeer.Connections(). That is O(n_our_connections)
// at best.
func (c *GossipChannel) gcSenders(connections ConnectionSet) {
for conn, sender := range c.senders {
if _, found := connections[conn]; !found {
delete(c.senders, conn)
@@ -132,28 +146,25 @@ func (c *GossipChannel) Send(srcName PeerName, data GossipData) {
break
}
}
for conn := range selectedConnections {
c.sendDown(conn, data)
}
}
func (c *GossipChannel) SendDown(conn Connection, data GossipData) {
c.Lock()
c.sendDown(conn, data)
c.Unlock()
}
// We have seen a couple of failures which suggest a >128GB slice was encountered.
// 100MB should be enough for anyone.
const maxFeasibleMessageLen = 100 * 1024 * 1024
func (c *GossipChannel) sendDown(conn Connection, data GossipData) {
sender, found := c.senders[conn]
if !found {
sender = NewGossipSender(func(pending GossipData) {
for _, msg := range pending.Encode() {
if len(msg) > maxFeasibleMessageLen {
panic(fmt.Sprintf("Gossip message too large: len=%d bytes; on channel '%s' from %+v", len(msg), c.name, pending))
}
protocolMsg := ProtocolMsg{ProtocolGossip, GobEncode(c.name, c.ourself.Name, msg)}
conn.(ProtocolSender).SendProtocolMsg(protocolMsg)
}
})
c.senders[conn] = sender
sender.Start()
}
sender.Send(data)
}
@@ -204,7 +215,6 @@ func (c *GossipChannel) relayBroadcast(srcName PeerName, update GossipData) erro
if !found {
broadcaster = NewGossipSender(func(pending GossipData) { c.sendBroadcast(srcName, pending) })
c.broadcasters[srcName] = broadcaster
broadcaster.Start()
}
broadcaster.Send(update)
return nil
@@ -229,3 +239,12 @@ func (c *GossipChannel) sendBroadcast(srcName PeerName, update GossipData) {
func (c *GossipChannel) log(args ...interface{}) {
log.Println(append(append([]interface{}{}, "[gossip "+c.name+"]:"), args...)...)
}
func GobEncode(items ...interface{}) []byte {
buf := new(bytes.Buffer)
enc := gob.NewEncoder(buf)
for _, i := range items {
checkFatal(enc.Encode(i))
}
return buf.Bytes()
}
+14 -20
View File
@@ -1,10 +1,12 @@
package router
package mesh
import (
"fmt"
"sync"
"testing"
"time"
"github.com/stretchr/testify/require"
wt "github.com/weaveworks/weave/testing"
)
@@ -16,15 +18,10 @@ type mockChannelConnection struct {
dest *Router
}
// Construct a "passive" Router, i.e. without any goroutines, except
// for Routes and GossipSenders.
func NewTestRouter(name string) *Router {
peerName, _ := PeerNameFromString(name)
router := NewRouter(Config{}, peerName, "")
// need to create a dummy channel otherwise tests hang on nil
// channels when the Router invoked ConnectionMaker.Refresh
router.ConnectionMaker.actionChan = make(chan ConnectionMakerAction, ChannelSize)
router.Routes.Start()
router := NewRouter(Config{}, peerName, "nick", nil)
router.Start()
return router
}
@@ -45,11 +42,8 @@ func sendPendingGossip(routers ...*Router) {
}
func (router *Router) AddTestChannelConnection(r *Router) {
fromName := router.Ourself.Peer.Name
toName := r.Ourself.Peer.Name
fromPeer := NewPeer(fromName, "", router.Ourself.Peer.UID, 0)
toPeer := NewPeer(toName, "", r.Ourself.Peer.UID, 0)
fromPeer := NewPeerFrom(router.Ourself.Peer)
toPeer := NewPeerFrom(r.Ourself.Peer)
r.Peers.FetchWithDefault(fromPeer) // Has side-effect of incrementing refcount
router.Peers.FetchWithDefault(toPeer) //
@@ -63,8 +57,8 @@ func (router *Router) DeleteTestChannelConnection(r *Router) {
fromName := router.Ourself.Peer.Name
toName := r.Ourself.Peer.Name
fromPeer, _ := r.Peers.Fetch(fromName)
toPeer, _ := router.Peers.Fetch(toName)
fromPeer := r.Peers.Fetch(fromName)
toPeer := router.Peers.Fetch(toName)
r.Peers.Dereference(fromPeer)
router.Peers.Dereference(toPeer)
@@ -83,13 +77,13 @@ func TestGossipTopology(t *testing.T) {
// the routers supplied as arguments, carrying across all UID and
// version information.
func (router *Router) tp(routers ...*Router) *Peer {
peer := NewPeer(router.Ourself.Peer.Name, "", router.Ourself.Peer.UID, 0)
peer := NewPeerFrom(router.Ourself.Peer)
connections := make(map[PeerName]Connection)
for _, r := range routers {
p := NewPeer(r.Ourself.Peer.Name, "", r.Ourself.Peer.UID, r.Ourself.Peer.version)
p := NewPeerFrom(r.Ourself.Peer)
connections[r.Ourself.Peer.Name] = newMockConnection(peer, p)
}
peer.version = router.Ourself.Peer.version
peer.Version = router.Ourself.Peer.Version
peer.connections = connections
return peer
}
@@ -215,7 +209,7 @@ func (g *testGossiper) OnGossipUnicast(sender PeerName, msg []byte) error {
return nil
}
func (g *testGossiper) OnGossipBroadcast(update []byte) (GossipData, error) {
func (g *testGossiper) OnGossipBroadcast(_ PeerName, update []byte) (GossipData, error) {
g.Lock()
defer g.Unlock()
for _, v := range update {
@@ -255,7 +249,7 @@ func (g *testGossiper) checkHas(t *testing.T, vs ...byte) {
defer g.RUnlock()
for _, v := range vs {
if _, found := g.state[v]; !found {
wt.Fatalf(t, "%d is missing", v)
require.FailNow(t, fmt.Sprintf("%d is missing", v))
}
}
}
+44 -78
View File
@@ -1,8 +1,7 @@
package router
package mesh
import (
"fmt"
"log"
"net"
"sync"
"time"
@@ -18,69 +17,14 @@ type LocalPeer struct {
type LocalPeerAction func()
func NewLocalPeer(name PeerName, nickName string, router *Router) *LocalPeer {
return &LocalPeer{Peer: NewPeer(name, nickName, 0, 0), router: router}
}
func (peer *LocalPeer) Start() {
actionChan := make(chan LocalPeerAction, ChannelSize)
peer.actionChan = actionChan
peer := &LocalPeer{
Peer: NewPeer(name, nickName, randomPeerUID(), 0, randomPeerShortID()),
router: router,
actionChan: actionChan,
}
go peer.actorLoop(actionChan)
}
func (peer *LocalPeer) Forward(dstPeer *Peer, df bool, frame []byte, dec *EthernetDecoder) error {
return peer.Relay(peer.Peer, dstPeer, df, frame, dec)
}
func (peer *LocalPeer) Broadcast(df bool, frame []byte, dec *EthernetDecoder) {
peer.RelayBroadcast(peer.Peer, df, frame, dec)
}
func (peer *LocalPeer) Relay(srcPeer, dstPeer *Peer, df bool, frame []byte, dec *EthernetDecoder) error {
relayPeerName, found := peer.router.Routes.Unicast(dstPeer.Name)
if !found {
// Not necessarily an error as there could be a race with the
// dst disappearing whilst the frame is in flight
log.Println("Received packet for unknown destination:", dstPeer)
return nil
}
conn, found := peer.ConnectionTo(relayPeerName)
if !found {
// Again, could just be a race, not necessarily an error
log.Println("Unable to find connection to relay peer", relayPeerName)
return nil
}
return conn.(*LocalConnection).Forward(df, &ForwardedFrame{
srcPeer: srcPeer,
dstPeer: dstPeer,
frame: frame},
dec)
}
func (peer *LocalPeer) RelayBroadcast(srcPeer *Peer, df bool, frame []byte, dec *EthernetDecoder) {
nextHops := peer.router.Routes.Broadcast(srcPeer.Name)
if len(nextHops) == 0 {
return
}
for _, conn := range peer.ConnectionsTo(nextHops) {
err := conn.(*LocalConnection).Forward(df, &ForwardedFrame{
srcPeer: srcPeer,
dstPeer: conn.Remote(),
frame: frame},
dec)
if err != nil {
if ftbe, ok := err.(FrameTooBigError); ok {
log.Printf("dropping too big DF broadcast frame (%v -> %v): PMTU= %v\n", dec.ip.DstIP, dec.ip.SrcIP, ftbe.EPMTU)
} else {
log.Println(err)
}
}
}
}
func (peer *LocalPeer) Info() string {
peer.RLock()
defer peer.RUnlock()
return peer.Peer.Info()
return peer
}
func (peer *LocalPeer) Connections() ConnectionSet {
@@ -118,22 +62,16 @@ func (peer *LocalPeer) CreateConnection(peerAddr string, acceptNewPeer bool) err
if err := peer.checkConnectionLimit(); err != nil {
return err
}
tcpAddr, tcpErr := net.ResolveTCPAddr("tcp4", peerAddr)
udpAddr, udpErr := net.ResolveUDPAddr("udp4", peerAddr)
if tcpErr != nil || udpErr != nil {
// they really should have the same value, but just in case...
if tcpErr == nil {
return udpErr
}
return tcpErr
tcpAddr, err := net.ResolveTCPAddr("tcp4", peerAddr)
if err != nil {
return err
}
tcpConn, err := net.DialTCP("tcp4", nil, tcpAddr)
if err != nil {
return err
}
connRemote := NewRemoteConnection(peer.Peer, nil, tcpConn.RemoteAddr().String(), true, false)
connLocal := NewLocalConnection(connRemote, tcpConn, udpAddr, peer.router)
connLocal.Start(acceptNewPeer)
StartLocalConnection(connRemote, tcpConn, peer.router, acceptNewPeer)
return nil
}
@@ -217,7 +155,10 @@ func (peer *LocalPeer) handleAddConnection(conn Connection) error {
conn.Log("connection added (new peer)")
peer.router.SendAllGossipDown(conn)
}
peer.router.Routes.Recalculate()
peer.broadcastPeerUpdate(conn.Remote())
return nil
}
@@ -231,6 +172,8 @@ func (peer *LocalPeer) handleConnectionEstablished(conn Connection) {
}
peer.connectionEstablished(conn)
conn.Log("connection fully established")
peer.router.Routes.Recalculate()
peer.broadcastPeerUpdate()
}
@@ -250,14 +193,20 @@ func (peer *LocalPeer) handleDeleteConnection(conn Connection) {
// Must do garbage collection first to ensure we don't send out an
// update with unreachable peers (can cause looping)
peer.router.Peers.GarbageCollect()
peer.router.Routes.Recalculate()
peer.broadcastPeerUpdate()
}
// helpers
func (peer *LocalPeer) broadcastPeerUpdate(peers ...*Peer) {
peer.router.Routes.Recalculate()
peer.router.TopologyGossip.GossipBroadcast(NewTopologyGossipData(peer.router.Peers, append(peers, peer.Peer)...))
// Some tests run without a router. This should be fixed so
// that the relevant part of Router can be easily run in the
// context of a test, but that will involve significant
// reworking of tests.
if peer.router != nil {
peer.router.BroadcastTopologyUpdate(append(peers, peer.Peer))
}
}
func (peer *LocalPeer) checkConnectionLimit() error {
@@ -272,20 +221,20 @@ func (peer *LocalPeer) addConnection(conn Connection) {
peer.Lock()
defer peer.Unlock()
peer.connections[conn.Remote().Name] = conn
peer.version++
peer.Version++
}
func (peer *LocalPeer) deleteConnection(conn Connection) {
peer.Lock()
defer peer.Unlock()
delete(peer.connections, conn.Remote().Name)
peer.version++
peer.Version++
}
func (peer *LocalPeer) connectionEstablished(conn Connection) {
peer.Lock()
defer peer.Unlock()
peer.version++
peer.Version++
}
func (peer *LocalPeer) connectionCount() int {
@@ -293,3 +242,20 @@ func (peer *LocalPeer) connectionCount() int {
defer peer.RUnlock()
return len(peer.connections)
}
func (peer *LocalPeer) setShortID(shortID PeerShortID) {
peer.Lock()
defer peer.Unlock()
peer.ShortID = shortID
peer.Version++
}
func (peer *LocalPeer) setVersionBeyond(version uint64) bool {
peer.Lock()
defer peer.Unlock()
if version >= peer.Version {
peer.Version = version + 1
return true
}
return false
}
+14 -13
View File
@@ -3,17 +3,20 @@
// It supplies some mock implementations to other unit tests, and is
// named "...test.go" so it is only compiled under `go test`.
package router
package mesh
import (
"fmt"
"testing"
wt "github.com/weaveworks/weave/testing"
"github.com/stretchr/testify/require"
)
// Add to peers a connection from peers.ourself to p
func (peers *Peers) AddTestConnection(p *Peer) {
toPeer := NewPeer(p.Name, "", p.UID, 0)
summary := p.PeerSummary
summary.Version = 0
toPeer := NewPeerFromSummary(summary)
peers.FetchWithDefault(toPeer) // Has side-effect of incrementing refcount
conn := newMockConnection(peers.ourself.Peer, toPeer)
peers.ourself.addConnection(conn)
@@ -22,18 +25,16 @@ func (peers *Peers) AddTestConnection(p *Peer) {
// Add to peers a connection from p1 to p2
func (peers *Peers) AddTestRemoteConnection(p1, p2 *Peer) {
fromName := p1.Name
fromPeer := NewPeer(fromName, "", p1.UID, 0)
fromPeer := NewPeerFrom(p1)
fromPeer = peers.FetchWithDefault(fromPeer)
toName := p2.Name
toPeer := NewPeer(toName, "", p2.UID, 0)
toPeer := NewPeerFrom(p2)
toPeer = peers.FetchWithDefault(toPeer)
peers.ourself.addConnection(&RemoteConnection{fromPeer, toPeer, "", false, false})
}
func (peers *Peers) DeleteTestConnection(p *Peer) {
toName := p.Name
toPeer, _ := peers.Fetch(toName)
toPeer := peers.Fetch(toName)
peers.Dereference(toPeer)
conn, _ := peers.ourself.ConnectionTo(toName)
peers.ourself.deleteConnection(conn)
@@ -58,18 +59,18 @@ func checkEqualConns(t *testing.T, ourName PeerName, got, wanted map[PeerName]Co
if _, found := checkConns[remoteName]; found {
delete(checkConns, remoteName)
} else {
wt.Fatalf(t, "Unexpected connection from %s to %s", ourName, remoteName)
require.FailNow(t, fmt.Sprintf("Unexpected connection from %s to %s", ourName, remoteName))
}
}
if len(checkConns) > 0 {
wt.Fatalf(t, "Expected connections not found: from %s to %v", ourName, checkConns)
require.FailNow(t, fmt.Sprintf("Expected connections not found: from %s to %v", ourName, checkConns))
}
}
// Get all the peers from a Peers in a slice
func (peers *Peers) allPeers() []*Peer {
var res []*Peer
for _, peer := range peers.table {
for _, peer := range peers.byName {
res = append(res, peer)
}
return res
@@ -105,10 +106,10 @@ func checkTopologyPeers(t *testing.T, checkConns bool, peers []*Peer, wantedPeer
}
delete(check, name)
} else {
wt.Fatalf(t, "Unexpected peer: %s", name)
require.FailNow(t, fmt.Sprintf("Unexpected peer: %s", name))
}
}
if len(check) > 0 {
wt.Fatalf(t, "Expected peers not found: %v", check)
require.FailNow(t, fmt.Sprintf("Expected peers not found: %v", check))
}
}
+116
View File
@@ -0,0 +1,116 @@
package mesh
import (
"net"
)
type Overlay interface {
// Enhance a features map with overlay-related features
AddFeaturesTo(map[string]string)
// Prepare on overlay connection. The connection should remain
// passive until it has been Confirm()ed.
PrepareConnection(OverlayConnectionParams) (OverlayConnection, error)
// Obtain diagnostic information specific to the overlay
Diagnostics() interface{}
}
type OverlayConnectionParams struct {
RemotePeer *Peer
// The local address of the corresponding TCP connection. Used to
// derive the local IP address for sending. May differ for
// different overlay connections.
LocalAddr *net.TCPAddr
// The remote address of the corresponding TCP connection. Used to
// determine the address to send to, but only if the TCP
// connection is outbound. Otherwise the Overlay needs to discover
// it (e.g. from incoming datagrams).
RemoteAddr *net.TCPAddr
// Is the corresponding TCP connection outbound?
Outbound bool
// Unique identifier for this connection
ConnUID uint64
// Session key, if connection is encrypted; nil otherwise.
//
// NB: overlay connections must take care not to use nonces which
// may collide with those of the main connection. These nonces are
// 192 bits, with the top most bit unspecified, the next bit set
// to 1, followed by 126 zero bits, and a message sequence number
// in the lowest 64 bits.
SessionKey *[32]byte
// Function to send a control message to the counterpart
// overlay connection.
SendControlMessage func(tag byte, msg []byte) error
// Features passed at connection initiation
Features map[string]string
}
// All of the machinery to manage overlay connectivity to a particular
// peer
type OverlayConnection interface {
// Confirm that the connection is really wanted, and so the
// Overlay should begin heartbeats etc. to verify the operation of
// the overlay connection.
Confirm()
// A channel indicating that the overlay connection is
// established, i.e. its operation has been confirmed.
EstablishedChannel() <-chan struct{}
// A channel indicating an error from the overlay connection. The
// overlay connection is not expected to be operational after the
// first error, so the channel only needs to buffer a single
// error.
ErrorChannel() <-chan error
Stop()
// Handle a message from the peer. 'tag' exists for
// compatibility, and should always be
// ProtocolOverlayControlMessage for non-sleeve overlays.
ControlMessage(tag byte, msg []byte)
// User facing overlay name
DisplayName() string
}
type NullOverlay struct{}
func (NullOverlay) AddFeaturesTo(map[string]string) {
}
func (NullOverlay) PrepareConnection(OverlayConnectionParams) (OverlayConnection, error) {
return NullOverlay{}, nil
}
func (NullOverlay) Diagnostics() interface{} {
return nil
}
func (NullOverlay) Confirm() {
}
func (NullOverlay) EstablishedChannel() <-chan struct{} {
return nil
}
func (NullOverlay) ErrorChannel() <-chan error {
return nil
}
func (NullOverlay) Stop() {
}
func (NullOverlay) ControlMessage(byte, []byte) {
}
func (NullOverlay) DisplayName() string {
return "null"
}
+78 -21
View File
@@ -1,51 +1,108 @@
package router
package mesh
import (
"crypto/rand"
"encoding/binary"
"fmt"
"sort"
"strconv"
)
func randBytes(n int) []byte {
buf := make([]byte, n)
_, err := rand.Read(buf)
checkFatal(err)
return buf
}
func randUint64() (r uint64) {
return binary.LittleEndian.Uint64(randBytes(8))
}
func randUint16() (r uint16) {
return binary.LittleEndian.Uint16(randBytes(2))
}
type PeerUID uint64
func randomPeerUID() PeerUID {
return PeerUID(randUint64())
}
func ParsePeerUID(s string) (PeerUID, error) {
uid, err := strconv.ParseUint(s, 10, 64)
return PeerUID(uid), err
}
// Short IDs exist for the sake of fast datapath. They are 12 bits,
// randomly assigned, but we detect and recover from collisions. This
// does limit us to 4096 peers, but that should be sufficient for a
// while.
type PeerShortID uint16
const PeerShortIDBits = 12
func randomPeerShortID() PeerShortID {
return PeerShortID(randUint16() & (1<<PeerShortIDBits - 1))
}
type PeerSummary struct {
NameByte []byte
NickName string
UID PeerUID
Version uint64
ShortID PeerShortID
HasShortID bool
}
type Peer struct {
Name PeerName
NameByte []byte
NickName string
UID PeerUID
version uint64
Name PeerName
PeerSummary
localRefCount uint64 // maintained by Peers
connections map[PeerName]Connection
}
type ListOfPeers []*Peer
func (lop ListOfPeers) Len() int {
return len(lop)
}
func (lop ListOfPeers) Swap(i, j int) {
lop[i], lop[j] = lop[j], lop[i]
}
func (lop ListOfPeers) Less(i, j int) bool {
return lop[i].Name < lop[j].Name
}
type ConnectionSet map[Connection]struct{}
func NewPeer(name PeerName, nickName string, uid PeerUID, version uint64) *Peer {
if uid == 0 {
uid = PeerUID(randUint64())
}
func NewPeerFromSummary(summary PeerSummary) *Peer {
return &Peer{
Name: name,
NameByte: name.Bin(),
NickName: nickName,
UID: uid,
version: version,
connections: make(map[PeerName]Connection)}
Name: PeerNameFromBin(summary.NameByte),
PeerSummary: summary,
connections: make(map[PeerName]Connection),
}
}
func NewPeer(name PeerName, nickName string, uid PeerUID, version uint64, shortID PeerShortID) *Peer {
return NewPeerFromSummary(PeerSummary{
NameByte: name.Bin(),
NickName: nickName,
UID: uid,
Version: version,
ShortID: shortID,
HasShortID: true,
})
}
func NewPeerFrom(peer *Peer) *Peer {
return NewPeerFromSummary(peer.PeerSummary)
}
func (peer *Peer) String() string {
return fmt.Sprint(peer.Name, "(", peer.NickName, ")")
}
func (peer *Peer) Info() string {
return fmt.Sprint(peer.String(), " (v", peer.version, ") (UID ", peer.UID, ")")
}
// Calculate the routing table from this peer to all peers reachable
// from it, returning a "next hop" map of PeerNameX -> PeerNameY,
// which says "in order to send a message to X, the peer should send
@@ -70,7 +127,7 @@ func (peer *Peer) Info() string {
// NB: This function should generally be invoked while holding a read
// lock on Peers and LocalPeer.
func (peer *Peer) Routes(stopAt *Peer, establishedAndSymmetric bool) (bool, map[PeerName]PeerName) {
routes := make(map[PeerName]PeerName)
routes := make(unicastRoutes)
routes[peer.Name] = UnknownPeerName
nextWorklist := []*Peer{peer}
for len(nextWorklist) > 0 {
@@ -3,7 +3,7 @@
// Let peer names be sha256 hashes...of anything (as long as it's
// unique).
package router
package mesh
import (
"crypto/sha256"
@@ -15,7 +15,7 @@
// name. In particular it doesn't actually have to be the MAC of, say,
// the network interface the peer is sniffing on.
package router
package mesh
import (
"net"
@@ -56,3 +56,20 @@ func (name PeerName) Bin() []byte {
func (name PeerName) String() string {
return intmac(uint64(name)).String()
}
func macint(mac net.HardwareAddr) (r uint64) {
for _, b := range mac {
r <<= 8
r |= uint64(b)
}
return
}
func intmac(key uint64) (r net.HardwareAddr) {
r = make([]byte, 6)
for i := 5; i >= 0; i-- {
r[i] = byte(key)
key >>= 8
}
return
}
+538
View File
@@ -0,0 +1,538 @@
package mesh
import (
"bytes"
"encoding/gob"
"fmt"
"io"
"math/rand"
"sync"
)
var void = struct{}{}
type Peers struct {
sync.RWMutex
ourself *LocalPeer
byName map[PeerName]*Peer
byShortID map[PeerShortID]ShortIDPeers
onGC []func(*Peer)
// Called when the mapping from short ids to peers changes
onInvalidateShortIDs []func()
}
type ShortIDPeers struct {
// If we know about a single peer with the short id, this is
// that peer. If there is a collision, this is the peer with
// the lowest Name.
peer *Peer
// In case of a collision, this holds the other peers.
others []*Peer
}
type UnknownPeerError struct {
Name PeerName
}
func (upe UnknownPeerError) Error() string {
return fmt.Sprint("Reference to unknown peer ", upe.Name)
}
type PeerNameSet map[PeerName]struct{}
type ConnectionSummary struct {
NameByte []byte
RemoteTCPAddr string
Outbound bool
Established bool
}
// Pending notifications due to changes to Peers that need to be sent
// out once the Peers is unlocked.
type PeersPendingNotifications struct {
// Peers that have been GCed
removed []*Peer
// The mapping from shorts ids to peers changed
invalidateShortIDs bool
// The local short ID needs reassigning due to a collision
reassignLocalShortID bool
// The local peer was modified
localPeerModified bool
}
func NewPeers(ourself *LocalPeer) *Peers {
peers := &Peers{
ourself: ourself,
byName: make(map[PeerName]*Peer),
byShortID: make(map[PeerShortID]ShortIDPeers),
}
peers.FetchWithDefault(ourself.Peer)
return peers
}
func (peers *Peers) OnGC(callback func(*Peer)) {
peers.Lock()
defer peers.Unlock()
// Although the array underlying peers.onGC might be accessed
// without holding the lock in unlockAndNotify, we don't
// support removing callbacks, so a simple append here is
// safe.
peers.onGC = append(peers.onGC, callback)
}
func (peers *Peers) OnInvalidateShortIDs(callback func()) {
peers.Lock()
defer peers.Unlock()
// Safe, as in OnGC
peers.onInvalidateShortIDs = append(peers.onInvalidateShortIDs, callback)
}
func (peers *Peers) unlockAndNotify(pending *PeersPendingNotifications) {
broadcastLocalPeer := (pending.reassignLocalShortID && peers.reassignLocalShortID(pending)) ||
pending.localPeerModified
onGC := peers.onGC
onInvalidateShortIDs := peers.onInvalidateShortIDs
peers.Unlock()
if pending.removed != nil {
for _, callback := range onGC {
for _, peer := range pending.removed {
callback(peer)
}
}
}
if pending.invalidateShortIDs {
for _, callback := range onInvalidateShortIDs {
callback()
}
}
if broadcastLocalPeer {
peers.ourself.broadcastPeerUpdate()
}
}
func (peers *Peers) addByShortID(peer *Peer, pending *PeersPendingNotifications) {
if !peer.HasShortID {
return
}
entry, ok := peers.byShortID[peer.ShortID]
if !ok {
entry = ShortIDPeers{peer: peer}
} else if entry.peer == nil {
// This short ID is free, but was used in the past.
// Because we are reusing it, it's an invalidation
// event.
entry.peer = peer
pending.invalidateShortIDs = true
} else if peer.Name < entry.peer.Name {
// Short ID collision, this peer becomes the principal
// peer for the short ID, bumping the previous one
// into others.
if entry.peer == peers.ourself.Peer {
// The bumped peer is peers.ourself, so we
// need to look for a new short id
pending.reassignLocalShortID = true
}
entry.others = append(entry.others, entry.peer)
entry.peer = peer
pending.invalidateShortIDs = true
} else {
// Short ID collision, this peer is secondary
entry.others = append(entry.others, peer)
}
peers.byShortID[peer.ShortID] = entry
}
func (peers *Peers) deleteByShortID(peer *Peer, pending *PeersPendingNotifications) {
if !peer.HasShortID {
return
}
entry := peers.byShortID[peer.ShortID]
var otherIndex int
if peer != entry.peer {
// peer is secondary, find its index in others
otherIndex = -1
for i, other := range entry.others {
if peer == other {
otherIndex = i
break
}
}
if otherIndex < 0 {
return
}
} else if len(entry.others) != 0 {
// need to find the peer with the lowest name to
// become the new principal one
otherIndex = 0
minName := entry.others[0].Name
for i := 1; i < len(entry.others); i++ {
otherName := entry.others[i].Name
if otherName < minName {
minName = otherName
otherIndex = i
}
}
entry.peer = entry.others[otherIndex]
pending.invalidateShortIDs = true
} else {
// This is the last peer with the short id. We clear
// the entry, don't delete it, in order to detect when
// it gets re-used.
peers.byShortID[peer.ShortID] = ShortIDPeers{}
return
}
entry.others[otherIndex] = entry.others[len(entry.others)-1]
entry.others = entry.others[:len(entry.others)-1]
peers.byShortID[peer.ShortID] = entry
}
func (peers *Peers) reassignLocalShortID(pending *PeersPendingNotifications) bool {
newShortID, ok := peers.chooseShortID()
if ok {
peers.setLocalShortID(newShortID, pending)
return true
}
// Otherwise we'll try again later on in garbageColleect
return false
}
func (peers *Peers) setLocalShortID(newShortID PeerShortID, pending *PeersPendingNotifications) {
peers.deleteByShortID(peers.ourself.Peer, pending)
peers.ourself.setShortID(newShortID)
peers.addByShortID(peers.ourself.Peer, pending)
}
// Choose an available short id at random
func (peers *Peers) chooseShortID() (PeerShortID, bool) {
rng := rand.New(rand.NewSource(int64(randUint64())))
// First, just try picking some short ids at random, and
// seeing if they are available:
for i := 0; i < 10; i++ {
shortID := PeerShortID(rng.Intn(1 << PeerShortIDBits))
if peers.byShortID[shortID].peer == nil {
return shortID, true
}
}
// Looks like most short ids are used. So count the number of
// unused ones, and pick one at random.
available := int(1 << PeerShortIDBits)
for _, entry := range peers.byShortID {
if entry.peer != nil {
available--
}
}
if available == 0 {
// All short ids are used.
return 0, false
}
n := rng.Intn(available)
var i PeerShortID
for {
if peers.byShortID[i].peer == nil {
if n == 0 {
return PeerShortID(i), true
}
n--
}
i++
}
}
func (peers *Peers) FetchWithDefault(peer *Peer) *Peer {
peers.Lock()
var pending PeersPendingNotifications
defer peers.unlockAndNotify(&pending)
if existingPeer, found := peers.byName[peer.Name]; found {
existingPeer.localRefCount++
return existingPeer
}
peers.byName[peer.Name] = peer
peers.addByShortID(peer, &pending)
peer.localRefCount++
return peer
}
func (peers *Peers) Fetch(name PeerName) *Peer {
peers.RLock()
defer peers.RUnlock()
return peers.byName[name]
}
func (peers *Peers) FetchAndAddRef(name PeerName) *Peer {
peers.Lock()
defer peers.Unlock()
peer := peers.byName[name]
if peer != nil {
peer.localRefCount++
}
return peer
}
func (peers *Peers) FetchByShortID(shortID PeerShortID) *Peer {
peers.RLock()
defer peers.RUnlock()
return peers.byShortID[shortID].peer
}
func (peers *Peers) Dereference(peer *Peer) {
peers.Lock()
defer peers.Unlock()
peer.localRefCount--
}
func (peers *Peers) ForEach(fun func(*Peer)) {
peers.RLock()
defer peers.RUnlock()
for _, peer := range peers.byName {
fun(peer)
}
}
// Merge an incoming update with our own topology.
//
// We add peers hitherto unknown to us, and update peers for which the
// update contains a more recent version than known to us. The return
// value is a) a representation of the received update, and b) an
// "improved" update containing just these new/updated elements.
func (peers *Peers) ApplyUpdate(update []byte) (PeerNameSet, PeerNameSet, error) {
peers.Lock()
var pending PeersPendingNotifications
defer peers.unlockAndNotify(&pending)
newPeers, decodedUpdate, decodedConns, err := peers.decodeUpdate(update)
if err != nil {
return nil, nil, err
}
// By this point, we know the update doesn't refer to any peers we
// have no knowledge of. We can now apply the update. Start by
// adding in any new peers.
for name, newPeer := range newPeers {
peers.byName[name] = newPeer
peers.addByShortID(newPeer, &pending)
}
// Now apply the updates
newUpdate := peers.applyUpdate(decodedUpdate, decodedConns, &pending)
peers.garbageCollect(&pending)
for _, peerRemoved := range pending.removed {
delete(newUpdate, peerRemoved.Name)
}
updateNames := make(PeerNameSet)
for _, peer := range decodedUpdate {
updateNames[peer.Name] = void
}
return updateNames, newUpdate, nil
}
func (peers *Peers) Names() PeerNameSet {
peers.RLock()
defer peers.RUnlock()
names := make(PeerNameSet)
for name := range peers.byName {
names[name] = void
}
return names
}
func (peers *Peers) EncodePeers(names PeerNameSet) []byte {
buf := new(bytes.Buffer)
enc := gob.NewEncoder(buf)
peers.RLock()
defer peers.RUnlock()
for name := range names {
if peer, found := peers.byName[name]; found {
if peer == peers.ourself.Peer {
peers.ourself.Encode(enc)
} else {
peer.Encode(enc)
}
}
}
return buf.Bytes()
}
func (peers *Peers) GarbageCollect() {
peers.Lock()
var pending PeersPendingNotifications
defer peers.unlockAndNotify(&pending)
peers.garbageCollect(&pending)
}
func (peers *Peers) garbageCollect(pending *PeersPendingNotifications) {
peers.ourself.RLock()
_, reached := peers.ourself.Routes(nil, false)
peers.ourself.RUnlock()
for name, peer := range peers.byName {
if _, found := reached[peer.Name]; !found && peer.localRefCount == 0 {
delete(peers.byName, name)
peers.deleteByShortID(peer, pending)
pending.removed = append(pending.removed, peer)
}
}
if peers.byShortID[peers.ourself.ShortID].peer != peers.ourself.Peer {
// The local peer doesn't own its short id. Garbage
// collection might have freed some up, so try to
// reassign.
pending.reassignLocalShortID = true
}
}
func (peers *Peers) decodeUpdate(update []byte) (newPeers map[PeerName]*Peer, decodedUpdate []*Peer, decodedConns [][]ConnectionSummary, err error) {
newPeers = make(map[PeerName]*Peer)
decodedUpdate = []*Peer{}
decodedConns = [][]ConnectionSummary{}
decoder := gob.NewDecoder(bytes.NewReader(update))
for {
peerSummary, connSummaries, decErr := decodePeer(decoder)
if decErr == io.EOF {
break
} else if decErr != nil {
err = decErr
return
}
newPeer := NewPeerFromSummary(peerSummary)
decodedUpdate = append(decodedUpdate, newPeer)
decodedConns = append(decodedConns, connSummaries)
if _, found := peers.byName[newPeer.Name]; !found {
newPeers[newPeer.Name] = newPeer
}
}
for _, connSummaries := range decodedConns {
for _, connSummary := range connSummaries {
remoteName := PeerNameFromBin(connSummary.NameByte)
if _, found := newPeers[remoteName]; found {
continue
}
if _, found := peers.byName[remoteName]; found {
continue
}
// Update refers to a peer which we have no knowledge
// of. Thus we can't apply the update. Abort.
err = UnknownPeerError{remoteName}
return
}
}
return
}
func (peers *Peers) applyUpdate(decodedUpdate []*Peer, decodedConns [][]ConnectionSummary, pending *PeersPendingNotifications) PeerNameSet {
newUpdate := make(PeerNameSet)
for idx, newPeer := range decodedUpdate {
connSummaries := decodedConns[idx]
name := newPeer.Name
// guaranteed to find peer in the peers.byName
switch peer := peers.byName[name]; peer {
case peers.ourself.Peer:
if newPeer.UID != peer.UID {
// The update contains information about an old
// incarnation of ourselves. We increase our version
// number beyond that which we received, so our
// information supersedes the old one when it is
// received by other peers.
pending.localPeerModified = peers.ourself.setVersionBeyond(newPeer.Version)
}
case newPeer:
peer.connections = makeConnsMap(peer, connSummaries, peers.byName)
newUpdate[name] = void
default: // existing peer
if newPeer.Version < peer.Version ||
(newPeer.Version == peer.Version && newPeer.UID <= peer.UID) {
continue
}
peer.Version = newPeer.Version
peer.UID = newPeer.UID
peer.connections = makeConnsMap(peer, connSummaries, peers.byName)
if newPeer.ShortID != peer.ShortID {
peers.deleteByShortID(peer, pending)
peer.ShortID = newPeer.ShortID
peers.addByShortID(peer, pending)
}
newUpdate[name] = void
}
}
return newUpdate
}
func (peer *Peer) Encode(enc *gob.Encoder) {
checkFatal(enc.Encode(peer.PeerSummary))
connSummaries := []ConnectionSummary{}
for _, conn := range peer.connections {
connSummaries = append(connSummaries, ConnectionSummary{
conn.Remote().NameByte,
conn.RemoteTCPAddr(),
conn.Outbound(),
conn.Established(),
})
}
checkFatal(enc.Encode(connSummaries))
}
func (peer *LocalPeer) Encode(enc *gob.Encoder) {
peer.RLock()
defer peer.RUnlock()
peer.Peer.Encode(enc)
}
func decodePeer(dec *gob.Decoder) (peerSummary PeerSummary, connSummaries []ConnectionSummary, err error) {
if err = dec.Decode(&peerSummary); err != nil {
return
}
if err = dec.Decode(&connSummaries); err != nil {
return
}
return
}
func makeConnsMap(peer *Peer, connSummaries []ConnectionSummary, byName map[PeerName]*Peer) map[PeerName]Connection {
conns := make(map[PeerName]Connection)
for _, connSummary := range connSummaries {
name := PeerNameFromBin(connSummary.NameByte)
remotePeer := byName[name]
conn := NewRemoteConnection(peer, remotePeer, connSummary.RemoteTCPAddr, connSummary.Outbound, connSummary.Established)
conns[name] = conn
}
return conns
}
+376
View File
@@ -0,0 +1,376 @@
package mesh
import (
"fmt"
"math/rand"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// TODO we should also test:
//
// - applying an incremental update, including the case where that
// leads to an UnknownPeerError
//
// - the "improved update" calculation
//
// - non-gc of peers that are only referenced locally
func newNode(name PeerName) (*Peer, *Peers) {
peer := NewLocalPeer(name, "", nil)
peers := NewPeers(peer)
return peer.Peer, peers
}
// Check that ApplyUpdate copies the whole topology from peers
func checkApplyUpdate(t *testing.T, peers *Peers) {
dummyName, _ := PeerNameFromString("99:00:00:01:00:00")
// We need a new node outside of the network, with a connection
// into it.
_, testBedPeers := newNode(dummyName)
testBedPeers.AddTestConnection(peers.ourself.Peer)
testBedPeers.ApplyUpdate(peers.EncodePeers(peers.Names()))
checkTopologyPeers(t, true, testBedPeers.allPeersExcept(dummyName), peers.allPeers()...)
}
func TestPeersEncoding(t *testing.T) {
const numNodes = 20
const numIters = 1000
var peer [numNodes]*Peer
var ps [numNodes]*Peers
for i := 0; i < numNodes; i++ {
name, _ := PeerNameFromString(fmt.Sprintf("%02d:00:00:01:00:00", i))
peer[i], ps[i] = newNode(name)
}
var conns []struct{ from, to int }
for i := 0; i < numIters; i++ {
oper := rand.Intn(2)
switch oper {
case 0:
from, to := rand.Intn(numNodes), rand.Intn(numNodes)
if from != to {
if _, found := peer[from].connections[peer[to].Name]; !found {
ps[from].AddTestConnection(peer[to])
conns = append(conns, struct{ from, to int }{from, to})
checkApplyUpdate(t, ps[from])
}
}
case 1:
if len(conns) > 0 {
n := rand.Intn(len(conns))
c := conns[n]
ps[c.from].DeleteTestConnection(peer[c.to])
ps[c.from].GarbageCollect()
checkApplyUpdate(t, ps[c.from])
conns = append(conns[:n], conns[n+1:]...)
}
}
}
}
func garbageCollect(peers *Peers) []*Peer {
var removed []*Peer
peers.OnGC(func(peer *Peer) { removed = append(removed, peer) })
peers.GarbageCollect()
return removed
}
func TestPeersGarbageCollection(t *testing.T) {
const (
peer1NameString = "01:00:00:01:00:00"
peer2NameString = "02:00:00:02:00:00"
peer3NameString = "03:00:00:03:00:00"
)
var (
peer1Name, _ = PeerNameFromString(peer1NameString)
peer2Name, _ = PeerNameFromString(peer2NameString)
peer3Name, _ = PeerNameFromString(peer3NameString)
)
// Create some peers with some connections to each other
p1, ps1 := newNode(peer1Name)
p2, ps2 := newNode(peer2Name)
p3, ps3 := newNode(peer3Name)
ps1.AddTestConnection(p2)
ps2.AddTestRemoteConnection(p1, p2)
ps2.AddTestConnection(p1)
ps2.AddTestConnection(p3)
ps3.AddTestConnection(p1)
ps1.AddTestConnection(p3)
ps2.AddTestRemoteConnection(p1, p3)
ps2.AddTestRemoteConnection(p3, p1)
// Every peer is referenced, so nothing should be dropped
require.Empty(t, garbageCollect(ps1), "peers removed")
require.Empty(t, garbageCollect(ps2), "peers removed")
require.Empty(t, garbageCollect(ps3), "peers removed")
// Drop the connection from 2 to 3, and 3 isn't garbage-collected
// because 1 has a connection to 3
ps2.DeleteTestConnection(p3)
require.Empty(t, garbageCollect(ps2), "peers removed")
// Drop the connection from 1 to 3, and 3 will get removed by
// garbage-collection
ps1.DeleteTestConnection(p3)
checkPeerArray(t, garbageCollect(ps1), p3)
}
func TestShortIDCollisions(t *testing.T) {
rng := rand.New(rand.NewSource(time.Now().UnixNano()))
_, peers := newNode(PeerName(1 << PeerShortIDBits))
// Make enough peers that short id collisions are
// overwhelmingly likely
ps := make([]*Peer, 1<<PeerShortIDBits)
for i := 0; i < 1<<PeerShortIDBits; i++ {
ps[i] = NewPeer(PeerName(i), "", PeerUID(i), 0,
PeerShortID(rng.Intn(1<<PeerShortIDBits)))
}
shuffle := func() {
for i := range ps {
j := rng.Intn(i + 1)
ps[i], ps[j] = ps[j], ps[i]
}
}
// Fill peers
shuffle()
var pending PeersPendingNotifications
for _, p := range ps {
peers.addByShortID(p, &pending)
}
// Check invariants
counts := make([]int, 1<<PeerShortIDBits)
saw := func(p *Peer) {
if p != peers.ourself.Peer {
counts[p.UID]++
}
}
for shortID, entry := range peers.byShortID {
if entry.peer == nil {
// no principal peer for this short id, so
// others must be empty
require.Empty(t, entry.others)
continue
}
require.Equal(t, shortID, entry.peer.ShortID)
saw(entry.peer)
for _, p := range entry.others {
saw(p)
require.Equal(t, shortID, p.ShortID)
// the principal peer should have the lowest name
require.True(t, p.Name > entry.peer.Name)
}
}
// Check that every peer was seen
for _, n := range counts {
require.Equal(t, 1, n)
}
// Delete all the peers
shuffle()
for _, p := range ps {
peers.deleteByShortID(p, &pending)
}
for _, entry := range peers.byShortID {
if entry.peer != peers.ourself.Peer {
require.Nil(t, entry.peer)
}
require.Empty(t, entry.others)
}
}
// Test the easy case of short id reassignment, when few short ids are taken
func TestShortIDReassignmentEasy(t *testing.T) {
rng := rand.New(rand.NewSource(time.Now().UnixNano()))
_, peers := newNode(PeerName(0))
for i := 1; i <= 10; i++ {
peers.FetchWithDefault(NewPeer(PeerName(i), "", PeerUID(i), 0,
PeerShortID(rng.Intn(1<<PeerShortIDBits))))
}
checkShortIDReassignment(t, peers)
}
// Test the hard case of short id reassignment, when most short ids are taken
func TestShortIDReassignmentHard(t *testing.T) {
rng := rand.New(rand.NewSource(time.Now().UnixNano()))
_, peers := newNode(PeerName(1 << PeerShortIDBits))
// Take all short ids
ps := make([]*Peer, 1<<PeerShortIDBits)
var pending PeersPendingNotifications
for i := 0; i < 1<<PeerShortIDBits; i++ {
ps[i] = NewPeer(PeerName(i), "", PeerUID(i), 0,
PeerShortID(i))
peers.addByShortID(ps[i], &pending)
}
// As all short ids are taken, an attempted reassigment won't
// do anything
oldShortID := peers.ourself.ShortID
require.False(t, peers.reassignLocalShortID(&pending))
require.Equal(t, oldShortID, peers.ourself.ShortID)
// Free up a few ids
for i := 0; i < 10; i++ {
x := rng.Intn(len(ps))
if ps[x] != nil {
peers.deleteByShortID(ps[x], &pending)
ps[x] = nil
}
}
checkShortIDReassignment(t, peers)
}
func checkShortIDReassignment(t *testing.T, peers *Peers) {
oldShortID := peers.ourself.ShortID
peers.reassignLocalShortID(&PeersPendingNotifications{})
require.NotEqual(t, oldShortID, peers.ourself.ShortID)
require.Equal(t, peers.ourself.Peer, peers.byShortID[peers.ourself.ShortID].peer)
}
func TestShortIDInvalidation(t *testing.T) {
_, peers := newNode(PeerName(1 << PeerShortIDBits))
// need to use a short id that is not the local peer's
shortID := peers.ourself.ShortID + 1
var pending PeersPendingNotifications
requireInvalidateShortIDs := func(expect bool) {
require.Equal(t, expect, pending.invalidateShortIDs)
pending.invalidateShortIDs = false
}
// The use of a fresh short id does not cause invalidation
a := NewPeer(PeerName(1), "", PeerUID(1), 0, shortID)
peers.addByShortID(a, &pending)
requireInvalidateShortIDs(false)
// An addition which does not change the mapping
b := NewPeer(PeerName(2), "", PeerUID(2), 0, shortID)
peers.addByShortID(b, &pending)
requireInvalidateShortIDs(false)
// An addition which does change the mapping
c := NewPeer(PeerName(0), "", PeerUID(0), 0, shortID)
peers.addByShortID(c, &pending)
requireInvalidateShortIDs(true)
// A deletion which does not change the mapping
peers.deleteByShortID(b, &pending)
requireInvalidateShortIDs(false)
// A deletion which does change the mapping
peers.deleteByShortID(c, &pending)
requireInvalidateShortIDs(true)
// Deleting the last peer with a short id does not cause invalidation
peers.deleteByShortID(a, &pending)
requireInvalidateShortIDs(false)
// .. but subsequent reuse of that short id does cause invalidation
peers.addByShortID(a, &pending)
requireInvalidateShortIDs(true)
}
func TestShortIDPropagation(t *testing.T) {
_, peers1 := newNode(PeerName(1))
_, peers2 := newNode(PeerName(2))
peers1.AddTestConnection(peers2.ourself.Peer)
peers1.ApplyUpdate(peers2.EncodePeers(peers2.Names()))
peers12 := peers1.Fetch(PeerName(2))
old := peers12.PeerSummary
require.True(t,
peers2.reassignLocalShortID(&PeersPendingNotifications{}))
peers1.ApplyUpdate(peers2.EncodePeers(peers2.Names()))
require.NotEqual(t, old.Version, peers12.Version)
require.NotEqual(t, old.ShortID, peers12.ShortID)
}
func TestShortIDCollision(t *testing.T) {
// Create 3 peers
_, peers1 := newNode(PeerName(1))
_, peers2 := newNode(PeerName(2))
_, peers3 := newNode(PeerName(3))
var pending PeersPendingNotifications
peers1.setLocalShortID(1, &pending)
peers2.setLocalShortID(2, &pending)
peers3.setLocalShortID(3, &pending)
peers2.AddTestConnection(peers1.ourself.Peer)
peers3.AddTestConnection(peers2.ourself.Peer)
// Propogate from 1 to 2 to 3
peers2.ApplyUpdate(peers1.EncodePeers(peers1.Names()))
peers3.ApplyUpdate(peers2.EncodePeers(peers2.Names()))
// Force the short id of peer 1 to collide with peer 2. Peer
// 1 has the lowest name, so it gets to keep the short id
peers1.setLocalShortID(2, &pending)
oldShortID := peers2.ourself.ShortID
_, updated, _ := peers2.ApplyUpdate(peers1.EncodePeers(peers1.Names()))
// peer 2 should have noticed the collision and resolved it
require.NotEqual(t, oldShortID, peers2.ourself.ShortID)
// The Peers do not have a Router, so broadcastPeerUpdate does
// nothing in the context of this test. So we fake what it
// would do.
updated[PeerName(2)] = void
// the update from peer 2 should include its short id change
peers3.ApplyUpdate(peers2.EncodePeers(updated))
require.Equal(t, peers2.ourself.ShortID,
peers3.Fetch(PeerName(2)).ShortID)
}
// Test the case where all short ids are taken, but then some peers go
// away, so the local peer reassigns
func TestDeferredShortIDReassignment(t *testing.T) {
rng := rand.New(rand.NewSource(time.Now().UnixNano()))
_, us := newNode(PeerName(1 << PeerShortIDBits))
// Connect us to other peers occupying all short ids
others := make([]*Peers, 1<<PeerShortIDBits)
var pending PeersPendingNotifications
for i := range others {
_, others[i] = newNode(PeerName(i))
others[i].setLocalShortID(PeerShortID(i), &pending)
us.AddTestConnection(others[i].ourself.Peer)
}
// Check that, as expected, the local peer does not own its
// short id
require.NotEqual(t, us.ourself.Peer,
us.byShortID[us.ourself.ShortID].peer)
// Disconnect one peer, and we should now be able to claim its
// short id
other := others[rng.Intn(1<<PeerShortIDBits)]
us.DeleteTestConnection(other.ourself.Peer)
us.GarbageCollect()
require.Equal(t, us.ourself.Peer, us.byShortID[us.ourself.ShortID].peer)
}
+348
View File
@@ -0,0 +1,348 @@
package mesh
import (
"bytes"
"encoding/gob"
"encoding/hex"
"fmt"
"io"
"time"
)
const (
Protocol = "weave"
ProtocolMinVersion = 1
ProtocolMaxVersion = 2
)
var (
ProtocolBytes = []byte(Protocol)
HeaderTimeout = 10 * time.Second
ProtocolV1Features = []string{
"ConnID",
"Name",
"NickName",
"PeerNameFlavour",
"UID",
}
ErrExpectedCrypto = fmt.Errorf("Password specified, but peer requested an unencrypted connection")
ErrExpectedNoCrypto = fmt.Errorf("No password specificed, but peer requested an encrypted connection")
)
// We don't need the full net.TCPConn to do the protocol intro. This
// interface contains just the parts we do need, to support testing
type ProtocolIntroConn interface {
// io.Reader
Read(b []byte) (n int, err error)
// io.Writer
Write(b []byte) (n int, err error)
// net.Conn's deadline methods
SetDeadline(t time.Time) error
SetReadDeadline(t time.Time) error
SetWriteDeadline(t time.Time) error
}
type ProtocolIntroParams struct {
MinVersion byte
MaxVersion byte
Features map[string]string
Conn ProtocolIntroConn
Password []byte
Outbound bool
}
type ProtocolIntroResults struct {
Features map[string]string
Receiver TCPReceiver
Sender TCPSender
SessionKey *[32]byte
Version byte
}
func (params ProtocolIntroParams) DoIntro() (res ProtocolIntroResults, err error) {
if err = params.Conn.SetDeadline(time.Now().Add(HeaderTimeout)); err != nil {
return
}
if res.Version, err = params.exchangeProtocolHeader(); err != nil {
return
}
var pubKey, privKey *[32]byte
if params.Password != nil {
if pubKey, privKey, err = GenerateKeyPair(); err != nil {
return
}
}
if err = params.Conn.SetWriteDeadline(time.Time{}); err != nil {
return
}
if err = params.Conn.SetReadDeadline(time.Now().Add(TCPHeartbeat * 2)); err != nil {
return
}
switch res.Version {
case 1:
err = res.doIntroV1(params, pubKey, privKey)
case 2:
err = res.doIntroV2(params, pubKey, privKey)
default:
panic("unhandled protocol version")
}
return
}
func (params ProtocolIntroParams) exchangeProtocolHeader() (byte, error) {
// Write in a separate goroutine to avoid the possibility of
// deadlock. The result channel is of size 1 so that the
// goroutine does not linger even if we encounter an error on
// the read side.
sendHeader := append(ProtocolBytes, params.MinVersion, params.MaxVersion)
writeDone := make(chan error, 1)
go func() {
_, err := params.Conn.Write(sendHeader)
writeDone <- err
}()
header := make([]byte, len(ProtocolBytes)+2)
if n, err := io.ReadFull(params.Conn, header); err != nil && n == 0 {
return 0, fmt.Errorf("failed to receive remote protocol header: %s", err)
} else if err != nil {
return 0, fmt.Errorf("received incomplete remote protocol header (%d octets instead of %d): %v; error: %s",
n, len(header), header[:n], err)
}
if !bytes.Equal(ProtocolBytes, header[:len(ProtocolBytes)]) {
return 0, fmt.Errorf("remote protocol header not recognised: %v", header[:len(ProtocolBytes)])
}
theirMinVersion := header[len(ProtocolBytes)]
minVersion := theirMinVersion
if params.MinVersion > minVersion {
minVersion = params.MinVersion
}
theirMaxVersion := header[len(ProtocolBytes)+1]
maxVersion := theirMaxVersion
if maxVersion > params.MaxVersion {
maxVersion = params.MaxVersion
}
if minVersion > maxVersion {
return 0, fmt.Errorf("remote version range [%d,%d] is incompatible with ours [%d,%d]",
theirMinVersion, theirMaxVersion,
params.MinVersion, params.MaxVersion)
}
if err := <-writeDone; err != nil {
return 0, err
}
return maxVersion, nil
}
// The V1 procotol consists of the protocol identification/version
// header, followed by a stream of gobified values. The first value
// is the encoded features map (never encrypted). The subsequent
// values are the messages on the connection (encrypted for an
// encrypted connection). For an encrypted connection, the public key
// is passed in the "PublicKey" feature as a string of hex digits.
func (res *ProtocolIntroResults) doIntroV1(params ProtocolIntroParams, pubKey, privKey *[32]byte) error {
features := filterV1Features(params.Features)
if pubKey != nil {
features["PublicKey"] = hex.EncodeToString(pubKey[:])
}
enc := gob.NewEncoder(params.Conn)
dec := gob.NewDecoder(params.Conn)
// Encode in a separate goroutine to avoid the possibility of
// deadlock. The result channel is of size 1 so that the
// goroutine does not linger even if we encounter an error on
// the read side.
encodeDone := make(chan error, 1)
go func() {
encodeDone <- enc.Encode(features)
}()
if err := dec.Decode(&res.Features); err != nil {
return err
}
if err := <-encodeDone; err != nil {
return err
}
res.Sender = NewGobTCPSender(enc)
res.Receiver = NewGobTCPReceiver(dec)
if pubKey == nil {
if _, present := res.Features["PublicKey"]; present {
return ErrExpectedNoCrypto
}
} else {
remotePubKeyStr, ok := res.Features["PublicKey"]
if !ok {
return ErrExpectedCrypto
}
remotePubKey, err := hex.DecodeString(remotePubKeyStr)
if err != nil {
return err
}
res.setupCrypto(params, remotePubKey, privKey)
}
res.Features = filterV1Features(res.Features)
return nil
}
// In the V1 protocol, the intro fields are sent unencrypted. So we
// restrict them to an established subset of fields that are assumed
// to be safe.
func filterV1Features(intro map[string]string) map[string]string {
safe := make(map[string]string)
for _, k := range ProtocolV1Features {
if val, ok := intro[k]; ok {
safe[k] = val
}
}
return safe
}
// The V2 procotol consists of the protocol identification/version
// header, followed by:
//
// - A single "encryption flag" byte: 0 for no encryption, 1 for
// encryption.
//
// - When the connection is encrypted, 32 bytes follow containing the
// public key.
//
// - Then a stream of length-prefixed messages, which are encrypted
// for an encrypted connection.
//
// The first message contains the encoded features map (so in contrast
// to V1, it will be encrypted on an encrypted connection).
func (res *ProtocolIntroResults) doIntroV2(params ProtocolIntroParams, pubKey, privKey *[32]byte) error {
// Public key exchange
var wbuf []byte
if pubKey == nil {
wbuf = []byte{0}
} else {
wbuf = make([]byte, 1+len(*pubKey))
wbuf[0] = 1
copy(wbuf[1:], (*pubKey)[:])
}
// Write in a separate goroutine to avoid the possibility of
// deadlock. The result channel is of size 1 so that the
// goroutine does not linger even if we encounter an error on
// the read side.
writeDone := make(chan error, 1)
go func() {
_, err := params.Conn.Write(wbuf)
writeDone <- err
}()
rbuf := make([]byte, 1)
if _, err := io.ReadFull(params.Conn, rbuf); err != nil {
return err
}
switch rbuf[0] {
case 0:
if pubKey != nil {
return ErrExpectedCrypto
}
res.Sender = NewLengthPrefixTCPSender(params.Conn)
res.Receiver = NewLengthPrefixTCPReceiver(params.Conn)
case 1:
if pubKey == nil {
return ErrExpectedNoCrypto
}
rbuf = make([]byte, len(pubKey))
if _, err := io.ReadFull(params.Conn, rbuf); err != nil {
return err
}
res.Sender = NewLengthPrefixTCPSender(params.Conn)
res.Receiver = NewLengthPrefixTCPReceiver(params.Conn)
res.setupCrypto(params, rbuf, privKey)
default:
return fmt.Errorf("Bad encryption flag %d", rbuf[0])
}
if err := <-writeDone; err != nil {
return err
}
// Features exchange
go func() {
buf := new(bytes.Buffer)
if err := gob.NewEncoder(buf).Encode(&params.Features); err != nil {
writeDone <- err
return
}
writeDone <- res.Sender.Send(buf.Bytes())
}()
rbuf, err := res.Receiver.Receive()
if err != nil {
return err
}
if err := gob.NewDecoder(bytes.NewReader(rbuf)).Decode(&res.Features); err != nil {
return err
}
if err := <-writeDone; err != nil {
return err
}
return nil
}
func (res *ProtocolIntroResults) setupCrypto(params ProtocolIntroParams, remotePubKey []byte, privKey *[32]byte) {
var remotePubKeyArr [32]byte
copy(remotePubKeyArr[:], remotePubKey)
res.SessionKey = FormSessionKey(&remotePubKeyArr, privKey, params.Password)
res.Sender = NewEncryptedTCPSender(res.Sender, res.SessionKey, params.Outbound)
res.Receiver = NewEncryptedTCPReceiver(res.Receiver, res.SessionKey, params.Outbound)
}
type ProtocolTag byte
const (
ProtocolHeartbeat = iota
ProtocolReserved1 // legacy overlay control messages
ProtocolReserved2 //
ProtocolReserved3 //
ProtocolGossip
ProtocolGossipUnicast
ProtocolGossipBroadcast
ProtocolOverlayControlMsg
)
type ProtocolMsg struct {
tag ProtocolTag
msg []byte
}
type ProtocolSender interface {
SendProtocolMsg(m ProtocolMsg)
}
+180
View File
@@ -0,0 +1,180 @@
package mesh
import (
"crypto/rand"
"crypto/sha256"
"encoding/binary"
"encoding/gob"
"fmt"
"io"
"sync"
"golang.org/x/crypto/nacl/box"
"golang.org/x/crypto/nacl/secretbox"
)
const MaxTCPMsgSize = 10 * 1024 * 1024
func GenerateKeyPair() (publicKey, privateKey *[32]byte, err error) {
return box.GenerateKey(rand.Reader)
}
func FormSessionKey(remotePublicKey, localPrivateKey *[32]byte, secretKey []byte) *[32]byte {
var sharedKey [32]byte
box.Precompute(&sharedKey, remotePublicKey, localPrivateKey)
sharedKeySlice := sharedKey[:]
sharedKeySlice = append(sharedKeySlice, secretKey...)
sessionKey := sha256.Sum256(sharedKeySlice)
return &sessionKey
}
// TCP Senders/Receivers
// The lowest 64 bits of the nonce contain the message sequence
// number. The top most bit indicates the connection polarity at the
// sender - '1' for outbound; the next indicates protocol type - '1'
// for TCP. The remaining 126 bits are zero. The polarity is needed so
// that the two ends of a connection do not use the same nonces; the
// protocol type so that the TCP connection nonces are distinct from
// nonces used by overlay connections, if they share the session key.
// This is a requirement of the NaCl Security Model; see
// http://nacl.cr.yp.to/box.html.
type TCPCryptoState struct {
sessionKey *[32]byte
nonce [24]byte
seqNo uint64
}
func NewTCPCryptoState(sessionKey *[32]byte, outbound bool) *TCPCryptoState {
s := &TCPCryptoState{sessionKey: sessionKey}
if outbound {
s.nonce[0] |= (1 << 7)
}
s.nonce[0] |= (1 << 6)
return s
}
func (s *TCPCryptoState) advance() {
s.seqNo++
binary.BigEndian.PutUint64(s.nonce[16:24], s.seqNo)
}
type TCPSender interface {
Send([]byte) error
}
type GobTCPSender struct {
encoder *gob.Encoder
}
type LengthPrefixTCPSender struct {
writer io.Writer
}
type EncryptedTCPSender struct {
sync.RWMutex
sender TCPSender
state *TCPCryptoState
}
func NewGobTCPSender(encoder *gob.Encoder) *GobTCPSender {
return &GobTCPSender{encoder: encoder}
}
func (sender *GobTCPSender) Send(msg []byte) error {
return sender.encoder.Encode(msg)
}
func NewLengthPrefixTCPSender(writer io.Writer) *LengthPrefixTCPSender {
return &LengthPrefixTCPSender{writer: writer}
}
func (sender *LengthPrefixTCPSender) Send(msg []byte) error {
l := len(msg)
if l > MaxTCPMsgSize {
return fmt.Errorf("outgoing message exceeds maximum size: %d > %d", l, MaxTCPMsgSize)
}
// We copy the message so we can send it in a single Write
// operation, thus making this thread-safe without locking.
prefixedMsg := make([]byte, 4+l)
binary.BigEndian.PutUint32(prefixedMsg, uint32(l))
copy(prefixedMsg[4:], msg)
_, err := sender.writer.Write(prefixedMsg)
return err
}
func NewEncryptedTCPSender(sender TCPSender, sessionKey *[32]byte, outbound bool) *EncryptedTCPSender {
return &EncryptedTCPSender{sender: sender, state: NewTCPCryptoState(sessionKey, outbound)}
}
func (sender *EncryptedTCPSender) Send(msg []byte) error {
sender.Lock()
defer sender.Unlock()
encodedMsg := secretbox.Seal(nil, msg, &sender.state.nonce, sender.state.sessionKey)
sender.state.advance()
return sender.sender.Send(encodedMsg)
}
type TCPReceiver interface {
Receive() ([]byte, error)
}
type GobTCPReceiver struct {
decoder *gob.Decoder
}
type LengthPrefixTCPReceiver struct {
reader io.Reader
}
type EncryptedTCPReceiver struct {
receiver TCPReceiver
state *TCPCryptoState
}
func NewGobTCPReceiver(decoder *gob.Decoder) *GobTCPReceiver {
return &GobTCPReceiver{decoder: decoder}
}
func (receiver *GobTCPReceiver) Receive() ([]byte, error) {
var msg []byte
err := receiver.decoder.Decode(&msg)
return msg, err
}
func NewLengthPrefixTCPReceiver(reader io.Reader) *LengthPrefixTCPReceiver {
return &LengthPrefixTCPReceiver{reader: reader}
}
func (receiver *LengthPrefixTCPReceiver) Receive() ([]byte, error) {
lenPrefix := make([]byte, 4)
if _, err := io.ReadFull(receiver.reader, lenPrefix); err != nil {
return nil, err
}
l := binary.BigEndian.Uint32(lenPrefix)
if l > MaxTCPMsgSize {
return nil, fmt.Errorf("incoming message exceeds maximum size: %d > %d", l, MaxTCPMsgSize)
}
msg := make([]byte, l)
_, err := io.ReadFull(receiver.reader, msg)
return msg, err
}
func NewEncryptedTCPReceiver(receiver TCPReceiver, sessionKey *[32]byte, outbound bool) *EncryptedTCPReceiver {
return &EncryptedTCPReceiver{receiver: receiver, state: NewTCPCryptoState(sessionKey, !outbound)}
}
func (receiver *EncryptedTCPReceiver) Receive() ([]byte, error) {
msg, err := receiver.receiver.Receive()
if err != nil {
return nil, err
}
decodedMsg, success := secretbox.Open(nil, msg, &receiver.state.nonce, receiver.state.sessionKey)
if !success {
return nil, fmt.Errorf("Unable to decrypt TCP msg")
}
receiver.state.advance()
return decodedMsg, nil
}
+96
View File
@@ -0,0 +1,96 @@
package mesh
import (
"io"
"testing"
"time"
"github.com/stretchr/testify/require"
)
type testConn struct {
io.Writer
io.Reader
}
func (testConn) SetDeadline(t time.Time) error {
return nil
}
func (testConn) SetReadDeadline(t time.Time) error {
return nil
}
func (testConn) SetWriteDeadline(t time.Time) error {
return nil
}
func connPair() (ProtocolIntroConn, ProtocolIntroConn) {
a := testConn{}
b := testConn{}
a.Reader, b.Writer = io.Pipe()
b.Reader, a.Writer = io.Pipe()
return &a, &b
}
func doIntro(t *testing.T, params ProtocolIntroParams) <-chan ProtocolIntroResults {
ch := make(chan ProtocolIntroResults, 1)
go func() {
res, err := params.DoIntro()
require.Nil(t, err)
ch <- res
}()
return ch
}
func doProtocolIntro(t *testing.T, aver, bver byte, password []byte) byte {
aconn, bconn := connPair()
aresch := doIntro(t, ProtocolIntroParams{
MinVersion: ProtocolMinVersion,
MaxVersion: aver,
Features: map[string]string{"Name": "A"},
Conn: aconn,
Outbound: true,
Password: password,
})
bresch := doIntro(t, ProtocolIntroParams{
MinVersion: ProtocolMinVersion,
MaxVersion: bver,
Features: map[string]string{"Name": "B"},
Conn: bconn,
Outbound: false,
Password: password,
})
ares := <-aresch
bres := <-bresch
// Check that features were conveyed
require.Equal(t, "B", ares.Features["Name"])
require.Equal(t, "A", bres.Features["Name"])
// Check that Senders and Receivers work
go func() {
require.Nil(t, ares.Sender.Send([]byte("Hello from A")))
require.Nil(t, bres.Sender.Send([]byte("Hello from B")))
}()
data, err := bres.Receiver.Receive()
require.Nil(t, err)
require.Equal(t, "Hello from A", string(data))
data, err = ares.Receiver.Receive()
require.Nil(t, err)
require.Equal(t, "Hello from B", string(data))
require.Equal(t, ares.Version, bres.Version)
return ares.Version
}
func TestProtocolIntro(t *testing.T) {
require.Equal(t, 2, int(doProtocolIntro(t, 2, 2, nil)))
require.Equal(t, 2, int(doProtocolIntro(t, 2, 2, []byte("sekr1t"))))
require.Equal(t, 1, int(doProtocolIntro(t, 1, 2, nil)))
require.Equal(t, 1, int(doProtocolIntro(t, 1, 2, []byte("pa55"))))
require.Equal(t, 1, int(doProtocolIntro(t, 2, 1, nil)))
require.Equal(t, 1, int(doProtocolIntro(t, 2, 1, []byte("w0rd"))))
}
+189
View File
@@ -0,0 +1,189 @@
package mesh
import (
"fmt"
"math"
"net"
"sync"
"time"
"github.com/weaveworks/weave/common"
)
const (
Port = 6783
ChannelSize = 16
TCPHeartbeat = 30 * time.Second
GossipInterval = 30 * time.Second
MaxDuration = time.Duration(math.MaxInt64)
acceptMaxTokens = 100 // [1]
acceptTokenDelay = 100 * time.Millisecond // [2]
)
// [1] capacity of token bucket for rate limiting accepts
// [2] control rate at which new tokens are added to the bucket
var (
log = common.Log
checkFatal = common.CheckFatal
checkWarn = common.CheckWarn
)
type Config struct {
Port int
ProtocolMinVersion byte
Password []byte
ConnLimit int
PeerDiscovery bool
}
type Router struct {
Config
Overlay Overlay
Ourself *LocalPeer
Peers *Peers
Routes *Routes
ConnectionMaker *ConnectionMaker
gossipLock sync.RWMutex
gossipChannels GossipChannels
TopologyGossip Gossip
acceptLimiter *TokenBucket
}
func NewRouter(config Config, name PeerName, nickName string, overlay Overlay) *Router {
router := &Router{Config: config, gossipChannels: make(GossipChannels)}
if overlay == nil {
overlay = NullOverlay{}
}
router.Overlay = overlay
router.Ourself = NewLocalPeer(name, nickName, router)
router.Peers = NewPeers(router.Ourself)
router.Peers.OnGC(func(peer *Peer) {
log.Println("Removed unreachable peer", peer)
})
router.Routes = NewRoutes(router.Ourself, router.Peers)
router.ConnectionMaker = NewConnectionMaker(router.Ourself, router.Peers, router.Port, router.PeerDiscovery)
router.TopologyGossip = router.NewGossip("topology", router)
router.acceptLimiter = NewTokenBucket(acceptMaxTokens, acceptTokenDelay)
return router
}
// Start listening for TCP connections. This is separate from
// NewRouter so that gossipers can register before we start forming
// connections.
func (router *Router) Start() {
router.listenTCP(router.Port)
}
func (router *Router) Stop() error {
// TODO: perform graceful shutdown...
return nil
}
func (router *Router) UsingPassword() bool {
return router.Password != nil
}
func (router *Router) listenTCP(localPort int) {
localAddr, err := net.ResolveTCPAddr("tcp4", fmt.Sprint(":", localPort))
checkFatal(err)
ln, err := net.ListenTCP("tcp4", localAddr)
checkFatal(err)
go func() {
defer ln.Close()
for {
tcpConn, err := ln.AcceptTCP()
if err != nil {
log.Errorln(err)
continue
}
router.acceptTCP(tcpConn)
router.acceptLimiter.Wait()
}
}()
}
func (router *Router) acceptTCP(tcpConn *net.TCPConn) {
remoteAddrStr := tcpConn.RemoteAddr().String()
log.Printf("->[%s] connection accepted", remoteAddrStr)
connRemote := NewRemoteConnection(router.Ourself.Peer, nil, remoteAddrStr, false, false)
StartLocalConnection(connRemote, tcpConn, router, true)
}
// Gossiper methods - the Router is the topology Gossiper
type TopologyGossipData struct {
peers *Peers
update PeerNameSet
}
func (router *Router) BroadcastTopologyUpdate(update []*Peer) {
names := make(PeerNameSet)
for _, p := range update {
names[p.Name] = void
}
router.TopologyGossip.GossipBroadcast(&TopologyGossipData{
peers: router.Peers,
update: names,
})
}
func (d *TopologyGossipData) Merge(other GossipData) {
for name := range other.(*TopologyGossipData).update {
d.update[name] = void
}
}
func (d *TopologyGossipData) Encode() [][]byte {
return [][]byte{d.peers.EncodePeers(d.update)}
}
func (router *Router) OnGossipUnicast(sender PeerName, msg []byte) error {
return fmt.Errorf("unexpected topology gossip unicast: %v", msg)
}
func (router *Router) OnGossipBroadcast(_ PeerName, update []byte) (GossipData, error) {
origUpdate, _, err := router.applyTopologyUpdate(update)
if err != nil || len(origUpdate) == 0 {
return nil, err
}
return &TopologyGossipData{peers: router.Peers, update: origUpdate}, nil
}
func (router *Router) Gossip() GossipData {
return &TopologyGossipData{peers: router.Peers, update: router.Peers.Names()}
}
func (router *Router) OnGossip(update []byte) (GossipData, error) {
_, newUpdate, err := router.applyTopologyUpdate(update)
if err != nil || len(newUpdate) == 0 {
return nil, err
}
return &TopologyGossipData{peers: router.Peers, update: newUpdate}, nil
}
func (router *Router) applyTopologyUpdate(update []byte) (PeerNameSet, PeerNameSet, error) {
origUpdate, newUpdate, err := router.Peers.ApplyUpdate(update)
if _, ok := err.(UnknownPeerError); err != nil && ok {
// That update contained a reference to a peer which wasn't
// itself included in the update, and we didn't know about
// already. We ignore this; eventually we should receive an
// update containing a complete topology.
log.Println("Topology gossip:", err)
return nil, nil, nil
}
if err != nil {
return nil, nil, err
}
if len(newUpdate) > 0 {
router.ConnectionMaker.Refresh()
router.Routes.Recalculate()
}
return origUpdate, newUpdate, nil
}
+89 -48
View File
@@ -1,20 +1,22 @@
package router
package mesh
import (
"bytes"
"fmt"
"math"
"sync"
)
type unicastRoutes map[PeerName]PeerName
type broadcastRoutes map[PeerName][]PeerName
type Routes struct {
sync.RWMutex
ourself *LocalPeer
peers *Peers
unicast map[PeerName]PeerName
unicastAll map[PeerName]PeerName // [1]
broadcast map[PeerName][]PeerName
broadcastAll map[PeerName][]PeerName // [1]
onChange []func()
unicast unicastRoutes
unicastAll unicastRoutes // [1]
broadcast broadcastRoutes
broadcastAll broadcastRoutes // [1]
recalculate chan<- *struct{}
wait chan<- chan struct{}
// [1] based on *all* connections, not just established &
@@ -22,26 +24,29 @@ type Routes struct {
}
func NewRoutes(ourself *LocalPeer, peers *Peers) *Routes {
recalculate := make(chan *struct{}, 1)
wait := make(chan chan struct{})
routes := &Routes{
ourself: ourself,
peers: peers,
unicast: make(map[PeerName]PeerName),
unicastAll: make(map[PeerName]PeerName),
broadcast: make(map[PeerName][]PeerName),
broadcastAll: make(map[PeerName][]PeerName)}
unicast: make(unicastRoutes),
unicastAll: make(unicastRoutes),
broadcast: make(broadcastRoutes),
broadcastAll: make(broadcastRoutes),
recalculate: recalculate,
wait: wait}
routes.unicast[ourself.Name] = UnknownPeerName
routes.unicastAll[ourself.Name] = UnknownPeerName
routes.broadcast[ourself.Name] = []PeerName{}
routes.broadcastAll[ourself.Name] = []PeerName{}
go routes.run(recalculate, wait)
return routes
}
func (routes *Routes) Start() {
recalculate := make(chan *struct{}, 1)
wait := make(chan chan struct{})
routes.recalculate = recalculate
routes.wait = wait
go routes.run(recalculate, wait)
func (routes *Routes) OnChange(callback func()) {
routes.Lock()
defer routes.Unlock()
routes.onChange = append(routes.onChange, callback)
}
func (routes *Routes) PeerNames() PeerNameSet {
@@ -65,21 +70,13 @@ func (routes *Routes) UnicastAll(name PeerName) (PeerName, bool) {
func (routes *Routes) Broadcast(name PeerName) []PeerName {
routes.RLock()
defer routes.RUnlock()
hops, found := routes.broadcast[name]
if !found {
return []PeerName{}
}
return hops
return routes.broadcast[name]
}
func (routes *Routes) BroadcastAll(name PeerName) []PeerName {
routes.RLock()
defer routes.RUnlock()
hops, found := routes.broadcastAll[name]
if !found {
return []PeerName{}
}
return hops
return routes.broadcastAll[name]
}
// Choose min(log2(n_peers), n_neighbouring_peers) neighbours, with a
@@ -112,23 +109,6 @@ func (routes *Routes) RandomNeighbours(except PeerName) PeerNameSet {
return res
}
func (routes *Routes) String() string {
var buf bytes.Buffer
routes.RLock()
defer routes.RUnlock()
fmt.Fprintln(&buf, "unicast:")
for name, hop := range routes.unicast {
fmt.Fprintf(&buf, "%s -> %s\n", name, hop)
}
fmt.Fprintln(&buf, "broadcast:")
for name, hops := range routes.broadcast {
fmt.Fprintf(&buf, "%s -> %v\n", name, hops)
}
// We don't include the 'all' routes here since they are of
// limited utility in troubleshooting
return buf.String()
}
// Request recalculation of the routing table. This is async but can
// effectively be made synchronous with a subsequent call to
// EnsureRecalculated.
@@ -169,6 +149,8 @@ func (routes *Routes) calculate() {
routes.peers.RLock()
routes.ourself.RLock()
var (
oldUnicast = routes.unicast
oldBroadcast = routes.broadcast
unicast = routes.calculateUnicast(true)
unicastAll = routes.calculateUnicast(false)
broadcast = routes.calculateBroadcast(true)
@@ -182,7 +164,14 @@ func (routes *Routes) calculate() {
routes.unicastAll = unicastAll
routes.broadcast = broadcast
routes.broadcastAll = broadcastAll
onChange := routes.onChange
routes.Unlock()
if !unicast.equals(oldUnicast) || !broadcast.equals(oldBroadcast) {
for _, callback := range onChange {
callback()
}
}
}
// Calculate all the routes for the question: if *we* want to send a
@@ -194,7 +183,7 @@ func (routes *Routes) calculate() {
// any knowledge of the MAC address at all. Thus there's no need
// to exchange knowledge of MAC addresses, nor any constraints on
// the routes that we construct.
func (routes *Routes) calculateUnicast(establishedAndSymmetric bool) map[PeerName]PeerName {
func (routes *Routes) calculateUnicast(establishedAndSymmetric bool) unicastRoutes {
_, unicast := routes.ourself.Routes(nil, establishedAndSymmetric)
return unicast
}
@@ -217,9 +206,9 @@ func (routes *Routes) calculateUnicast(establishedAndSymmetric bool) map[PeerNam
// Y =/= Z /\ X.Routes(Y) <= X.Routes(Z) =>
// X.Routes(Y) u [P | Y.HasSymmetricConnectionTo(P)] <= X.Routes(Z)
// where <= is the subset relationship on keys of the returned map.
func (routes *Routes) calculateBroadcast(establishedAndSymmetric bool) map[PeerName][]PeerName {
broadcast := make(map[PeerName][]PeerName)
for _, peer := range routes.peers.table {
func (routes *Routes) calculateBroadcast(establishedAndSymmetric bool) broadcastRoutes {
broadcast := make(broadcastRoutes)
for _, peer := range routes.peers.byName {
hops := []PeerName{}
if found, reached := peer.Routes(routes.ourself.Peer, establishedAndSymmetric); found {
routes.ourself.ForEachConnectedPeer(establishedAndSymmetric, reached,
@@ -229,3 +218,55 @@ func (routes *Routes) calculateBroadcast(establishedAndSymmetric bool) map[PeerN
}
return broadcast
}
func (a unicastRoutes) equals(b unicastRoutes) bool {
for key, aval := range a {
if bval, ok := b[key]; !ok || bval != aval {
return false
}
}
for key := range b {
if _, ok := a[key]; !ok {
return false
}
}
return true
}
func (a broadcastRoutes) equals(b broadcastRoutes) bool {
set := make(map[PeerName]struct{})
for key, aval := range a {
bval, ok := b[key]
if !ok {
return false
}
for _, peer := range aval {
set[peer] = struct{}{}
}
for _, peer := range bval {
if _, ok := set[peer]; !ok {
return false
}
delete(set, peer)
}
if len(set) != 0 {
return false
}
}
for key := range b {
if _, ok := a[key]; !ok {
return false
}
}
return true
}
+193
View File
@@ -0,0 +1,193 @@
package mesh
import (
"fmt"
)
type Status struct {
Protocol string
ProtocolMinVersion int
ProtocolMaxVersion int
Encryption bool
PeerDiscovery bool
Name string
NickName string
Port int
Peers []PeerStatus
UnicastRoutes []UnicastRouteStatus
BroadcastRoutes []BroadcastRouteStatus
Connections []LocalConnectionStatus
Targets []string
OverlayDiagnostics interface{}
}
type PeerStatus struct {
Name string
NickName string
UID PeerUID
ShortID PeerShortID
Version uint64
Connections []ConnectionStatus
}
type ConnectionStatus struct {
Name string
NickName string
Address string
Outbound bool
Established bool
}
type UnicastRouteStatus struct {
Dest, Via string
}
type BroadcastRouteStatus struct {
Source string
Via []string
}
type LocalConnectionStatus struct {
Address string
Outbound bool
State string
Info string
}
func NewStatus(router *Router) *Status {
return &Status{
Protocol,
ProtocolMinVersion,
ProtocolMaxVersion,
router.UsingPassword(),
router.PeerDiscovery,
router.Ourself.Name.String(),
router.Ourself.NickName,
router.Port,
NewPeerStatusSlice(router.Peers),
NewUnicastRouteStatusSlice(router.Routes),
NewBroadcastRouteStatusSlice(router.Routes),
NewLocalConnectionStatusSlice(router.ConnectionMaker),
NewTargetSlice(router.ConnectionMaker),
router.Overlay.Diagnostics()}
}
func NewPeerStatusSlice(peers *Peers) []PeerStatus {
var slice []PeerStatus
peers.ForEach(func(peer *Peer) {
var connections []ConnectionStatus
if peer == peers.ourself.Peer {
for conn := range peers.ourself.Connections() {
connections = append(connections, newConnectionStatus(conn))
}
} else {
// Modifying peer.connections requires a write lock on
// Peers, and since we are holding a read lock (due to the
// ForEach), access without locking the peer is safe.
for _, conn := range peer.connections {
connections = append(connections, newConnectionStatus(conn))
}
}
slice = append(slice, PeerStatus{
peer.Name.String(),
peer.NickName,
peer.UID,
peer.ShortID,
peer.Version,
connections})
})
return slice
}
func newConnectionStatus(c Connection) ConnectionStatus {
return ConnectionStatus{
c.Remote().Name.String(),
c.Remote().NickName,
c.RemoteTCPAddr(),
c.Outbound(),
c.Established()}
}
func NewUnicastRouteStatusSlice(routes *Routes) []UnicastRouteStatus {
routes.RLock()
defer routes.RUnlock()
var slice []UnicastRouteStatus
for dest, via := range routes.unicast {
slice = append(slice, UnicastRouteStatus{dest.String(), via.String()})
}
return slice
}
func NewBroadcastRouteStatusSlice(routes *Routes) []BroadcastRouteStatus {
routes.RLock()
defer routes.RUnlock()
var slice []BroadcastRouteStatus
for source, via := range routes.broadcast {
var hops []string
for _, hop := range via {
hops = append(hops, hop.String())
}
slice = append(slice, BroadcastRouteStatus{source.String(), hops})
}
return slice
}
func NewLocalConnectionStatusSlice(cm *ConnectionMaker) []LocalConnectionStatus {
resultChan := make(chan []LocalConnectionStatus, 0)
cm.actionChan <- func() bool {
var slice []LocalConnectionStatus
for conn := range cm.connections {
state := "pending"
if conn.Established() {
state = "established"
}
lc, _ := conn.(*LocalConnection)
info := fmt.Sprintf("%-6v %v", lc.OverlayConn.DisplayName(), conn.Remote())
slice = append(slice, LocalConnectionStatus{conn.RemoteTCPAddr(), conn.Outbound(), state, info})
}
for address, target := range cm.targets {
add := func(state, info string) {
slice = append(slice, LocalConnectionStatus{address, true, state, info})
}
switch target.state {
case TargetWaiting:
until := "never"
if !target.tryAfter.IsZero() {
until = target.tryAfter.String()
}
if target.lastError == nil { // shouldn't happen
add("waiting", "until: "+until)
} else {
add("failed", target.lastError.Error()+", retry: "+until)
}
case TargetAttempting:
if target.lastError == nil {
add("connecting", "")
} else {
add("retrying", target.lastError.Error())
}
case TargetConnected:
}
}
resultChan <- slice
return false
}
return <-resultChan
}
func NewTargetSlice(cm *ConnectionMaker) []string {
resultChan := make(chan []string, 0)
cm.actionChan <- func() bool {
var slice []string
for peer := range cm.directPeers {
slice = append(slice, peer)
}
resultChan <- slice
return false
}
return <-resultChan
}
@@ -1,4 +1,4 @@
package router
package mesh
type SurrogateGossipData struct {
messages [][]byte
@@ -24,7 +24,7 @@ func (*SurrogateGossiper) OnGossipUnicast(sender PeerName, msg []byte) error {
return nil
}
func (*SurrogateGossiper) OnGossipBroadcast(update []byte) (GossipData, error) {
func (*SurrogateGossiper) OnGossipBroadcast(_ PeerName, update []byte) (GossipData, error) {
return NewSurrogateGossipData(update), nil
}
@@ -1,4 +1,4 @@
package router
package mesh
import (
"time"
-128
View File
@@ -1,128 +0,0 @@
package nameserver
import (
"bytes"
"fmt"
"net"
)
// A zone record
// Note: we will try to keep all the zone records information is concentrated here. Maybe not all queries will
// use things like priorities or weights, but we do not want to create a hierarchy for dealing with all
// possible queries...
type ZoneRecord interface {
Name() string // The name for this IP
IP() net.IP // The IP (v4) address
Priority() int // The priority
Weight() int // The weight
TTL() int // TTL
}
type ZoneLookup interface {
// Lookup for a name
LookupName(name string) ([]ZoneRecord, error)
// Lookup for an address
LookupInaddr(inaddr string) ([]ZoneRecord, error)
}
type ZoneLookupFunc func(target string) ([]ZoneRecord, error)
type ZoneObservable interface {
// Observe anything that affects a particular name in the zone
ObserveName(name string, observer ZoneRecordObserver) error
// Observe anything that affects a particular IP in the zone
ObserveInaddr(inaddr string, observer ZoneRecordObserver) error
}
type ZoneObserverFunc func(string, ZoneRecordObserver) error
type ZoneRecordObserver func()
/////////////////////////////////////////////////////////////
// A basic record struct (satisfying the ZoneRecord interface)
type Record struct {
name string
ip net.IP
priority int
weight int
ttl int
}
func (r Record) Name() string { return r.name }
func (r Record) IP() net.IP { return r.ip }
func (r Record) Priority() int { return r.priority }
func (r Record) Weight() int { return r.weight }
func (r Record) TTL() int { return r.ttl }
func (r Record) String() string {
var buf bytes.Buffer
if len(r.Name()) > 0 {
fmt.Fprintf(&buf, "%s", r.Name())
}
if !r.IP().IsUnspecified() {
fmt.Fprintf(&buf, "[%s]", r.IP())
}
if r.Priority() > 0 {
fmt.Fprintf(&buf, "/P:%d", r.Priority())
}
if r.Weight() > 0 {
fmt.Fprintf(&buf, "/W:%d", r.Weight())
}
if r.TTL() > 0 {
fmt.Fprintf(&buf, "/TTL:%d", r.TTL())
}
return buf.String()
}
/////////////////////////////////////////////////////////////
func newParseError(reason string, addr string) *net.ParseError {
return &net.ParseError{Type: reason, Text: addr}
}
// simple IPv4 type that can be used as a key in a map (in contrast with net.IP), used for sets of IPs, etc...
type IPv4 [4]byte
func (ip IPv4) toNetIP() net.IP { return net.IPv4(ip[0], ip[1], ip[2], ip[3]) }
func (ip IPv4) String() string { return ip.toNetIP().String() }
func ipToIPv4(nip net.IP) IPv4 { ip := nip.To4(); return IPv4([4]byte{ip[0], ip[1], ip[2], ip[3]}) }
// Parse an address (eg, "10.13.12.11") and return the corresponding IPv4 (eg, "10.13.12.11")
func addrToIPv4(addr string) (IPv4, error) {
ip := net.ParseIP(addr)
if ip == nil {
return IPv4{}, newParseError("IP address", addr)
}
return ipToIPv4(ip), nil
}
// Parse a reverse address (eg, "11.12.13.10.in-addr.arpa.") and return the corresponding IPv4 (eg, "10.13.12.11")
func raddrToIPv4(addr string) (IPv4, error) {
l := len(addr)
if l < rdnsDomainLen+1 {
return IPv4{}, newParseError("too short reverse IP address", addr)
}
suffixLen := l - rdnsDomainLen
suffix := addr[suffixLen+1:]
if suffix != RDNSDomain {
return IPv4{}, newParseError("suffix of reverse IP address", addr)
}
ipStr := addr[:suffixLen]
revIP := net.ParseIP(ipStr)
if revIP == nil {
return IPv4{}, newParseError("reverse IP address", addr)
}
revIP4 := revIP.To4()
return IPv4([4]byte{revIP4[3], revIP4[2], revIP4[1], revIP4[0]}), nil
}
func raddrToIP(addr string) (net.IP, error) {
r, err := raddrToIPv4(addr)
if err != nil {
return net.IP{}, err
}
return r.toNetIP(), nil
}
-32
View File
@@ -1,32 +0,0 @@
package nameserver
import (
"net"
"testing"
. "github.com/weaveworks/weave/common"
wt "github.com/weaveworks/weave/testing"
)
func TestAddrs(t *testing.T) {
InitDefaultLogging(testing.Verbose())
Info.Println("TestAddrs starting")
ip, err := addrToIPv4("10.13.12.11")
wt.AssertNoErr(t, err)
wt.AssertTrue(t, net.ParseIP("10.13.12.11").Equal(ip.toNetIP()), "IP")
ip, err = raddrToIPv4("11.12.13.10.in-addr.arpa.")
wt.AssertNoErr(t, err)
wt.AssertTrue(t, net.ParseIP("10.13.12.11").Equal(ip.toNetIP()), "IP")
// some malformed addresses
ip, err = addrToIPv4("10.13.12")
wt.AssertTrue(t, err != nil, "when parsing malformed address")
ip, err = addrToIPv4("10.13.AA.12")
wt.AssertTrue(t, err != nil, "when parsing malformed address")
ip, err = raddrToIPv4("11.12.13.10.in-axxx.arpa.")
wt.AssertTrue(t, err != nil, "when parsing malformed address")
ip, err = raddrToIPv4("11.12.13.10.in-addr")
wt.AssertTrue(t, err != nil, "when parsing malformed address")
}
-382
View File
@@ -1,382 +0,0 @@
package nameserver
import (
"bytes"
"container/heap"
"errors"
"fmt"
"math"
"sync"
"time"
"github.com/benbjohnson/clock"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
)
var (
errInvalidCapacity = errors.New("Invalid cache capacity")
errCouldNotResolve = errors.New("Could not resolve")
errTimeout = errors.New("Timeout while waiting for resolution")
errNoLocalReplies = errors.New("No local replies")
)
const (
defPendingTimeout int = 5 // timeout for a resolution
)
const nullTTL = 0 // a null TTL
type entryStatus uint8
const (
stPending entryStatus = iota // someone is waiting for the resolution
stResolved entryStatus = iota // resolved
)
var statusToString = map[entryStatus]string{
stPending: "pending",
stResolved: "resolved",
}
const (
CacheNoLocalReplies uint8 = 1 << iota // not found in local network (stored in the cache so we skip another local lookup or some time)
)
// a cache entry
type cacheEntry struct {
Status entryStatus // status of the entry
Flags uint8 // some extra flags
ReplyLen int
question dns.Question
protocol dnsProtocol
reply dns.Msg
validUntil time.Time // obtained from the reply and stored here for convenience/speed
putTime time.Time
index int // for fast lookups in the heap
}
func newCacheEntry(question *dns.Question, now time.Time) *cacheEntry {
e := &cacheEntry{
Status: stPending,
validUntil: now.Add(time.Second * time.Duration(defPendingTimeout)),
question: *question,
index: -1,
}
return e
}
// Get a copy of the reply stored in the entry, but with some values adjusted like the TTL
func (e *cacheEntry) getReply(request *dns.Msg, maxLen int, now time.Time) (*dns.Msg, error) {
if e.Status != stResolved {
return nil, nil
}
// if the reply has expired or is invalid, force the caller to start a new resolution
if e.hasExpired(now) {
return nil, nil
}
if e.Flags&CacheNoLocalReplies != 0 {
return nil, errNoLocalReplies
}
if e.ReplyLen >= maxLen {
Debug.Printf("[cache msgid %d] returning truncated reponse: %d > %d", request.MsgHdr.Id, e.ReplyLen, maxLen)
return makeTruncatedReply(request), nil
}
// create a copy of the reply, with values for this particular query
reply := e.reply.Copy()
reply.SetReply(request)
// adjust the TTLs
passedSecs := uint32(now.Sub(e.putTime).Seconds())
for _, rr := range reply.Answer {
hdr := rr.Header()
ttl := hdr.Ttl
if passedSecs < ttl {
hdr.Ttl = ttl - passedSecs
} else {
return nil, nil // it is expired: do not spend more time and return nil...
}
}
reply.Rcode = e.reply.Rcode
reply.Authoritative = true
return reply, nil
}
func (e cacheEntry) hasExpired(now time.Time) bool {
return e.validUntil.Before(now) || e.validUntil == now
}
// set the reply for the entry
// returns True if the entry has changed the validUntil time
func (e *cacheEntry) setReply(reply *dns.Msg, ttl int, flags uint8, now time.Time) bool {
var prevValidUntil time.Time
if e.Status == stResolved {
if reply != nil {
Debug.Printf("[cache msgid %d] replacing response in cache", reply.MsgHdr.Id)
}
prevValidUntil = e.validUntil
}
// make sure we do not overwrite noLocalReplies entries
if flags&CacheNoLocalReplies != 0 {
if e.Flags&CacheNoLocalReplies != 0 {
return false
}
}
if ttl != nullTTL {
e.validUntil = now.Add(time.Second * time.Duration(ttl))
} else if reply != nil {
// calculate the validUntil from the reply TTL
var minTTL uint32 = math.MaxUint32
for _, rr := range reply.Answer {
ttl := rr.Header().Ttl
if ttl < minTTL {
minTTL = ttl // TODO: improve the minTTL calculation (maybe we should skip some RRs)
}
}
e.validUntil = now.Add(time.Second * time.Duration(minTTL))
} else {
Warning.Printf("[cache] no valid TTL could be calculated")
}
e.Status = stResolved
e.Flags = flags
e.putTime = now
if reply != nil {
e.reply = *reply.Copy()
e.ReplyLen = reply.Len()
}
return (prevValidUntil != e.validUntil)
}
func (e *cacheEntry) String() string {
var buf bytes.Buffer
q := e.question
fmt.Fprintf(&buf, "'%s'[%s]: ", q.Name, dns.TypeToString[q.Qtype])
if e.Flags&CacheNoLocalReplies != 0 {
fmt.Fprintf(&buf, "neg-local")
} else {
fmt.Fprintf(&buf, "%s", statusToString[e.Status])
}
fmt.Fprintf(&buf, "(%d bytes)", e.ReplyLen)
return buf.String()
}
//////////////////////////////////////////////////////////////////////////////////////
// An entriesPtrHeap is a min-heap of cache entries.
type entriesPtrsHeap []*cacheEntry
func (h entriesPtrsHeap) Len() int { return len(h) }
func (h entriesPtrsHeap) Less(i, j int) bool { return h[i].validUntil.Before(h[j].validUntil) }
func (h entriesPtrsHeap) Swap(i, j int) {
h[i], h[j] = h[j], h[i]
h[i].index = i
h[j].index = j
}
func (h *entriesPtrsHeap) Push(x interface{}) {
// Push and Pop use pointer receivers because they modify the slice's length,
// not just its contents.
n := len(*h)
entry := x.(*cacheEntry)
entry.index = n
*h = append(*h, entry)
}
func (h *entriesPtrsHeap) Pop() interface{} {
old := *h
n := len(old)
item := old[n-1]
item.index = -1 // for safety
*h = old[0 : n-1]
return item
}
//////////////////////////////////////////////////////////////////////////////////////
// The cache interface
type ZoneCache interface {
// Get from the cache
Get(request *dns.Msg, maxLen int) (reply *dns.Msg, err error)
// Put in the cache
Put(request *dns.Msg, reply *dns.Msg, ttl int, flags uint8) int
// Remove from the cache
Remove(question *dns.Question)
// Purge all expired entries from the cache
Purge()
// Remove all entries
Clear()
// Return the cache length
Len() int
// Return the max capacity
Capacity() int
}
type cacheKey dns.Question
type entries map[cacheKey]*cacheEntry
// Cache is a thread-safe fixed capacity LRU cache.
type Cache struct {
capacity int
entries entries
entriesH entriesPtrsHeap // len(entriesH) <= len(entries), as pending entries can be in entries but not in entriesH
clock clock.Clock
lock sync.RWMutex
}
// NewCache creates a cache of the given capacity
func NewCache(capacity int, clk clock.Clock) (*Cache, error) {
if capacity <= 0 {
return nil, errInvalidCapacity
}
c := &Cache{
capacity: capacity,
entries: make(entries, capacity),
clock: clk,
}
if c.clock == nil {
c.clock = clock.New()
}
heap.Init(&c.entriesH)
return c, nil
}
// Clear removes all the entries in the cache
func (c *Cache) Clear() {
c.lock.Lock()
defer c.lock.Unlock()
c.entries = make(entries, c.capacity)
heap.Init(&c.entriesH)
}
// Purge removes the old elements in the cache
func (c *Cache) Purge() {
c.lock.Lock()
defer c.lock.Unlock()
now := c.clock.Now()
for i, entry := range c.entriesH {
if entry.hasExpired(now) {
heap.Remove(&c.entriesH, i)
delete(c.entries, cacheKey(entry.question))
} else {
return // all remaining entries must be still valid...
}
}
}
// Add adds a reply to the cache.
// When `ttl` is equal to `nullTTL`, the cache entry will be valid until the closest TTL in the `reply`
func (c *Cache) Put(request *dns.Msg, reply *dns.Msg, ttl int, flags uint8) int {
c.lock.Lock()
defer c.lock.Unlock()
now := c.clock.Now()
question := request.Question[0]
key := cacheKey(question)
ent, found := c.entries[key]
if found {
updated := ent.setReply(reply, ttl, flags, now)
if updated {
heap.Fix(&c.entriesH, ent.index)
}
} else {
// If we will add a new item and the capacity has been exceeded, make some room...
if len(c.entriesH) >= c.capacity {
lowestEntry := heap.Pop(&c.entriesH).(*cacheEntry)
delete(c.entries, cacheKey(lowestEntry.question))
}
ent = newCacheEntry(&question, now)
ent.setReply(reply, ttl, flags, now)
heap.Push(&c.entriesH, ent)
c.entries[key] = ent
}
return ent.ReplyLen
}
// Look up for a question's reply from the cache.
// If no reply is stored in the cache, it returns a `nil` reply and no error.
func (c *Cache) Get(request *dns.Msg, maxLen int) (reply *dns.Msg, err error) {
c.lock.Lock()
defer c.lock.Unlock()
now := c.clock.Now()
question := request.Question[0]
key := cacheKey(question)
if ent, found := c.entries[key]; found {
reply, err = ent.getReply(request, maxLen, now)
if ent.hasExpired(now) {
Debug.Printf("[cache msgid %d] expired: removing", request.MsgHdr.Id)
if ent.index > 0 {
heap.Remove(&c.entriesH, ent.index)
}
delete(c.entries, key)
reply = nil
}
} else {
// we are the first asking for this name: create an entry with no reply... the caller must wait
Debug.Printf("[cache msgid %d] addind in pending state", request.MsgHdr.Id)
c.entries[key] = newCacheEntry(&question, now)
}
return
}
// Remove removes the provided question from the cache.
func (c *Cache) Remove(question *dns.Question) {
c.lock.Lock()
defer c.lock.Unlock()
key := cacheKey(*question)
if entry, found := c.entries[key]; found {
Debug.Printf("[cache] removing %s-response for '%s'", dns.TypeToString[question.Qtype], question.Name)
if entry.index > 0 {
heap.Remove(&c.entriesH, entry.index)
}
delete(c.entries, key)
}
}
// Len returns the number of entries in the cache.
func (c *Cache) Len() int {
c.lock.RLock()
defer c.lock.RUnlock()
return len(c.entries)
}
// Return the max capacity
func (c *Cache) Capacity() int {
c.lock.RLock()
defer c.lock.RUnlock()
return c.capacity
}
// Return the max capacity
func (c *Cache) String() string {
c.lock.RLock()
defer c.lock.RUnlock()
var buf bytes.Buffer
for _, entry := range c.entries {
fmt.Fprintf(&buf, "%s\n", entry)
}
return buf.String()
}
-230
View File
@@ -1,230 +0,0 @@
package nameserver
import (
"fmt"
"net"
"testing"
"time"
"github.com/benbjohnson/clock"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
wt "github.com/weaveworks/weave/testing"
)
// Check that the cache keeps its intended capacity constant
func TestCacheLength(t *testing.T) {
InitDefaultLogging(testing.Verbose())
Info.Println("TestCacheLength starting")
const cacheLen = 128
l, err := NewCache(cacheLen, nil)
wt.AssertNoErr(t, err)
insTime := time.Now()
t.Logf("Inserting 256 questions in the cache at '%s', with TTL from 0 to 255", insTime)
for i := 0; i < cacheLen*2; i++ {
name := fmt.Sprintf("name%d", i)
questionMsg := new(dns.Msg)
questionMsg.SetQuestion(name, dns.TypeA)
questionMsg.RecursionDesired = true
question := &questionMsg.Question[0]
ip := net.ParseIP(fmt.Sprintf("10.0.1.%d", i))
records := []ZoneRecord{Record{name, ip, 0, 0, 0}}
reply := makeAddressReply(questionMsg, question, records, DefaultLocalTTL)
reply.Answer[0].Header().Ttl = uint32(i)
l.Put(questionMsg, reply, 0, 0)
}
wt.AssertEqualInt(t, l.Len(), cacheLen, "cache length")
minExpectedTime := insTime.Add(time.Duration(cacheLen) * time.Second)
t.Logf("Checking all remaining entries expire after insert_time + %d secs='%s'", cacheLen, minExpectedTime)
for _, entry := range l.entries {
if entry.validUntil.Before(minExpectedTime) {
t.Fatalf("Entry valid until %s", entry.validUntil)
}
}
}
// Check that the cache entries are ok
func TestCacheEntries(t *testing.T) {
InitDefaultLogging(testing.Verbose())
Info.Println("TestCacheEntries starting")
Info.Println("Checking cache consistency")
const cacheLen = 128
clk := clock.NewMock()
l, err := NewCache(cacheLen, clk)
wt.AssertNoErr(t, err)
questionMsg := new(dns.Msg)
questionMsg.SetQuestion("some.name", dns.TypeA)
questionMsg.RecursionDesired = true
question := &questionMsg.Question[0]
t.Logf("Trying to get a name")
resp, err := l.Get(questionMsg, minUDPSize)
wt.AssertNoErr(t, err)
if resp != nil {
t.Logf("Got\n%s", resp)
t.Fatalf("ERROR: Did not expect a reponse from Get() yet")
}
t.Logf("Trying to get it again")
resp, err = l.Get(questionMsg, minUDPSize)
wt.AssertNoErr(t, err)
if resp != nil {
t.Logf("Got\n%s", resp)
t.Fatalf("ERROR: Did not expect a reponse from Get() yet")
}
t.Logf("Inserting the reply")
records := []ZoneRecord{Record{"some.name", net.ParseIP("10.0.1.1"), 0, 0, 0}}
reply1 := makeAddressReply(questionMsg, question, records, DefaultLocalTTL)
l.Put(questionMsg, reply1, nullTTL, 0)
t.Logf("Checking we can Get() the reply now")
resp, err = l.Get(questionMsg, minUDPSize)
wt.AssertNoErr(t, err)
wt.AssertTrue(t, resp != nil, "reponse from Get()")
t.Logf("Received '%s'", resp.Answer[0])
wt.AssertType(t, resp.Answer[0], (*dns.A)(nil), "DNS record")
ttlGet1 := resp.Answer[0].Header().Ttl
clk.Add(time.Duration(1) * time.Second)
t.Logf("Checking that a second Get(), after 1 second, gets the same result, but with reduced TTL")
resp, err = l.Get(questionMsg, minUDPSize)
wt.AssertNoErr(t, err)
wt.AssertTrue(t, resp != nil, "reponse from a second Get()")
t.Logf("Received '%s'", resp.Answer[0])
wt.AssertType(t, resp.Answer[0], (*dns.A)(nil), "DNS record")
ttlGet2 := resp.Answer[0].Header().Ttl
wt.AssertEqualInt(t, int(ttlGet1-ttlGet2), 1, "TTL difference")
clk.Add(time.Duration(DefaultLocalTTL) * time.Second)
t.Logf("Checking that a third Get(), after %d second, gets no result", DefaultLocalTTL)
resp, err = l.Get(questionMsg, minUDPSize)
wt.AssertNoErr(t, err)
if resp != nil {
t.Logf("Got\n%s", resp)
t.Fatalf("ERROR: Did NOT expect a reponse from the second Get()")
}
t.Logf("Checking that an Remove() results in Get() returning nothing")
records = []ZoneRecord{Record{"some.name", net.ParseIP("10.0.9.9"), 0, 0, 0}}
replyTemp := makeAddressReply(questionMsg, question, records, DefaultLocalTTL)
l.Put(questionMsg, replyTemp, nullTTL, 0)
lenBefore := l.Len()
l.Remove(question)
wt.AssertEqualInt(t, l.Len(), lenBefore-1, "cache length")
l.Remove(question) // do it again: should have no effect...
wt.AssertEqualInt(t, l.Len(), lenBefore-1, "cache length")
resp, err = l.Get(questionMsg, minUDPSize)
wt.AssertNoErr(t, err)
wt.AssertTrue(t, resp == nil, "reponse from the Get() after a Remove()")
t.Logf("Inserting a two replies for the same query")
records = []ZoneRecord{Record{"some.name", net.ParseIP("10.0.1.2"), 0, 0, 0}}
reply2 := makeAddressReply(questionMsg, question, records, DefaultLocalTTL)
l.Put(questionMsg, reply2, nullTTL, 0)
clk.Add(time.Duration(1) * time.Second)
records = []ZoneRecord{Record{"some.name", net.ParseIP("10.0.1.3"), 0, 0, 0}}
reply3 := makeAddressReply(questionMsg, question, records, DefaultLocalTTL)
l.Put(questionMsg, reply3, nullTTL, 0)
t.Logf("Checking we get the last one...")
resp, err = l.Get(questionMsg, minUDPSize)
wt.AssertNoErr(t, err)
wt.AssertTrue(t, resp != nil, "reponse from the Get()")
t.Logf("Received '%s'", resp.Answer[0])
wt.AssertType(t, resp.Answer[0], (*dns.A)(nil), "DNS record")
wt.AssertEqualString(t, resp.Answer[0].(*dns.A).A.String(), "10.0.1.3", "IP address")
wt.AssertEqualInt(t, int(resp.Answer[0].Header().Ttl), int(DefaultLocalTTL), "TTL")
clk.Add(time.Duration(DefaultLocalTTL-1) * time.Second)
resp, err = l.Get(questionMsg, minUDPSize)
wt.AssertNoErr(t, err)
wt.AssertTrue(t, resp != nil, "reponse from the Get()")
t.Logf("Received '%s'", resp.Answer[0])
wt.AssertType(t, resp.Answer[0], (*dns.A)(nil), "DNS record")
wt.AssertEqualString(t, resp.Answer[0].(*dns.A).A.String(), "10.0.1.3", "IP address")
wt.AssertEqualInt(t, int(resp.Answer[0].Header().Ttl), 1, "TTL")
t.Logf("Checking we get empty replies when they are expired...")
lenBefore = l.Len()
clk.Add(time.Duration(DefaultLocalTTL) * time.Second)
resp, err = l.Get(questionMsg, minUDPSize)
wt.AssertNoErr(t, err)
if resp != nil {
t.Logf("Got\n%s", resp.Answer[0])
t.Fatalf("ERROR: Did NOT expect a reponse from the Get()")
}
wt.AssertEqualInt(t, l.Len(), lenBefore-1, "cache length (after getting an expired entry)")
questionMsg2 := new(dns.Msg)
questionMsg2.SetQuestion("some.other.name", dns.TypeA)
questionMsg2.RecursionDesired = true
question2 := &questionMsg2.Question[0]
t.Logf("Trying to Get() a name")
resp, err = l.Get(questionMsg2, minUDPSize)
wt.AssertNoErr(t, err)
wt.AssertNil(t, resp, "reponse from Get() yet")
t.Logf("Checking that an Remove() between Get() and Put() does not break things")
records = []ZoneRecord{Record{"some.name", net.ParseIP("10.0.9.9"), 0, 0, 0}}
replyTemp2 := makeAddressReply(questionMsg2, question2, records, DefaultLocalTTL)
l.Remove(question2)
l.Put(questionMsg2, replyTemp2, nullTTL, 0)
resp, err = l.Get(questionMsg2, minUDPSize)
wt.AssertNoErr(t, err)
wt.AssertNotNil(t, resp, "reponse from Get()")
questionMsg3 := new(dns.Msg)
questionMsg3.SetQuestion("some.other.name", dns.TypeA)
questionMsg3.RecursionDesired = true
question3 := &questionMsg3.Question[0]
t.Logf("Checking that a entry with CacheNoLocalReplies return an error")
l.Put(questionMsg3, nil, DefaultLocalTTL, CacheNoLocalReplies)
resp, err = l.Get(questionMsg3, minUDPSize)
wt.AssertNil(t, resp, "Get() response with CacheNoLocalReplies")
wt.AssertTrue(t, err == errNoLocalReplies, "Get() error with CacheNoLocalReplies")
t.Logf("Checking that more Put's do not make the negative TTL longer")
clk.Add(time.Duration(DefaultLocalTTL/2) * time.Second)
l.Put(questionMsg3, nil, DefaultLocalTTL, CacheNoLocalReplies)
clk.Add(time.Duration(1) * time.Second)
l.Put(questionMsg3, nil, DefaultLocalTTL, CacheNoLocalReplies)
clk.Add(time.Duration(1) * time.Second)
l.Put(questionMsg3, nil, DefaultLocalTTL, CacheNoLocalReplies)
resp, err = l.Get(questionMsg3, minUDPSize)
wt.AssertNil(t, resp, "Get() response with CacheNoLocalReplies")
wt.AssertTrue(t, err == errNoLocalReplies, "Get() error with CacheNoLocalReplies")
clk.Add(time.Duration(DefaultLocalTTL/2) * time.Second)
t.Logf("Checking that we get an expired response after %f seconds", DefaultLocalTTL)
clk.Add(time.Second * time.Duration(DefaultLocalTTL+1))
t.Logf("Checking that we get an expired response after %d seconds", DefaultLocalTTL)
resp, err = l.Get(questionMsg3, minUDPSize)
wt.AssertNil(t, resp, "expired Get() response with CacheNoLocalReplies")
wt.AssertNil(t, err, "expired Get() error with CacheNoLocalReplies")
l.Remove(question3)
t.Logf("Checking that Put&Get with CacheNoLocalReplies with a Remove in the middle returns nothing")
l.Put(questionMsg3, nil, nullTTL, CacheNoLocalReplies)
l.Remove(question3)
resp, err = l.Get(questionMsg3, minUDPSize)
wt.AssertNil(t, resp, "Get() reponse with CacheNoLocalReplies")
wt.AssertNil(t, err, "Get() error with CacheNoLocalReplies")
}
+267 -86
View File
@@ -1,117 +1,298 @@
package nameserver
import (
"bytes"
"fmt"
"math/rand"
"net"
"sort"
"strings"
"time"
"github.com/miekg/dns"
"github.com/weaveworks/weave/net/address"
)
const (
minUDPSize = 512
maxUDPSize = 65535
topDomain = "."
reverseDNSdomain = "in-addr.arpa."
etcResolvConf = "/etc/resolv.conf"
udpBuffSize = uint16(4096)
minUDPSize = 512
DefaultListenAddress = "0.0.0.0:53"
DefaultTTL = 1
DefaultClientTimeout = 5 * time.Second
)
func makeHeader(r *dns.Msg, q *dns.Question, ttl int) *dns.RR_Header {
return &dns.RR_Header{
Name: q.Name,
Rrtype: q.Qtype,
type DNSServer struct {
ns *Nameserver
domain string
ttl uint32
address string
servers []*dns.Server
upstream *dns.ClientConfig
tcpClient *dns.Client
udpClient *dns.Client
}
func filter(ss []string, s string) []string {
for i := 0; i < len(ss); {
if ss[i] == s {
ss = append(ss[:i], ss[i+1:]...)
continue
}
i++
}
return ss
}
func NewDNSServer(ns *Nameserver, domain, address, effectiveAddress string, ttl uint32, clientTimeout time.Duration) (*DNSServer, error) {
s := &DNSServer{
ns: ns,
domain: dns.Fqdn(domain),
ttl: ttl,
address: address,
tcpClient: &dns.Client{Net: "tcp", ReadTimeout: clientTimeout},
udpClient: &dns.Client{Net: "udp", ReadTimeout: clientTimeout, UDPSize: udpBuffSize},
}
var err error
if s.upstream, err = dns.ClientConfigFromFile(etcResolvConf); err != nil {
return nil, err
}
if s.upstream != nil {
s.upstream.Servers = filter(s.upstream.Servers, effectiveAddress)
}
err = s.listen(address)
return s, err
}
func (d *DNSServer) String() string {
var buf bytes.Buffer
fmt.Fprintf(&buf, "WeaveDNS (%s)\n", d.ns.ourName)
fmt.Fprintf(&buf, " listening on %s, for domain %s\n", d.address, d.domain)
fmt.Fprintf(&buf, " response ttl %d\n", d.ttl)
return buf.String()
}
func (d *DNSServer) listen(address string) error {
udpListener, err := net.ListenPacket("udp", address)
if err != nil {
return err
}
udpServer := &dns.Server{PacketConn: udpListener, Handler: d.createMux(d.udpClient, minUDPSize)}
tcpListener, err := net.Listen("tcp", address)
if err != nil {
udpServer.Shutdown()
return err
}
tcpServer := &dns.Server{Listener: tcpListener, Handler: d.createMux(d.tcpClient, -1)}
d.servers = []*dns.Server{udpServer, tcpServer}
return nil
}
func (d *DNSServer) ActivateAndServe() {
for _, server := range d.servers {
go func(server *dns.Server) {
server.ActivateAndServe()
}(server)
}
}
func (d *DNSServer) Stop() error {
for _, server := range d.servers {
if err := server.Shutdown(); err != nil {
return err
}
}
return nil
}
type handler struct {
*DNSServer
maxResponseSize int
client *dns.Client
}
func (d *DNSServer) createMux(client *dns.Client, defaultMaxResponseSize int) *dns.ServeMux {
m := dns.NewServeMux()
h := &handler{
DNSServer: d,
maxResponseSize: defaultMaxResponseSize,
client: client,
}
m.HandleFunc(d.domain, h.handleLocal)
m.HandleFunc(reverseDNSdomain, h.handleReverse)
m.HandleFunc(topDomain, h.handleRecursive)
return m
}
func (h *handler) handleLocal(w dns.ResponseWriter, req *dns.Msg) {
h.ns.debugf("local request: %+v", *req)
if len(req.Question) != 1 || req.Question[0].Qtype != dns.TypeA {
h.nameError(w, req)
return
}
hostname := dns.Fqdn(req.Question[0].Name)
if strings.Count(hostname, ".") == 1 {
hostname = hostname + h.domain
}
addrs := h.ns.Lookup(hostname)
if len(addrs) == 0 {
h.nameError(w, req)
return
}
header := dns.RR_Header{
Name: req.Question[0].Name,
Rrtype: dns.TypeA,
Class: dns.ClassINET,
Ttl: uint32(ttl),
Ttl: h.ttl,
}
}
func makeReply(r *dns.Msg, as []dns.RR) *dns.Msg {
m := new(dns.Msg)
m.SetReply(r)
m.RecursionAvailable = true
m.Answer = as
return m
}
func makeTruncatedReply(r *dns.Msg) *dns.Msg {
// for truncated response, we create a minimal reply with the Truncated bit set
reply := new(dns.Msg)
reply.SetReply(r)
reply.Truncated = true
return reply
}
type DNSResponseBuilder func(r *dns.Msg, q *dns.Question, addrs []ZoneRecord, ttl int) *dns.Msg
func makeAddressReply(r *dns.Msg, q *dns.Question, addrs []ZoneRecord, ttl int) *dns.Msg {
answers := make([]dns.RR, len(addrs))
header := makeHeader(r, q, ttl)
count := 0
for _, addr := range addrs {
ip := addr.IP()
ip4 := ip.To4()
switch q.Qtype {
case dns.TypeA:
if ip4 != nil {
answers[count] = &dns.A{Hdr: *header, A: ip}
count++
}
case dns.TypeAAAA:
if ip4 == nil {
answers[count] = &dns.AAAA{Hdr: *header, AAAA: ip}
count++
}
}
for i, addr := range addrs {
ip := addr.IP4()
answers[i] = &dns.A{Hdr: header, A: ip}
}
return makeReply(r, answers[:count])
shuffleAnswers(&answers)
h.respond(w, h.makeResponse(req, answers))
}
func makePTRReply(r *dns.Msg, q *dns.Question, names []ZoneRecord, ttl int) *dns.Msg {
answers := make([]dns.RR, len(names))
header := makeHeader(r, q, ttl)
for i, name := range names {
answers[i] = &dns.PTR{Hdr: *header, Ptr: name.Name()}
func (h *handler) handleReverse(w dns.ResponseWriter, req *dns.Msg) {
h.ns.debugf("reverse request: %+v", *req)
if len(req.Question) != 1 || req.Question[0].Qtype != dns.TypePTR {
h.nameError(w, req)
return
}
return makeReply(r, answers)
}
func makeDNSFailResponse(r *dns.Msg) *dns.Msg {
m := new(dns.Msg)
m.SetReply(r)
m.RecursionAvailable = true
m.Rcode = dns.RcodeNameError
return m
}
// get the maximum UDP-reply length
func getMaxReplyLen(r *dns.Msg, proto dnsProtocol) int {
maxLen := minUDPSize
if proto == protTCP {
maxLen = maxUDPSize
} else if opt := r.IsEdns0(); opt != nil {
maxLen = int(opt.UDPSize())
ipStr := strings.TrimSuffix(req.Question[0].Name, "."+reverseDNSdomain)
ip, err := address.ParseIP(ipStr)
if err != nil {
h.nameError(w, req)
return
}
return maxLen
hostname, err := h.ns.ReverseLookup(ip.Reverse())
if err != nil {
h.handleRecursive(w, req)
return
}
header := dns.RR_Header{
Name: req.Question[0].Name,
Rrtype: dns.TypePTR,
Class: dns.ClassINET,
Ttl: h.ttl,
}
answers := []dns.RR{&dns.PTR{
Hdr: header,
Ptr: hostname,
}}
h.respond(w, h.makeResponse(req, answers))
}
func init() {
rand.Seed(time.Now().UTC().UnixNano())
}
func (h *handler) handleRecursive(w dns.ResponseWriter, req *dns.Msg) {
h.ns.debugf("recursive request: %+v", *req)
// shuffleAnswers reorders answers for very basic load balancing
func shuffleAnswers(answers []dns.RR) []dns.RR {
if len(answers) > 1 {
for i := range answers {
j := rand.Intn(i + 1)
answers[i], answers[j] = answers[j], answers[i]
// Resolve unqualified names locally
if len(req.Question) == 1 && req.Question[0].Qtype == dns.TypeA {
hostname := dns.Fqdn(req.Question[0].Name)
if strings.Count(hostname, ".") == 1 {
h.handleLocal(w, req)
return
}
}
return answers
for _, server := range h.upstream.Servers {
reqCopy := req.Copy()
reqCopy.Id = dns.Id()
response, _, err := h.client.Exchange(reqCopy, fmt.Sprintf("%s:%s", server, h.upstream.Port))
if (err != nil && err != dns.ErrTruncated) || response == nil {
h.ns.debugf("error trying %s: %v", server, err)
continue
}
response.Id = req.Id
if h.responseTooBig(req, response) {
response.Compress = true
}
h.respond(w, response)
return
}
h.respond(w, h.makeErrorResponse(req, dns.RcodeServerFailure))
}
// only take the first `num` answers
func pruneAnswers(answers []dns.RR, num int) []dns.RR {
if num > 0 && len(answers) > num {
// TODO: we should have some prefer locally-introduced answers, etc...
return answers[:num]
func (h *handler) makeResponse(req *dns.Msg, answers []dns.RR) *dns.Msg {
response := &dns.Msg{}
response.SetReply(req)
response.RecursionAvailable = true
response.Authoritative = true
response.Answer = answers
if !h.responseTooBig(req, response) {
return response
}
// search for smallest i that is too big
maxSize := h.getMaxResponseSize(req)
i := sort.Search(len(answers), func(i int) bool {
// return true if too big
response.Answer = answers[:i+1]
return response.Len() > maxSize
})
response.Answer = answers[:i]
if i < len(answers) {
response.Truncated = true
}
return response
}
func (h *handler) makeErrorResponse(req *dns.Msg, code int) *dns.Msg {
response := &dns.Msg{}
response.SetReply(req)
response.RecursionAvailable = true
response.Rcode = code
return response
}
func (h *handler) responseTooBig(req, response *dns.Msg) bool {
return len(response.Answer) > 1 && h.maxResponseSize > 0 && response.Len() > h.getMaxResponseSize(req)
}
func (h *handler) respond(w dns.ResponseWriter, response *dns.Msg) {
h.ns.debugf("response: %+v", response)
if err := w.WriteMsg(response); err != nil {
h.ns.infof("error responding: %v", err)
}
}
func (h *handler) nameError(w dns.ResponseWriter, req *dns.Msg) {
h.respond(w, h.makeErrorResponse(req, dns.RcodeNameError))
}
func (h *handler) getMaxResponseSize(req *dns.Msg) int {
if opt := req.IsEdns0(); opt != nil {
return int(opt.UDPSize())
}
return h.maxResponseSize
}
func shuffleAnswers(answers *[]dns.RR) {
if len(*answers) <= 1 {
return
}
for i := range *answers {
j := rand.Intn(i + 1)
(*answers)[i], (*answers)[j] = (*answers)[j], (*answers)[i]
}
return answers
}
+156 -23
View File
@@ -1,39 +1,172 @@
package nameserver
import (
"fmt"
"math/rand"
"net"
"strconv"
"testing"
"time"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
wt "github.com/weaveworks/weave/testing"
"github.com/stretchr/testify/require"
"github.com/weaveworks/weave/mesh"
"github.com/weaveworks/weave/net/address"
)
// Check that we can prune an answer
func TestPrune(t *testing.T) {
InitDefaultLogging(testing.Verbose())
Info.Println("TestPrune starting")
func startServer(t *testing.T, upstream *dns.ClientConfig) (*DNSServer, *Nameserver, int, int) {
peername, err := mesh.PeerNameFromString("00:00:00:02:00:00")
require.Nil(t, err)
nameserver := New(peername, "", func(mesh.PeerName) bool { return true })
dnsserver, err := NewDNSServer(nameserver, "weave.local.", "0.0.0.0:0", "", 30, 5*time.Second)
require.Nil(t, err)
udpPort := dnsserver.servers[0].PacketConn.LocalAddr().(*net.UDPAddr).Port
tcpPort := dnsserver.servers[1].Listener.Addr().(*net.TCPAddr).Port
if upstream != nil {
dnsserver.upstream = upstream
}
go dnsserver.ActivateAndServe()
return dnsserver, nameserver, udpPort, tcpPort
}
questionMsg := new(dns.Msg)
questionMsg.SetQuestion("name", dns.TypeA)
questionMsg.RecursionDesired = true
question := &questionMsg.Question[0]
records := []ZoneRecord{
Record{"name", net.ParseIP("10.0.1.1"), 0, 0, 0},
Record{"name", net.ParseIP("10.0.1.2"), 0, 0, 0},
Record{"name", net.ParseIP("10.0.1.3"), 0, 0, 0},
Record{"name", net.ParseIP("10.0.1.4"), 0, 0, 0},
func TestTruncation(t *testing.T) {
//common.SetLogLevel("debug")
dnsserver, nameserver, udpPort, tcpPort := startServer(t, nil)
defer dnsserver.Stop()
// Add 100 mappings to nameserver
addrs := []address.Address{}
for i := address.Address(0); i < 100; i++ {
addrs = append(addrs, i)
nameserver.AddEntry("foo.weave.local.", "", mesh.UnknownPeerName, i)
}
reply := makeAddressReply(questionMsg, question, records, DefaultLocalTTL)
reply.Answer[0].Header().Ttl = DefaultLocalTTL
doRequest := func(client *dns.Client, request *dns.Msg, port int, expectedErr error) *dns.Msg {
request.SetQuestion("foo.weave.local.", dns.TypeA)
response, _, err := client.Exchange(request, fmt.Sprintf("127.0.0.1:%d", port))
require.Equal(t, expectedErr, err)
return response
}
pruned := pruneAnswers(reply.Answer, 1)
wt.AssertEqualInt(t, len(pruned), 1, "wrong number of answers")
// do a udp query, ensure we get a truncated response
{
udpClient := dns.Client{Net: "udp", UDPSize: minUDPSize}
response := doRequest(&udpClient, &dns.Msg{}, udpPort, dns.ErrTruncated)
require.True(t, response.MsgHdr.Truncated)
require.True(t, len(response.Answer) < 100)
}
pruned = pruneAnswers(reply.Answer, 2)
wt.AssertEqualInt(t, len(pruned), 2, "wrong number of answers")
// do a udp query with big size, ensure we don't get a truncated response
{
udpClient := dns.Client{Net: "udp", UDPSize: 65535}
request := &dns.Msg{}
request.SetEdns0(65535, false)
response := doRequest(&udpClient, request, udpPort, nil)
require.False(t, response.MsgHdr.Truncated)
require.Equal(t, len(response.Answer), 100)
}
pruned = pruneAnswers(reply.Answer, 0)
wt.AssertEqualInt(t, len(pruned), len(records), "wrong number of answers")
// do a tcp query, ensure we don't get a truncated response
{
tcpClient := dns.Client{Net: "tcp"}
response := doRequest(&tcpClient, &dns.Msg{}, tcpPort, nil)
require.False(t, response.MsgHdr.Truncated)
require.Equal(t, len(response.Answer), 100)
}
}
func TestTruncateResponse(t *testing.T) {
header := dns.RR_Header{
Name: "host.domain.com",
Rrtype: dns.TypePTR,
Class: dns.ClassINET,
Ttl: 30,
}
for i := 0; i < 10000; i++ {
// generate a random answer set
numAnswers := 40 + rand.Intn(200)
answers := make([]dns.RR, numAnswers)
for j := 0; j < numAnswers; j++ {
answers[j] = &dns.A{Hdr: header, A: address.Address(j).IP4()}
}
// pick a random max size, truncate response to that, check it
maxSize := 512 + rand.Intn(2*512)
h := handler{maxResponseSize: maxSize}
response := h.makeResponse(&dns.Msg{}, answers)
require.True(t, response.Len() <= maxSize)
}
}
func TestRecursiveCompress(t *testing.T) {
const (
hostname = "foo.example."
maxSize = 512
)
// Construct a response that is >512 when uncompressed, <512 when compressed
response := dns.Msg{}
response.Authoritative = true
response.Answer = []dns.RR{}
header := dns.RR_Header{
Name: hostname,
Rrtype: dns.TypeA,
Class: dns.ClassINET,
Ttl: 10,
}
for response.Len() <= maxSize {
ip := address.Address(rand.Uint32()).IP4()
response.Answer = append(response.Answer, &dns.A{Hdr: header, A: ip})
}
response.Compress = true
require.True(t, response.Len() <= maxSize)
// A dns server that returns the above response
var gotRequest = make(chan struct{}, 1)
handleRecursive := func(w dns.ResponseWriter, req *dns.Msg) {
gotRequest <- struct{}{}
require.Equal(t, req.Question[0].Name, hostname)
response.SetReply(req)
err := w.WriteMsg(&response)
require.Nil(t, err)
}
mux := dns.NewServeMux()
mux.HandleFunc(topDomain, handleRecursive)
udpListener, err := net.ListenPacket("udp", "0.0.0.0:0")
require.Nil(t, err)
udpServer := &dns.Server{PacketConn: udpListener, Handler: mux}
udpServerPort := udpListener.LocalAddr().(*net.UDPAddr).Port
go udpServer.ActivateAndServe()
defer udpServer.Shutdown()
// The weavedns server, pointed at the above server
dnsserver, _, udpPort, _ := startServer(t, &dns.ClientConfig{
Servers: []string{"127.0.0.1"},
Port: strconv.Itoa(udpServerPort),
Ndots: 1,
Timeout: 5,
Attempts: 2,
})
defer dnsserver.Stop()
// Now do lookup, check its what we expected.
// NB this doesn't really test golang's resolver behaves correctly, as I can't see
// a way to point golangs resolver at a specific hosts.
req := new(dns.Msg)
req.Id = dns.Id()
req.RecursionDesired = true
req.Question = make([]dns.Question, 1)
req.Question[0] = dns.Question{
Name: hostname,
Qtype: dns.TypeA,
Qclass: dns.ClassINET,
}
c := new(dns.Client)
res, _, err := c.Exchange(req, fmt.Sprintf("127.0.0.1:%d", udpPort))
require.Nil(t, err)
require.True(t, len(gotRequest) > 0)
require.True(t, res.Len() > maxSize)
}
+276
View File
@@ -0,0 +1,276 @@
package nameserver
import (
"bytes"
"encoding/gob"
"fmt"
"sort"
"strings"
"time"
"github.com/weaveworks/weave/mesh"
"github.com/weaveworks/weave/net/address"
)
var now = func() int64 { return time.Now().Unix() }
type Entry struct {
ContainerID string
Origin mesh.PeerName
Addr address.Address
Hostname string // as supplied
lHostname string // lowercased (not exported, so not encoded by gob)
Version int
Tombstone int64 // timestamp of when it was deleted
}
type Entries []Entry
type CaseSensitive Entries
type CaseInsensitive Entries
type SortableEntries interface {
sort.Interface
Get(i int) Entry
}
// Gossip messages are sorted in a case sensitive order...
func (es CaseSensitive) Len() int { return len(es) }
func (es CaseSensitive) Swap(i, j int) { es[i], es[j] = es[j], es[i] }
func (es CaseSensitive) Get(i int) Entry { return es[i] }
func (es CaseSensitive) Less(i, j int) bool { return es[i].less(&es[j]) }
// ... but we store entries in a case insensitive order.
func (es CaseInsensitive) Len() int { return len(es) }
func (es CaseInsensitive) Swap(i, j int) { es[i], es[j] = es[j], es[i] }
func (es CaseInsensitive) Get(i int) Entry { return es[i] }
func (es CaseInsensitive) Less(i, j int) bool { return es[i].insensitiveLess(&es[j]) }
func (e1 Entry) equal(e2 Entry) bool {
return e1.ContainerID == e2.ContainerID &&
e1.Origin == e2.Origin &&
e1.Addr == e2.Addr &&
e1.Hostname == e2.Hostname
}
func (e1 *Entry) less(e2 *Entry) bool {
// Entries are kept sorted by Hostname, Origin, ContainerID then address
switch {
case e1.Hostname != e2.Hostname:
return e1.Hostname < e2.Hostname
case e1.Origin != e2.Origin:
return e1.Origin < e2.Origin
case e1.ContainerID != e2.ContainerID:
return e1.ContainerID < e2.ContainerID
default:
return e1.Addr < e2.Addr
}
}
func (e1 *Entry) insensitiveLess(e2 *Entry) bool {
// Entries are kept sorted by Hostname, Origin, ContainerID then address
e1Hostname, e2Hostname := e1.lHostname, e2.lHostname
switch {
case e1Hostname != e2Hostname:
return e1Hostname < e2Hostname
case e1.Origin != e2.Origin:
return e1.Origin < e2.Origin
case e1.ContainerID != e2.ContainerID:
return e1.ContainerID < e2.ContainerID
default:
return e1.Addr < e2.Addr
}
}
// returns true to indicate a change
func (e1 *Entry) merge(e2 *Entry) bool {
// we know container id, origin, add and hostname are equal
if e2.Version > e1.Version {
e1.Version = e2.Version
e1.Tombstone = e2.Tombstone
return true
}
return false
}
func (e1 *Entry) String() string {
return fmt.Sprintf("%s -> %s", e1.Hostname, e1.Addr.String())
}
func (e1 *Entry) addLowercase() {
e1.lHostname = strings.ToLower(e1.Hostname)
}
func check(es SortableEntries) error {
if !sort.IsSorted(es) {
return fmt.Errorf("Not sorted!")
}
for i := 1; i < es.Len(); i++ {
if es.Get(i).equal(es.Get(i - 1)) {
return fmt.Errorf("Duplicate entry: %d:%v and %d:%v", i-1, es.Get(i-1), i, es.Get(i))
}
}
return nil
}
func checkAndPanic(es SortableEntries) {
if err := check(es); err != nil {
panic(err)
}
}
func (es *Entries) checkAndPanic() *Entries {
checkAndPanic(CaseInsensitive(*es))
return es
}
func (es *Entries) add(hostname, containerid string, origin mesh.PeerName, addr address.Address) Entry {
defer es.checkAndPanic().checkAndPanic()
entry := Entry{Hostname: hostname, lHostname: strings.ToLower(hostname),
Origin: origin, ContainerID: containerid, Addr: addr}
i := sort.Search(len(*es), func(i int) bool {
return !(*es)[i].insensitiveLess(&entry)
})
if i < len(*es) && (*es)[i].equal(entry) {
if (*es)[i].Tombstone > 0 {
(*es)[i].Tombstone = 0
(*es)[i].Version++
}
} else {
*es = append(*es, Entry{})
copy((*es)[i+1:], (*es)[i:])
(*es)[i] = entry
}
return (*es)[i]
}
func (es *Entries) merge(incoming Entries) Entries {
defer es.checkAndPanic().checkAndPanic()
incoming.checkAndPanic()
newEntries := Entries{}
i := 0
for _, entry := range incoming {
for i < len(*es) && (*es)[i].insensitiveLess(&entry) {
i++
}
if i < len(*es) && (*es)[i].equal(entry) {
if (*es)[i].merge(&entry) {
newEntries = append(newEntries, entry)
}
} else {
*es = append(*es, Entry{})
copy((*es)[i+1:], (*es)[i:])
(*es)[i] = entry
newEntries = append(newEntries, entry)
}
}
return newEntries
}
// f returning true means keep the entry.
func (es *Entries) tombstone(ourname mesh.PeerName, f func(*Entry) bool) Entries {
defer es.checkAndPanic().checkAndPanic()
tombstoned := Entries{}
for i, e := range *es {
if f(&e) && e.Origin == ourname {
e.Version++
e.Tombstone = now()
(*es)[i] = e
tombstoned = append(tombstoned, e)
}
}
return tombstoned
}
func (es *Entries) filter(f func(*Entry) bool) {
defer es.checkAndPanic().checkAndPanic()
i := 0
for _, e := range *es {
if !f(&e) {
continue
}
(*es)[i] = e
i++
}
*es = (*es)[:i]
}
func (es Entries) lookup(hostname string) Entries {
es.checkAndPanic()
lowerHostname := strings.ToLower(hostname)
i := sort.Search(len(es), func(i int) bool {
return es[i].lHostname >= lowerHostname
})
if i >= len(es) || es[i].lHostname != lowerHostname {
return Entries{}
}
j := sort.Search(len(es)-i, func(j int) bool {
return es[i+j].lHostname > lowerHostname
})
return es[i : i+j]
}
func (es Entries) first(f func(*Entry) bool) (*Entry, error) {
es.checkAndPanic()
for _, e := range es {
if f(&e) {
return &e, nil
}
}
return nil, fmt.Errorf("Not found")
}
func (es Entries) addLowercase() {
for i := range es {
es[i].addLowercase()
}
}
type GossipData struct {
Timestamp int64
Entries
}
func (g *GossipData) Merge(o mesh.GossipData) {
other := o.(*GossipData)
g.Entries.merge(other.Entries)
if g.Timestamp < other.Timestamp {
g.Timestamp = other.Timestamp
}
}
func (g *GossipData) Decode(msg []byte) error {
if err := gob.NewDecoder(bytes.NewReader(msg)).Decode(g); err != nil {
return err
}
g.Entries.addLowercase() // lowercase strings not sent on the wire
sort.Sort(CaseInsensitive(g.Entries))
return nil
}
func (g *GossipData) Encode() [][]byte {
// Make a copy so we can sort: all outgoing data is sent in case-sensitive order
g2 := GossipData{Timestamp: g.Timestamp, Entries: make(Entries, len(g.Entries))}
copy(g2.Entries, g.Entries)
sort.Sort(CaseSensitive(g2.Entries))
buf := &bytes.Buffer{}
if err := gob.NewEncoder(buf).Encode(g2); err != nil {
panic(err)
}
return [][]byte{buf.Bytes()}
}
+123
View File
@@ -0,0 +1,123 @@
package nameserver
import (
"testing"
"github.com/stretchr/testify/require"
"github.com/weaveworks/weave/mesh"
"github.com/weaveworks/weave/net/address"
)
func l(es Entries) Entries {
es.addLowercase()
return es
}
func makeEntries(values string) Entries {
entries := make(Entries, len(values))
for i, c := range values {
entries[i] = Entry{Hostname: string(c)}
}
return l(entries)
}
func TestAdd(t *testing.T) {
oldNow := now
defer func() { now = oldNow }()
now = func() int64 { return 1234 }
entries := Entries{}
entries.add("A", "", mesh.UnknownPeerName, address.Address(0))
expected := l(Entries{
Entry{Hostname: "A", Origin: mesh.UnknownPeerName, Addr: address.Address(0)},
})
require.Equal(t, entries, expected)
entries.tombstone(mesh.UnknownPeerName, func(e *Entry) bool { return e.Hostname == "A" })
expected = l(Entries{
Entry{Hostname: "A", Origin: mesh.UnknownPeerName, Addr: address.Address(0), Version: 1, Tombstone: 1234},
})
require.Equal(t, entries, expected)
entries.add("A", "", mesh.UnknownPeerName, address.Address(0))
expected = l(Entries{
Entry{Hostname: "A", Origin: mesh.UnknownPeerName, Addr: address.Address(0), Version: 2},
})
require.Equal(t, entries, expected)
}
func TestMerge(t *testing.T) {
e1 := makeEntries("ACDF")
e2 := makeEntries("BEF")
diff := e1.merge(e2)
require.Equal(t, makeEntries("BE"), diff)
require.Equal(t, makeEntries("ABCDEF"), e1)
diff = e1.merge(e1)
require.Equal(t, Entries{}, diff)
}
func TestOldMerge(t *testing.T) {
e1 := l(Entries{Entry{Hostname: "A", Version: 0}})
diff := e1.merge(l(Entries{Entry{Hostname: "A", Version: 1}}))
require.Equal(t, l(Entries{Entry{Hostname: "A", Version: 1}}), diff)
require.Equal(t, l(Entries{Entry{Hostname: "A", Version: 1}}), e1)
diff = e1.merge(l(Entries{Entry{Hostname: "A", Version: 0}}))
require.Equal(t, Entries{}, diff)
require.Equal(t, l(Entries{Entry{Hostname: "A", Version: 1}}), e1)
}
func TestTombstone(t *testing.T) {
oldNow := now
defer func() { now = oldNow }()
now = func() int64 { return 1234 }
es := makeEntries("AB")
es.tombstone(mesh.UnknownPeerName, func(e *Entry) bool {
return e.Hostname == "B"
})
expected := l(Entries{
Entry{Hostname: "A"},
Entry{Hostname: "B", Version: 1, Tombstone: 1234},
})
require.Equal(t, expected, es)
}
func TestDelete(t *testing.T) {
es := makeEntries("AB")
es.filter(func(e *Entry) bool {
return e.Hostname != "A"
})
require.Equal(t, makeEntries("B"), es)
}
func TestLookup(t *testing.T) {
es := l(Entries{
Entry{Hostname: "A"},
Entry{Hostname: "B", ContainerID: "bar"},
Entry{Hostname: "B", ContainerID: "foo"},
Entry{Hostname: "C"},
})
have := es.lookup("B")
want := l(Entries{
Entry{Hostname: "B", ContainerID: "bar"},
Entry{Hostname: "B", ContainerID: "foo"},
})
require.Equal(t, have, want)
}
func TestGossipDataMerge(t *testing.T) {
g1 := GossipData{Entries: makeEntries("AcDf")}
g2 := GossipData{Entries: makeEntries("BEf")}
g1.Merge(&g2)
require.Equal(t, GossipData{Entries: makeEntries("ABcDEf")}, g1)
}
+64 -90
View File
@@ -1,125 +1,99 @@
package nameserver
import (
"encoding/json"
"fmt"
"log"
"net"
"net/http"
"github.com/gorilla/mux"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/common/docker"
"github.com/weaveworks/weave/net/address"
)
func httpErrorAndLog(level *log.Logger, w http.ResponseWriter, msg string,
status int, logmsg string, logargs ...interface{}) {
http.Error(w, msg, status)
level.Printf("[http] "+logmsg, logargs...)
func (n *Nameserver) badRequest(w http.ResponseWriter, err error) {
http.Error(w, err.Error(), http.StatusBadRequest)
n.infof("%v", err)
}
func extractFQDN(r *http.Request) (string, string) {
if fqdnStr := r.FormValue("fqdn"); fqdnStr != "" {
return fqdnStr, fqdnStr
}
return "*", ""
}
func ServeHTTP(listener net.Listener, version string, server *DNSServer, dockerCli *docker.Client) {
muxRouter := mux.NewRouter()
muxRouter.Methods("GET").Path("/status").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "weave DNS", version)
fmt.Fprintln(w, server.Status())
fmt.Fprintln(w, server.Zone.Status())
func (n *Nameserver) HandleHTTP(router *mux.Router, dockerCli *docker.Client) {
router.Methods("GET").Path("/domain").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, n.domain)
})
muxRouter.Methods("GET").Path("/domain").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, server.Zone.Domain())
})
muxRouter.Methods("PUT").Path("/name/{id:.+}/{ip:.+}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
reqError := func(msg string, logmsg string, logargs ...interface{}) {
httpErrorAndLog(Warning, w, msg, http.StatusBadRequest, logmsg, logargs...)
}
vars := mux.Vars(r)
idStr := vars["id"]
ipStr := vars["ip"]
name := r.FormValue("fqdn")
if name == "" {
reqError("Invalid FQDN", "Invalid FQDN in request: %s, %s", r.URL, r.Form)
router.Methods("PUT").Path("/name/{container}/{ip}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var (
vars = mux.Vars(r)
container = vars["container"]
ipStr = vars["ip"]
hostname = dns.Fqdn(r.FormValue("fqdn"))
ip, err = address.ParseIP(ipStr)
)
if err != nil {
n.badRequest(w, err)
return
}
ip := net.ParseIP(ipStr)
if ip == nil {
reqError("Invalid IP", "Invalid IP in request: %s", ipStr)
if !dns.IsSubDomain(n.domain, hostname) {
n.infof("Ignoring registration %s %s %s (not a subdomain of %s)", hostname, ipStr, container, n.domain)
return
}
domain := server.Zone.Domain()
if !dns.IsSubDomain(domain, name) {
Info.Printf("[http] Ignoring name %s, not in %s", name, domain)
if err := n.AddEntry(hostname, container, n.ourName, ip); err != nil {
n.badRequest(w, fmt.Errorf("Unable to add entry: %v", err))
return
}
Info.Printf("[http] Adding %s -> %s", name, ipStr)
if err := server.Zone.AddRecord(idStr, name, ip); err != nil {
if _, ok := err.(DuplicateError); !ok {
httpErrorAndLog(
Error, w, "Internal error", http.StatusInternalServerError,
"Unexpected error from DB: %s", err)
return
} // oh, I already know this. whatever.
}
if dockerCli != nil && dockerCli.IsContainerNotRunning(idStr) {
Info.Printf("[http] '%s' is not running: removing", idStr)
server.Zone.DeleteRecords(idStr, name, ip)
if r.FormValue("check-alive") == "true" && dockerCli != nil && dockerCli.IsContainerNotRunning(container) {
n.infof("container '%s' is not running: removing", container)
if err := n.Delete(hostname, container, ipStr, ip); err != nil {
n.infof("failed to remove: %v", err)
}
}
})
muxRouter.Methods("DELETE").Path("/name/{id:.+}/{ip:.+}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
idStr := vars["id"]
ipStr := vars["ip"]
ip := net.ParseIP(ipStr)
if ip == nil {
httpErrorAndLog(
Warning, w, "Invalid IP in request", http.StatusBadRequest,
"Invalid IP in request: %s", ipStr)
return
}
fqdnStr, fqdn := extractFQDN(r)
Info.Printf("[http] Deleting ID %s, IP %s, FQDN %s", idStr, ipStr, fqdnStr)
server.Zone.DeleteRecords(idStr, fqdn, ip)
w.WriteHeader(204)
})
muxRouter.Methods("DELETE").Path("/name/{id:.+}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
deleteHandler := func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
idStr := vars["id"]
fqdnStr, fqdn := extractFQDN(r)
Info.Printf("[http] Deleting ID %s, IP *, FQDN %s", idStr, fqdnStr)
server.Zone.DeleteRecords(idStr, fqdn, net.IP{})
})
muxRouter.Methods("DELETE").Path("/name").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fqdnStr, fqdn := extractFQDN(r)
hostname := r.FormValue("fqdn")
if hostname == "" {
hostname = "*"
} else {
hostname = dns.Fqdn(hostname)
}
Info.Printf("[http] Deleting ID *, IP *, FQDN %s", fqdnStr)
server.Zone.DeleteRecords("", fqdn, net.IP{})
})
container, ok := vars["container"]
if !ok {
container = "*"
}
http.Handle("/", muxRouter)
ipStr, ok := vars["ip"]
ip, err := address.ParseIP(ipStr)
if ok && err != nil {
n.badRequest(w, err)
return
} else if !ok {
ipStr = "*"
}
if err := http.Serve(listener, nil); err != nil {
Error.Fatal("[http] Unable to serve http: ", err)
if err := n.Delete(hostname, container, ipStr, ip); err != nil {
n.badRequest(w, fmt.Errorf("Unable to delete entries: %v", err))
return
}
w.WriteHeader(204)
}
router.Methods("DELETE").Path("/name/{container}/{ip}").HandlerFunc(deleteHandler)
router.Methods("DELETE").Path("/name/{container}").HandlerFunc(deleteHandler)
router.Methods("DELETE").Path("/name").HandlerFunc(deleteHandler)
router.Methods("GET").Path("/name").Headers("Accept", "application/json").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
n.RLock()
defer n.RUnlock()
if err := json.NewEncoder(w).Encode(n.entries); err != nil {
n.badRequest(w, fmt.Errorf("Error marshalling response: %v", err))
}
})
}
-130
View File
@@ -1,130 +0,0 @@
package nameserver
import (
"fmt"
"net"
"net/http"
"net/url"
"strings"
"testing"
wt "github.com/weaveworks/weave/testing"
)
func genForm(method string, url string, data url.Values) (resp *http.Response, err error) {
req, err := http.NewRequest(method, url, strings.NewReader(data.Encode()))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
return http.DefaultClient.Do(req)
}
func TestHttp(t *testing.T) {
var (
containerID = "deadbeef"
testDomain = "weave.local."
successTestName = "test1." + testDomain
testAddr1 = "10.2.2.1/24"
testAddr2 = "10.2.2.2/24"
dockerIP = "9.8.7.6"
)
zone, err := NewZoneDb(ZoneConfig{Domain: testDomain})
wt.AssertNoErr(t, err)
err = zone.Start()
wt.AssertNoErr(t, err)
defer zone.Stop()
httpListener, err := net.Listen("tcp", ":0")
if err != nil {
t.Fatal("Unable to create http listener: ", err)
}
// Create a useless server... it will not even be started
srv, _ := NewDNSServer(DNSServerConfig{Zone: zone})
port := httpListener.Addr().(*net.TCPAddr).Port
go ServeHTTP(httpListener, "", srv, nil)
// Ask the http server to add our test address into the database
addrParts := strings.Split(testAddr1, "/")
addrURL := fmt.Sprintf("http://localhost:%d/name/%s/%s", port, containerID, addrParts[0])
resp, err := genForm("PUT", addrURL,
url.Values{"fqdn": {successTestName}, "local_ip": {dockerIP}, "routing_prefix": {addrParts[1]}})
wt.AssertNoErr(t, err)
wt.AssertStatus(t, resp.StatusCode, http.StatusOK, "http response")
// Check that the address is now there.
ip, _, _ := net.ParseCIDR(testAddr1)
foundIP, err := zone.LookupName(successTestName)
wt.AssertNoErr(t, err)
if !foundIP[0].IP().Equal(ip) {
t.Fatalf("Unexpected result for %s: received %s, expected %s", successTestName, foundIP, ip)
}
// Adding exactly the same address should be OK
resp, err = genForm("PUT", addrURL,
url.Values{"fqdn": {successTestName}, "local_ip": {dockerIP}, "routing_prefix": {addrParts[1]}})
wt.AssertNoErr(t, err)
wt.AssertStatus(t, resp.StatusCode, http.StatusOK, "http success response for duplicate add")
// Now try adding the same address again with a different ident --
// again should be fine
otherURL := fmt.Sprintf("http://localhost:%d/name/%s/%s", port, "other", addrParts[0])
resp, err = genForm("PUT", otherURL,
url.Values{"fqdn": {successTestName}, "local_ip": {dockerIP}, "routing_prefix": {addrParts[1]}})
wt.AssertNoErr(t, err)
wt.AssertStatus(t, resp.StatusCode, http.StatusOK, "http response")
// Adding a new IP for the same name should be OK
addrParts2 := strings.Split(testAddr2, "/")
addrURL2 := fmt.Sprintf("http://localhost:%d/name/%s/%s", port, containerID, addrParts2[0])
resp, err = genForm("PUT", addrURL2,
url.Values{"fqdn": {successTestName}, "local_ip": {dockerIP}, "routing_prefix": {addrParts2[1]}})
wt.AssertNoErr(t, err)
wt.AssertStatus(t, resp.StatusCode, http.StatusOK, "http success response for second IP")
// Check that we can get two IPs for that name
ip2, _, _ := net.ParseCIDR(testAddr2)
foundIP, err = zone.LookupName(successTestName)
wt.AssertNoErr(t, err)
if len(foundIP) != 2 {
t.Logf("IPs found: %s", foundIP)
t.Fatalf("Unexpected result length: received %d responses", len(foundIP))
}
if !(foundIP[0].IP().Equal(ip) || foundIP[0].IP().Equal(ip2)) {
t.Logf("IPs found: %s", foundIP)
t.Fatalf("Unexpected result for %s: received %s, expected %s", successTestName, foundIP, ip)
}
if !(foundIP[1].IP().Equal(ip) || foundIP[1].IP().Equal(ip2)) {
t.Logf("IPs found: %s", foundIP)
t.Fatalf("Unexpected result for %s: received %s, expected %s", successTestName, foundIP, ip)
}
// Delete the address
resp, err = genForm("DELETE", addrURL, nil)
wt.AssertNoErr(t, err)
wt.AssertStatus(t, resp.StatusCode, http.StatusOK, "http response")
// Check that the address is still resolvable.
x, err := zone.LookupName(successTestName)
t.Logf("Got %s", x)
wt.AssertNoErr(t, err)
// Delete the address record mentioning the other container
resp, err = genForm("DELETE", otherURL, nil)
wt.AssertNoErr(t, err)
wt.AssertStatus(t, resp.StatusCode, http.StatusOK, "http response")
// Delete the second IP
resp, err = genForm("DELETE", addrURL2, nil)
wt.AssertNoErr(t, err)
wt.AssertStatus(t, resp.StatusCode, http.StatusOK, "http response")
// Check that the address is gone
x, err = zone.LookupName(successTestName)
t.Logf("Got %s", x)
wt.AssertErrorType(t, err, (*LookupError)(nil), "fully-removed address")
// Would like to shut down the http server at the end of this test
// but it's complicated.
// See https://groups.google.com/forum/#!topic/golang-nuts/vLHWa5sHnCE
}
-28
View File
@@ -1,28 +0,0 @@
package nameserver
import "net"
// A basic mDNS service
type ZoneMDNS interface {
// Start the service
Start(*net.Interface) error
// Stop the service
Stop() error
}
// A mDNS server
type ZoneMDNSServer interface {
ZoneMDNS
// Return the Zone database used by the server
Zone() Zone
}
// A mDNS client interface
type ZoneMDNSClient interface {
ZoneMDNS
ZoneLookup
// Perform an insistent lookup for a name
InsistentLookupName(name string) ([]ZoneRecord, error)
// Perform an insistent lookup for a reverse address
InsistentLookupInaddr(inaddr string) ([]ZoneRecord, error)
}
-290
View File
@@ -1,290 +0,0 @@
package nameserver
import (
"bytes"
"fmt"
"math"
"net"
"time"
"github.com/miekg/dns"
)
// Portions of this code taken from github.com/armon/mdns
const (
ipv4mdns = "224.0.0.251" // link-local multicast address
mdnsPort = 5353 // mDNS assigned port
// We wait this long to hear a response from other mDNS servers on
// the network.
mDNSTimeout = 500 * time.Millisecond
MaxDuration = time.Duration(math.MaxInt64)
MailboxSize = 16
)
var (
ipv4Addr = &net.UDPAddr{
IP: net.ParseIP(ipv4mdns),
Port: mdnsPort,
}
)
type Response struct {
name string
addr net.IP
ttl int
err error
}
func (r Response) Name() string { return r.name }
func (r Response) IP() net.IP { return r.addr }
func (r Response) Priority() int { return 0 }
func (r Response) Weight() int { return 0 }
func (r Response) TTL() int { return r.ttl }
func (r Response) Equal(r2 *Response) bool {
if r.name != r2.name {
return false
}
if !r.addr.Equal(r2.addr) {
return false
}
if r.err != r2.err {
return false
}
return true
}
func (r Response) String() string {
var buf bytes.Buffer
if r.err != nil {
fmt.Fprintf(&buf, "%s", r.err)
} else {
if len(r.Name()) > 0 {
fmt.Fprintf(&buf, "%s", r.Name())
}
if !r.IP().IsUnspecified() {
fmt.Fprintf(&buf, "[%s]", r.IP())
}
if r.ttl > 0 {
fmt.Fprintf(&buf, "(TTL:%d)", r.TTL())
}
}
return buf.String()
}
type responseInfo struct {
timeout time.Time // if no answer by this time, give up
insistent bool // insistent queries are not removed on the first reply
ch chan<- *Response
}
// Represents one query that we have sent for one name.
// If we, internally, get several requests for the same name while we have
// a query in flight, then we don't want to send more queries out.
// Invariant on responseInfos: they are in ascending order of timeout.
type inflightQuery struct {
name string
id uint16 // the DNS message ID
responseInfos []*responseInfo
}
type MDNSClient struct {
// note unorthodox use of 'Server' class in client logic - using it to
// listen on a multicast socket and call callback whenever a message comes in.
listener *dns.Server
conn *net.UDPConn
addr *net.UDPAddr
inflight map[string]*inflightQuery
actionChan chan<- MDNSAction
}
type mDNSQueryInfo struct {
name string
querytype uint16
responseCh chan<- *Response
}
func NewMDNSClient() (*MDNSClient, error) {
return &MDNSClient{
addr: ipv4Addr,
inflight: make(map[string]*inflightQuery)}, nil
}
func (c *MDNSClient) Start(ifi *net.Interface) (err error) {
if c.conn, err = net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4zero, Port: 0}); err != nil {
return err
}
multicast, err := LinkLocalMulticastListener(ifi)
if err != nil {
return err
}
handleMDNS := func(w dns.ResponseWriter, r *dns.Msg) {
// Don't want to handle queries here, so filter anything out that isn't a response
if len(r.Answer) > 0 {
c.ResponseCallback(r)
}
}
actionChan := make(chan MDNSAction, MailboxSize)
c.actionChan = actionChan
go c.actorLoop(actionChan)
c.listener = &dns.Server{Unsafe: true, PacketConn: multicast, Handler: dns.HandlerFunc(handleMDNS)}
go c.listener.ActivateAndServe()
return nil
}
func (c *MDNSClient) Stop() error {
c.actionChan <- nil
return nil
}
func LinkLocalMulticastListener(ifi *net.Interface) (net.PacketConn, error) {
return net.ListenMulticastUDP("udp", ifi, ipv4Addr)
}
// ACTOR client API
type MDNSAction func()
// Async
func (c *MDNSClient) SendQuery(name string, querytype uint16, insistent bool, responseCh chan<- *Response) {
c.actionChan <- func() {
query, found := c.inflight[name]
if !found {
m := new(dns.Msg)
m.SetQuestion(name, querytype)
m.RecursionDesired = false
buf, err := m.Pack()
if err != nil {
responseCh <- &Response{err: err}
close(responseCh)
return
}
query = &inflightQuery{
name: name,
id: m.Id,
}
if _, err = c.conn.WriteTo(buf, c.addr); err != nil {
responseCh <- &Response{err: err}
close(responseCh)
return
}
c.inflight[name] = query
}
info := &responseInfo{
ch: responseCh,
timeout: time.Now().Add(mDNSTimeout),
insistent: insistent,
}
// Invariant on responseInfos: they are in ascending order of
// timeout. Since we use a fixed interval from Now(), this
// must be after all existing timeouts.
query.responseInfos = append(query.responseInfos, info)
}
}
// Async - called from dns library multiplexer
func (c *MDNSClient) ResponseCallback(r *dns.Msg) {
c.actionChan <- func() {
for _, answer := range r.Answer {
var name string
var res *Response
switch rr := answer.(type) {
case *dns.A:
name = rr.Hdr.Name
res = &Response{name: name, addr: rr.A, ttl: int(rr.Hdr.Ttl)}
case *dns.PTR:
name = rr.Hdr.Name
raddr, _ := raddrToIP(name)
res = &Response{name: rr.Ptr, addr: raddr, ttl: int(rr.Hdr.Ttl)}
default:
return
}
if query, found := c.inflight[name]; found {
var newResponseInfos []*responseInfo
for _, resp := range query.responseInfos {
resp.ch <- res
// insistent queries are not removed on the first reply, but on the timeout
if resp.insistent {
newResponseInfos = append(newResponseInfos, resp)
} else {
close(resp.ch)
}
}
if len(newResponseInfos) == 0 {
delete(c.inflight, name)
} else {
query.responseInfos = newResponseInfos
}
} else {
// We've received a response that didn't match a query
// Do we want to cache it?
}
}
}
}
// ACTOR server
// Check all in-flight queries, close all that have already timed out,
// and return the duration until the next timeout
func (c *MDNSClient) checkInFlightQueries() time.Duration {
now := time.Now()
after := MaxDuration
for name, query := range c.inflight {
// Invariant on responseInfos: they are in ascending order of timeout.
numClosed := 0
for _, item := range query.responseInfos {
duration := item.timeout.Sub(now)
if duration <= 0 { // timed out
close(item.ch)
numClosed++
} else {
if duration < after {
after = duration
}
break // don't need to look at any more for this query
}
}
// Remove timed-out items from the slice
query.responseInfos = query.responseInfos[numClosed:]
if len(query.responseInfos) == 0 {
delete(c.inflight, name)
}
}
return after
}
func (c *MDNSClient) actorLoop(actionChan <-chan MDNSAction) {
timer := time.NewTimer(MaxDuration)
run := func() { timer.Reset(c.checkInFlightQueries()) }
loop:
for {
select {
case action := <-actionChan:
if action == nil {
c.listener.Shutdown()
break loop
} else {
action()
run()
}
case <-timer.C:
run()
}
}
// Close all response channels at termination
for _, query := range c.inflight {
for _, item := range query.responseInfos {
close(item.ch)
}
}
}
-198
View File
@@ -1,198 +0,0 @@
package nameserver
import (
"log"
"net"
"testing"
"time"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
wt "github.com/weaveworks/weave/testing"
)
var (
successTestName = "test1.weave."
failTestName = "test2.weave."
testAddr = net.ParseIP("9.8.7.6")
testInAddr = "6.7.8.9.in-addr.arpa."
)
func minimalServer(w dns.ResponseWriter, req *dns.Msg) {
sendAnswer := func(m *dns.Msg) {
buf, err := m.Pack()
CheckFatal(err)
if buf == nil {
log.Fatal("Nil buffer")
}
//log.Println("minimalServer sending:", buf)
// This is a bit of a kludge - per the RFC we should send responses from 5353, but that doesn't seem to work
sendconn, err := net.DialUDP("udp", nil, ipv4Addr)
CheckFatal(err)
_, err = sendconn.Write(buf)
sendconn.Close()
CheckFatal(err)
}
//log.Println("minimalServer received:", req)
if len(req.Answer) > 0 {
return // Only interested in questions.
}
if len(req.Question) != 1 {
return // We only handle single-question messages
}
if req.Question[0].Qtype == dns.TypeA && req.Question[0].Name == successTestName {
m := new(dns.Msg)
m.SetReply(req)
hdr := dns.RR_Header{Name: m.Question[0].Name, Rrtype: dns.TypeA,
Class: dns.ClassINET, Ttl: 3600}
a := &dns.A{Hdr: hdr, A: testAddr}
m.Answer = append(m.Answer, a)
sendAnswer(m)
} else if req.Question[0].Qtype == dns.TypePTR && req.Question[0].Name == testInAddr {
m := new(dns.Msg)
m.SetReply(req)
hdr := dns.RR_Header{Name: m.Question[0].Name, Rrtype: dns.TypePTR,
Class: dns.ClassINET, Ttl: 3600}
a := &dns.PTR{Hdr: hdr, Ptr: successTestName}
m.Answer = append(m.Answer, a)
sendAnswer(m)
}
}
func RunLocalMulticastServer() (*dns.Server, error) {
multicast, err := LinkLocalMulticastListener(nil)
if err != nil {
return nil, err
}
server := &dns.Server{
PacketConn: multicast,
Handler: dns.HandlerFunc(minimalServer),
ReadTimeout: 100 * time.Millisecond}
go server.ActivateAndServe()
return server, nil
}
func setup(t *testing.T) (*MDNSClient, *dns.Server, error) {
InitDefaultLogging(testing.Verbose())
server, err := RunLocalMulticastServer()
if err != nil {
t.Fatalf("Unable to run test server: %s. No default multicast interface?", err)
}
mdnsClient, err := NewMDNSClient()
wt.AssertNoErr(t, err)
err = mdnsClient.Start(nil)
wt.AssertNoErr(t, err)
return mdnsClient, server, err
}
func (c *MDNSClient) Shutdown() {
c.actionChan <- nil
}
type testContext struct {
receivedAddr net.IP
receivedCount int
channel chan *Response
}
func newTestContext() *testContext {
return &testContext{channel: make(chan *Response, 4)}
}
func (c *testContext) checkResponse(t *testing.T, channelOk bool, resp *Response) {
if !channelOk {
c.channel = nil
return
}
wt.AssertNoErr(t, resp.err)
log.Printf("Got address response %s addr %s", resp.Name(), resp.IP())
c.receivedAddr = resp.IP()
c.receivedCount++
}
func TestSimpleQuery(t *testing.T) {
log.Println("TestSimpleQuery starting")
mdnsClient, server, _ := setup(t)
defer mdnsClient.Shutdown()
defer server.Shutdown()
context := newTestContext()
// First, a test we expect to succeed
mdnsClient.SendQuery(successTestName, dns.TypeA, false, context.channel)
for resp := range context.channel {
context.checkResponse(t, true, resp)
}
if !context.receivedAddr.Equal(testAddr) || context.receivedCount != 1 {
t.Fatal("Unexpected result for", successTestName, context.receivedAddr, context.receivedCount)
}
// Now, a test we expect to time out with no responses
context = newTestContext()
mdnsClient.SendQuery("test2.weave.", dns.TypeA, false, context.channel)
for resp := range context.channel {
context.checkResponse(t, true, resp)
}
if context.receivedCount > 0 {
t.Fatal("Unexpected result for test2.weave", context.receivedAddr)
}
}
func TestParallelQuery(t *testing.T) {
log.Println("TestParallelQuery starting")
mdnsClient, server, _ := setup(t)
defer mdnsClient.Shutdown()
defer server.Shutdown()
context1 := newTestContext()
context2 := newTestContext()
go mdnsClient.SendQuery(successTestName, dns.TypeA, false, context1.channel)
go mdnsClient.SendQuery(successTestName, dns.TypeA, false, context2.channel)
timeout := time.After(2 * time.Second)
outerloop:
for context1.channel != nil || context2.channel != nil {
select {
case resp, ok := <-context1.channel:
context1.checkResponse(t, ok, resp)
case resp, ok := <-context2.channel:
context2.checkResponse(t, ok, resp)
case <-timeout:
break outerloop
}
}
if !context1.receivedAddr.Equal(testAddr) || !context2.receivedAddr.Equal(testAddr) || context1.receivedCount != 1 || context2.receivedCount != 1 {
t.Fatal("Unexpected result for", successTestName, context1.receivedAddr, context2.receivedAddr, context1.receivedCount, context2.receivedCount)
}
}
func TestAsLookup(t *testing.T) {
mdnsClient, server, _ := setup(t)
defer mdnsClient.Shutdown()
defer server.Shutdown()
ips, err := mdnsClient.LookupName(successTestName)
wt.AssertNoErr(t, err)
if !testAddr.Equal(ips[0].IP()) {
t.Fatalf("Returned address incorrect %s", ips)
}
ips, err = mdnsClient.LookupName("foo.example.com.")
wt.AssertErrorType(t, err, (*LookupError)(nil), "unknown hostname")
names, err := mdnsClient.LookupInaddr(testInAddr)
wt.AssertNoErr(t, err)
if !(successTestName == names[0].Name()) {
t.Fatalf("Expected name %s, got %s", successTestName, names)
}
}
-71
View File
@@ -1,71 +0,0 @@
package nameserver
import (
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
)
func mdnsLookup(client *MDNSClient, name string, qtype uint16, insistent bool) ([]ZoneRecord, error) {
var responses []*Response
channel := make(chan *Response)
client.SendQuery(name, qtype, insistent, channel)
for resp := range channel {
if err := resp.err; err != nil {
Debug.Printf("[mdns] Error for query type %s name %s: %s", dns.TypeToString[qtype], name, err)
return nil, err
}
Debug.Printf("[mdns] Got response name for %s-query about name '%s': name '%s', addr '%s'",
dns.TypeToString[qtype], name, resp.name, resp.addr)
if !insistent {
return []ZoneRecord{resp}, nil // non-background queries return the first response...
}
// add the response, avoiding duplicates
// TODO: replace this linear search by something better... (not a big deal as we don't expect long responses)
duplicate := false
for _, r := range responses {
if r.Equal(resp) {
duplicate = true
break
}
}
if !duplicate {
responses = append(responses, resp)
}
}
if len(responses) == 0 {
return nil, LookupError(name)
}
// convert []Response to []ZoneRecord
res := make([]ZoneRecord, len(responses))
for i, rr := range responses {
res[i] = ZoneRecord(rr)
}
return res, nil
}
// Perform a lookup for a name, returning either an IP address or an error
func (client *MDNSClient) LookupName(name string) ([]ZoneRecord, error) {
return mdnsLookup(client, name, dns.TypeA, false)
}
// Perform a lookup for an IP address, returning either a name or an error
func (client *MDNSClient) LookupInaddr(inaddr string) ([]ZoneRecord, error) {
return mdnsLookup(client, inaddr, dns.TypePTR, false)
}
// Perform lookup for a name, returning either a list of IP addresses or an error
// Insistent queries will wait up to `mDNSTimeout` milliseconds for responses
func (client *MDNSClient) InsistentLookupName(name string) ([]ZoneRecord, error) {
return mdnsLookup(client, name, dns.TypeA, true)
}
// Perform an insistent lookup for an IP address, returning either a list of names or an error
// Insistent queries will wait up to `mDNSTimeout` milliseconds for responses
func (client *MDNSClient) InsistentLookupInaddr(inaddr string) ([]ZoneRecord, error) {
return mdnsLookup(client, inaddr, dns.TypePTR, true)
}
-148
View File
@@ -1,148 +0,0 @@
package nameserver
import (
"net"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
)
type MDNSServer struct {
localAddrs []net.Addr
sendconn *net.UDPConn
srv *dns.Server
zone Zone
allowLocal bool
ttl int
}
// Create a new mDNS server
// Nothing will be done (including port bindings) until you `Start()` the server
func NewMDNSServer(zone Zone, local bool, ttl int) (*MDNSServer, error) {
return &MDNSServer{
zone: zone,
allowLocal: local,
ttl: ttl,
}, nil
}
// Return true if testaddr is a UDP address with IP matching my local i/f
func (s *MDNSServer) addrIsLocal(testaddr net.Addr) bool {
if udpaddr, ok := testaddr.(*net.UDPAddr); ok {
for _, localaddr := range s.localAddrs {
if ipnetlocal, ok := localaddr.(*net.IPNet); ok {
if ipnetlocal.IP.Equal(udpaddr.IP) {
return true
}
}
}
}
return false
}
// Start the mDNS server
func (s *MDNSServer) Start(ifi *net.Interface) (err error) {
// This is a bit of a kludge - per the RFC we should send responses from 5353, but that doesn't seem to work
s.sendconn, err = net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4zero, Port: 0})
if err != nil {
return err
}
conn, err := LinkLocalMulticastListener(ifi)
if err != nil {
return err
}
if ifi == nil {
s.localAddrs, err = net.InterfaceAddrs()
} else {
s.localAddrs, err = ifi.Addrs()
}
if err != nil {
return err
}
handleLocal := s.makeHandler(dns.TypeA,
func(zone ZoneLookup, r *dns.Msg, q *dns.Question) *dns.Msg {
if ips, err := zone.LookupName(q.Name); err == nil {
return makeAddressReply(r, q, ips, s.ttl)
}
return nil
})
handleReverse := s.makeHandler(dns.TypePTR,
func(zone ZoneLookup, r *dns.Msg, q *dns.Question) *dns.Msg {
if names, err := zone.LookupInaddr(q.Name); err == nil {
return makePTRReply(r, q, names, s.ttl)
}
return nil
})
mux := dns.NewServeMux()
mux.HandleFunc(s.zone.Domain(), handleLocal)
mux.HandleFunc(RDNSDomain, handleReverse)
s.srv = &dns.Server{
Listener: nil,
PacketConn: conn,
Handler: mux,
}
go s.srv.ActivateAndServe()
return err
}
// Stop the mDNS server
func (s *MDNSServer) Stop() error {
return s.srv.Shutdown()
}
func (s *MDNSServer) Zone() Zone {
return s.zone
}
type LookupFunc func(ZoneLookup, *dns.Msg, *dns.Question) *dns.Msg
func (s *MDNSServer) makeHandler(qtype uint16, lookup LookupFunc) dns.HandlerFunc {
return func(rw dns.ResponseWriter, r *dns.Msg) {
// Handle only questions, ignore answers. We might also ignore
// questions that arise locally (i.e., that come from an IP we
// think is local), but in the interest of avoiding
// complication, and easier testing, this is elided on the
// assumption that the client wouldn't ask if it already knew
// the answer, and if it does ask, it'll be happy to get an
// answer.
remoteAddr := rw.RemoteAddr()
if s.addrIsLocal(remoteAddr) && !s.allowLocal {
Debug.Printf("[mdns] srv: mDNS query from local host ('%s'): ignored", remoteAddr)
return
}
// TODO: use any Answer in the query for adding records in the Zone database...
if len(r.Answer) == 0 && len(r.Question) > 0 {
q := &r.Question[0]
if q.Qtype == qtype {
Debug.Printf("[mdns msgid %d] srv: trying to answer to mDNS query '%s'", r.MsgHdr.Id, q.Name)
if m := lookup(s.zone, r, q); m != nil {
Debug.Printf("[mdns msgid %d] srv: found local answer to mDNS query '%s'", r.MsgHdr.Id, q.Name)
if err := s.sendResponse(m); err != nil {
Warning.Printf("[mdns msgid %d] srv: error writing mDNS response to %v", r.MsgHdr.Id, s.sendconn)
} else {
Debug.Printf("[mdns msgid %d] srv: response sent: %d answers", r.MsgHdr.Id, len(m.Answer))
}
} else {
Debug.Printf("[mdns msgid %d] srv: no local answer for answering mDNS query '%s'",
r.MsgHdr.Id, q.Name)
}
}
}
}
}
func (s *MDNSServer) sendResponse(m *dns.Msg) error {
buf, err := m.Pack()
if err != nil {
return err
}
_, err = s.sendconn.WriteTo(buf, ipv4Addr)
return err
}
-124
View File
@@ -1,124 +0,0 @@
package nameserver
import (
"net"
"testing"
"time"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
wt "github.com/weaveworks/weave/testing"
)
func TestServerSimpleQuery(t *testing.T) {
var (
testRecord1 = Record{"test.weave.local.", net.ParseIP("10.20.20.10"), 0, 0, 0}
testRecord2 = Record{"test.weave.local.", net.ParseIP("10.20.20.20"), 0, 0, 0}
testInAddr1 = "10.20.20.10.in-addr.arpa."
)
InitDefaultLogging(testing.Verbose())
Info.Println("TestServerSimpleQuery starting")
mzone := newMockedZoneWithRecords([]ZoneRecord{testRecord1, testRecord2})
mdnsServer, err := NewMDNSServer(mzone, true, DefaultLocalTTL)
wt.AssertNoErr(t, err)
err = mdnsServer.Start(nil)
wt.AssertNoErr(t, err)
defer mdnsServer.Stop()
var receivedAddrs []net.IP
receivedName := ""
recvChan := make(chan interface{})
receivedCount := 0
// Implement a minimal listener for responses
multicast, err := LinkLocalMulticastListener(nil)
wt.AssertNoErr(t, err)
handleMDNS := func(w dns.ResponseWriter, r *dns.Msg) {
// Only handle responses here
if len(r.Answer) > 0 {
t.Logf("Received %d answer(s)", len(r.Answer))
for _, answer := range r.Answer {
recvChan <- answer
}
recvChan <- "ok"
}
}
sendQuery := func(name string, querytype uint16) {
receivedAddrs = make([]net.IP, 0)
receivedName = ""
receivedCount = 0
m := new(dns.Msg)
m.SetQuestion(name, querytype)
m.RecursionDesired = false
buf, err := m.Pack()
wt.AssertNoErr(t, err)
conn, err := net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4zero, Port: 0})
wt.AssertNoErr(t, err)
Debug.Printf("Sending UDP packet to %s", ipv4Addr)
_, err = conn.WriteTo(buf, ipv4Addr)
wt.AssertNoErr(t, err)
Debug.Printf("Waiting for response")
for {
select {
case x := <-recvChan:
switch rr := x.(type) {
case *dns.A:
t.Logf("... A:\n%+v", rr)
receivedAddrs = append(receivedAddrs, rr.A)
receivedCount++
case *dns.PTR:
t.Logf("... PTR:\n%+v", rr)
receivedName = rr.Ptr
receivedCount++
case string:
return
}
case <-time.After(100 * time.Millisecond):
Debug.Printf("Timeout while waiting for response")
return
}
}
}
listener := &dns.Server{
Unsafe: true,
PacketConn: multicast,
Handler: dns.HandlerFunc(handleMDNS),
ReadTimeout: 100 * time.Millisecond}
go listener.ActivateAndServe()
defer listener.Shutdown()
time.Sleep(100 * time.Millisecond) // Allow for server to get going
Debug.Printf("Checking that we get 2 IPs fo name '%s' [A]", testRecord1.Name())
sendQuery(testRecord1.Name(), dns.TypeA)
if receivedCount != 2 {
t.Fatalf("Unexpected result count %d for %s", receivedCount, testRecord1.Name())
}
if !(receivedAddrs[0].Equal(testRecord1.IP()) || receivedAddrs[0].Equal(testRecord2.IP())) {
t.Fatalf("Unexpected result %s for %s", receivedAddrs, testRecord1.Name())
}
if !(receivedAddrs[1].Equal(testRecord1.IP()) || receivedAddrs[1].Equal(testRecord2.IP())) {
t.Fatalf("Unexpected result %s for %s", receivedAddrs, testRecord1.Name())
}
Debug.Printf("Checking that 'testfail.weave.' [A] gets no answers")
sendQuery("testfail.weave.", dns.TypeA)
if receivedCount != 0 {
t.Fatalf("Unexpected result count %d for testfail.weave", receivedCount)
}
Debug.Printf("Checking that '%s' [PTR] gets one name", testInAddr1)
sendQuery(testInAddr1, dns.TypePTR)
if receivedCount != 1 {
t.Fatalf("Expected an answer to %s, got %d answers", testInAddr1, receivedCount)
} else if !(testRecord1.Name() == receivedName) {
t.Fatalf("Expected answer %s to query for %s, got %s", testRecord1.Name(), testInAddr1, receivedName)
}
}
-170
View File
@@ -1,170 +0,0 @@
package nameserver
import (
"net"
"testing"
"time"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
wt "github.com/weaveworks/weave/testing"
)
// Check that we can use a regular mDNS server with a regular mDNS client
func TestClientServerSimpleQuery(t *testing.T) {
InitDefaultLogging(testing.Verbose())
testRecord1 := Record{"test.weave.local.", net.ParseIP("10.2.2.1"), 0, 0, 0}
testInAddr1 := "1.2.2.10.in-addr.arpa."
mzone := newMockedZoneWithRecords([]ZoneRecord{testRecord1})
mdnsServer, err := NewMDNSServer(mzone, true, DefaultLocalTTL)
wt.AssertNoErr(t, err)
err = mdnsServer.Start(nil)
wt.AssertNoErr(t, err)
defer mdnsServer.Stop()
var receivedAddr net.IP
var receivedName string
receivedCount := 0
mdnsCli, err := NewMDNSClient()
wt.AssertNoErr(t, err)
err = mdnsCli.Start(nil)
wt.AssertNoErr(t, err)
sendQuery := func(name string, querytype uint16) {
receivedAddr = nil
receivedName = ""
receivedCount = 0
Debug.Printf("Sending query...")
switch querytype {
case dns.TypeA:
r, err := mdnsCli.LookupName(name)
if len(r) > 0 {
receivedAddr = r[0].IP()
if err == nil {
receivedCount++
}
}
case dns.TypePTR:
r, err := mdnsCli.LookupInaddr(name)
if len(r) > 0 {
receivedName = r[0].Name()
if err == nil {
receivedCount++
}
}
}
}
time.Sleep(100 * time.Millisecond) // Allow for server to get going
Debug.Printf("Query: %s dns.TypeA", testRecord1.Name())
sendQuery(testRecord1.Name(), dns.TypeA)
if receivedCount != 1 {
t.Fatalf("Unexpected result count %d for %s", receivedCount, testRecord1.Name())
}
if !receivedAddr.Equal(testRecord1.IP()) {
t.Fatalf("Unexpected result %s for %s", receivedAddr, testRecord1.Name())
}
Debug.Printf("Query: testfail.weave. dns.TypeA")
sendQuery("testfail.weave.", dns.TypeA)
if receivedCount != 0 {
t.Fatalf("Unexpected result count %d for testfail.weave", receivedCount)
}
Debug.Printf("Query: %s dns.TypePTR", testInAddr1)
sendQuery(testInAddr1, dns.TypePTR)
if receivedCount != 1 {
t.Fatalf("Expected an answer to %s, got %d answers", testInAddr1, receivedCount)
} else if !(testRecord1.Name() == receivedName) {
t.Fatalf("Expected answer %s to query for %s, got %s", testRecord1.Name(), testInAddr1, receivedName)
}
}
// Check that we can use a use "insistent" queries
func TestClientServerInsistentQuery(t *testing.T) {
InitDefaultLogging(testing.Verbose())
testRecord1 := Record{"test.weave.local.", net.ParseIP("10.2.2.1"), 0, 0, 0}
testInAddr1 := "1.2.2.10.in-addr.arpa."
testRecord2 := Record{"test.weave.local.", net.ParseIP("10.2.2.2"), 0, 0, 0}
testInAddr2 := "2.2.2.10.in-addr.arpa."
mzone1 := newMockedZoneWithRecords([]ZoneRecord{testRecord1})
mdnsServer1, err := NewMDNSServer(mzone1, true, DefaultLocalTTL)
wt.AssertNoErr(t, err)
err = mdnsServer1.Start(nil)
wt.AssertNoErr(t, err)
defer mdnsServer1.Stop()
mzone2 := newMockedZoneWithRecords([]ZoneRecord{testRecord2})
mdnsServer2, err := NewMDNSServer(mzone2, true, DefaultLocalTTL)
wt.AssertNoErr(t, err)
err = mdnsServer2.Start(nil)
wt.AssertNoErr(t, err)
defer mdnsServer2.Stop()
// create a third server with exactly the same info as the second server (so we can test duplicates removals)
mdnsServer3, err := NewMDNSServer(mzone2, true, DefaultLocalTTL)
wt.AssertNoErr(t, err)
err = mdnsServer3.Start(nil)
wt.AssertNoErr(t, err)
defer mdnsServer3.Stop()
var receivedAddrs []ZoneRecord
var receivedNames []ZoneRecord
receivedCount := 0
mdnsCli, err := NewMDNSClient()
wt.AssertNoErr(t, err)
err = mdnsCli.Start(nil)
wt.AssertNoErr(t, err)
sendQuery := func(name string, querytype uint16) {
receivedAddrs = nil
receivedNames = nil
receivedCount = 0
Debug.Printf("Sending query...")
switch querytype {
case dns.TypeA:
receivedAddrs, err = mdnsCli.InsistentLookupName(name)
if err == nil {
receivedCount = len(receivedAddrs)
}
case dns.TypePTR:
receivedNames, err = mdnsCli.InsistentLookupInaddr(name)
if err == nil {
receivedCount = len(receivedNames)
}
}
}
time.Sleep(100 * time.Millisecond) // Allow for server to get going
Debug.Printf("Query: %s dns.TypeA", testRecord1.Name())
sendQuery(testRecord1.Name(), dns.TypeA)
if receivedCount != 2 {
t.Fatalf("Unexpected result count %d for %s", receivedCount, testRecord1.Name())
}
Debug.Printf("Query: testfail.weave. dns.TypeA")
sendQuery("testfail.weave.", dns.TypeA)
if receivedCount != 0 {
t.Fatalf("Unexpected result count %d for testfail.weave", receivedCount)
}
Debug.Printf("Query: %s dns.TypePTR", testInAddr1)
sendQuery(testInAddr1, dns.TypePTR)
if receivedCount != 1 {
t.Fatalf("Expected an answer to %s, got %d answers", testInAddr1, receivedCount)
}
Debug.Printf("Query: %s dns.TypePTR", testInAddr2)
sendQuery(testInAddr2, dns.TypePTR)
if receivedCount != 1 {
t.Fatalf("Expected an answer to %s, got %d answers", testInAddr2, receivedCount)
}
}
-507
View File
@@ -1,507 +0,0 @@
package nameserver
import (
"fmt"
"net"
"strconv"
"sync"
"testing"
"time"
"github.com/benbjohnson/clock"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
wt "github.com/weaveworks/weave/testing"
)
const (
testSocketTimeout = 100 // in millisecs
)
// Warn about some methods that some day should be implemented...
func notImplWarn() { Warning.Printf("Mocked method. Not implemented.") }
// A mocked Zone that always returns the same records
// * it does not send/receive any mDNS query
type mockedZoneWithRecords struct {
sync.RWMutex
records []ZoneRecord
// Statistics
NumLookupsName int
NumLookupsInaddr int
}
func newMockedZoneWithRecords(zr []ZoneRecord) *mockedZoneWithRecords {
return &mockedZoneWithRecords{records: zr}
}
func (mz *mockedZoneWithRecords) Domain() string { return DefaultLocalDomain }
func (mz *mockedZoneWithRecords) LookupName(name string) ([]ZoneRecord, error) {
Debug.Printf("[mocked zone]: LookupName: returning records %s", mz.records)
mz.Lock()
defer mz.Unlock()
mz.NumLookupsName++
var res []ZoneRecord
for _, r := range mz.records {
if r.Name() == name {
res = append(res, r)
}
}
return res, nil
}
func (mz *mockedZoneWithRecords) LookupInaddr(inaddr string) ([]ZoneRecord, error) {
Debug.Printf("[mocked zone]: LookupInaddr: returning records %s", mz.records)
mz.Lock()
defer mz.Unlock()
mz.NumLookupsInaddr++
var res []ZoneRecord
for _, r := range mz.records {
revIP, err := raddrToIP(inaddr)
if err != nil {
return nil, newParseError("lookup address", inaddr)
}
if r.IP().Equal(revIP) {
res = append(res, r)
}
}
return res, nil
}
func (mz *mockedZoneWithRecords) DomainLookupName(name string) ([]ZoneRecord, error) {
return mz.LookupName(name)
}
func (mz *mockedZoneWithRecords) DomainLookupInaddr(inaddr string) ([]ZoneRecord, error) {
return mz.LookupInaddr(inaddr)
}
// the following methods are not currently needed...
func (mz *mockedZoneWithRecords) AddRecord(ident string, name string, ip net.IP) error {
notImplWarn()
return nil
}
func (mz *mockedZoneWithRecords) DeleteRecords(ident string, name string, ip net.IP) int {
notImplWarn()
return 0
}
func (mz *mockedZoneWithRecords) Status() string { notImplWarn(); return "nothing" }
func (mz *mockedZoneWithRecords) ObserveName(name string, observer ZoneRecordObserver) error {
notImplWarn()
return nil
}
func (mz *mockedZoneWithRecords) ObserveInaddr(inaddr string, observer ZoneRecordObserver) error {
notImplWarn()
return nil
}
//////////////////////////////////////////////////////////////////
// A mocked mDNS server
type mockedMDNSServer struct {
zone Zone
}
func newMockedMDNSServer(z Zone) *mockedMDNSServer { return &mockedMDNSServer{zone: z} }
func (ms *mockedMDNSServer) Start(_ *net.Interface) error { return nil }
func (ms *mockedMDNSServer) Stop() error { return nil }
func (ms *mockedMDNSServer) Zone() Zone { return ms.zone }
// a useful mix of NewMockedZone and NewMockedMDNSServer
func newMockedMDNSServerWithRecord(zr ZoneRecord) *mockedMDNSServer {
return newMockedMDNSServer(newMockedZoneWithRecords([]ZoneRecord{zr}))
}
func newMockedMDNSServerWithRecords(zrs []ZoneRecord) *mockedMDNSServer {
return newMockedMDNSServer(newMockedZoneWithRecords(zrs))
}
//////////////////////////////////////////////////////////////////
// A mocked mDNS client.
// This mock asks a group of (potentially mocked) mDNS servers
type mockedMDNSClient struct {
sync.RWMutex
servers map[ZoneMDNSServer]ZoneMDNSServer
// Statistics
NumLookupsName int
NumLookupsInaddr int
NumInsLookupsName int
NumInsLookupsInaddr int
}
func newMockedMDNSClient(srvrs []*mockedMDNSServer) *mockedMDNSClient {
r := mockedMDNSClient{
servers: make(map[ZoneMDNSServer]ZoneMDNSServer),
}
if srvrs != nil {
for _, server := range srvrs {
r.servers[server] = server
}
}
return &r
}
func (mc *mockedMDNSClient) Start(ifi *net.Interface) error { return nil }
func (mc *mockedMDNSClient) Stop() error { return nil }
func (mc *mockedMDNSClient) Domain() string { return DefaultLocalDomain }
func (mc *mockedMDNSClient) LookupName(name string) ([]ZoneRecord, error) {
mc.Lock()
defer mc.Unlock()
// get a random result
mc.NumLookupsName++
for _, server := range mc.servers {
r, err := server.Zone().LookupName(name)
if err == nil {
return r, nil // return the first answer
}
}
return nil, LookupError(name)
}
func (mc *mockedMDNSClient) LookupInaddr(inaddr string) ([]ZoneRecord, error) {
mc.Lock()
defer mc.Unlock()
// get a random result
mc.NumLookupsInaddr++
for _, server := range mc.servers {
r, err := server.Zone().LookupInaddr(inaddr)
if err == nil {
return r, nil // return the first answer
}
}
return nil, LookupError(inaddr)
}
func (mc *mockedMDNSClient) InsistentLookupName(name string) ([]ZoneRecord, error) {
mc.Lock()
defer mc.Unlock()
var res []ZoneRecord
mc.NumInsLookupsName++
for _, server := range mc.servers {
r, err := server.Zone().LookupName(name)
if err == nil {
res = append(res, r...)
}
}
if len(res) == 0 {
return nil, LookupError(name)
}
return res, nil
}
func (mc *mockedMDNSClient) InsistentLookupInaddr(inaddr string) ([]ZoneRecord, error) {
mc.Lock()
defer mc.Unlock()
var res []ZoneRecord
mc.NumInsLookupsInaddr++
for _, server := range mc.servers {
r, err := server.Zone().LookupInaddr(inaddr)
if err == nil {
res = append(res, r...)
}
}
if len(res) == 0 {
return nil, LookupError(inaddr)
}
return res, nil
}
func (mc *mockedMDNSClient) AddServer(srv ZoneMDNSServer) {
mc.Lock()
defer mc.Unlock()
mc.servers[srv] = srv
}
func (mc *mockedMDNSClient) RemoveServer(srv ZoneMDNSServer) {
mc.Lock()
defer mc.Unlock()
delete(mc.servers, srv)
}
//////////////////////////////////////////////////////////////////
type zbmEntry struct {
Server *mockedMDNSServer
Client *mockedMDNSClient
Zone *ZoneDb
}
type zoneDbsWithMockedMDns []*zbmEntry
// Creates a group of (real) zone databases linked through mocked mDNS servers and clients
// This effectively creates a cluster of WeaveDNS peers
func newZoneDbsWithMockedMDns(num int, config ZoneConfig) zoneDbsWithMockedMDns {
res := make(zoneDbsWithMockedMDns, num)
for i := 0; i < num; i++ {
res[i] = new(zbmEntry)
Debug.Printf("[test] Creating mocked mDNS server #%d", i)
res[i].Server = newMockedMDNSServer(nil)
Debug.Printf("[test] Creating mocked mDNS client #%d", i)
res[i].Client = newMockedMDNSClient([]*mockedMDNSServer{})
cfg := config
cfg.MDNSClient = res[i].Client
cfg.MDNSServer = res[i].Server
Debug.Printf("[test] Creating ZoneDb #%d", i)
res[i].Zone, _ = NewZoneDb(cfg)
// link the mDNS server to its zone
// ZoneDbs are connected to mDNS in two directions: they use their mDNS
// clients for asking other peers about names/IPs, and export their records
// through a mDNS server.
res[i].Server.zone = res[i].Zone
}
// link all the clients to all the servers
for i := 0; i < num; i++ {
for j := 0; j < num; j++ {
// with this check, the local client will not ask to the local server
// this is not what we currently do, but can be convenient in some cases
// for finding some bugs...
if i != j {
Debug.Printf("[test] Linking mocked mDNS client #%d to server #%d", i, j)
res[i].Client.AddServer(res[j].Server)
}
}
}
return res
}
func (zbs zoneDbsWithMockedMDns) Start() {
for _, entry := range zbs {
entry.Zone.Start()
}
}
func (zbs zoneDbsWithMockedMDns) Stop() {
for _, entry := range zbs {
entry.Zone.Stop()
}
}
func (zbs zoneDbsWithMockedMDns) Flush() {
for _, entry := range zbs {
entry.Zone.refreshScheds.Flush()
}
}
//////////////////////////////////////////////////////////////////
// A mocked cache where we never find a single thing... ;)
type mockedCache struct {
NumGets int
NumPuts int
NumRemovals int
}
func newMockedCache() *mockedCache { return &mockedCache{} }
func (c *mockedCache) Get(request *dns.Msg, maxLen int) (reply *dns.Msg, err error) {
c.NumGets++
return nil, nil
}
func (c *mockedCache) Put(request *dns.Msg, reply *dns.Msg, ttl int, flags uint8) int {
c.NumPuts++
return 0
}
func (c *mockedCache) Remove(question *dns.Question) { c.NumRemovals++ }
func (c *mockedCache) Purge() {}
func (c *mockedCache) Clear() {}
func (c *mockedCache) Len() int { return 0 }
func (c *mockedCache) Capacity() int { return DefaultCacheLen }
//////////////////////////////////////////////////////////////////
// A mocked fallback server
type mockedFallback struct {
CliConfig *dns.ClientConfig
Addr string
Port int
udpSrv *dns.Server
tcpSrv *dns.Server
}
// Create a mocked fallback server
// You can use the server's `CliConfig` as the `UpstreamCfg` of a `DNSServer`...
func newMockedFallback(udpH dns.HandlerFunc, tcpH dns.HandlerFunc) (*mockedFallback, error) {
udpSrv, fallbackAddr, err := runLocalUDPServer("127.0.0.1:0", udpH)
if err != nil {
return nil, err
}
_, fallbackPort, err := net.SplitHostPort(fallbackAddr)
if err != nil {
return nil, err
}
fallbackPortI, _ := strconv.Atoi(fallbackPort)
res := mockedFallback{
udpSrv: udpSrv,
Addr: fallbackAddr,
Port: fallbackPortI,
CliConfig: &dns.ClientConfig{
Servers: []string{"127.0.0.1"},
Port: fallbackPort,
},
}
if tcpH != nil {
fallbackTCPAddr := fmt.Sprintf("127.0.0.1:%s", fallbackPort)
res.tcpSrv, _, err = runLocalTCPServer(fallbackTCPAddr, tcpH)
if err != nil {
return nil, err
}
}
return &res, nil
}
// Start the fallback server
func (mf *mockedFallback) Start() error {
return nil
}
// Stop the fallback server
func (mf *mockedFallback) Stop() error {
if mf.tcpSrv != nil {
mf.udpSrv.Shutdown()
mf.udpSrv = nil
}
if mf.tcpSrv != nil {
mf.tcpSrv.Shutdown()
mf.udpSrv = nil
}
return nil
}
// Run a UDP fallback server
func runLocalUDPServer(laddr string, handler dns.HandlerFunc) (*dns.Server, string, error) {
Debug.Printf("[mocked fallback] Starting fallback UDP server at %s", laddr)
pc, err := net.ListenPacket("udp", laddr)
if err != nil {
return nil, "", err
}
server := &dns.Server{PacketConn: pc, Handler: handler, ReadTimeout: testSocketTimeout * time.Millisecond}
go func() {
server.ActivateAndServe()
pc.Close()
}()
Debug.Printf("[mocked fallback] Fallback UDP server listening at %s", pc.LocalAddr())
return server, pc.LocalAddr().String(), nil
}
// Run a TCP fallback server
func runLocalTCPServer(laddr string, handler dns.HandlerFunc) (*dns.Server, string, error) {
Debug.Printf("[mocked fallback] Starting fallback TCP server at %s", laddr)
laddrTCP, err := net.ResolveTCPAddr("tcp", laddr)
if err != nil {
return nil, "", err
}
l, err := net.ListenTCP("tcp", laddrTCP)
if err != nil {
return nil, "", err
}
server := &dns.Server{Listener: l, Handler: handler, ReadTimeout: testSocketTimeout * time.Millisecond}
go func() {
server.ActivateAndServe()
l.Close()
}()
Debug.Printf("[mocked fallback] Fallback TCP server listening at %s", l.Addr().String())
return server, l.Addr().String(), nil
}
//////////////////////////////////////////////////////////////////
type mockedClock struct {
*clock.Mock
}
func newMockedClock() *mockedClock {
return &mockedClock{clock.NewMock()}
}
// Note: moving the clock forward does not take into account the data that could
// be waiting in channels. So your code should not depend on the channels
// for creating new timers. Otherwise, time travelling will not be reliable...
func (clk *mockedClock) Forward(secs int) {
Debug.Printf(">>>>>>> Moving clock forward %d seconds - Time traveling >>>>>>>", secs)
clk.Add(time.Duration(secs) * time.Second)
Debug.Printf("<<<<<<< Time travel finished! We are at %s <<<<<<<", clk.Now())
}
//////////////////////////////////////////////////////////////////
// Perform a DNS query and assert the reply code, number or answers, etc
func assertExchange(t *testing.T, z string, ty uint16, port int, minAnswers int, maxAnswers int, expErr int) (*dns.Msg, *dns.Msg) {
wt.AssertNotEqualInt(t, port, 0, "invalid DNS server port")
c := &dns.Client{
UDPSize: testUDPBufSize,
}
m := new(dns.Msg)
m.RecursionDesired = true
m.SetQuestion(z, ty)
m.SetEdns0(testUDPBufSize, false) // we don't want to play with truncation here...
lstAddr := fmt.Sprintf("127.0.0.1:%d", port)
r, _, err := c.Exchange(m, lstAddr)
t.Logf("Response from '%s':\n%+v\n", lstAddr, r)
if err != nil {
t.Errorf("Error when querying DNS server at %s: %s", lstAddr, err)
}
wt.AssertNoErr(t, err)
if minAnswers == 0 && maxAnswers == 0 {
wt.AssertStatus(t, r.Rcode, expErr, "DNS response code")
} else {
wt.AssertStatus(t, r.Rcode, dns.RcodeSuccess, "DNS response code")
}
answers := len(r.Answer)
if minAnswers >= 0 && answers < minAnswers {
wt.Fatalf(t, "Number of answers >= %d", minAnswers)
}
if maxAnswers >= 0 && answers > maxAnswers {
wt.Fatalf(t, "Number of answers <= %d", maxAnswers)
}
return m, r
}
// Assert that we have a response in the cache for a query `q`
func assertInCache(t *testing.T, cache ZoneCache, q *dns.Msg, desc string) {
r, err := cache.Get(q, maxUDPSize)
wt.AssertNoErr(t, err)
wt.AssertNotNil(t, r, fmt.Sprintf("value in the cache: %s", desc))
}
// Assert that we have a response in the cache for a query `q`
func assertNotLocalInCache(t *testing.T, cache ZoneCache, q *dns.Msg, desc string) {
r, err := cache.Get(q, maxUDPSize)
if !(r == nil && err == errNoLocalReplies) {
t.Fatalf("Cache does not return a noLocalReplies error for query %s", q)
}
}
// Assert that we do not have a response in the cache for a query `q`
func assertNotInCache(t *testing.T, cache ZoneCache, q *dns.Msg, desc string) {
r, err := cache.Get(q, maxUDPSize)
wt.AssertNoErr(t, err)
wt.AssertNil(t, r, fmt.Sprintf("value in the cache: %s\n%s", desc, r))
}
+244
View File
@@ -0,0 +1,244 @@
package nameserver
import (
"fmt"
"sync"
"time"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/mesh"
"github.com/weaveworks/weave/net/address"
)
const (
// Tombstones do not need to survive long periods of peer disconnection, as
// we delete entries for disconnected peers. Therefore they just need to hang
// around to account for propagation delay through gossip. We do need them
// to hang around longer than the allowed clock skew (defined next). 30mins
// tombstone timeout allows for 15mins max clock skew.
tombstoneTimeout = time.Minute * 30
// Maximum age of acceptable gossip messages (to account for clock skew)
gossipWindow = int64(tombstoneTimeout/time.Second) / 2
// Used by prog/weaver/main.go and proxy/create_container_interceptor.go
DefaultDomain = "weave.local."
)
// Nameserver: gossip-based, in memory nameserver.
// - Holds a sorted list of (hostname, peer, container id, ip) tuples for the whole cluster.
// - This list is gossiped & merged around the cluser.
// - Lookup-by-hostname are O(nlogn), and return a (copy of a) slice of the entries
// - Update is O(n) for now
type Nameserver struct {
sync.RWMutex
ourName mesh.PeerName
domain string
gossip mesh.Gossip
entries Entries
isKnownPeer func(mesh.PeerName) bool
quit chan struct{}
}
func New(ourName mesh.PeerName, domain string, isKnownPeer func(mesh.PeerName) bool) *Nameserver {
return &Nameserver{
ourName: ourName,
domain: dns.Fqdn(domain),
isKnownPeer: isKnownPeer,
quit: make(chan struct{}),
}
}
func (n *Nameserver) SetGossip(gossip mesh.Gossip) {
n.gossip = gossip
}
func (n *Nameserver) Start() {
go func() {
ticker := time.Tick(tombstoneTimeout)
for {
select {
case <-n.quit:
return
case <-ticker:
n.deleteTombstones()
}
}
}()
}
func (n *Nameserver) Stop() {
n.quit <- struct{}{}
}
func (n *Nameserver) broadcastEntries(es ...Entry) error {
if n.gossip != nil {
return n.gossip.GossipBroadcast(&GossipData{
Entries: Entries(es),
Timestamp: now(),
})
}
return nil
}
func (n *Nameserver) AddEntry(hostname, containerid string, origin mesh.PeerName, addr address.Address) error {
n.infof("adding entry %s -> %s", hostname, addr.String())
n.Lock()
entry := n.entries.add(hostname, containerid, origin, addr)
n.Unlock()
return n.broadcastEntries(entry)
}
func (n *Nameserver) Lookup(hostname string) []address.Address {
n.RLock()
defer n.RUnlock()
entries := n.entries.lookup(hostname)
result := []address.Address{}
for _, e := range entries {
if e.Tombstone > 0 {
continue
}
result = append(result, e.Addr)
}
n.debugf("lookup %s -> %s", hostname, &result)
return result
}
func (n *Nameserver) ReverseLookup(ip address.Address) (string, error) {
n.RLock()
defer n.RUnlock()
match, err := n.entries.first(func(e *Entry) bool {
return e.Tombstone == 0 && e.Addr == ip
})
if err != nil {
return "", err
}
n.debugf("reverse lookup %s -> %s", ip, match.Hostname)
return match.Hostname, nil
}
func (n *Nameserver) ContainerStarted(ident string) {}
func (n *Nameserver) ContainerDied(ident string) {
n.Lock()
entries := n.entries.tombstone(n.ourName, func(e *Entry) bool {
if e.ContainerID == ident {
n.infof("container %s died; tombstoning entry %s", ident, e.String())
return true
}
return false
})
n.Unlock()
if len(entries) > 0 {
if err := n.broadcastEntries(entries...); err != nil {
n.errorf("failed to broadcast container %s death: %v", ident, err)
}
}
}
func (n *Nameserver) PeerGone(peer mesh.PeerName) {
n.infof("peer %s gone", peer.String())
n.Lock()
defer n.Unlock()
n.entries.filter(func(e *Entry) bool {
return e.Origin != peer
})
}
func (n *Nameserver) Delete(hostname, containerid, ipStr string, ip address.Address) error {
n.Lock()
n.infof("tombstoning hostname=%s, container=%s, ip=%s", hostname, containerid, ipStr)
entries := n.entries.tombstone(n.ourName, func(e *Entry) bool {
if hostname != "*" && e.Hostname != hostname {
return false
}
if containerid != "*" && e.ContainerID != containerid {
return false
}
if ipStr != "*" && e.Addr != ip {
return false
}
n.infof("tombstoning entry %v", e)
return true
})
n.Unlock()
return n.broadcastEntries(entries...)
}
func (n *Nameserver) deleteTombstones() {
n.Lock()
defer n.Unlock()
now := time.Now().Unix()
n.entries.filter(func(e *Entry) bool {
return e.Tombstone == 0 || now-e.Tombstone <= int64(tombstoneTimeout/time.Second)
})
}
func (n *Nameserver) Gossip() mesh.GossipData {
n.RLock()
defer n.RUnlock()
gossip := &GossipData{
Entries: make(Entries, len(n.entries)),
Timestamp: now(),
}
copy(gossip.Entries, n.entries)
return gossip
}
func (n *Nameserver) OnGossipUnicast(sender mesh.PeerName, msg []byte) error {
return nil
}
func (n *Nameserver) receiveGossip(msg []byte) (mesh.GossipData, mesh.GossipData, error) {
var gossip GossipData
if err := gossip.Decode(msg); err != nil {
return nil, nil, err
}
if delta := gossip.Timestamp - now(); delta > gossipWindow || delta < -gossipWindow {
return nil, nil, fmt.Errorf("clock skew of %d detected", delta)
}
n.Lock()
defer n.Unlock()
gossip.Entries.filter(func(e *Entry) bool {
return n.isKnownPeer(e.Origin)
})
newEntries := n.entries.merge(gossip.Entries)
if len(newEntries) > 0 {
return &GossipData{Entries: newEntries, Timestamp: now()}, &gossip, nil
}
return nil, &gossip, nil
}
// merge received data into state and return "everything new I've
// just learnt", or nil if nothing in the received data was new
func (n *Nameserver) OnGossip(msg []byte) (mesh.GossipData, error) {
newEntries, _, err := n.receiveGossip(msg)
return newEntries, err
}
// merge received data into state and return a representation of
// the received data, for further propagation
func (n *Nameserver) OnGossipBroadcast(_ mesh.PeerName, msg []byte) (mesh.GossipData, error) {
_, entries, err := n.receiveGossip(msg)
return entries, err
}
func (n *Nameserver) infof(fmt string, args ...interface{}) {
Log.Infof("[nameserver %s] "+fmt, append([]interface{}{n.ourName}, args...)...)
}
func (n *Nameserver) debugf(fmt string, args ...interface{}) {
Log.Debugf("[nameserver %s] "+fmt, append([]interface{}{n.ourName}, args...)...)
}
func (n *Nameserver) errorf(fmt string, args ...interface{}) {
Log.Errorf("[nameserver %s] "+fmt, append([]interface{}{n.ourName}, args...)...)
}
+279
View File
@@ -0,0 +1,279 @@
package nameserver
import (
"fmt"
"math/rand"
"reflect"
"sort"
"strings"
"testing"
"time"
"github.com/stretchr/testify/require"
"github.com/weaveworks/weave/mesh"
"github.com/weaveworks/weave/net/address"
wt "github.com/weaveworks/weave/testing"
"github.com/weaveworks/weave/testing/gossip"
)
func makeNameserver(name mesh.PeerName) *Nameserver {
return New(name, "", func(mesh.PeerName) bool { return true })
}
func makeNetwork(size int) ([]*Nameserver, *gossip.TestRouter) {
gossipRouter := gossip.NewTestRouter(0.0)
nameservers := make([]*Nameserver, size)
for i := 0; i < size; i++ {
name, _ := mesh.PeerNameFromString(fmt.Sprintf("%02d:00:00:02:00:00", i))
nameserver := makeNameserver(name)
nameserver.SetGossip(gossipRouter.Connect(nameserver.ourName, nameserver))
nameserver.Start()
nameservers[i] = nameserver
}
return nameservers, gossipRouter
}
func stopNetwork(nameservers []*Nameserver, grouter *gossip.TestRouter) {
for _, nameserver := range nameservers {
nameserver.Stop()
}
grouter.Stop()
}
type pair struct {
origin mesh.PeerName
addr address.Address
}
type mapping struct {
hostname string
addrs []pair
}
type addrs []address.Address
func (a addrs) Len() int { return len(a) }
func (a addrs) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
func (a addrs) Less(i, j int) bool { return a[i] < a[j] }
func (a addrs) String() string {
ss := []string{}
for _, addr := range a {
ss = append(ss, addr.String())
}
return strings.Join(ss, " ")
}
func (m mapping) Addrs() []address.Address {
want := addrs{}
for _, p := range m.addrs {
want = append(want, p.addr)
}
sort.Sort(want)
return want
}
func TestNameservers(t *testing.T) {
wt.RunWithTimeout(t, 2*time.Minute, func() {
testNameservers(t)
})
}
func testNameservers(t *testing.T) {
//common.SetLogLevel("debug")
lookupTimeout := 10 // ms
nameservers, grouter := makeNetwork(30)
defer stopNetwork(nameservers, grouter)
// This subset will sometimes lose touch with some of the others
badNameservers := nameservers[25:]
// This subset will remain well-connected, and we will deal mainly with them
nameservers = nameservers[:25]
nameserversByName := map[mesh.PeerName]*Nameserver{}
for _, n := range nameservers {
nameserversByName[n.ourName] = n
}
mappings := []mapping{}
check := func(nameserver *Nameserver, expected mapping) {
have := []address.Address{}
for i := 0; i < lookupTimeout; i++ {
have = nameserver.Lookup(expected.hostname)
sort.Sort(addrs(have))
if reflect.DeepEqual(have, expected.Addrs()) {
return
}
time.Sleep(1 * time.Millisecond)
}
want := expected.Addrs()
require.Equal(t, addrs(want).String(), addrs(have).String())
}
addMapping := func() {
nameserver := nameservers[rand.Intn(len(nameservers))]
addr := address.Address(rand.Int31())
// Create a hostname which has some upper and lowercase letters,
// and a unique number so we don't have to check if we allocated it already
randomBits := rand.Int63()
firstLetter := 'H' + (randomBits&1)*32
secondLetter := 'O' + (randomBits&2)*16
randomBits = randomBits >> 2
hostname := fmt.Sprintf("%c%cstname%d", firstLetter, secondLetter, randomBits)
mapping := mapping{hostname, []pair{{nameserver.ourName, addr}}}
mappings = append(mappings, mapping)
require.Nil(t, nameserver.AddEntry(hostname, "", nameserver.ourName, addr))
check(nameserver, mapping)
}
addExtraMapping := func() {
if len(mappings) <= 0 {
return
}
nameserver := nameservers[rand.Intn(len(nameservers))]
i := rand.Intn(len(mappings))
mapping := mappings[i]
addr := address.Address(rand.Int31())
mapping.addrs = append(mapping.addrs, pair{nameserver.ourName, addr})
mappings[i] = mapping
require.Nil(t, nameserver.AddEntry(mapping.hostname, "", nameserver.ourName, addr))
check(nameserver, mapping)
}
loseConnection := func() {
nameserver1 := badNameservers[rand.Intn(len(badNameservers))]
nameserver2 := nameservers[rand.Intn(len(nameservers))]
nameserver1.PeerGone(nameserver2.ourName)
}
deleteMapping := func() {
if len(mappings) <= 0 {
return
}
i := rand.Intn(len(mappings))
mapping := mappings[i]
if len(mapping.addrs) <= 0 {
return
}
j := rand.Intn(len(mapping.addrs))
pair := mapping.addrs[j]
mapping.addrs = append(mapping.addrs[:j], mapping.addrs[j+1:]...)
mappings[i] = mapping
nameserver := nameserversByName[pair.origin]
require.Nil(t, nameserver.Delete(mapping.hostname, "*", pair.addr.String(), pair.addr))
check(nameserver, mapping)
}
doLookup := func() {
if len(mappings) <= 0 {
return
}
mapping := mappings[rand.Intn(len(mappings))]
nameserver := nameservers[rand.Intn(len(nameservers))]
check(nameserver, mapping)
}
doReverseLookup := func() {
if len(mappings) <= 0 {
return
}
mapping := mappings[rand.Intn(len(mappings))]
if len(mapping.addrs) <= 0 {
return
}
nameserver := nameservers[rand.Intn(len(nameservers))]
hostname := ""
var err error
for i := 0; i < lookupTimeout; i++ {
hostname, err = nameserver.ReverseLookup(mapping.addrs[0].addr)
if err != nil && mapping.hostname == hostname {
return
}
time.Sleep(1 * time.Millisecond)
}
require.Nil(t, err)
require.Equal(t, mapping.hostname, hostname)
}
for i := 0; i < 800; i++ {
r := rand.Float32()
switch {
case r < 0.1:
addMapping()
case 0.1 <= r && r < 0.2:
addExtraMapping()
case 0.2 <= r && r < 0.3:
deleteMapping()
case 0.3 <= r && r < 0.35:
loseConnection()
case 0.35 <= r && r < 0.9:
doLookup()
case 0.9 <= r:
doReverseLookup()
}
grouter.Flush()
}
}
func TestContainerAndPeerDeath(t *testing.T) {
peername, err := mesh.PeerNameFromString("00:00:00:02:00:00")
require.Nil(t, err)
nameserver := makeNameserver(peername)
err = nameserver.AddEntry("hostname", "containerid", peername, address.Address(0))
require.Nil(t, err)
require.Equal(t, []address.Address{0}, nameserver.Lookup("hostname"))
nameserver.ContainerDied("containerid")
require.Equal(t, []address.Address{}, nameserver.Lookup("hostname"))
err = nameserver.AddEntry("hostname", "containerid", peername, address.Address(0))
require.Nil(t, err)
require.Equal(t, []address.Address{0}, nameserver.Lookup("hostname"))
nameserver.PeerGone(peername)
require.Equal(t, []address.Address{}, nameserver.Lookup("hostname"))
}
func TestTombstoneDeletion(t *testing.T) {
oldNow := now
defer func() { now = oldNow }()
now = func() int64 { return 1234 }
peername, err := mesh.PeerNameFromString("00:00:00:02:00:00")
require.Nil(t, err)
nameserver := makeNameserver(peername)
err = nameserver.AddEntry("hostname", "containerid", peername, address.Address(0))
require.Nil(t, err)
require.Equal(t, []address.Address{0}, nameserver.Lookup("hostname"))
nameserver.deleteTombstones()
require.Equal(t, []address.Address{0}, nameserver.Lookup("hostname"))
err = nameserver.Delete("hostname", "containerid", "", address.Address(0))
require.Nil(t, err)
require.Equal(t, []address.Address{}, nameserver.Lookup("hostname"))
require.Equal(t, l(Entries{Entry{
ContainerID: "containerid",
Origin: peername,
Addr: address.Address(0),
Hostname: "hostname",
Version: 1,
Tombstone: 1234,
}}), nameserver.entries)
now = func() int64 { return 1234 + int64(tombstoneTimeout/time.Second) + 1 }
nameserver.deleteTombstones()
require.Equal(t, Entries{}, nameserver.entries)
}
-416
View File
@@ -1,416 +0,0 @@
package nameserver
import (
"bytes"
"fmt"
"net"
"strconv"
"sync"
"time"
"github.com/benbjohnson/clock"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
)
const (
DefaultServerPort = 53 // The default server port
DefaultHTTPPort = 6785 // The default http port
DefaultCLICfgFile = "/etc/resolv.conf" // default "resolv.conf" file to try to load
DefaultLocalTTL = 30 // default TTL for responses for local domain queries
DefaultUDPBuflen = 4096 // bigger than the default 512
DefaultCacheLen = 8192 // default cache capacity
DefaultTimeout = 5000 // default timeout for DNS resolutions (millisecs)
DefaultMaxAnswers = 1 // default number of answers provided to users
)
type DNSServerConfig struct {
// The zone
Zone Zone
// (Optional) client config file for resolving upstream servers
UpstreamCfgFile string
// (Optional) DNS client config for the fallback server(s)
UpstreamCfg *dns.ClientConfig
// (Optional) port number (for TCP and UDP)
Port int
// (Optional) cache size
CacheLen int
// (Optional) disable the cache
CacheDisabled bool
// (Optional) timeout for DNS queries
Timeout int
// (Optional) UDP buffer length
UDPBufLen int
// (Optional) force a specific cache
Cache ZoneCache
// (Optional) TTL for local domain responses
LocalTTL int
// (Optional) TTL for negative results in the local domain (defaults to LocalTTL)
CacheNegLocalTTL int
// (Optional) for a specific clock provider
Clock clock.Clock
// (Optional) Listening socket read timeout (in milliseconds)
ListenReadTimeout int
// Maximum number of answers provided to users
MaxAnswers int
}
type dnsProtocol uint8
const (
protUDP dnsProtocol = iota // UDP protocol
protTCP dnsProtocol = iota // TCP protocol
)
func (proto dnsProtocol) String() string {
switch proto {
case protUDP:
return "UDP"
case protTCP:
return "TCP"
}
return "unknown"
}
// get a new dns.Client for a protocol
func (proto dnsProtocol) GetNewClient(bufsize int, timeout time.Duration) *dns.Client {
Debug.Printf("[dns] Creating %s DNS client with %s timeout", proto, timeout)
switch proto {
case protTCP:
return &dns.Client{Net: "tcp", ReadTimeout: timeout}
case protUDP:
return &dns.Client{Net: "udp", ReadTimeout: timeout, UDPSize: uint16(bufsize)}
}
return nil
}
// a DNS server
type DNSServer struct {
Zone Zone
Upstream *dns.ClientConfig
Domain string // the local domain
ListenAddr string // the address the server is listening at
udpSrv *dns.Server
tcpSrv *dns.Server
pc net.PacketConn
lst net.Listener
cache ZoneCache
cacheDisabled bool
maxAnswers int
localTTL int
negLocalTTL int
timeout time.Duration
readTimeout time.Duration
udpBuf int
listenersWg *sync.WaitGroup
clock clock.Clock
}
// Creates a new DNS server
func NewDNSServer(config DNSServerConfig) (s *DNSServer, err error) {
s = &DNSServer{
Zone: config.Zone,
Domain: DefaultLocalDomain,
ListenAddr: fmt.Sprintf(":%d", config.Port),
listenersWg: new(sync.WaitGroup),
timeout: DefaultTimeout * time.Millisecond,
readTimeout: DefaultTimeout * time.Millisecond,
cacheDisabled: false,
maxAnswers: DefaultMaxAnswers,
localTTL: DefaultLocalTTL,
clock: config.Clock,
}
// check some basic parameters are valid
if s.Zone == nil {
return nil, fmt.Errorf("No valid Zone provided in server initialization")
}
if len(s.Domain) == 0 {
return nil, fmt.Errorf("No valid Domain provided in server initialization")
}
if s.clock == nil {
s.clock = clock.New()
}
// fill empty parameters with defaults...
if config.UpstreamCfg != nil {
s.Upstream = config.UpstreamCfg
} else {
cfgFile := DefaultCLICfgFile
if len(config.UpstreamCfgFile) > 0 {
cfgFile = config.UpstreamCfgFile
}
if s.Upstream, err = dns.ClientConfigFromFile(cfgFile); err != nil {
return nil, err
}
}
if config.Timeout > 0 {
s.timeout = time.Duration(config.Timeout) * time.Millisecond
}
if config.ListenReadTimeout > 0 {
s.readTimeout = time.Duration(config.ListenReadTimeout) * time.Millisecond
}
if config.UDPBufLen > 0 {
s.udpBuf = config.UDPBufLen
}
if config.MaxAnswers > 0 {
s.maxAnswers = config.MaxAnswers
}
if config.LocalTTL > 0 {
s.localTTL = config.LocalTTL
}
if config.CacheNegLocalTTL > 0 {
s.negLocalTTL = config.CacheNegLocalTTL
} else {
s.negLocalTTL = s.localTTL
}
if config.CacheDisabled {
s.cacheDisabled = true
}
if !s.cacheDisabled {
if config.Cache != nil {
s.cache = config.Cache
} else {
cacheLen := DefaultCacheLen
if config.CacheLen > 0 {
cacheLen = config.CacheLen
}
if s.cache, err = NewCache(cacheLen, s.clock); err != nil {
return
}
}
}
return
}
// Start the DNS server
func (s *DNSServer) Start() error {
Info.Printf("[dns] Upstream server(s): %+v", s.Upstream)
if s.cacheDisabled {
Info.Printf("[dns] Cache: disabled")
} else {
Info.Printf("[dns] Cache: %d entries", s.cache.Capacity())
}
pc, err := net.ListenPacket("udp", s.ListenAddr)
if err != nil {
return err
}
s.pc = pc
_, port, err := net.SplitHostPort(pc.LocalAddr().String())
if err != nil {
return err
}
s.ListenAddr = fmt.Sprintf(":%s", port)
s.udpSrv = &dns.Server{PacketConn: s.pc, Handler: s.createMux(protUDP), ReadTimeout: s.readTimeout}
// Bind the TCP socket at the same port, aborting otherwise
l, err := net.Listen("tcp", s.ListenAddr)
if err != nil {
s.Stop()
return err
}
s.lst = l
s.tcpSrv = &dns.Server{Listener: l, Handler: s.createMux(protTCP), ReadTimeout: s.readTimeout}
s.listenersWg.Add(2)
return nil
}
func (s *DNSServer) ActivateAndServe() {
go func() {
defer s.listenersWg.Done()
Info.Printf("[dns] Listening for DNS on %s (UDP)", s.ListenAddr)
err := s.udpSrv.ActivateAndServe()
CheckFatal(err)
Debug.Printf("[dns] DNS UDP server exiting...")
}()
go func() {
defer s.listenersWg.Done()
Info.Printf("[dns] Listening for DNS on %s (TCP)", s.ListenAddr)
err := s.tcpSrv.ActivateAndServe()
CheckFatal(err)
Debug.Printf("[dns] DNS TCP server exiting...")
}()
// Waiting for all goroutines to finish (otherwise they die as main routine dies)
s.listenersWg.Wait()
Info.Printf("[dns] Server exiting...")
}
// Return status string
func (s *DNSServer) Status() string {
var buf bytes.Buffer
fmt.Fprintln(&buf, "Listen address", s.ListenAddr)
fmt.Fprintln(&buf, "Fallback DNS config", s.Upstream)
return buf.String()
}
// Perform a graceful shutdown
func (s *DNSServer) Stop() error {
// Stop the listeners/handlers
if s.tcpSrv != nil {
if err := s.tcpSrv.Shutdown(); err != nil {
return err
}
s.lst.Close()
s.tcpSrv = nil
}
if s.udpSrv != nil {
if err := s.udpSrv.Shutdown(); err != nil {
return err
}
s.pc.Close()
s.udpSrv = nil
}
s.listenersWg.Wait()
return nil
}
// Create a multiplexer for requests
// We must create two DNS request multiplexers, depending on the protocol used by
// clients (as we use the same protocol for asking upstream servers)
func (s *DNSServer) createMux(proto dnsProtocol) *dns.ServeMux {
failFallback := func(w dns.ResponseWriter, r *dns.Msg) {
w.WriteMsg(makeDNSFailResponse(r))
}
notUsHandler := s.notUsHandler(proto)
notUsFallback := func(w dns.ResponseWriter, r *dns.Msg) {
Info.Printf("[dns msgid %d] -> sending to fallback server", r.MsgHdr.Id)
notUsHandler(w, r)
}
// create the multiplexer
m := dns.NewServeMux()
m.HandleFunc(s.Zone.Domain(), s.localHandler(proto, "Query", dns.TypeA,
s.Zone.DomainLookupName, makeAddressReply, s.Zone.ObserveName, failFallback))
m.HandleFunc(RDNSDomain, s.localHandler(proto, "Reverse query", dns.TypePTR,
s.Zone.DomainLookupInaddr, makePTRReply, s.Zone.ObserveInaddr, notUsFallback))
m.HandleFunc(".", s.notUsHandler(proto))
return m
}
// Process a request for a name/address in the '.weave.local.' zone
func (s *DNSServer) localHandler(proto dnsProtocol, kind string, qtype uint16,
lookup func(inaddr string) ([]ZoneRecord, error),
msgBuilder DNSResponseBuilder, observer ZoneObserverFunc, fallback dns.HandlerFunc) dns.HandlerFunc {
return func(w dns.ResponseWriter, r *dns.Msg) {
q := r.Question[0]
Debug.Printf("[dns msgid %d] %s: %+v", r.MsgHdr.Id, kind, q)
maxLen := getMaxReplyLen(r, proto)
// cache a response if the cache is enabled, and observe the name/IP
maybeCache := func(m *dns.Msg, ttl int, flags uint8) {
if !s.cacheDisabled {
s.cache.Put(r, m, ttl, flags)
// any change in the Zone database for this IP/name will lead to this
// cache entry being removed...
// TODO: this closure results in unnecessary `Remove`s and some wasted
// mem... but we can live with that.
observer(q.Name, func() { s.cache.Remove(&q) })
}
}
if !s.cacheDisabled {
reply, err := s.cache.Get(r, maxLen)
if err != nil {
if err == errNoLocalReplies {
Debug.Printf("[dns msgid %d] Cached no-local-replies", r.MsgHdr.Id)
fallback(w, r)
} else {
Debug.Printf("[dns msgid %d] Error from cache: %s", r.MsgHdr.Id, err)
w.WriteMsg(makeDNSFailResponse(r))
}
return
}
if reply != nil {
reply.Answer = pruneAnswers(shuffleAnswers(reply.Answer), s.maxAnswers)
Debug.Printf("[dns msgid %d] Returning reply from cache: %s/%d answers",
r.MsgHdr.Id, dns.RcodeToString[reply.MsgHdr.Rcode], len(reply.Answer))
w.WriteMsg(reply)
return
}
}
// catch unsupported queries
if q.Qtype != qtype {
Debug.Printf("[dns msgid %d] Unsupported query type %s", r.MsgHdr.Id, dns.TypeToString[q.Qtype])
m := makeDNSFailResponse(r)
maybeCache(m, s.negLocalTTL, 0)
w.WriteMsg(m)
return
}
if answers, err := lookup(q.Name); err != nil {
Info.Printf("[dns msgid %d] No results for type %s query for '%s' [caching no-local]",
r.MsgHdr.Id, dns.TypeToString[q.Qtype], q.Name)
maybeCache(nil, s.negLocalTTL, CacheNoLocalReplies)
fallback(w, r)
} else {
m := msgBuilder(r, &q, answers, s.localTTL)
m.Authoritative = true
maybeCache(m, nullTTL, 0)
m.Answer = pruneAnswers(shuffleAnswers(m.Answer), s.maxAnswers)
Debug.Printf("[dns msgid %d] Sending response: %s/%d answers [code:%s]",
m.MsgHdr.Id, dns.TypeToString[q.Qtype], len(m.Answer), dns.RcodeToString[m.Rcode])
w.WriteMsg(m)
}
}
}
// When we receive a request for a name outside of our '.weave.local.'
// domain, ask the configured DNS server as a fallback.
func (s *DNSServer) notUsHandler(proto dnsProtocol) dns.HandlerFunc {
dnsClient := proto.GetNewClient(DefaultUDPBuflen, s.timeout)
return func(w dns.ResponseWriter, r *dns.Msg) {
q := r.Question[0]
// announce our max payload size as the max payload our client supports
maxLen := getMaxReplyLen(r, proto)
rcopy := r
rcopy.SetEdns0(uint16(maxLen), false)
Debug.Printf("[dns msgid %d] Fallback query: %+v [%s, max:%d bytes]", rcopy.MsgHdr.Id, q, proto, maxLen)
for _, server := range s.Upstream.Servers {
reply, _, err := dnsClient.Exchange(rcopy, fmt.Sprintf("%s:%s", server, s.Upstream.Port))
if err != nil {
Debug.Printf("[dns msgid %d] Network error trying %s (%s)",
r.MsgHdr.Id, server, err)
continue
}
if reply != nil && reply.Rcode != dns.RcodeSuccess {
Debug.Printf("[dns msgid %d] Failure reported by %s for query %s",
r.MsgHdr.Id, server, q.Name)
continue
}
Debug.Printf("[dns msgid %d] Given answer by %s for query %s",
r.MsgHdr.Id, server, q.Name)
if reply.Len() > maxLen {
reply.Compress = true
}
w.WriteMsg(reply)
return
}
Warning.Printf("[dns msgid %d] Failed lookup for external name %s", r.MsgHdr.Id, q.Name)
w.WriteMsg(makeDNSFailResponse(r))
}
}
// Get the listen port
func (s *DNSServer) GetPort() (int, error) {
_, portS, err := net.SplitHostPort(s.ListenAddr)
if err != nil {
return 0, err
}
return strconv.Atoi(portS)
}
-351
View File
@@ -1,351 +0,0 @@
package nameserver
import (
"fmt"
"net"
"testing"
"time"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
wt "github.com/weaveworks/weave/testing"
)
// Check that AddRecord/DeleteRecord/... in the Zone database lead to cache invalidations
func TestServerDbCacheInvalidation(t *testing.T) {
const (
containerID = "somecontainer"
testName1 = "first.weave.local."
testName2 = "second.weave.local."
)
InitDefaultLogging(testing.Verbose())
Info.Println("TestServerDbCacheInvalidation starting")
clk := newMockedClock()
Debug.Printf("Creating mocked mDNS client and server")
mdnsServer1 := newMockedMDNSServerWithRecord(Record{testName1, net.ParseIP("10.2.2.9"), 0, 0, 0})
mdnsCli1 := newMockedMDNSClient([]*mockedMDNSServer{mdnsServer1})
Debug.Printf("Creating zone database with the mocked mDNS client and server")
zoneConfig := ZoneConfig{
MDNSServer: mdnsServer1,
MDNSClient: mdnsCli1,
Clock: clk,
}
zone, err := NewZoneDb(zoneConfig)
wt.AssertNoErr(t, err)
err = zone.Start()
wt.AssertNoErr(t, err)
defer zone.Stop()
Debug.Printf("Creating a cache")
cache, err := NewCache(1024, clk)
wt.AssertNoErr(t, err)
fallbackHandler := func(w dns.ResponseWriter, req *dns.Msg) {
m := new(dns.Msg)
m.SetReply(req)
if len(req.Question) == 1 {
m.Rcode = dns.RcodeNameError
}
w.WriteMsg(m)
}
// Run another DNS server for fallback
fallback, err := newMockedFallback(fallbackHandler, nil)
wt.AssertNoErr(t, err)
fallback.Start()
defer fallback.Stop()
Debug.Printf("Creating a real DNS server with a mocked cache")
srv, err := NewDNSServer(DNSServerConfig{
Zone: zone,
Cache: cache,
Clock: clk,
ListenReadTimeout: testSocketTimeout,
UpstreamCfg: fallback.CliConfig,
MaxAnswers: 4,
})
wt.AssertNoErr(t, err)
err = srv.Start()
wt.AssertNoErr(t, err)
go srv.ActivateAndServe()
defer srv.Stop()
time.Sleep(100 * time.Millisecond) // Allow server goroutine to start
testPort, err := srv.GetPort()
wt.AssertNoErr(t, err)
wt.AssertNotEqualInt(t, testPort, 0, "invalid listen port")
Debug.Printf("Adding two IPs to %s", testName1)
zone.AddRecord(containerID, testName1, net.ParseIP("10.2.2.1"))
zone.AddRecord(containerID, testName1, net.ParseIP("10.2.2.2"))
q, _ := assertExchange(t, testName1, dns.TypeA, testPort, 2, 2, 0)
assertInCache(t, cache, q, fmt.Sprintf("after asking for %s", testName1))
// Zone database at this point:
// first.weave.local = 10.2.2.1 10.2.2.2
zone.AddRecord(containerID, testName2, net.ParseIP("10.9.9.1"))
assertInCache(t, cache, q, fmt.Sprintf("after adding a new IP for %s", testName2))
// we should have an entry in the cache for this query
// if we add another IP, that cache entry should be removed
Debug.Printf("Adding a new IP to %s: the cache entry should be removed", testName1)
zone.AddRecord(containerID, testName1, net.ParseIP("10.2.2.3"))
assertNotInCache(t, cache, q, fmt.Sprintf("after adding a new IP for %s", testName1))
// Zone database at this point:
// first.weave.local = 10.2.2.1 10.2.2.2 10.2.2.3
// second.weave.local = 10.9.9.1
Debug.Printf("Querying again (so a cache entry will be created)")
q, _ = assertExchange(t, testName1, dns.TypeA, testPort, 3, 4, 0)
assertInCache(t, cache, q, "after asking about the name")
Debug.Printf("... and removing one of the IP addresses")
zone.DeleteRecords(containerID, "", net.ParseIP("10.2.2.2"))
assertNotInCache(t, cache, q, "after deleting IP for 10.2.2.2")
// Zone database at this point:
// first.weave.local = 10.2.2.1 10.2.2.3
// second.weave.local = 10.9.9.1
// generate cache responses
Debug.Printf("Querying for a raddr")
qname, _ := assertExchange(t, testName1, dns.TypeA, testPort, 2, 2, 0)
qptr, _ := assertExchange(t, "1.2.2.10.in-addr.arpa.", dns.TypePTR, testPort, 1, 1, 0)
qotherName, _ := assertExchange(t, testName2, dns.TypeA, testPort, 1, 1, 0)
qotherPtr, _ := assertExchange(t, "1.9.9.10.in-addr.arpa.", dns.TypePTR, testPort, 1, 1, 0)
qwrongName, _ := assertExchange(t, "wrong.weave.local.", dns.TypeA, testPort, 0, 0, dns.RcodeNameError)
assertInCache(t, cache, qname, "after asking for name")
assertInCache(t, cache, qptr, "after asking for address")
assertInCache(t, cache, qotherName, "after asking for second name")
assertInCache(t, cache, qotherPtr, "after asking for second address")
assertNotLocalInCache(t, cache, qwrongName, "after asking for a wrong name")
// now we will check if a removal affects all the responses
Debug.Printf("... and removing an IP should invalidate both the cached responses for name and raddr")
zone.DeleteRecords(containerID, "", net.ParseIP("10.2.2.1"))
assertNotInCache(t, cache, qptr, "after deleting record")
assertNotInCache(t, cache, qname, "after deleting record")
assertInCache(t, cache, qotherName, "after deleting record")
// Zone database at this point:
// first.weave.local = 10.2.2.3
// second.weave.local = 10.9.9.1
// generate cache responses
Debug.Printf("Querying for a raddr")
qptr, _ = assertExchange(t, "3.2.2.10.in-addr.arpa.", dns.TypePTR, testPort, 1, 1, 0)
qname, _ = assertExchange(t, testName1, dns.TypeA, testPort, 1, 1, 0)
qotherName, _ = assertExchange(t, testName2, dns.TypeA, testPort, 1, 1, 0)
qotherPtr, _ = assertExchange(t, "1.9.9.10.in-addr.arpa.", dns.TypePTR, testPort, 1, 1, 0)
assertInCache(t, cache, qname, "after asking for name")
assertInCache(t, cache, qptr, "after asking for PTR")
assertInCache(t, cache, qotherName, "after asking for second name")
assertInCache(t, cache, qotherPtr, "after asking for second address")
// let's repeat this, but adding an IP
Debug.Printf("... and adding a new IP should invalidate both the cached responses for the name")
zone.AddRecord(containerID, testName1, net.ParseIP("10.2.2.7"))
assertNotInCache(t, cache, qname, "after adding a new IP")
assertInCache(t, cache, qotherName, "after adding a new IP")
assertInCache(t, cache, qotherPtr, "after adding a new IP")
// check that after some time, the cache entry is expired
clk.Forward(int(DefaultLocalTTL) + 1)
assertNotInCache(t, cache, qotherName, "after passing some time")
assertNotInCache(t, cache, qwrongName, "after passing some time")
// Zone database at this point:
// first.weave.local = 10.2.2.3 10.2.2.7
// second.weave.local = 10.9.9.1
zone.DeleteRecords(containerID, "", net.IP{})
assertNotInCache(t, cache, qotherName, "after removing container")
assertNotInCache(t, cache, qotherPtr, "after removing container")
}
// Check if the entries in the cache expire
func TestServerCacheExpiration(t *testing.T) {
const (
containerID = "somecontainer"
testName1 = "first.weave.local."
testName2 = "second.weave.local."
negativeLocalTTL = 10
)
InitDefaultLogging(testing.Verbose())
Info.Println("TestServerCacheExpiration starting")
clk := newMockedClock()
Debug.Printf("Creating 2 zone databases")
zoneConfig := ZoneConfig{
RefreshInterval: 0, // no name updates
Clock: clk,
}
dbs := newZoneDbsWithMockedMDns(2, zoneConfig)
dbs.Start()
defer dbs.Stop()
Debug.Printf("Creating a cache")
cache, err := NewCache(1024, clk)
wt.AssertNoErr(t, err)
Debug.Printf("Creating a real DNS server for the first zone database and with the cache")
srv, err := NewDNSServer(DNSServerConfig{
Zone: dbs[0].Zone,
Cache: cache,
Clock: clk,
ListenReadTimeout: testSocketTimeout,
CacheNegLocalTTL: negativeLocalTTL,
})
wt.AssertNoErr(t, err)
err = srv.Start()
wt.AssertNoErr(t, err)
go srv.ActivateAndServe()
defer srv.Stop()
time.Sleep(100 * time.Millisecond) // Allow server goroutine to start
testPort, err := srv.GetPort()
wt.AssertNoErr(t, err)
wt.AssertNotEqualInt(t, testPort, 0, "invalid listen port")
// Check that the DNS server knows nothing about testName1
qName1, _ := assertExchange(t, testName1, dns.TypeA, testPort, 0, 0, dns.RcodeNameError)
// there should be a negative reply cached during [0, negativeLocalTTL]
assertNotLocalInCache(t, cache, qName1, fmt.Sprintf("after asking for %s", testName1))
clk.Forward(negativeLocalTTL - 1)
assertNotLocalInCache(t, cache, qName1, fmt.Sprintf("after asking for %s", testName1))
// after negativeLocalTTL, there should be absolutely nothing in the cache
clk.Forward(2)
assertNotInCache(t, cache, qName1, fmt.Sprintf("after asking for %s", testName1))
// We add the IP in the second peer right after
Debug.Printf("Adding an IPs to %s in database #2", testName1)
dbs[1].Zone.AddRecord(containerID, testName1, net.ParseIP("10.2.2.1"))
t.Logf("Zone database #2:\n%s", dbs[1].Zone)
// Check that the DNS server returns updated information, and it will be
// cached during [0, DefaultLocalTTL]
qName1, _ = assertExchange(t, testName1, dns.TypeA, testPort, 1, 1, 0)
assertInCache(t, cache, qName1, fmt.Sprintf("after asking for %s", testName1))
clk.Forward(int(DefaultLocalTTL) - 1)
assertInCache(t, cache, qName1, fmt.Sprintf("after asking for %s", testName1))
// and it will expire after DefaultLocalTTL
clk.Forward(2)
assertNotInCache(t, cache, qName1, fmt.Sprintf("after asking for %s", testName1))
// We delete the IP in the second peer right after
Debug.Printf("Removing the IPs from %s in database #2", testName1)
dbs[1].Zone.DeleteRecords(containerID, "", net.ParseIP("10.2.2.1"))
// Check that we return to the initial state: a neg-local entry in the cache
qName1, _ = assertExchange(t, testName1, dns.TypeA, testPort, 0, 0, dns.RcodeNameError)
assertNotLocalInCache(t, cache, qName1, fmt.Sprintf("after asking for %s", testName1))
clk.Forward(negativeLocalTTL + 1)
// Check that all entries should be outdated
cache.Purge()
if cache.Len() != 0 {
t.Fatalf("Some entries still valid in cache:\n%s", cache)
}
}
// Check if the names updates lead to cache invalidations
func TestServerCacheRefresh(t *testing.T) {
const (
containerID = "somecontainer"
testName1 = "first.weave.local."
testName2 = "second.weave.local."
refreshInterval = int(DefaultLocalTTL) / 3
)
InitDefaultLogging(testing.Verbose())
Info.Println("TestServerCacheRefresh starting")
clk := newMockedClock()
Debug.Printf("Creating 2 zone databases")
zoneConfig := ZoneConfig{
RefreshInterval: refreshInterval,
Clock: clk,
}
dbs := newZoneDbsWithMockedMDns(2, zoneConfig)
dbs.Start()
defer dbs.Stop()
Debug.Printf("Creating a cache")
cache, err := NewCache(1024, clk)
wt.AssertNoErr(t, err)
Debug.Printf("Creating a real DNS server for the first zone database and with the cache")
srv, err := NewDNSServer(DNSServerConfig{
Zone: dbs[0].Zone,
Cache: cache,
Clock: clk,
ListenReadTimeout: testSocketTimeout,
MaxAnswers: 4,
})
wt.AssertNoErr(t, err)
err = srv.Start()
wt.AssertNoErr(t, err)
go srv.ActivateAndServe()
defer srv.Stop()
time.Sleep(100 * time.Millisecond) // Allow sever goroutine to start
testPort, err := srv.GetPort()
wt.AssertNoErr(t, err)
wt.AssertNotEqualInt(t, testPort, 0, "listen port")
Debug.Printf("Adding an IPs to %s", testName1)
dbs[1].Zone.AddRecord(containerID, testName1, net.ParseIP("10.2.2.1"))
dbs.Flush() // ensure the immediate refresh is done
// Zone database #2 at this point:
// first.weave.local = 10.2.2.1
// testName1 and testName2 should have no IPs yet
qName1, _ := assertExchange(t, testName1, dns.TypeA, testPort, 1, 1, 0)
qName2, _ := assertExchange(t, testName2, dns.TypeA, testPort, 0, 0, dns.RcodeNameError)
assertInCache(t, cache, qName1, "after asking for first name")
assertNotLocalInCache(t, cache, qName2, "after asking for second name")
clk.Forward(refreshInterval / 2)
dbs.Flush()
Debug.Printf("Adding an IP to %s and to %s", testName1, testName2)
dbs[1].Zone.AddRecord(containerID, testName1, net.ParseIP("10.2.2.2"))
dbs[1].Zone.AddRecord(containerID, testName2, net.ParseIP("10.9.9.2"))
// Zone database #2 at this point:
// first.weave.local = 10.2.2.1 10.2.2.2
// second.weave.local = 10.9.9.2
clk.Forward(refreshInterval/2 + 2)
dbs.Flush()
// at this point, the testName1 should have been refreshed
// so it should have two IPs, and the cache entry should have been invalidated
assertNotInCache(t, cache, qName1, fmt.Sprintf("after asking for %s", testName1))
assertNotLocalInCache(t, cache, qName2, fmt.Sprintf("after asking for %s", testName2))
qName1, _ = assertExchange(t, testName1, dns.TypeA, testPort, 2, 2, 0)
qName2, _ = assertExchange(t, testName2, dns.TypeA, testPort, 0, 0, dns.RcodeNameError)
assertInCache(t, cache, qName1, fmt.Sprintf("after asking for %s", testName1))
assertNotLocalInCache(t, cache, qName2, "after asking for a unknown name")
// delete the IPs, and some time passes by so the cache should be purged...
dbs[1].Zone.DeleteRecords(containerID, "", net.ParseIP("10.2.2.1"))
dbs[1].Zone.DeleteRecords(containerID, "", net.ParseIP("10.2.2.2"))
clk.Forward(refreshInterval + 1)
dbs.Flush()
qName1, _ = assertExchange(t, testName1, dns.TypeA, testPort, 0, 0, dns.RcodeNameError)
qName2, _ = assertExchange(t, testName2, dns.TypeA, testPort, 0, 0, dns.RcodeNameError)
assertNotLocalInCache(t, cache, qName1, "after asking for a unknown name")
assertNotLocalInCache(t, cache, qName2, "after asking for a unknown name")
}
-273
View File
@@ -1,273 +0,0 @@
package nameserver
import (
"encoding/binary"
"fmt"
"math/rand"
"net"
"testing"
"time"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/ipam/address"
wt "github.com/weaveworks/weave/testing"
)
const (
testRDNSsuccess = "1.2.2.10.in-addr.arpa."
testRDNSfail = "4.3.2.1.in-addr.arpa."
testRDNSnonlocal = "8.8.8.8.in-addr.arpa."
testUDPBufSize = 16384
)
func setupForTest(t *testing.T) {
// fail early if we cannot find a default multicast interface
multicast, err := LinkLocalMulticastListener(nil)
if err != nil {
t.Fatalf("Unable to create multicast listener: %s. No default multicast interface?", err)
}
multicast.Close()
}
func TestUDPDNSServer(t *testing.T) {
setupForTest(t)
const (
successTestName = "test1.weave.local."
failTestName = "fail.weave.local."
compressedTestName = "compressed.weave.works."
nonLocalName = "weave.works."
testAddr1 = "10.2.2.1"
containerID = "somecontainer"
)
testCIDR1 := testAddr1 + "/24"
InitDefaultLogging(testing.Verbose())
Info.Println("TestUDPDNSServer starting")
zone, err := NewZoneDb(ZoneConfig{})
wt.AssertNoErr(t, err)
err = zone.Start()
wt.AssertNoErr(t, err)
defer zone.Stop()
ip, _, _ := net.ParseCIDR(testCIDR1)
zone.AddRecord(containerID, successTestName, ip)
fallbackHandler := func(w dns.ResponseWriter, req *dns.Msg) {
if len(req.Question) == 0 {
return // ignore empty queries (sent when shutting down the server)
}
m := new(dns.Msg)
m.SetReply(req)
q := req.Question[0]
if q.Name == nonLocalName && q.Qtype == dns.TypeMX {
m.Answer = make([]dns.RR, 1)
m.Answer[0] = &dns.MX{Hdr: dns.RR_Header{Name: m.Question[0].Name, Rrtype: dns.TypeMX, Class: dns.ClassINET, Ttl: 0}, Mx: "mail." + nonLocalName}
} else if q.Name == nonLocalName && q.Qtype == dns.TypeANY {
m.Answer = make([]dns.RR, 512/len("mailn."+nonLocalName)+1)
for i := range m.Answer {
m.Answer[i] = &dns.MX{Hdr: dns.RR_Header{Name: m.Question[0].Name, Rrtype: dns.TypeMX, Class: dns.ClassINET, Ttl: 0}, Mx: fmt.Sprintf("mail%d.%s", i, nonLocalName)}
}
} else if q.Name == testRDNSnonlocal && q.Qtype == dns.TypePTR {
m.Answer = make([]dns.RR, 1)
m.Answer[0] = &dns.PTR{Hdr: dns.RR_Header{Name: m.Question[0].Name, Rrtype: dns.TypePTR, Class: dns.ClassINET, Ttl: 0}, Ptr: "ns1.google.com."}
} else if q.Name == testRDNSfail && q.Qtype == dns.TypePTR {
m.Rcode = dns.RcodeNameError
} else if q.Name == compressedTestName {
// Construct a response that is >512 when uncompressed, <512 when compressed
header := makeHeader(m, &q, 300)
for m.Len() <= minUDPSize {
ip := address.Address(rand.Uint32()).IP4()
m.Answer = append(m.Answer, &dns.A{Hdr: *header, A: ip})
}
m.Compress = true
wt.AssertTrue(t, m.Len() <= minUDPSize, fmt.Sprintf("length: %d", m.Len()))
}
w.WriteMsg(m)
}
// Run another DNS server for fallback
fallback, err := newMockedFallback(fallbackHandler, nil)
wt.AssertNoErr(t, err)
fallback.Start()
defer fallback.Stop()
srv, err := NewDNSServer(DNSServerConfig{
Zone: zone,
UpstreamCfg: fallback.CliConfig,
CacheDisabled: true,
ListenReadTimeout: testSocketTimeout,
})
wt.AssertNoErr(t, err)
err = srv.Start()
wt.AssertNoErr(t, err)
go srv.ActivateAndServe()
defer srv.Stop()
time.Sleep(100 * time.Millisecond) // Allow sever goroutine to start
var r *dns.Msg
testPort, err := srv.GetPort()
wt.AssertNoErr(t, err)
wt.AssertNotEqualInt(t, testPort, 0, "invalid listen port")
_, r = assertExchange(t, successTestName, dns.TypeA, testPort, 1, 1, 0)
wt.AssertType(t, r.Answer[0], (*dns.A)(nil), "DNS record")
wt.AssertEqualString(t, r.Answer[0].(*dns.A).A.String(), testAddr1, "IP address")
assertExchange(t, failTestName, dns.TypeA, testPort, 0, 0, dns.RcodeNameError)
_, r = assertExchange(t, testRDNSsuccess, dns.TypePTR, testPort, 1, 1, 0)
wt.AssertType(t, r.Answer[0], (*dns.PTR)(nil), "DNS record")
wt.AssertEqualString(t, r.Answer[0].(*dns.PTR).Ptr, successTestName, "IP address")
assertExchange(t, testRDNSfail, dns.TypePTR, testPort, 0, 0, dns.RcodeNameError)
// This should fail because we don't handle MX records
assertExchange(t, successTestName, dns.TypeMX, testPort, 0, 0, dns.RcodeNameError)
// This non-local query for an MX record should succeed by being
// passed on to the fallback server
assertExchange(t, nonLocalName, dns.TypeMX, testPort, 1, -1, 0)
// Now ask a query that we expect to return a lot of data.
assertExchange(t, nonLocalName, dns.TypeANY, testPort, 5, -1, 0)
assertExchange(t, testRDNSnonlocal, dns.TypePTR, testPort, 1, -1, 0)
_, r = assertExchange(t, compressedTestName, dns.TypeA, testPort, -1, -1, 0)
wt.AssertTrue(t, r.Len() > minUDPSize, "unexpected message length")
// Not testing MDNS functionality of server here (yet), since it
// needs two servers, each listening on its own address
}
func TestTCPDNSServer(t *testing.T) {
setupForTest(t)
const (
numAnswers = 512
nonLocalName = "weave.works."
)
InitDefaultLogging(testing.Verbose())
Info.Println("TestTCPDNSServer starting")
zone, err := NewZoneDb(ZoneConfig{})
wt.AssertNoErr(t, err)
err = zone.Start()
wt.AssertNoErr(t, err)
defer zone.Stop()
// generate a list of `numAnswers` IP addresses
var addrs []ZoneRecord
bs := make([]byte, 4)
for i := 0; i < numAnswers; i++ {
binary.LittleEndian.PutUint32(bs, uint32(i))
ip := net.IPv4(bs[0], bs[1], bs[2], bs[3])
addrs = append(addrs, ZoneRecord(Record{"", ip, 0, 0, 0}))
}
// handler for the fallback server: it will just return a very long response
fallbackUDPHandler := func(w dns.ResponseWriter, req *dns.Msg) {
if len(req.Question) == 0 {
return // ignore empty queries (sent when shutting down the server)
}
maxLen := getMaxReplyLen(req, protUDP)
t.Logf("Fallback UDP server got asked: returning %d answers", numAnswers)
q := req.Question[0]
m := makeAddressReply(req, &q, addrs, DefaultLocalTTL)
mLen := m.Len()
m.SetEdns0(uint16(maxLen), false)
if mLen > maxLen {
t.Logf("... truncated response (%d > %d)", mLen, maxLen)
m.Truncated = true
}
w.WriteMsg(m)
}
fallbackTCPHandler := func(w dns.ResponseWriter, req *dns.Msg) {
if len(req.Question) == 0 {
return // ignore empty queries (sent when shutting down the server)
}
t.Logf("Fallback TCP server got asked: returning %d answers", numAnswers)
q := req.Question[0]
m := makeAddressReply(req, &q, addrs, DefaultLocalTTL)
w.WriteMsg(m)
}
t.Logf("Running a DNS fallback server with UDP")
fallback, err := newMockedFallback(fallbackUDPHandler, fallbackTCPHandler)
wt.AssertNoErr(t, err)
fallback.Start()
defer fallback.Stop()
t.Logf("Creating a WeaveDNS server instance, falling back to 127.0.0.1:%d", fallback.Port)
srv, err := NewDNSServer(DNSServerConfig{
Zone: zone,
UpstreamCfg: fallback.CliConfig,
CacheDisabled: true,
ListenReadTimeout: testSocketTimeout,
})
wt.AssertNoErr(t, err)
err = srv.Start()
wt.AssertNoErr(t, err)
go srv.ActivateAndServe()
defer srv.Stop()
time.Sleep(100 * time.Millisecond) // Allow sever goroutine to start
testPort, err := srv.GetPort()
wt.AssertNoErr(t, err)
wt.AssertNotEqualInt(t, testPort, 0, "listen port")
dnsAddr := fmt.Sprintf("127.0.0.1:%d", testPort)
t.Logf("Creating a UDP and a TCP client")
uc := new(dns.Client)
uc.UDPSize = minUDPSize
tc := new(dns.Client)
tc.Net = "tcp"
t.Logf("Creating DNS query message")
m := new(dns.Msg)
m.RecursionDesired = true
m.SetQuestion(nonLocalName, dns.TypeA)
t.Logf("Checking the fallback server at %s returns a truncated response with UDP", fallback.Addr)
r, _, err := uc.Exchange(m, fallback.Addr)
t.Logf("Got response from fallback server (UDP) with %d answers", len(r.Answer))
t.Logf("Response:\n%+v\n", r)
wt.AssertNoErr(t, err)
wt.AssertTrue(t, r.MsgHdr.Truncated, "DNS truncated reponse flag")
wt.AssertNotEqualInt(t, len(r.Answer), numAnswers, "number of answers (UDP)")
t.Logf("Checking the WeaveDNS server at %s returns a truncated reponse with UDP", dnsAddr)
r, _, err = uc.Exchange(m, dnsAddr)
t.Logf("UDP Response:\n%+v\n", r)
wt.AssertNoErr(t, err)
wt.AssertNotNil(t, r, "response")
t.Logf("%d answers", len(r.Answer))
wt.AssertTrue(t, r.MsgHdr.Truncated, "DNS truncated reponse flag")
wt.AssertNotEqualInt(t, len(r.Answer), numAnswers, "number of answers (UDP)")
t.Logf("Checking the WeaveDNS server at %s does not return a truncated reponse with TCP", dnsAddr)
r, _, err = tc.Exchange(m, dnsAddr)
t.Logf("TCP Response:\n%+v\n", r)
wt.AssertNoErr(t, err)
wt.AssertNotNil(t, r, "response")
t.Logf("%d answers", len(r.Answer))
wt.AssertFalse(t, r.MsgHdr.Truncated, "DNS truncated response flag")
wt.AssertEqualInt(t, len(r.Answer), numAnswers, "number of answers (TCP)")
t.Logf("Checking the WeaveDNS server at %s does not return a truncated reponse with UDP with a bigger buffer", dnsAddr)
m.SetEdns0(testUDPBufSize, false)
r, _, err = uc.Exchange(m, dnsAddr)
t.Logf("UDP-large Response:\n%+v\n", r)
wt.AssertNoErr(t, err)
wt.AssertNotNil(t, r, "response")
t.Logf("%d answers", len(r.Answer))
wt.AssertNoErr(t, err)
wt.AssertFalse(t, r.MsgHdr.Truncated, "DNS truncated response flag")
wt.AssertEqualInt(t, len(r.Answer), numAnswers, "number of answers (UDP-long)")
}
+43
View File
@@ -0,0 +1,43 @@
package nameserver
type Status struct {
Domain string
Address string
TTL uint32
Entries []EntryStatus
}
type EntryStatus struct {
Hostname string
Origin string
ContainerID string
Address string
Version int
Tombstone int64
}
func NewStatus(ns *Nameserver, dnsServer *DNSServer) *Status {
if dnsServer == nil {
return nil
}
ns.RLock()
defer ns.RUnlock()
var entryStatusSlice []EntryStatus
for _, entry := range ns.entries {
entryStatusSlice = append(entryStatusSlice, EntryStatus{
entry.Hostname,
entry.Origin.String(),
entry.ContainerID,
entry.Addr.String(),
entry.Version,
entry.Tombstone})
}
return &Status{
dnsServer.domain,
dnsServer.address,
dnsServer.ttl,
entryStatusSlice}
}
-739
View File
@@ -1,739 +0,0 @@
package nameserver
import (
"bytes"
"fmt"
"net"
"sync"
"time"
"github.com/benbjohnson/clock"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
)
const (
RDNSDomain = "in-addr.arpa."
)
const (
DefaultLocalDomain = "weave.local." // The default name used for the local domain
DefaultRefreshInterval = int(DefaultLocalTTL) // Period for background updates with mDNS
DefaultRelevantTime = 60 // When to forget info about remote info if nobody asks...
defaultRemoteIdent = "weave:remote" // Ident used for info obtained from mDNS
)
// +1 to also exclude a dot
var rdnsDomainLen = len(RDNSDomain) + 1
type LookupError string
func (ops LookupError) Error() string {
return "Unable to find " + string(ops)
}
type DuplicateError struct {
}
func (dup DuplicateError) Error() string {
return "Tried to add a duplicate entry"
}
type Zone interface {
ZoneObservable
ZoneLookup
// The domain where we operate (eg, "weave.local.")
Domain() string
// Add a record in the local database
AddRecord(ident string, name string, ip net.IP) error
// Delete matching records (uninitialised values act as wildcards)
DeleteRecords(ident string, name string, ip net.IP) (count int)
// Lookup for a name in the whole domain
DomainLookupName(name string) ([]ZoneRecord, error)
// Lookup for an address in the whole domain
DomainLookupInaddr(inaddr string) ([]ZoneRecord, error)
// Return a status string
Status() string
}
///////////////////////////////////////////////////////////////////
//
// Zone database overview:
//
// idents names entry
// *-------* *------* *------*
// zone -> | ident | --*--> | name | --*-> | IPv4 |
// *-------* | *------* | *------*
// | ident | *--> | name | *-> | IPv4 |
// *-------* *------* *------*
// ... ...
//
// The zone database keeps an ident per container, but also a special
// ident for names/IPs obtained from remote peers.
//
// Each name can have multiple IPs (so far, we only consider IPv4s), and
// each IP is stored in a `recordEntry` in the database. Entries store
// some additional information like priority and weight of the IP (for
// example, for future use with SRV records)
//
// An entry in the zone database
type recordEntry struct {
Record
observers []ZoneRecordObserver // the observers for this record
insTime time.Time // time when this record was inserted
}
func newRecordEntry(zr ZoneRecord, now time.Time) *recordEntry {
return &recordEntry{
Record: Record{
name: zr.Name(),
ip: zr.IP(),
priority: zr.Priority(),
weight: zr.Weight(),
ttl: zr.TTL(),
},
insTime: now,
observers: make([]ZoneRecordObserver, 0),
}
}
func (re *recordEntry) addIPObserver(zro ZoneRecordObserver) {
re.observers = append(re.observers, zro)
}
// Notify all the IP observers and then remove them all
// IP address observers are notified when
// - the IP is removed
// - the name is removed
func (re *recordEntry) notifyIPObservers() {
numObservers := len(re.observers)
if numObservers > 0 {
Debug.Printf("[zonedb] Notifying %d observers of '%s'", numObservers, re.ip)
for _, observer := range re.observers {
observer()
}
re.observers = make([]ZoneRecordObserver, 0)
}
}
func (re *recordEntry) TTL() int {
// if we never set the TTL (eg, when using AddRecord()), return the standard value...
if re.ttl == 0 {
return int(DefaultLocalTTL)
}
return re.ttl
}
// Check if the info in this record is still valid according to the TTL
func (re *recordEntry) hasExpired(now time.Time) bool {
return re.ttl > 0 && now.Sub(re.insTime).Seconds() > float64(re.ttl)
}
func (re *recordEntry) String() string {
var buf bytes.Buffer
fmt.Fprintf(&buf, "%s", re.Record)
lobs := len(re.observers)
if lobs > 0 {
fmt.Fprintf(&buf, "/OBS:%d", lobs)
}
return buf.String()
}
///////////////////////////////////////////////////////////////////
// a name, with group of records by IPv4
type name struct {
name string
ipv4 map[IPv4]*recordEntry // all the IPv4 records for this name
observers []ZoneRecordObserver // the observers for this name
lastRefreshTime time.Time // last time this name was updated by broadcasting a query
lastAccessTime time.Time // last time a Lookup* used info from this name
}
func newName(n string) *name {
return &name{
name: n,
ipv4: make(map[IPv4]*recordEntry),
observers: make([]ZoneRecordObserver, 0),
}
}
func (n *name) len() int { return len(n.ipv4) }
func (n *name) empty() bool { return n.len() == 0 }
func (n *name) hasIP(ip net.IP) bool { return n.hasIPv4(ipToIPv4(ip)) }
func (n *name) hasIPv4(ip IPv4) bool { _, found := n.ipv4[ip]; return found }
// Get all the entries for this name
func (n *name) getAllEntries() (res []*recordEntry) {
for _, entry := range n.ipv4 {
res = append(res, entry)
}
return
}
// Get the entry for an IP in this name
func (n *name) getEntryForIP(ip IPv4) (res *recordEntry) {
if entry, found := n.ipv4[ip]; found {
res = entry
}
return
}
// Add a new IPv4 to this name
func (n *name) addIP(zr ZoneRecord, now time.Time) (*recordEntry, error) {
ne := newRecordEntry(zr, now)
ipv4 := ipToIPv4(zr.IP())
n.ipv4[ipv4] = ne
n.notifyNameObservers()
return ne, nil
}
// Delete by IP. Pass uninitialized value for wildcard
func (n *name) deleteIP(ip net.IP) int {
count := 0
if !ip.Equal(net.IP{}) {
ipv4 := ipToIPv4(ip)
if ipv4V, found := n.ipv4[ipv4]; found {
ipv4V.notifyIPObservers()
delete(n.ipv4, ipv4)
count++
}
} else {
for ipv4K, ipv4V := range n.ipv4 {
ipv4V.notifyIPObservers()
delete(n.ipv4, ipv4K)
count++
}
}
if count > 0 {
n.notifyNameObservers()
}
return count
}
// Update the list of IPs for this name, adding new ones and removing old IPs...
func (n *name) updateIPs(records []ZoneRecord, now time.Time) (added int, removed int) {
ipsMap := make(map[IPv4]bool)
// add the new IPs
for _, record := range records {
ipv4 := ipToIPv4(record.IP())
ipsMap[ipv4] = true
if !n.hasIPv4(ipv4) {
n.addIP(record, now)
added++
}
}
// remove the old IPs
for ipv4, ipRecord := range n.ipv4 {
if _, isNew := ipsMap[ipv4]; !isNew {
ipRecord.notifyIPObservers() // we must notify the observers before removing the IP record...
delete(n.ipv4, ipv4)
removed++
}
}
// notify observers and update times
if added > 0 || removed > 0 {
n.lastRefreshTime = now
n.notifyNameObservers()
}
return
}
// Add a observer for this name
func (n *name) addNameObserver(observer ZoneRecordObserver) {
n.observers = append(n.observers, observer)
}
// Notify and flush all the observers
// Name observers are notified when
// - an IP is added
// - an IP is removed
// - the name is removed
func (n *name) notifyNameObservers() {
numObservers := len(n.observers)
if numObservers > 0 {
Debug.Printf("[zonedb] Notifying %d observers of '%s'", numObservers, n.name)
for _, observer := range n.observers {
observer()
}
n.observers = make([]ZoneRecordObserver, 0)
}
}
// Convert a set of IP records to a comma-separated string
func (n *name) String() string {
var buf bytes.Buffer
for _, ip := range n.ipv4 {
fmt.Fprintf(&buf, "%s, ", ip)
}
l := buf.Len()
if l > 2 {
buf.Truncate(l - 2)
}
return buf.String()
}
///////////////////////////////////////////////////////////////////
// all the names in a ident
type nameSet struct {
names map[string]*name
}
func newNameSet() *nameSet { return &nameSet{names: make(map[string]*name)} }
func (ns *nameSet) len() int { return len(ns.names) }
func (ns *nameSet) empty() bool { return ns.len() == 0 }
func (ns *nameSet) hasName(name string) bool {
n, found := ns.names[name]
if !found {
return false
}
// the name must have some valid IPs in order to be considered as "existent"
return len(n.ipv4) > 0
}
// Get the zone entries for a name
// If the name does not exist and `create` is true, a new name will be created
func (ns *nameSet) getName(name string, create bool) (n *name) {
n, found := ns.names[name]
if !found && create {
newName := newName(name)
ns.names[name] = newName
return newName
}
return n
}
// Get all the entries for a name
func (ns *nameSet) getEntriesForName(name string) (res []*recordEntry) {
n := ns.getName(name, false)
if n != nil {
res = n.getAllEntries()
}
return
}
// Get all the entries for an IP, for all names
func (ns *nameSet) getEntriesForIP(ip IPv4) (res []*recordEntry) {
for _, name := range ns.names {
entry := name.getEntryForIP(ip)
if entry != nil {
res = append(res, entry)
}
}
return
}
// Add a new IPv4 to a name
func (ns *nameSet) addIPToName(zr ZoneRecord, now time.Time) (*recordEntry, error) {
n := ns.getName(zr.Name(), true)
ipv4 := ipToIPv4(zr.IP())
if n.hasIPv4(ipv4) {
return n.getEntryForIP(ipv4), DuplicateError{}
}
ns.touchName(zr.Name(), now)
return n.addIP(zr, now)
}
// Delete by name and IP. Pass uninitialized values for wildcard
func (ns nameSet) deleteNameIP(name string, ip net.IP) int {
count := 0
if name != "" {
if nameV, found := ns.names[name]; found {
count += nameV.deleteIP(ip)
if nameV.empty() {
delete(ns.names, name)
}
}
} else {
for nameK, nameV := range ns.names {
count += nameV.deleteIP(ip)
if nameV.empty() {
delete(ns.names, nameK)
}
}
}
return count
}
// Get the name query time
func (ns *nameSet) getNameLastAccess(n string) time.Time {
if name, found := ns.names[n]; found {
return name.lastAccessTime
}
return time.Time{}
}
// Touch the last access time for a name
// The access time is saved only for locally-introduced records (otherwise it could
// be lost when names are irrelevant...)
func (ns *nameSet) touchName(name string, now time.Time) {
Debug.Printf("[zonedb] Touching name %s", name)
n := ns.getName(name, false)
if n != nil {
n.lastAccessTime = now
}
}
func (ns *nameSet) String() string {
var buf bytes.Buffer
for _, name := range ns.names {
fmt.Fprintf(&buf, "%s, ", name)
}
l := buf.Len()
if l > 2 {
buf.Truncate(l - 2)
}
return buf.String()
}
///////////////////////////////////////////////////////////////////
// Name sets, by ident name
type identRecordSet map[string]*nameSet
func (irs identRecordSet) String() string {
var buf bytes.Buffer
for ident, nameset := range irs {
fmt.Fprintf(&buf, "%.12s: %s\n", ident, nameset)
}
return buf.String()
}
///////////////////////////////////////////////////////////////////
// Invariant on updateRequests: they are in ascending order of time.
type refreshRequest struct {
name string
time time.Time
}
type ZoneConfig struct {
// The local domain
Domain string
// The interface where mDNS operates
Iface *net.Interface
// Refresh interval for local names in the zone database (disabled by default) (in seconds)
RefreshInterval int
// Forget about remote info if nobody asks for it in this time (in seconds)
RelevantTime int
// TTL returnined in local asnwers (with mDNS)
LocalTTL int
// Force a specific mDNS client
MDNSClient ZoneMDNSClient
// Force a specific mDNS server
MDNSServer ZoneMDNSServer
// For a specific clock provider
Clock clock.Clock
}
// Zone database
// The zone database contains the locally known information about the zone.
// The zone database uses some other mechanisms (eg, mDNS) for finding unknown
// information or for keeping current information up to date.
type ZoneDb struct {
mx sync.RWMutex
idents identRecordSet
mdnsCli ZoneMDNSClient
mdnsSrv ZoneMDNSServer
iface *net.Interface
domain string // the local domain
relevantLimit time.Duration // if no one asks for something in this time, it is not relevant a anymore...
clock clock.Clock
refreshScheds *SchedQueue
refreshCloseChan chan bool
refreshInterval time.Duration
}
// Create a new zone database
func NewZoneDb(config ZoneConfig) (zone *ZoneDb, err error) {
zone = &ZoneDb{
domain: config.Domain,
idents: make(identRecordSet),
mdnsCli: config.MDNSClient,
mdnsSrv: config.MDNSServer,
iface: config.Iface,
clock: config.Clock,
relevantLimit: time.Duration(DefaultRelevantTime) * time.Second,
refreshCloseChan: make(chan bool),
}
// fix the default configuration parameters
if zone.clock == nil {
zone.clock = clock.New()
}
if len(zone.domain) == 0 {
zone.domain = DefaultLocalDomain
}
if config.RefreshInterval > 0 {
zone.refreshInterval = time.Duration(config.RefreshInterval) * time.Second
}
if config.RelevantTime > 0 {
zone.relevantLimit = time.Duration(config.RelevantTime) * time.Second
}
if zone.refreshInterval > 0 {
zone.refreshScheds = NewSchedQueue(zone.clock)
}
// Create the mDNS client and server
if zone.mdnsCli == nil {
if zone.mdnsCli, err = NewMDNSClient(); err != nil {
return
}
}
if zone.mdnsSrv == nil {
mdnsTTL := DefaultLocalTTL
if config.LocalTTL > 0 {
mdnsTTL = config.LocalTTL
}
if zone.mdnsSrv, err = NewMDNSServer(zone, false, mdnsTTL); err != nil {
return
}
}
return
}
// Start the zone database
func (zone *ZoneDb) Start() (err error) {
if zone.iface != nil {
Info.Printf("[zonedb] Using mDNS on %+v", zone.iface)
} else {
Info.Printf("[zonedb] Using mDNS on all interfaces")
}
if err = zone.mdnsCli.Start(zone.iface); err != nil {
return
}
if err = zone.mdnsSrv.Start(zone.iface); err != nil {
return
}
if zone.refreshInterval > 0 {
zone.refreshScheds.Start()
}
return
}
// Perform a graceful shutdown of the zone database
func (zone *ZoneDb) Stop() error {
if zone.refreshInterval > 0 {
Debug.Printf("[zonedb] Closing background updaters...")
zone.refreshScheds.Stop()
}
Debug.Printf("[zonedb] Exiting mDNS client and server...")
zone.mdnsCli.Stop()
zone.mdnsSrv.Stop()
return nil
}
// Obtain the domain where this database keeps information for
func (zone *ZoneDb) Domain() string {
return zone.domain
}
// Return the status string
func (zone *ZoneDb) Status() string {
var buf bytes.Buffer
fmt.Fprintln(&buf, "Local domain", zone.domain)
fmt.Fprintln(&buf, "Interface", zone.iface)
fmt.Fprintf(&buf, "Zone database:\n%s", zone)
return buf.String()
}
// Add a record with `name` pointing to `ip`
func (zone *ZoneDb) AddRecord(ident string, name string, ip net.IP) (err error) {
zone.mx.Lock()
defer zone.mx.Unlock()
Debug.Printf("[zonedb] Adding record: '%s'/'%s'[%s]", ident, name, ip)
record := Record{dns.Fqdn(name), ip, 0, 0, 0}
_, err = zone.getNameSet(ident).addIPToName(record, zone.clock.Now())
return
}
// Delete matching records (uninitialised values act as wildcards)
func (zone *ZoneDb) DeleteRecords(ident string, name string, ip net.IP) int {
if name != "" {
name = dns.Fqdn(name)
}
zone.mx.Lock()
defer zone.mx.Unlock()
return zone.deleteIdentNameIP(ident, name, ip)
}
// Observe a name.
// The name must have at least one valid IP address.
// Name observers are notified when
// - an IP is added
// - an IP is removed
// - the name is removed
// The observer will be invoked on any change/removal that affects this name.
// Each observer will be invoked at least once (but possibly more). After that, they will be removed.
// The observer should not try to lock the ZoneDB (you will get a deadlock)
func (zone *ZoneDb) ObserveName(name string, observer ZoneRecordObserver) (err error) {
zone.mx.Lock()
defer zone.mx.Unlock()
cnt := 0
name = dns.Fqdn(name)
for _, nameset := range zone.idents {
if n := nameset.getName(name, false); n != nil {
n.addNameObserver(observer)
cnt++
}
}
if cnt == 0 {
err = LookupError(fmt.Sprintf("%s", name))
}
return
}
// Observe a IP address.
// The IP address must be exit in the database.
// IP address observers are notified when
// - the IP is removed
// - the name is removed
// The observer will be invoked on any change/removal that affects this IP.
// Each observer will be invoked at least once (but possibly more). After that, they will be removed.
// The observer should not try to lock the ZoneDB (you will get a deadlock)
func (zone *ZoneDb) ObserveInaddr(inaddr string, observer ZoneRecordObserver) (err error) {
zone.mx.Lock()
defer zone.mx.Unlock()
cnt := 0
revIP, err := raddrToIPv4(inaddr)
if err != nil {
return newParseError("lookup address", inaddr)
}
for _, nameset := range zone.idents {
for _, entry := range nameset.getEntriesForIP(revIP) {
entry.addIPObserver(observer)
cnt++
}
}
if cnt == 0 {
err = LookupError(fmt.Sprintf("%+v/%+v", inaddr, revIP))
}
return
}
// Get the string representation of a zone
func (zone *ZoneDb) String() string {
zone.mx.RLock()
defer zone.mx.RUnlock()
return zone.idents.String()
}
// Notify that a container has died
func (zone *ZoneDb) ContainerDied(ident string) error {
Info.Printf("[zonedb] Container %s down. Removing records", ident)
zone.DeleteRecords(ident, "", net.IP{})
return nil
}
// Return true if a remote name is still relevant
func (zone *ZoneDb) IsNameRelevant(n string) bool {
zone.mx.Lock()
defer zone.mx.Unlock()
lastAccess := time.Time{}
for _, nameset := range zone.idents {
t := nameset.getNameLastAccess(n)
if !t.IsZero() && (lastAccess.IsZero() || t.After(lastAccess)) {
lastAccess = t
}
}
return !lastAccess.IsZero() && zone.clock.Now().Sub(lastAccess) <= zone.relevantLimit
}
// Returns true if the info we have for a remote name has expired. Local names are never expired
// Returns false if we do not have info for this name from remote peers.
func (zone *ZoneDb) IsNameExpired(n string) bool {
zone.mx.Lock()
defer zone.mx.Unlock()
now := zone.clock.Now()
remoteNameSet := zone.getNameSet(defaultRemoteIdent)
entries := remoteNameSet.getEntriesForName(n)
if entries != nil {
for _, entry := range entries {
if entry.hasExpired(now) {
return true
}
}
}
return false
}
func (zone *ZoneDb) HasNameLocalInfo(n string) bool {
zone.mx.Lock()
defer zone.mx.Unlock()
remoteNameSet := zone.getNameSet(defaultRemoteIdent)
for _, nameset := range zone.idents {
if nameset != remoteNameSet {
if nameset.hasName(n) {
return true
}
}
}
return false
}
// Return true if the we have obtained some information for a name from remote peers
// We can have both local (eg, introduced through the HTTP API) and remote (eg, mDNS)
// information for a name. This method only checks if there is remote information...
func (zone *ZoneDb) HasNameRemoteInfo(n string) bool {
zone.mx.Lock()
defer zone.mx.Unlock()
remoteNameSet := zone.getNameSet(defaultRemoteIdent)
if remoteNameSet != nil {
return remoteNameSet.hasName(n)
}
return false
}
// Get or create a name set
func (zone *ZoneDb) getNameSet(ident string) *nameSet {
ns, found := zone.idents[ident]
if !found {
ns = newNameSet()
zone.idents[ident] = ns
}
return ns
}
// Delete by ident, name and IP. Pass uninitialized value for wildcard
func (zone *ZoneDb) deleteIdentNameIP(ident string, name string, ip net.IP) int {
count := 0
if ident != "" {
if identV, found := zone.idents[ident]; found {
count += identV.deleteNameIP(name, ip)
if identV.empty() {
delete(zone.idents, ident)
}
}
} else {
for identK, identV := range zone.idents {
count += identV.deleteNameIP(name, ip)
if identV.empty() {
delete(zone.idents, identK)
}
}
}
return count
}
-293
View File
@@ -1,293 +0,0 @@
package nameserver
import (
"net"
"time"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
)
type uniqZoneRecordKey struct {
name string
ipv4 IPv4
}
// A group of ZoneRecords where there are no duplicates (according to the name & IPv4)
type uniqZoneRecords map[uniqZoneRecordKey]ZoneRecord
func newUniqZoneRecords() uniqZoneRecords {
return make(uniqZoneRecords, 0)
}
// Add a new ZoneRecord to the group
func (uzr *uniqZoneRecords) add(zr ZoneRecord) {
key := uniqZoneRecordKey{zr.Name(), ipToIPv4(zr.IP())}
(*uzr)[key] = zr
}
// Return the group as an slice
func (uzr *uniqZoneRecords) toSlice() []ZoneRecord {
res := make([]ZoneRecord, len(*uzr))
i := 0
for _, r := range *uzr {
res[i] = r
i++
}
return res
}
//////////////////////////////////////////////////////////////////////////////
// Lookup in the database for locally-introduced information
func (zone *ZoneDb) lookup(target string, lfun func(ns *nameSet) []*recordEntry) (res []ZoneRecord, err error) {
uniq := newUniqZoneRecords()
for identName, nameset := range zone.idents {
if identName != defaultRemoteIdent {
for _, ze := range lfun(nameset) {
uniq.add(ze)
}
}
}
if len(uniq) == 0 {
return nil, LookupError(target)
}
return uniq.toSlice(), nil
}
// Perform a lookup for a name in the zone
// The name can be resolved locally with the local database
func (zone *ZoneDb) LookupName(name string) (res []ZoneRecord, err error) {
zone.mx.RLock()
defer zone.mx.RUnlock()
// note: LookupName() is usually called from the mDNS server, so we do not touch the name
name = dns.Fqdn(name)
Debug.Printf("[zonedb] Looking for name '%s' in local database", name)
return zone.lookup(name, func(ns *nameSet) []*recordEntry { return ns.getEntriesForName(name) })
}
// Perform a lookup for a IP address in the zone
// The address can be resolved locally with the local database
func (zone *ZoneDb) LookupInaddr(inaddr string) (res []ZoneRecord, err error) {
zone.mx.RLock()
defer zone.mx.RUnlock()
// note: LookupInaddr() is usually called from the mDNS server, so we do not touch the name
revIPv4, err := raddrToIPv4(inaddr)
if err != nil {
return nil, newParseError("lookup address", inaddr)
}
Debug.Printf("[zonedb] Looking for address in local database: '%s' (%s)", revIPv4, inaddr)
return zone.lookup(inaddr, func(ns *nameSet) []*recordEntry { return ns.getEntriesForIP(revIPv4) })
}
// Perform a domain lookup with mDNS
func (zone *ZoneDb) domainLookup(target string, lfun ZoneLookupFunc) (res []ZoneRecord, err error) {
// no local results have been obtained in the local database: try with a mDNS query
Debug.Printf("[zonedb] '%s' not in local database... trying with mDNS", target)
lanswers, err := lfun(target)
if err != nil {
Debug.Printf("[zonedb] mDNS lookup error for '%s': %s", target, err)
return nil, err
}
// if the request has been successful, save the IP in the local database and return the corresponding ZoneRecord
// (we do not get the remote ident in the mDNS reply, so we save it in a "remote" ident)
Debug.Printf("[zonedb] adding '%s' (obtained with mDNS) to '%s'", lanswers, target)
res = make([]ZoneRecord, len(lanswers))
zone.mx.Lock()
now := zone.clock.Now()
uniq := newUniqZoneRecords()
remoteIdent := zone.getNameSet(defaultRemoteIdent)
for _, answer := range lanswers {
r, err := remoteIdent.addIPToName(answer, now)
if err != nil {
if _, ok := err.(DuplicateError); !ok {
zone.mx.Unlock()
Warning.Printf("[zonedb] '%s' insertion for %s failed: %s", answer, target, err)
return nil, err
}
}
uniq.add(r)
}
zone.mx.Unlock()
return uniq.toSlice(), nil
}
// Perform a lookup for a name in the zone
// The name can be resolved locally with the local database or with some other resolution method (eg, a mDNS query)
func (zone *ZoneDb) DomainLookupName(name string) (res []ZoneRecord, err error) {
name = dns.Fqdn(name)
Debug.Printf("[zonedb] Looking for name '%s' in local(&remote) database", name)
zone.mx.Lock()
now := zone.clock.Now()
uniq := newUniqZoneRecords()
for identName, nameset := range zone.idents {
for _, ze := range nameset.getEntriesForName(name) {
// filter the entries with expired TTL
// locally introduced entries are never expired: they always have TTL=0
if ze.hasExpired(now) {
Debug.Printf("[zonedb] '%s': expired entry '%s' ignored: removing", name, ze)
nameset.deleteNameIP(name, net.IP{})
} else {
uniq.add(ze)
}
}
if identName != defaultRemoteIdent {
nameset.touchName(name, now)
}
}
zone.mx.Unlock()
if len(uniq) > 0 {
Debug.Printf("[zonedb] '%s' resolved in local database", name)
res = uniq.toSlice()
} else {
res, err = zone.domainLookup(name, zone.mdnsCli.LookupName)
}
if len(res) > 0 {
zone.startUpdatingName(name)
return res, nil
}
return nil, LookupError(name)
}
// Perform a lookup for a IP address in the zone
// The address can be resolved either with the local database or
// with some other resolution method (eg, a mDNS query)
func (zone *ZoneDb) DomainLookupInaddr(inaddr string) (res []ZoneRecord, err error) {
revIPv4, err := raddrToIPv4(inaddr)
if err != nil {
return nil, newParseError("lookup address", inaddr)
}
Debug.Printf("[zonedb] Looking for address in local(&remote) database: '%s' (%s)", revIPv4, inaddr)
zone.mx.Lock()
now := zone.clock.Now()
uniq := newUniqZoneRecords()
for identName, nameset := range zone.idents {
for _, ze := range nameset.getEntriesForIP(revIPv4) {
// filter the entries with expired TTL
// locally introduced entries are never expired: they always have TTL=0
if ze.hasExpired(now) {
Debug.Printf("[zonedb] '%s': expired entry '%s' ignored: removing", revIPv4, ze)
nameset.deleteNameIP("", revIPv4.toNetIP())
} else {
uniq.add(ze)
if identName != defaultRemoteIdent {
nameset.touchName(ze.Name(), now)
}
}
}
}
zone.mx.Unlock()
if len(uniq) > 0 {
Debug.Printf("[zonedb] '%s' resolved in local database", inaddr)
res = uniq.toSlice()
} else {
res, err = zone.domainLookup(inaddr, zone.mdnsCli.LookupInaddr)
}
if len(res) > 0 {
// note: even for reverse addresses, we perform the background updates in the name, not in the IP
// this simplifies the process and produces basically the same results...
// note: we do not spend time trying to update names that did not return an initial response...
for _, r := range res {
zone.startUpdatingName(r.Name())
}
return res, nil
}
return nil, LookupError(inaddr)
}
//////////////////////////////////////////////////////////////////////////////
// Names updates try to find all the IPs for a given name with a mDNS query
//
// There are two types of names updates:
//
// - immediate updates.
// After a `DomainLookup*()` for a name not in the database we will return the
// first IP we can get with mDNS from other peers. Waiting for more responses would
// mean more latency in the response to the client, so we send only one answer BUT
// we also trigger an immediate update request for that name in order to get all
// the other IPs we didn't wait for...
//
// - periodic updates
// once we have obtained the first group of IPs for a name, we schedule a periodic
// refresh for that name, so we keep the list of IPs for that name up to date.
//
// These names updates are repeated until either
//
// a) there is no interest in the name, determined by a global 'relevant time'
// and the last time some local client asked about the name,
// or
// b) no peers answer one of our refresh requests (because the name has probably
// disappeared from the network)
//
// Check if we must start updating a name and, in that case, trigger a immediate update
func (zone *ZoneDb) startUpdatingName(name string) {
if zone.refreshInterval > 0 {
zone.mx.Lock()
// check if we should enqueue a refresh request for this name
n := zone.getNameSet(defaultRemoteIdent).getName(name, true)
if n.lastRefreshTime.IsZero() {
now := zone.clock.Now()
n.lastRefreshTime = now
zone.mx.Unlock()
Debug.Printf("[zonedb] Creating new immediate refresh request for '%s'", name)
zone.refreshScheds.Add(func() time.Time { return zone.updater(name) }, now)
} else {
zone.mx.Unlock()
}
}
}
// Update the IPs we have for a name
func (zone *ZoneDb) updater(name string) (nextTime time.Time) {
deleteRemoteInfo := func() {
zone.mx.Lock()
zone.getNameSet(defaultRemoteIdent).deleteNameIP(name, net.IP{})
zone.mx.Unlock()
}
// if nobody has asked for this name for long time, just forget about it...
if !zone.IsNameRelevant(name) || zone.IsNameExpired(name) {
Debug.Printf("[zonedb] '%s' seem to be irrelevant now: removing any remote information", name)
deleteRemoteInfo()
return
}
// perform the refresh for this name
fullName := dns.Fqdn(name)
startTime := zone.clock.Now()
Debug.Printf("[zonedb] Refreshing name '%s' with mDNS...", fullName)
res, _ := zone.mdnsCli.InsistentLookupName(fullName)
if res != nil && len(res) > 0 {
numIps := len(res)
zone.mx.Lock()
now := zone.clock.Now()
added, removed := zone.getNameSet(defaultRemoteIdent).getName(name, true).updateIPs(res, now)
zone.mx.Unlock()
Debug.Printf("[zonedb] Obtained %d IPs for name '%s' with mDNS: %d added, %d removed",
numIps, name, added, removed)
// once the name has been updated, we re-schedule the update
nextTime = startTime.Add(zone.refreshInterval)
Debug.Printf("[zonedb] Rescheduling update for '%s' in %s", name, nextTime.Sub(zone.clock.Now()))
} else {
Debug.Printf("[zonedb] nobody knows about '%s'... removing", name)
deleteRemoteInfo()
}
return
}
-116
View File
@@ -1,116 +0,0 @@
package nameserver
import (
"net"
"testing"
"time"
. "github.com/weaveworks/weave/common"
wt "github.com/weaveworks/weave/testing"
)
// Check that the refreshing mechanism works as expected
func TestZoneRefresh(t *testing.T) {
const (
refreshInterval = 5
relevantTime = 30
)
var (
name = "mysql.weave.local."
addr1 = "10.2.8.4"
addr2 = "10.2.8.5"
addr3 = "10.2.8.6"
addr4 = "10.2.8.7"
)
InitDefaultLogging(testing.Verbose())
Info.Println("TestZoneRefresh starting")
clk := newMockedClock()
// Create multiple zone databases, linked through mocked mDNS connections
zc := ZoneConfig{
RefreshInterval: refreshInterval,
RelevantTime: relevantTime,
Clock: clk,
}
dbs := newZoneDbsWithMockedMDns(2, zc)
dbs.Start()
defer dbs.Stop()
time.Sleep(100 * time.Millisecond) // allow for server to get going
Debug.Printf("Adding '%s' to Db #1", name)
dbs[0].Zone.AddRecord("someident", name, net.ParseIP(addr1))
Debug.Printf("Checking that the name %s is relevant (as it has been locally inserted) and not remote", name)
wt.AssertTrue(t, dbs[0].Zone.IsNameRelevant(name), "name relevant")
wt.AssertTrue(t, dbs[0].Zone.HasNameLocalInfo(name), "local name info")
wt.AssertFalse(t, dbs[0].Zone.HasNameRemoteInfo(name), "remote name info")
Debug.Printf("Asking for '%s' to Db #1: should get 1 IP from the local database...", name)
res, err := dbs[0].Zone.DomainLookupName(name)
wt.AssertNoErr(t, err)
Debug.Printf("Got: %s", res)
t.Logf("Db #1 after the lookup:\n%s", dbs[0].Zone)
wt.AssertEqualInt(t, len(res), 1, "lookup result")
clk.Forward(refreshInterval / 2)
dbs.Flush()
Debug.Printf("A couple of seconds later, we should still have one IP for that name")
res, err = dbs[0].Zone.DomainLookupName(name)
wt.AssertNoErr(t, err)
wt.AssertEqualInt(t, len(res), 1, "lookup result")
Debug.Printf("And then we add 2 IPs for that name at ZoneDb 2")
clk.Forward(1)
dbs.Flush()
Debug.Printf("Adding 2 IPs to '%s' in Db #2", name)
dbs[1].Zone.AddRecord("someident", name, net.ParseIP(addr2))
dbs[1].Zone.AddRecord("someident", name, net.ParseIP(addr3))
Debug.Printf("Perform a lookup, to ensure the name will be updated in the background...")
dbs[0].Zone.DomainLookupName(name)
Debug.Printf("Wait for a while, until a refresh is performed...")
clk.Forward(refreshInterval + 1)
dbs.Flush()
Debug.Printf("A refresh should have been scheduled now: we should have 3 IPs:")
Debug.Printf("the first (local) IP and the others obtained from zone2 with a mDNS query")
Debug.Printf("Asking for '%s' again... we should have 3 IPs now", name)
res, err = dbs[0].Zone.DomainLookupName(name)
Debug.Printf("Got: %s", res)
t.Logf("Db #1 after the second lookup:\n%s", dbs[0].Zone)
wt.AssertEqualInt(t, len(res), 3, "lookup result length")
Debug.Printf("We will not ask for `name` for a while, so it will become irrelevant and will be removed...")
clk.Forward(refreshInterval + relevantTime + 1)
dbs.Flush()
// the name should be irrelevant now, and all remote info should have been
// removed from the zone database
Debug.Printf("Name '%s' should not be in the remote database in ZoneDb 1", name)
wt.AssertFalse(t, dbs[0].Zone.IsNameRelevant(name), "name still relevant after some inactivity time")
wt.AssertTrue(t, dbs[0].Zone.HasNameLocalInfo(name), "local name info")
wt.AssertFalse(t, dbs[0].Zone.HasNameRemoteInfo(name), "remote name info")
Debug.Printf("There is no remote info about this name at zone 1: a new IP appears remotely meanwhile...")
clk.Forward(1)
dbs.Flush()
Debug.Printf("Adding '%s' to Db #2", name)
dbs[1].Zone.AddRecord("someident", name, net.ParseIP(addr4))
Debug.Printf("When we ask about this name again, we get 4 IPs (1 local, 3 remote)")
Debug.Printf("Asking for '%s' again... the first lookup will return only the local results", name)
res, err = dbs[0].Zone.DomainLookupName(name)
wt.AssertEqualInt(t, len(res), 1, "lookup result length")
Debug.Printf("... but a second lookup should return all the results in the network")
clk.Forward(refreshInterval + 1)
dbs.Flush()
res, err = dbs[0].Zone.DomainLookupName(name)
Debug.Printf("Got: %s", res)
wt.AssertEqualInt(t, len(res), 4, "lookup result length")
}
-159
View File
@@ -1,159 +0,0 @@
package nameserver
import (
"fmt"
"net"
"testing"
. "github.com/weaveworks/weave/common"
wt "github.com/weaveworks/weave/testing"
)
func TestZone(t *testing.T) {
var (
container1 = "deadbeef"
container2 = "cowjuice"
name1 = "test1.weave."
name1Addr1 = "10.9.2.1/24"
name1Addr2 = "10.9.2.2/24"
revName1Addr1 = "1.2.9.10.in-addr.arpa."
)
InitDefaultLogging(testing.Verbose())
zone, err := NewZoneDb(ZoneConfig{})
wt.AssertNoErr(t, err)
err = zone.Start()
wt.AssertNoErr(t, err)
defer zone.Stop()
ip1, _, _ := net.ParseCIDR(name1Addr1)
t.Logf("Adding '%s'/%s to '%s'", name1, ip1, container1)
err = zone.AddRecord(container1, name1, ip1)
wt.AssertNoErr(t, err)
// Add a few more records to make the job harder
t.Logf("Adding 'adummy.weave.' to 'abcdef0123'")
err = zone.AddRecord("abcdef0123", "adummy.weave.", net.ParseIP("10.9.0.1"))
wt.AssertNoErr(t, err)
t.Logf("Adding 'zdummy.weave.' to '0123abcdef'")
err = zone.AddRecord("0123abcdef", "zdummy.weave.", net.ParseIP("10.9.0.2"))
wt.AssertNoErr(t, err)
t.Logf("Zone database:\n%s", zone)
t.Logf("Checking if we can find the name '%s'", name1)
foundIPs, err := zone.LookupName(name1)
wt.AssertNoErr(t, err)
if !foundIPs[0].IP().Equal(ip1) {
t.Fatal("Unexpected result for", name1, foundIPs)
}
t.Logf("Checking if we cannot find some silly name like 'something.wrong'")
foundIPs, err = zone.LookupName("something.wrong")
wt.AssertErrorType(t, err, (*LookupError)(nil), fmt.Sprintf("unknown name: %+v", foundIPs))
ip2, _, _ := net.ParseCIDR(name1Addr2)
t.Logf("Adding a second IP for '%s'/%s to '%s'", name1, ip2, container1)
err = zone.AddRecord(container1, name1, ip2)
wt.AssertNoErr(t, err)
t.Logf("Checking if we can find both the old IP and the new IP for '%s'", name1)
foundIPs, err = zone.LookupName(name1)
wt.AssertNoErr(t, err)
if !(foundIPs[0].IP().Equal(ip1) || foundIPs[1].IP().Equal(ip1)) {
t.Fatal("Unexpected result for", name1, foundIPs)
}
if !(foundIPs[0].IP().Equal(ip2) || foundIPs[1].IP().Equal(ip2)) {
t.Fatal("Unexpected result for", name1, foundIPs)
}
t.Logf("Checking if we can find the address by IP '1.2.9.10.in-addr.arpa.'")
foundNames, err := zone.LookupInaddr("1.2.9.10.in-addr.arpa.")
wt.AssertNoErr(t, err)
if foundNames[0].Name() != name1 {
t.Fatal("Unexpected result for", ip1, foundNames)
}
t.Logf("Checking we can not find an unknown address '30.20.10.1.in-addr.arpa.'")
foundNames, err = zone.LookupInaddr("30.20.10.1.in-addr.arpa.")
wt.AssertErrorType(t, err, (*LookupError)(nil), fmt.Sprintf("unknown IP: %+v", foundNames))
t.Logf("Checking if adding again '%s'/%s in %s results in an error", name1, ip1, container1)
err = zone.AddRecord(container1, name1, ip1)
wt.AssertErrorType(t, err, (*DuplicateError)(nil), "duplicate add")
t.Logf("Adding '%s'/%s in %s too", name1, ip1, container2)
err = zone.AddRecord(container2, name1, ip1)
wt.AssertNoErr(t, err)
name1Removed := 0
err = zone.ObserveName(name1, func() { t.Logf("Observer #1 for '%s' notified.", name1); name1Removed++ })
wt.AssertNoErr(t, err)
err = zone.ObserveName(name1, func() { t.Logf("Observer #2 for '%s' notified.", name1); name1Removed++ })
wt.AssertNoErr(t, err)
err = zone.ObserveInaddr(revName1Addr1, func() { t.Logf("Observer #1 for '%s' notified.", revName1Addr1); name1Removed++ })
wt.AssertNoErr(t, err)
t.Logf("Zone database:\n%s", zone)
t.Logf("Deleting the %s in %s", ip1, container1)
count := zone.DeleteRecords(container1, "", ip1)
wt.AssertEqualInt(t, count, 1, "delete failed")
t.Logf("Zone database:\n%s", zone)
t.Logf("Checking %s's observers have been notified on removal", name1)
if name1Removed < 3 {
t.Logf("Zone database:\n%s", zone)
t.Fatalf("Unexpected number (%d) of calls to observers", name1Removed)
}
t.Logf("Checking %s can be found", name1)
_, err = zone.LookupName(name1)
wt.AssertNoErr(t, err)
t.Logf("Checking %s is not found after removing %s it in %s and %s in %s",
name1, ip1, container2, ip2, container1)
count = zone.DeleteRecords(container1, "", ip2)
wt.AssertEqualInt(t, count, 1, "delete failed")
count = zone.DeleteRecords(container2, "", ip1)
wt.AssertEqualInt(t, count, 1, "delete failed")
t.Logf("Zone database:\n%s", zone)
_, err = zone.LookupName(name1)
wt.AssertErrorType(t, err, (*LookupError)(nil), "after deleting record")
t.Logf("Checking if removing an unknown record results in an error")
count = zone.DeleteRecords(container1, "", net.ParseIP("0.0.0.0"))
wt.AssertEqualInt(t, count, 0, "delete failed")
}
func TestDeleteRecords(t *testing.T) {
var (
id = "foobar"
name = "foo.weave."
addr1 = "10.2.2.3/24"
addr2 = "10.2.7.8/24"
)
InitDefaultLogging(testing.Verbose())
zone, err := NewZoneDb(ZoneConfig{})
wt.AssertNoErr(t, err)
err = zone.Start()
wt.AssertNoErr(t, err)
defer zone.Stop()
for _, addr := range []string{addr1, addr2} {
ip, _, _ := net.ParseCIDR(addr)
err := zone.AddRecord(id, name, ip)
wt.AssertNoErr(t, err)
}
_, err = zone.LookupName(name)
wt.AssertNoErr(t, err)
count := zone.DeleteRecords(id, "", net.IP{})
wt.AssertEqualInt(t, count, 2, "wildcard delete failed")
_, err = zone.LookupName(name)
wt.AssertErrorType(t, err, (*LookupError)(nil), "after deleting records for ident")
}
@@ -19,10 +19,21 @@ func NewRange(start Address, size Offset) Range {
return Range{Start: start, End: Add(start, size)}
}
func (r Range) Size() Offset { return Subtract(r.End, r.Start) }
func (r Range) String() string { return fmt.Sprintf("[%s-%s)", r.Start, r.End) }
func (r Range) String() string { return fmt.Sprintf("%s-%s", r.Start, r.End-1) }
func (r Range) Overlaps(or Range) bool { return !(r.Start >= or.End || r.End <= or.Start) }
func (r Range) Contains(addr Address) bool { return addr >= r.Start && addr < r.End }
func (r Range) AsCIDRString() string {
prefixLen := 32
for size := r.Size(); size > 1; size = size / 2 {
if size%2 != 0 { // Size not a power of two; cannot be expressed as a CIDR.
return r.String()
}
prefixLen--
}
return CIDR{Start: r.Start, PrefixLen: prefixLen}.String()
}
type CIDR struct {
Start Address
PrefixLen int
@@ -79,6 +90,10 @@ func (addr Address) IP4() (r net.IP) {
return
}
func (addr Address) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf("%q", addr.String())), nil
}
func (addr Address) String() string {
return addr.IP4().String()
}
@@ -98,3 +113,10 @@ func Min(a, b Offset) Offset {
}
return a
}
func (addr Address) Reverse() Address {
return ((addr >> 24) & 0xff) | // move byte 3 to byte 0
((addr << 8) & 0xff0000) | // move byte 1 to byte 2
((addr >> 8) & 0xff00) | // move byte 2 to byte 1
((addr << 24) & 0xff000000) // byte 0 to byte 3
}
+78 -17
View File
@@ -2,30 +2,54 @@ package net
import (
"fmt"
"math"
"net"
"time"
"syscall"
"github.com/vishvananda/netlink/nl"
)
const (
sleepTime = 100 * time.Millisecond
)
// Wait for an interface to come up.
func EnsureInterface(ifaceName string) (*net.Interface, error) {
s, err := nl.Subscribe(syscall.NETLINK_ROUTE, syscall.RTNLGRP_LINK)
if err != nil {
return nil, err
}
defer s.Close()
iface, err := ensureInterface(s, ifaceName)
if err != nil {
return nil, err
}
return iface, err
}
// Wait `wait` seconds for an interface to come up. Pass zero to check once
// and return immediately, or a negative value to wait indefinitely.
func EnsureInterface(ifaceName string, wait int) (iface *net.Interface, err error) {
if iface, err = findInterface(ifaceName); err == nil || wait == 0 {
return
func ensureInterface(s *nl.NetlinkSocket, ifaceName string) (*net.Interface, error) {
if iface, err := findInterface(ifaceName); err == nil {
return iface, nil
}
i := int64(math.MaxInt64)
if wait > 0 {
i = int64(time.Duration(wait) * time.Second / sleepTime)
netlinkReceiveUntil(s, matchIfName(ifaceName))
iface, err := findInterface(ifaceName)
return iface, err
}
// Wait for an interface to come up and have a route added to the multicast subnet.
// This matches the behaviour in 'weave attach', which is the only context in which
// we expect this to be called. If you change one, change the other to match.
func EnsureInterfaceAndMcastRoute(ifaceName string) (*net.Interface, error) {
s, err := nl.Subscribe(syscall.NETLINK_ROUTE, syscall.RTNLGRP_LINK, syscall.RTNLGRP_IPV4_ROUTE)
if err != nil {
return nil, err
}
for ; err != nil && i > 0; i-- {
time.Sleep(sleepTime)
iface, err = findInterface(ifaceName)
defer s.Close()
iface, err := ensureInterface(s, ifaceName)
if err != nil {
return nil, err
}
return
dest := net.IPv4(224, 0, 0, 0)
if CheckRouteExists(ifaceName, dest) {
return iface, err
}
netlinkReceiveUntil(s, matchRoute(ifaceName, dest))
return iface, err
}
func findInterface(ifaceName string) (iface *net.Interface, err error) {
@@ -37,3 +61,40 @@ func findInterface(ifaceName string) (iface *net.Interface, err error) {
}
return
}
func netlinkReceiveUntil(s *nl.NetlinkSocket, f func(syscall.NetlinkMessage) (bool, error)) error {
for {
msgs, err := s.Receive()
if err != nil {
return err
}
for _, m := range msgs {
if done, err := f(m); done {
return err
}
}
}
}
func matchIfName(ifaceName string) func(m syscall.NetlinkMessage) (bool, error) {
return func(m syscall.NetlinkMessage) (bool, error) {
switch m.Header.Type {
case syscall.RTM_NEWLINK: // receive this type for link 'up'
ifmsg := nl.DeserializeIfInfomsg(m.Data)
attrs, err := syscall.ParseNetlinkRouteAttr(&m)
if err != nil {
return true, err
}
name := ""
for _, attr := range attrs {
if attr.Attr.Type == syscall.IFA_LABEL {
name = string(attr.Value[:len(attr.Value)-1])
}
}
if ifaceName == name && ifmsg.Flags&syscall.IFF_UP != 0 {
return true, nil
}
}
return false, nil
}
}
+96 -9
View File
@@ -3,26 +3,113 @@ package net
import (
"fmt"
"net"
"syscall"
"github.com/vishvananda/netlink"
"github.com/vishvananda/netlink/nl"
)
// A network is considered free if it does not overlap any existing
// routes on this host. This is the same approach taken by Docker.
func CheckNetworkFree(subnet *net.IPNet) error {
routes, err := netlink.RouteList(nil, netlink.FAMILY_V4)
if err != nil {
return err
}
for _, route := range routes {
func CheckNetworkFree(subnet *net.IPNet, ignoreIfaceNames map[string]struct{}) error {
return forEachRoute(ignoreIfaceNames, func(name string, route netlink.Route) error {
if route.Dst != nil && overlaps(route.Dst, subnet) {
return fmt.Errorf("network %s would overlap with route %s", subnet, route.Dst)
return fmt.Errorf("Network %s overlaps with existing route %s on host.", subnet, route.Dst)
}
}
return nil
return nil
})
}
// Two networks overlap if the start-point of one is inside the other.
func overlaps(n1, n2 *net.IPNet) bool {
return n1.Contains(n2.IP) || n2.Contains(n1.IP)
}
// For a specific address, we only care if it is actually *inside* an
// existing route, because weave-local traffic never hits IP routing.
func CheckAddressOverlap(addr net.IP, ignoreIfaceNames map[string]struct{}) error {
return forEachRoute(ignoreIfaceNames, func(name string, route netlink.Route) error {
if route.Dst != nil && route.Dst.Contains(addr) {
return fmt.Errorf("Address %s overlaps with existing route %s on host.", addr, route.Dst)
}
return nil
})
}
func forEachRoute(ignoreIfaceNames map[string]struct{}, check func(name string, r netlink.Route) error) error {
routes, err := netlink.RouteList(nil, netlink.FAMILY_V4)
if err != nil {
return err
}
for _, route := range routes {
link, err := netlink.LinkByIndex(route.LinkIndex)
if err == nil {
if _, found := ignoreIfaceNames[link.Attrs().Name]; found {
continue
}
}
if err := check(link.Attrs().Name, route); err != nil {
return err
}
}
return nil
}
func CheckRouteExists(ifaceName string, dest net.IP) bool {
found := false
forEachRoute(map[string]struct{}{}, func(name string, route netlink.Route) error {
if name == ifaceName && route.Dst.IP.Equal(dest) {
found = true
}
return nil
})
return found
}
func matchRoute(ifaceName string, dest net.IP) func(m syscall.NetlinkMessage) (bool, error) {
return func(m syscall.NetlinkMessage) (bool, error) {
switch m.Header.Type {
case syscall.RTM_NEWROUTE:
route, err := deserializeRoute(m.Data)
if err != nil {
return true, err
}
link, _ := netlink.LinkByIndex(route.LinkIndex)
if link.Attrs().Name == ifaceName && route.Dst.IP.Equal(dest) {
return true, nil
}
}
return false, nil
}
}
// This code proposed for addition to the vishvananda/netlink library
// deserializeRoute decodes a binary netlink message into a Route struct
func deserializeRoute(m []byte) (netlink.Route, error) {
route := netlink.Route{}
msg := nl.DeserializeRtMsg(m)
attrs, err := nl.ParseRouteAttr(m[msg.Len():])
if err != nil {
return route, err
}
route.Scope = netlink.Scope(msg.Scope)
native := nl.NativeEndian()
for _, attr := range attrs {
switch attr.Attr.Type {
case syscall.RTA_GATEWAY:
route.Gw = net.IP(attr.Value)
case syscall.RTA_PREFSRC:
route.Src = net.IP(attr.Value)
case syscall.RTA_DST:
route.Dst = &net.IPNet{
IP: attr.Value,
Mask: net.CIDRMask(int(msg.Dst_len), 8*len(attr.Value)),
}
case syscall.RTA_OIF:
routeIndex := int(native.Uint32(attr.Value[0:4]))
route.LinkIndex = routeIndex
}
}
return route, nil
}
+10 -10
View File
@@ -4,7 +4,7 @@ import (
"net"
"testing"
wt "github.com/weaveworks/weave/testing"
"github.com/stretchr/testify/require"
)
func TestOverlap(t *testing.T) {
@@ -13,13 +13,13 @@ func TestOverlap(t *testing.T) {
_, subnet3, _ := net.ParseCIDR("10.0.3.128/25")
_, subnet4, _ := net.ParseCIDR("10.0.3.192/25")
_, universe, _ := net.ParseCIDR("10.0.0.0/8")
wt.AssertEquals(t, overlaps(subnet1, subnet2), false)
wt.AssertEquals(t, overlaps(subnet2, subnet1), false)
wt.AssertEquals(t, overlaps(subnet1, subnet1), true)
wt.AssertEquals(t, overlaps(subnet1, subnet3), true)
wt.AssertEquals(t, overlaps(subnet1, subnet4), true)
wt.AssertEquals(t, overlaps(subnet2, subnet4), false)
wt.AssertEquals(t, overlaps(subnet4, subnet2), false)
wt.AssertEquals(t, overlaps(universe, subnet1), true)
wt.AssertEquals(t, overlaps(subnet1, universe), true)
require.False(t, overlaps(subnet1, subnet2))
require.False(t, overlaps(subnet2, subnet1))
require.True(t, overlaps(subnet1, subnet1))
require.True(t, overlaps(subnet1, subnet3))
require.True(t, overlaps(subnet1, subnet4))
require.False(t, overlaps(subnet2, subnet4))
require.False(t, overlaps(subnet4, subnet2))
require.True(t, overlaps(universe, subnet1))
require.True(t, overlaps(subnet1, universe))
}
+77
View File
@@ -0,0 +1,77 @@
// docker_tls_args: find the docker daemon's tls args
// This reimplements pgrep, because pgrep will only look in /proc, but not in
// $PROCFS.
package main
import (
"bytes"
"fmt"
"io/ioutil"
"os"
"path/filepath"
"strconv"
"strings"
)
func main() {
procRoot := os.Getenv("PROCFS")
if procRoot == "" {
procRoot = "/proc"
}
dirEntries, err := ioutil.ReadDir(procRoot)
checkErr(err)
for _, dirEntry := range dirEntries {
dirName := dirEntry.Name()
if _, err := strconv.Atoi(dirName); err != nil {
continue
}
if comm, err := ioutil.ReadFile(filepath.Join(procRoot, dirName, "comm")); err != nil || string(comm) != "docker\n" {
continue
}
cmdline, err := ioutil.ReadFile(filepath.Join(procRoot, dirName, "cmdline"))
if err != nil {
continue
}
isDaemon := false
tlsArgs := []string{}
args := bytes.Split(cmdline, []byte{'\000'})
for i := 0; i < len(args); i++ {
arg := string(args[i])
switch {
case arg == "-d" || arg == "daemon":
isDaemon = true
break
case arg == "--tls", arg == "--tlsverify":
tlsArgs = append(tlsArgs, arg)
case strings.HasPrefix(arg, "--tls"):
tlsArgs = append(tlsArgs, arg)
if len(args) > i+1 &&
!strings.Contains(arg, "=") &&
!strings.HasPrefix(string(args[i+1]), "-") {
tlsArgs = append(tlsArgs, string(args[i+1]))
i++
}
}
}
if !isDaemon {
continue
}
fmt.Println(strings.Join(tlsArgs, " "))
os.Exit(0)
}
os.Exit(1)
}
func checkErr(err error) {
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
+30 -7
View File
@@ -1,4 +1,4 @@
/* netcheck: check whether a given network overlaps with any existing routes */
/* netcheck: check whether a given network or address overlaps with any existing routes */
package main
import (
@@ -6,25 +6,48 @@ import (
"net"
"os"
"github.com/docker/docker/pkg/mflag"
weavenet "github.com/weaveworks/weave/net"
)
func fatal(err error) {
fmt.Println(err)
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
type stringList map[string]struct{}
func (ss *stringList) Set(value string) error {
(*ss)[value] = struct{}{}
return nil
}
func (ss *stringList) String() string {
return fmt.Sprintf("%v", *ss)
}
func main() {
if len(os.Args) <= 1 {
os.Exit(0)
ignoreIfaceNames := make(stringList)
mflag.Var(&ignoreIfaceNames, []string{"-ignore-iface"}, "name of interface to ignore)")
mflag.Parse()
if len(mflag.Args()) < 1 {
fmt.Fprintln(os.Stderr, "usage: netcheck [--ignore-iface <iface-name>] network-cidr")
os.Exit(1)
}
ipRangeStr := os.Args[1]
_, ipnet, err := net.ParseCIDR(ipRangeStr)
cidrStr := mflag.Args()[0]
addr, ipnet, err := net.ParseCIDR(cidrStr)
if err != nil {
fatal(err)
}
if err := weavenet.CheckNetworkFree(ipnet); err != nil {
if ipnet.IP.Equal(addr) {
err = weavenet.CheckNetworkFree(ipnet, ignoreIfaceNames)
} else {
err = weavenet.CheckAddressOverlap(addr, ignoreIfaceNames)
}
if err != nil {
fatal(err)
}
os.Exit(0)
-6
View File
@@ -1,6 +0,0 @@
FROM scratch
MAINTAINER Weaveworks Inc <help@weave.works>
WORKDIR /home/weave
ADD ./weavedns /home/weave/
EXPOSE 53/udp
ENTRYPOINT ["/home/weave/weavedns"]
-186
View File
@@ -1,186 +0,0 @@
package main
import (
"flag"
"fmt"
"net"
"os"
"strconv"
"github.com/miekg/dns"
. "github.com/weaveworks/weave/common"
"github.com/weaveworks/weave/common/docker"
weavedns "github.com/weaveworks/weave/nameserver"
weavenet "github.com/weaveworks/weave/net"
)
var version = "(unreleased version)"
func main() {
var (
justVersion bool
dockerCli *docker.Client
ifaceName string
apiPath string
domain string
dnsPort int
httpIfaceName string
httpPort int
wait int
ttl int
negTTL int
timeout int
udpbuf int
fallback string
refreshInterval int
relevantTime int
maxAnswers int
cacheLen int
cacheDisabled bool
watch bool
debug bool
err error
)
flag.BoolVar(&justVersion, "version", false, "print version and exit")
flag.StringVar(&ifaceName, "iface", "", "name of interface to use for multicast")
flag.StringVar(&apiPath, "api", "unix:///var/run/docker.sock", "path to Docker API socket")
flag.StringVar(&domain, "domain", weavedns.DefaultLocalDomain, "local domain (ie, 'weave.local.')")
flag.IntVar(&wait, "wait", -1, "number of seconds to wait for interfaces to come up (0=don't wait, -1=wait forever)")
flag.IntVar(&dnsPort, "dnsport", weavedns.DefaultServerPort, "port to listen to DNS requests")
flag.StringVar(&httpIfaceName, "httpiface", "", "interface on which to listen for HTTP requests (defaults to empty string which listens on all interfaces)")
flag.IntVar(&httpPort, "httpport", weavedns.DefaultHTTPPort, "port to listen to HTTP requests")
flag.IntVar(&cacheLen, "cache", weavedns.DefaultCacheLen, "cache length")
flag.IntVar(&ttl, "ttl", weavedns.DefaultLocalTTL, "TTL (in secs) for responses for local names")
flag.BoolVar(&watch, "watch", true, "watch the docker socket for container events")
flag.BoolVar(&debug, "debug", false, "output debugging info to stderr")
// advanced options
flag.IntVar(&negTTL, "neg-ttl", 0, "negative TTL (in secs) for unanswered queries for local names (0=same value as -ttl")
flag.IntVar(&refreshInterval, "refresh", weavedns.DefaultRefreshInterval, "refresh interval (in secs) for local names (0=disable)")
flag.IntVar(&maxAnswers, "max-answers", weavedns.DefaultMaxAnswers, "maximum number of answers returned to clients (0=unlimited)")
flag.IntVar(&relevantTime, "relevant", weavedns.DefaultRelevantTime, "life time for info in the absence of queries (in secs)")
flag.IntVar(&udpbuf, "udpbuf", weavedns.DefaultUDPBuflen, "UDP buffer length")
flag.IntVar(&timeout, "timeout", weavedns.DefaultTimeout, "timeout for resolutions (in millisecs)")
flag.BoolVar(&cacheDisabled, "no-cache", false, "disable the cache")
flag.StringVar(&fallback, "fallback", "", "force a fallback server (ie, '8.8.8.8:53') (instead of /etc/resolv.conf values)")
flag.Parse()
if justVersion {
fmt.Printf("weave DNS %s\n", version)
os.Exit(0)
}
InitDefaultLogging(debug)
Info.Printf("[main] WeaveDNS version %s\n", version) // first thing in log: the version
var iface *net.Interface
if ifaceName != "" {
var err error
Info.Println("[main] Waiting for mDNS interface", ifaceName, "to come up")
iface, err = weavenet.EnsureInterface(ifaceName, wait)
if err != nil {
Error.Fatal(err)
} else {
Info.Println("[main] Interface", ifaceName, "is up")
}
}
var httpIP string
if httpIfaceName == "" {
httpIP = "0.0.0.0"
} else {
Info.Println("[main] Waiting for HTTP interface", httpIfaceName, "to come up")
httpIface, err := weavenet.EnsureInterface(httpIfaceName, wait)
if err != nil {
Error.Fatal(err)
}
Info.Println("[main] Interface", httpIfaceName, "is up")
addrs, err := httpIface.Addrs()
if err != nil {
Error.Fatal(err)
}
if len(addrs) == 0 {
Error.Fatal("[main] No addresses on HTTP interface")
}
ip, _, err := net.ParseCIDR(addrs[0].String())
if err != nil {
Error.Fatal(err)
}
httpIP = ip.String()
}
httpAddr := net.JoinHostPort(httpIP, strconv.Itoa(httpPort))
zoneConfig := weavedns.ZoneConfig{
Domain: domain,
Iface: iface,
LocalTTL: ttl,
RefreshInterval: refreshInterval,
RelevantTime: relevantTime,
}
zone, err := weavedns.NewZoneDb(zoneConfig)
if err != nil {
Error.Fatal("[main] Unable to initialize the Zone database", err)
}
if err := zone.Start(); err != nil {
Error.Fatal("[main] Unable to start the Zone database", err)
}
defer zone.Stop()
dockerCli, err = docker.NewClient(apiPath)
if err != nil {
Error.Fatal("[main] Unable to start docker client: ", err)
}
if watch {
err := dockerCli.AddObserver(zone)
if err != nil {
Error.Fatal("[main] Unable to start watcher", err)
}
}
srvConfig := weavedns.DNSServerConfig{
Zone: zone,
Port: dnsPort,
CacheLen: cacheLen,
LocalTTL: ttl,
CacheNegLocalTTL: negTTL,
MaxAnswers: maxAnswers,
Timeout: timeout,
UDPBufLen: udpbuf,
CacheDisabled: cacheDisabled,
}
if len(fallback) > 0 {
fallbackHost, fallbackPort, err := net.SplitHostPort(fallback)
if err != nil {
Error.Fatal("[main] Could not parse fallback host and port", err)
}
srvConfig.UpstreamCfg = &dns.ClientConfig{Servers: []string{fallbackHost}, Port: fallbackPort}
Debug.Printf("[main] DNS fallback at %s:%s", fallbackHost, fallbackPort)
}
srv, err := weavedns.NewDNSServer(srvConfig)
if err != nil {
Error.Fatal("[main] Failed to initialize the WeaveDNS server", err)
}
httpListener, err := net.Listen("tcp", httpAddr)
if err != nil {
Error.Fatal("[main] Unable to create http listener: ", err)
}
Info.Println("[main] HTTP API listening on", httpAddr)
go SignalHandlerLoop(srv)
go weavedns.ServeHTTP(httpListener, version, srv, dockerCli)
err = srv.Start()
if err != nil {
Error.Fatal("[main] Failed to start the WeaveDNS server: ", err)
}
srv.ActivateAndServe()
}
+11 -11
View File
@@ -1,6 +1,12 @@
FROM gliderlabs/alpine
FROM alpine
MAINTAINER Weaveworks Inc <help@weave.works>
LABEL works.weave.role=system
WORKDIR /home/weave
VOLUME /w
VOLUME /w-noop
ENTRYPOINT ["/home/weave/sigproxy", "/home/weave/weave"]
RUN apk add --update \
curl \
@@ -9,17 +15,11 @@ RUN apk add --update \
iproute2 \
util-linux \
conntrack-tools \
bind-tools \
&& rm -rf /var/cache/apk/*
WORKDIR /home/weave
ADD ./weave /home/weave/
ADD ./sigproxy /home/weave/
ADD ./weaveproxy /home/weave/
ADD ./netcheck /usr/bin/
ADD ./weave ./sigproxy ./weaveproxy /home/weave/
ADD ./netcheck ./docker_tls_args /usr/bin/
ADD ./weavewait /w/w
ADD ./weavewait_noop /w-noop/w
ADD ./docker.tgz /
VOLUME /w
ENTRYPOINT ["/home/weave/sigproxy", "/home/weave/weave"]
Loaded 100 of 230 files, more files were not shown because too many files have changed in this diff. Show more