From 35ee71097a5a68df3c1ada57132378c2dbbdf938 Mon Sep 17 00:00:00 2001 From: Michael Bridgen Date: Fri, 10 Oct 2014 15:25:30 +0100 Subject: [PATCH] Took out things that will appear in the gh-pages site --- README.md | 444 ------------------------------------------------------ 1 file changed, 444 deletions(-) diff --git a/README.md b/README.md index 18f3c38c..f7378e12 100644 --- a/README.md +++ b/README.md @@ -120,447 +120,6 @@ Similarly, on $HOST2... So there we have it, two containers on separate hosts happily talking to each other. -## Features - -Weave has a few more features beyond those illustrated by the basic -example above... - -### Virtual Ethernet Switch - -To application containers, the network established by weave looks -like a giant Ethernet switch to which all the containers are -connected. - -Containers can easily access services from each other; e.g. in the -container on $HOST1 we can start a netcat "service" with - - root@28841bd02eff:/# nc -lk -p 4422 - -and then connect to it from the container on $HOST2 with - - root@f76829496120:/# echo 'Hello, world.' | nc 10.0.1.1 4422 - -Note that *any* protocol is supported. Doesn't even have to be over -TCP/IP, e.g. a netcat UDP service would be run with - - root@28841bd02eff:/# nc -lu -p 5533 - - root@f76829496120:/# echo 'Hello, world.' | nc -u 10.0.1.1 5533 - -We can deploy the entire arsenal of standard network tools and -applications, developed over decades, to configure, secure, monitor, -and troubleshoot our container network. To put it another way, we can -now re-use the same tools and techniques when deploying applications -as containers as we would have done when deploying them 'on metal' in -our data centre. - -### Application isolation - -A single weave network can host multiple, isolated applications, with -each application's containers being able to communicate with each -other but not containers of other applications. - -To accomplish that, we assign each application a different subnet. So, -in the above example, if we wanted to add another application similar -to, but isolated from, our first, we'd launch the containers with... - - host1# D=$(weave run 10.0.2.1/24 -t -i ubuntu) - host2# D=$(weave run 10.0.2.2/24 -t -i ubuntu) - -A quick 'ping' test in the containers confirms that they can talk to -each other but not the containers of our first application... - - host1# docker attach $D - - root@da50502598d5:/# ping -c 1 -q 10.0.2.2 - PING 10.0.2.2 (10.0.2.2): 48 data bytes - --- 10.0.2.2 ping statistics --- - 1 packets transmitted, 1 packets received, 0% packet loss - round-trip min/avg/max/stddev = 0.562/0.562/0.562/0.000 ms - - root@da50502598d5:/# ping -c 1 -q 10.0.1.1 - PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data. - --- 10.0.1.1 ping statistics --- - 1 packets transmitted, 0 received, 100% packet loss, time 0ms - - root@da50502598d5:/# ping -c 1 -q 10.0.1.2 - PING 10.0.1.2 (10.0.1.2) 56(84) bytes of data. - --- 10.0.1.2 ping statistics --- - 1 packets transmitted, 0 received, 100% packet loss, time 0ms - -This isolation-through-subnets scheme is an example of carrying over a -well-known technique from the 'on metal' days to containers. - -### Dynamic network attachment - -In some scenarios containers are started independently, e.g. via some -existing tool chain, or require more complex startup sequences than -provided by `weave run`. And sometimes the decision which application -network a container should be part of is made post-startup. For these -situations, weave allows an existing, running container to be attached -to the weave network. To illustrate, we can achieve the same effect as -the first example with - - host1# C=$(docker run -d -t -i ubuntu) - host1# weave attach 10.0.1.1/24 $C - -There is a matching `weave detach` command: - - host1# weave detach 10.0.1.1/24 $C - -You can detach a container from one application network and attach it -to another: - - host1# weave detach 10.0.1.1/24 $C - host1# weave attach 10.0.2.1/24 $C - -or attach a container to multiple application networks, effectively -sharing it between applications: - - host1# weave attach 10.0.1.1/24 $C - host1# weave attach 10.0.2.1/24 $C - -### Security - -In order to connect containers across untrusted networks, weave peers -can be told to encrypt traffic by supplying a `-password` option when -launching weave, e.g. - - host1# weave launch 10.0.0.1/16 -password wEaVe - -The same password must be specified for all weave peers; it is a -component in the creation of ephemeral session keys for connections -between peers. - -### Host network integration - -Weave application networks can be integrated with a host's network, -establishing connectivity between the host and application containers -anywhere. - -Let's say that in our example we want $HOST2 to have access to the -application containers. On $HOST2 we run - - host2# weave expose 10.0.1.102/24 - -choosing an unused IP address in the application subnet. (There is a -corresponding 'hide' command to revert this step.) - -Now - - host2# ping 10.0.1.2 - -will work. And, more interestingly, - - host2# ping 10.0.1.1 - -will work too, which is talking to a container that resides on $HOST1. - -### Service export - -Services running in containers on a weave network can be made -accessible to the outside world (and, more generally, other networks) -from any weave host, irrespective of where the service containers are -located. - -Say we want to make our example netcat "service", which is running in -a container on $HOST1, accessible to the outside world via $HOST2. - -First we need to expose the application network to $HOST2, as -explained [above](#host-network-integration), i.e. - - host2# weave expose 10.0.1.102/24 - -Then we add a NAT rule to route from the outside world to the -destination container service. - - host2# iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 2211 \ - -j DNAT --to-destination 10.0.1.1:4422 - -Here we are assuming that the "outside world" is connecting to $HOST2 -via 'eth0'. We want TCP traffic to port 2211 on the external IPs to be -routed to our 'nc' service, which is running on port 4422 in the -container with IP 10.0.1.1. - -With the above in place, we can connect to our 'nc' service from -anywhere with - - echo 'Hello, world.' | nc $HOST2 2211 - -(NB: due to the way routing is handled in the Linux kernel, this won't -work when run *on* $HOST2.) - -Similar NAT rules to the above can used to expose services not just to -the outside world but also other, internal, networks. - -### Service import - -Applications running in containers on a weave network can be given -access to services which are only reachable from certain weave hosts, -irrespective of where the application containers are located. - -Say that, as an extension of our example, we have a netcat service -running on $HOST3, port 2211, and that $HOST3 is not part of the weave -network and is only reachable from $HOST1, not $HOST2. Nevertheless we -want to make the service accessible to an application running in a -container on $HOST2. - -First we need to expose the application network to the host, as -explained [above](#host-network-integration), this time on $HOST1, -i.e. - - host1# weave expose 10.0.1.101/24 - -Then we add a NAT rule to route from the above IP to the destination -service. - - host1# iptables -t nat -A PREROUTING -p tcp -d 10.0.1.101 --dport 3322 \ - -j DNAT --to-destination $HOST3:2211 - -This allows any application container to reach the service by -connecting to 10.0.1.101:3322. So if $HOST3 is indeed running a netcat -service on port 2211, e.g. - - host3# nc -kl 2211 - -then we can connect to it from our application container on $HOST2 with - - root@f76829496120:/# echo 'Hello, world.' | nc 10.0.1.101 3322 - -The same command will work from any application container. - -### Service binding - -Importing a service provides a degree of indirection that allows late -and dynamic binding, similar to what can be achieved with a proxy. In -our example, application containers are unaware that the service they -are accessing at `10.0.1.101:3322` is in fact residing on -`$HOST3:2211`. We can point application containers at another service -location by changing the above NAT rule, without having to alter the -applications themselves. - -### Service routing - -The [service export](#service-export) and -[service import](#service-import) features can be combined to -establish connectivity between applications and services residing on -disjoint networks, even when those networks are separated by firewalls -and might have overlapping IP ranges. Each network imports its -services into weave, and in turn exports from weave services required -by its applications. There are no application containers in this -scenario (though of course there could be); weave is acting purely as -an address translation and routing facility, using the weave -application network as an intermediary. - -In our example above, the netcat service on $HOST3 is imported into -weave via $HOST1. We can export it on $HOST2 by first exposing the -application network with - - host2# weave expose 10.0.1.102/24 - -and then adding a NAT rule which routes traffic from the $HOST2 -network (i.e. anything which can connect to $HOST2) to the service -endpoint in the weave network - - host2# iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 4433 \ - -j DNAT --to-destination 10.0.1.101:3322 - -Now any host on the same network as $HOST2 can access the service with - - echo 'Hello, world.' | nc $HOST2 4433 - -Furthermore, as explained in [service-binding](#service-binding), we -can dynamically alter the service locations without having to touch -the applications that access them, e.g. we could move the example -netcat service to $HOST4:2211 while retaining its 10.0.1.101:3322 -endpoint in the weave network. - -### Multi-cloud networking - -Weave can network containers hosted in different cloud providers / -data centres. So, for example, one could run an application consisting -of containers on GCE, EC2 and in local data centres. - -To enable this, the network must be configured to permit TCP and UDP -connections to port 6783 of the docker hosts. - -### Multi-hop routing - -A network of containers across more than two hosts can be established -even when there is only partial connectivity between the hosts. Weave -is able to route traffic between containers as long as there is at -least one *path* of connected hosts between them. - -So, for example, if a docker host in a local data centre can connect -to hosts in GCE and EC2, but the latter two cannot connect to each -other, containers in the latter two can still communicate; weave will -route the traffic via the local data centre. - -### Dynamic topologies - -To add a host to an existing weave network, one simply launches -weave on the host, supplying the address of at least one existing -host. Weave will automatically discover the other hosts in the other -network and establish connections to them if it can (in order to avoid -unnecessary multi-hop routing). - -### Container mobility - -Containers can be moved between hosts without requiring any -reconfiguration or, in many cases, restarts of other containers. All -that is required is for the migrated container to be started with the -same IP address as it was given originally. - -### Fault tolerance - -Weave peers continually exchange topology information, and monitor -and (re)establish network connections to other peers. So if hosts or -networks fail, weave can "route around" the problem. This includes -network partitions; containers on either side of a partition can -continue to communicate, with full connectivity being restored when -the partition heals. - -The weave container is very light-weight - just over 8MB image size -and a few 10s of MBs of runtime memory - and disposable. I.e. should -weave ever run into difficulty, one can simply stop it (with `weave -stop`) and restart it. Application containers do *not* have to be -restarted in that event, and indeed may not even experience a -temporary connectivity failure if the weave container is restarted -quickly enough. - -## Troubleshooting - -Make sure you are running the latest version - you can download it -with - - docker pull zettio/weave - -Check the weave container logs with - - docker logs weave - -A reasonable amount of information, and all errors, get logged there. - -The log verbosity can be increased by supplying the `-debug` flag when -launching weave. Be warned, this will log information on a per-packet -basis, so can produce a lot of output. - -One can ask a weave router to report its status with - - weave status - -To stop weave, run - - weave stop - -Note that this leaves the local application container network intact; -containers on the local host can continue to communicate, whereas -communication with containers on different hosts, as well as service -export/import, is disrupted but resumes when weave is relaunched. - -To stop weave and completely remove all traces of the weave network on -the local host, run - - weave reset - -Any running application containers will permanently lose connectivity -with the weave network and have to be restarted in order to -re-connect. - -### Reboots - -When a host reboots, docker's default behaviour is to restart any -containers that were running. Since weave relies on special network -configuration outside of the containers, the weave network will not -function in this state. - -To remedy this, stop and re-launch the weave container, and re-attach -the application containers with `weave attach`. - -For a more permanent solution, -[disable Docker's auto-restart feature](https://docs.docker.com/articles/host_integration/) -and create appropriate startup scripts to launch weave and run -application containers from your favourite process manager. - -## Installation with Boot2Docker - -If you are running Docker inside the Boot2Docker VM, e.g. because you -are on a Mac, then the following changes are required to these -instructions: - -Assuming you have fetched the 'weave' script via curl or similar, and -it is in the current directory, transfer it to the Boot2Docker VM and -make it executable like this: - - host1$ boot2docker ssh "cat > weave" < weave - host1$ boot2docker ssh "chmod a+x weave" - -Then, if we were trying to create the same containers as in the first -example above, the 'launch' command would be run like this: - - host1$ boot2docker ssh "sudo ./weave launch 10.0.0.1/16" - -and the 'run' command like this: - - host1# C=$(boot2docker ssh "sudo ./weave run 10.0.1.1/24 -t -i ubuntu") - -## Building - -(NB. This is only necessary if you want to work on weave. Also, these -instructions have only been tested on Ubuntu.) - -To build weave you need `libpcap-dev` and `docker` installed. And `go` -(and `git` and `hg` to fetch dependencies). - -The package name is `github.com/zettio/weave`, so assuming `$GOPATH` -is set: - - $ cd $GOPATH - $ WEAVE=github.com/zettio/weave - $ git clone https://$WEAVE src/$WEAVE - $ cd src/$WEAVE - -Then simply run - - $ make -C weaver - -This will build the weave router, produce a docker image -`zettio/weave` and export that image to /tmp/weave.tar - -## How does it work? - -Weave creates a network bridge on the host. Each container is -connected to that bridge via a veth pair, the container side of which -is given the IP address & netmask supplied in 'weave run'. Also -connected to the bridge is the weave router container, which is given -the IP address & netmask supplied in 'weave launch'. - -A weave router captures Ethernet packets from its bridge-connected -interface in promiscuous mode, using 'pcap'. This typically excludes -traffic between local containers, and between the host and local -containers, all of which is routed straight over the bridge by the -kernel. Captured packets are forwarded over UDP to weave router peers -running on other hosts. On receipt of such a packet, a router injects -the packet on its bridge interface using 'pcap' and/or forwards the -packet to peers. - -Weave routers learn which peer host a particular MAC address resides -on. They combine this knowledge with topology information in order to -make routing decisions and thus avoid forwarding every packet to every -peer. The topology information captures which peers are connected to -which other peers; weave can route packets in partially connected -networks with changing topology. - -Weave routers establish TCP connections to each other, over which they -perform a protocol handshake and subsequently exchange topology -information. These connections are encrypted if so configured. Peers -also establish UDP "connections", possibly encrypted, for the -aforementioned packet forwarding. These "connections" are duplex and -can traverse firewalls. - -More details on the inner workings of weave can be found in the -[architecture documentation](docs/architecture.txt). - ## Contact Us Found a bug, want to suggest a feature, or have a question? @@ -573,6 +132,3 @@ Follow weave on Twitter: Read the Weave blog: [Weaveblog](http://weaveblog.com/). - -Discuss weave on -[Hacker News](https://news.ycombinator.com/item?id=8289786).