Added documentation website content, and a workflow to publish it to GitHub Pages

* Copied, rearranged and modified the contents of the old site directory for Jekyll static site generator, into the website directory
* Added some Jekyll-specific things for looks, including:
  * _config.yaml, Gemfile.* and 404.html files
  * _data, _layouts and assets directories and their content
* Added .github/workflows/github-pages-publish.yml workflow
This commit is contained in:
Raj Chaudhuri committed 2024-03-18 20:37:53 +05:30
1 parent 92779f3e57
commit e2db8a25bd
82 files changed
+6162

No files matched your search

+58
View File
@@ -0,0 +1,58 @@
---
title: Autoscaling
menu_order: 40
search_type: Documentation
---
### Bootstrapping
An autoscaling configuration begins with a fixed cluster:
* Configured as per the [Uniform Fixed Cluster]({{ '/operational-guide/uniform-fixed-cluster' | relative_url }})
scenario.
* Hosted on reserved or protected instances to ensure long-term
stability.
* Ideally sized at a minimum of three or five nodes (you can make your
fixed cluster bigger to accommodate base load as required; a minimum
is specified here in the interests of resilience only.)
Building on this foundation, arbitrary numbers of dynamic peers can be
added or removed concurrently as desired, without requiring any
changes to the configuration of the fixed cluster. As with the fixed
cluster, dynamically added nodes recover automatically from reboots
and partitions.
### Scaling Out
On the additional dynamic peer, at boot, via
[systemd]({{ '/install/systemd' | relative_url }}) or equivalent:
weave launch --no-restart --ipalloc-init=observer $PEERS
Where,
* `$PEERS` means all peers in the _fixed cluster_, initial
and subsequently added, which have not been explicitly removed. It
should include fixed peers which are temporarily offline or stopped.
You do not have to keep track of and specify the addresses
of other dynamic peers in `$PEERS` - they will discover and connect to
each other via the fixed cluster.
>**Note:** The use of `--ipalloc-init=observer` prevents dynamic peers from
coming to a consensus on their own - this is important to stop a
clique forming amongst a group of dynamically added peers if they
become partitioned from the fixed cluster after having learned about
each other via discovery.
### Scaling In
On the dynamic peer to be removed:
weave reset
If for any reason you cannot arrange for `weave reset` to be run on
the peer before the underlying host is destroyed (for example when
using spot instances that can be destroyed without notice), you will
need an asynchronous process to [reclaim lost IP address
space]({{ '/operational-guide/tasks#detect-reclaim-ipam' | relative_url }}).
+161
View File
@@ -0,0 +1,161 @@
---
title: Concepts
menu_order: 10
search_type: Documentation
---
This section describes some of the essential concepts with which you will
need to be familiar before continuing to the example deployment
scenarios.
The following concepts are described:
* [Host](#host)
* [Peer](#peer)
* [Peer Discovery](#peer-discovery)
* [Network Partition](#network-partition)
* [IP Address Manager- IPAM](#ip-address-manager)
* [Seeding](#seeding)
* [Consensus](#consensus)
* [Observers](#observers)
* [Persistence](#persistence)
## <a name="host"></a>Host
For the purposes of this documentation a host is an
installation of the Linux operating system that is running an
instance of the Docker Engine. The host may be executing directly on bare
hardware or inside a virtual machine.
## <a name="peer"></a>Peer
A peer is a running instance of Weave Net, typically one per host.
## <a name="peer-name"></a>Peer Name
Weave Net peers are identified by a 48-bit value formatted like an
ethernet MAC address, for example, `01:23:45:67:89:ab`. The 'peer
name' is used for various purposes:
* Routing of packets between containers on the overlay network
* Recording the origin peer of DNS entries
* Recording ownership of IP address ranges
While it is desirable for the peer name to remain stable across
restarts, it is essential that it is unique. If two or more peers
share the same name chaos will ensue, which includes but is not limited to
double allocation of addresses and the inability to route packets on the
overlay network.
When the router is launched on a host, it derives its peer name in order of preference:
* From the command line, where the user is responsible for uniqueness and
stability
* From Weave's data volume container (`weavedb`), if you upgrade your
Weave Net install from a pre-1.9 version and the underlying machine
IDs still match.
* From a unique machine ID. This is generated by combining IDs from
[systemd](https://www.freedesktop.org/software/systemd/man/machine-id.html)
or [D-Bus](https://dbus.freedesktop.org/doc/dbus-uuidgen.1.html)
with the BIOS product UUID or hypervisor UUID. Most machines have
one or two of these IDs, and the combination is practically stable
across restarts unless a user re-generates it.
* If none of the machine IDs are found, a random value is generated.
This is practically unique across different physical hardware and
cloned VMs but not stable across restarts.
The appropriate strategy for assigning peer names depends on the type
and method of your particular deployment and is discussed in more
detail below.
## <a name="peer-discovery"></a>Peer Discovery
Peer discovery is a mechanism that allows peers to learn about new
Weave Net hosts from existing peers without being explicitly told. Peer
discovery is
[enabled by default]({{ '/tasks/manage/finding-adding-hosts-dynamically' | relative_url }}).
## <a name="network-partition"></a>Network Partition
A network partition is a transient condition whereby some arbitrary
subsets of peers are unable to communicate with each other for the
duration - perhaps because a network switch has failed, or a fibre
optic line severed. Weave Net is designed to allow peers and their
containers to make maximum safe progress under conditions of
partition, healing automatically once the partition is over.
## <a name="ip-address-manager"></a>IP Address Manager (IPAM)
[IPAM]({{ '/tasks/ipam/ipam' | relative_url }}) is the subsystem responsible for dividing up a
large contiguous block of IP addresses (known as the IP allocation
range) amongst peers so that individual addresses may be uniquely
assigned to containers anywhere on the overlay network.
When a new network is formed an initial division of the IP allocation
range must be made. Two (mutually exclusive) mechanisms with different
tradeoffs are provided to perform this task: seeding and consensus.
### <a name="seeding"></a>Seeding
Seeding requires each peer to be told the list of peer names amongst
which the address space is to be divided initially. There are some
constraints and consequences:
* Every peer added to the network _must_ receive the same seed list,
for all time, or they will not be able to join together to form a
single cohesive whole.
* Because the 'product UUID' and 'random value' methods of peer name
assignment are unpredictable, the end user must by necessity also
specify peer names.
* Even though every peer _must_ receive the same seed, that seed does
_not_ have to include every peer in the network, nor does it have to
be updated when new peers are added (in fact due to the first
constraint above it may not be).
Example configurations are given in the section on deployment
scenarios:
* [Uniform Dynamic Cluster]({{ '/operational-guide/uniform-dynamic-cluster' | relative_url }})
### <a name="consensus"></a>Consensus
Alternatively, when a new network is formed for the first time, peers
can be configured to co-ordinate amongst themselves to automatically
divide up the IP allocation range. This process is known as consensus
and it requires each peer to be told the total number of expected peers
(the 'initial peer count') in order to prevent the formation of disjoint
peer groups which would, ultimately, result in duplicate IP
addresses.
Example configurations are given in the section on deployment
scenarios:
* [Interactive Deployment]({{ '/operational-guide/interactive' | relative_url }})
* [Uniform Fixed Cluster]({{ '/operational-guide/uniform-fixed-cluster' | relative_url }})
### <a name="observers"></a>Observers
Finally, an option is provided to start a peer as an _observer_. Such
peers do not require a seed peer name list or an initial peer
count; instead they rely on the existence of other peers in the
network which have been so configured. When an observer needs address
space, it asks for it from one of the peers which partook of the
initial division, triggering consensus if necessary.
Example configurations are given in the section on deployment
scenarios:
* [Autoscaling]({{ '/operational-guide/autoscaling' | relative_url }})
## <a name="persistence"></a>Persistence
Certain information is remembered between launches of Weave Net (for
example across reboots):
* The division of the IP allocation range amongst peers
* Allocation of addresses to containers
The persistence of this information is managed transparently in a
volume container but can be
[destroyed explicitly]({{ '/operational-guide/tasks#reset' | relative_url }})
if necessary.
+33
View File
@@ -0,0 +1,33 @@
---
title: Operational Guide
menu_order: 45
search_type: Documentation
---
This operational guide is intended to give you an overview of how to
operate and manage a Weave Network in production. It consists of three
main parts:
* A [glossary of concepts]({{ '/operational-guide/concepts' | relative_url }}) with
which you will need to be familiar
* Detailed instructions for safely bootstrapping, growing and
shrinking Weave networks in a number of different deployment
scenarios:
* An [interactive
installation]({{ '/operational-guide/interactive' | relative_url }}), suitable
for evaluation and development
* A [uniformly configured
cluster]({{ '/operational-guide/uniform-fixed-cluster' | relative_url }}) with
a fixed number of initial nodes, suitable for automated
provisioning but requiring manual intervention for resizing
* A [heterogenous cluster]({{ '/operational-guide/autoscaling' | relative_url }})
comprising fixed and autoscaling components, suitable for a base
load with automated scale-out/scale-in
* A [uniformly configured
cluster]({{ '/operational-guide/uniform-dynamic-cluster' | relative_url }})
with dynamic nodes, suitable for automated provisioning and
resizing.
* A list of [common administrative
tasks]({{ '/operational-guide/tasks' | relative_url }}), such as configuring Weave
Net to start on boot, upgrading clusters, cleaning up peers and
reclaiming IP address space
+66
View File
@@ -0,0 +1,66 @@
---
title: Interactive Deployment
menu_order: 20
search_type: Documentation
---
Weave Net can be launched interactively on the command line, and as
long as Docker is configured to start on boot, the network will survive
host reboots without the use of a systemd. However, since launching
Weave Net commands in interactive mode is not amenable to automation
and configuration management, it is recommended that deploying Weave Net
in this mode be reserved for exploration and evaluation only.
### Bootstrapping
On the initial peer:
weave launch
### Adding a Peer
On the new peer:
weave launch <extant peers>
Where,
* `<extant peers>` indicates all peers in the network, initial and
subsequently added, which have not been explicitly removed. It should
include peers that are temporarily offline or stopped.
To ensure that the new peer has joined the existing network,
execute the following:
weave prime
Before adding any new peers, you _must_ wait for this to complete.
If this command waits and does not exit, it means that there is some
issue (such as a network partition or failed peers) that is preventing
a quorum from being reached - you will need to [address
that]({{ '/troubleshooting' | relative_url }}) before moving on.
### Stopping a Peer
A peer can be stopped temporarily with the following command:
weave stop
A temporarily stopped peer will remember IP address allocation information on the
next `weave launch` but will forget any discovered peers or
modifications to the initial peer list that were made with `weave
connect` or `weave forget`. Note that if the host reboots, Docker
automatically restarts the peer.
### Removing a Peer
On the peer to be removed:
weave reset
Then optionally on each remaining peer:
weave forget <removed peer>
This step is not mandatory, but it will eliminate log noise and
spurious network traffic by stopping reconnection attempts and
preventing further connection attempts after a restart.
+120
View File
@@ -0,0 +1,120 @@
---
title: Administrative Tasks
menu_order: 60
search_type: Documentation
---
The following administrative tasks are discussed:
* [Configuring Weave Net to Start Automatically on Boot](#start-on-boot)
* [Detecting and Reclaiming Lost IP Address Space](#detect-reclaim-ipam)
* [Manually Reclaiming Lost Address Space](#manually-reclaim-address-space)
* [Upgrading a Cluster](#cluster-upgrade)
* [Resetting Persisted Data](#reset)
## <a name="start-on-boot"></a>Configuring Weave Net to Start Automatically on Boot
`weave launch` runs all of Weave Net's containers with a Docker restart
policy set to `always`. If you have launched Weave Net manually at least
once and your system is configured to start Docker on boot, then Weave Net
will start automatically on system restarts.
If you are aiming for a non-interactive installation, use
[systemd]({{ '/install/systemd' | relative_url }}) or a similar init system to
launch Weave using the `--no-restart` flag after Docker has been started.
## <a name="detect-reclaim-ipam"></a>Detecting and Reclaiming Lost IP Address Space
The recommended method of removing a peer is to run `weave reset` on that
peer before the underlying host is decommissioned or repurposed. This
ensures that the portion of the IPAM allocation range assigned to the
peer is released for reuse.
Under certain circumstances this operation may not be successful,
or possible:
* If the peer is partitioned from the rest of the network
when `weave reset` is executed on it
* If the underlying host is no longer available to execute `weave
reset` due to a hardware failure or other unplanned termination (for
example when using autoscaling with spot-instances that can be
destroyed without notice)
In some cases you may already be aware of the problem, as you were
unable to execute `weave reset` successfully or because you know
through other channels that the host has died. In these cases you can
proceed straight to [Manually Reclaiming Lost Address Space](#manually-reclaim-address-space).
However in some scenarios it may not be obvious that space has been
lost, in which case you can check for it periodically with the
following command on any peer:
weave status ipam
This command displays the peer names and nicknames, absolute quantity/percentage of allocation
range managed by peer and also identifies the names of unreachable peers. If you are satisfied
that the peer is truly gone, rather than temporarily unreachable due to a
partition, you can reclaim their space manually.
### <a name="manually-reclaim-address-space"></a>Manually Reclaiming Address Space
When a peer dies unexpectedly the remaining peers will consider its
address space to be unavailable even after it has remained unreachable
for prolonged periods. There is no universally applicable time limit
after which one of the remaining peers could decide unilaterally that
it is safe to appropriate the space for itself, and so an
administrative action is required to reclaim it.
The [`weave rmpeer`]({{ '/tasks/ipam/stop-remove-peers-ipam' | relative_url }})
command is provided to perform this task, and must
be executed on _one_ of the remaining peers. That peer will then take
ownership of the freed address space.
## <a name="cluster-upgrade"></a>Upgrading a Cluster
Protocol versioning and feature negotiation are employed in Weave Net
to enable incremental rolling upgrades. Each major maintains
the ability to speak to the preceding major release at a minimum, and
connected peers only utilize features which both support.
The general upgrade procedure is as follows:
On each peer:
* Stop the old Weave Net with `weave stop` (or `systemctl stop weave` if
you're using a systemd unit file)
* Download the new Weave Net script and replace the existing one
* Start the new Weave with `weave launch <existing peer list>` (or
`systemctl start weave` if you're using a systemd unit file)
To minimize downtime while the new script is pulling the new container images:
* Download the new Weave Net script to a temporary location, for example,
`/path/to/new/weave`
* Pull the new images with `/path/to/new/weave setup`
* Stop the old Weave Net with `weave stop` (or `systemctl stop weave` if
you're using a systemd unit file)
* Replace the existing script with the new one
* Start the new Weave Net with `weave launch <existing peer list>` (or
`systemctl start weave` if you're using a systemd unit file)
>**Note:** Always check the Release Notes for specific versions in case
there are any special caveats or deviations from the standard
procedure.
## <a name="reset"></a>Resetting Persisted Data
Weave Net persists information in a data volume container named
`weavedb`. If you wish to start from a completely clean slate (for
example to withdraw a peer from one network and join it to another)
you can issue the following command:
weave reset
**See Also**
* [Allocating IP Addresses]({{ '/tasks/ipam/ipam' | relative_url }})
* [Troubleshooting the IP Allocator]({{ '/tasks/ipam/troubleshooting-ipam' | relative_url }})
@@ -0,0 +1,97 @@
---
title: Uniform Dynamic Clusters
menu_order: 50
search_type: Documentation
---
A uniform dynamic cluster has the following characteristics:
* Recovers automatically after reboots and partitions.
* Identical configuration for all of its peers.
* Initial cluster peers can make progress from the outset even if
bootstrapping occurs under conditions of partition.
* Once an initial cluster has been bootstrapped, arbitrary numbers of
new peers can be added in parallel without coordination. This makes
it ideally suited for use with convergent provisioning tools that
operate across multiple hosts in an asynchronous fashion.
## Bootstrapping
On each initial peer N, at boot, via
[systemd]({{ '/install/systemd' | relative_url }}):
hostN$ weave launch --no-restart --name ::N --ipalloc-init seed=$SEED $PEERS
Where,
* `$SEED` is a comma separated list of the _names_ of the initial
peers, for example, `::1,::2,::3` and,
* `$PEERS` is obtained from `/etc/sysconfig/weave` as described in the linked systemd
documentation, and includes the complete set of initial peer
_addresses_.
For example, if you have three initial peers you would specify the
following:
host1$ weave launch --no-restart --name ::1 --ipalloc-init seed=$SEED $PEERS
host2$ weave launch --no-restart --name ::2 --ipalloc-init seed=$SEED $PEERS
host3$ weave launch --no-restart --name ::3 --ipalloc-init seed=$SEED $PEERS
Where
SEED="::1,::2,::3"
PEERS="host1 host2 host3"
## Adding a Peer
On each new peer, at boot, via
[systemd]({{ '/install/systemd' | relative_url }}):
hostN$ weave launch --no-restart --name ::N --ipalloc-init seed=$SEED $PEERS
Where,
* `--no-restart` disables the Docker restart policy, since this will be
handled by systemd.
* `--name` specifies a unique name for this new peer.
* `--ipalloc-init seed` specifies the names of only those peers that were
involved in the initial cluster bootstrap - even if they have been
subsequently removed from the cluster. You can view this as a kind
of 'cluster identity',where peers may only interoperate in the same
cluster if they share the same seed.
* `$PEERS` is obtained from `/etc/sysconfig/weave` as described in the
linked systemd documentation. For convenience, this may contain the
address of the peer that is being launched, so that you don't have
to compute separate lists of 'other' peers tailored to each peer -
just supply the same complete list of peer addresses to every peer.
Note that unlike [Interactive]({{ '/operational-guide/interactive' | relative_url }})
and [Uniform Fixed Cluster]({{ '/operational-guide/uniform-fixed-cluster' | relative_url }}) deployments
there is no `weave prime` step. You can add as many new peers in
parallel as you like, even under conditions of partition, and they
will all (eventually) join safely. This is ideal for use in
conjunction with asynchronous provisioning systems such as puppet or
chef.
For maximum robustness, you should distribute an updated
`/etc/sysconfig/weave` file including the new peer to all existing
peers.
### Removing a Peer
On the peer to be removed:
weave reset
You may remove a seed peer, as long as there is at least one other
seed peer left in the network.
Then, distribute an updated `/etc/sysconfig/weave` to the remaining
peers, omitting the removed peer from `$PEERS`.
On each remaining peer:
weave forget <removed peer>
This step is not mandatory, but it will eliminate log noise and
spurious network traffic by stopping reconnection attempts.
@@ -0,0 +1,88 @@
---
title: Uniform Fixed Clusters
menu_order: 30
search_type: Documentation
---
This scenario describes a production deployment of a fixed number of N
nodes (N=1 in the simplest case).
A uniform fixed cluster has the following characteristics:
* Recovers automatically from reboots and partitions.
* All peers have identical configuration.
* There is a controlled process for adding or removing nodes, however
the end user is responsible for ensuring that only one instance of
the process is in-flight at a time. While it is possible to
automate, the potentially-blocking `weave prime` operation and the
need for global serialization make it non-trivial. It is however
relatively straightforward for a human to provide the necessary
guarantees and exception handling manually, and so this scenario is best
suited to deployments which change size infrequently as a planned
maintenance event.
### Bootstrapping
On each initial peer, at boot, via
[systemd]({{ '/install/systemd' | relative_url }}):
weave launch --no-restart $PEERS
Where,
* `--no-restart` disables the Docker restart policy, since this will be
handled by systemd.
* `$PEERS` is obtained from `/etc/sysconfig/weave` as described in the
linked systemd documentation. For convenience, this may contain the
address of the peer which is being launched, so that you don't have
to compute separate lists of 'other' peers tailored to each peer -
just supply the same complete list of peer addresses to every peer.
Then on _any_ peer run the following to force consensus:
weave prime
>**Note:** You can run this safely on more than one or
even all peers, but it's only strictly necessary to run it on one of
them.
Once this command completes successfully, IP address
allocations can proceed under partition and it is safe to add new
peers. If this command waits without exiting, it means that there is an issue (such
as a network partition or failed peers) that is preventing a quorum
from being reached – you will need to [address
that]({{ '/troubleshooting' | relative_url }}) before moving on.
### Adding a Peer
On the new peer, at boot, via
[systemd]({{ '/install/systemd' | relative_url }}) run:
weave launch --no-restart $PEERS
Where,
* `$PEERS` is the new peer plus all other peers in the network,
initial and subsequently added, which have not been explicitly
removed. It should include peers which are temporarily offline or
stopped.
For maximum robustness, distribute an updated
`/etc/sysconfig/weave` file including the new peer to all existing
peers.
### Removing a Peer
On the peer to be removed:
weave reset
Then distribute an updated `/etc/sysconfig/weave` to the remaining
peers, omitting the removed peer from `$PEERS`.
On each remaining peer:
weave forget <removed peer>
This final step is not mandatory, but it will eliminate log noise and
spurious network traffic by stopping any reconnection attempts.