This commit is contained in:
Neil Alexander 2022-05-11 15:39:36 +01:00
parent 9599b3686e
commit 19a9166eb0
No known key found for this signature in database
GPG key ID: A02A2019A2BB0944
39 changed files with 1483 additions and 944 deletions

View file

@ -0,0 +1,110 @@
---
title: Planning your installation
parent: Installation
nav_order: 1
permalink: /installation/planning
---
# Planning your installation
## Modes
Dendrite can be run in one of two configurations:
* **Monolith mode**: All components run in the same process. In this mode,
it is possible to run an in-process NATS Server instead of running a standalone deployment.
This will usually be the preferred model for low-to-mid volume deployments, providing the best
balance between performance and resource usage.
* **Polylith mode**: A cluster of individual components running in their own processes, dealing
with different aspects of the Matrix protocol. Components communicate with each other using
internal HTTP APIs and NATS Server. This will almost certainly be the preferred model for very
large deployments but scalability comes with a cost. API calls are expensive and therefore a
polylith deployment may end up using disproportionately more resources for a smaller number of
users compared to a monolith deployment.
At present, we **recommend monolith mode deployments** in all cases.
## Databases
Dendrite can run with either a PostgreSQL or a SQLite backend. There are considerable tradeoffs
to consider:
* **PostgreSQL**: Needs to run separately to Dendrite, needs to be installed and configured separately
and and will use more resources over all, but will be **considerably faster** than SQLite. PostgreSQL
has much better write concurrency which will allow Dendrite to process more tasks in parallel. This
will be necessary for federated deployments to perform adequately.
* **SQLite**: Built into Dendrite, therefore no separate database engine is necessary and is quite
a bit easier to set up, but will be much slower than PostgreSQL in most cases. SQLite only allows a
single writer on a database at a given time, which will significantly restrict Dendrite's ability
to process multiple tasks in parallel.
At this time, we **recommend the PostgreSQL database engine** for all production deployments.
## Requirements
Dendrite will run on Linux, macOS and Windows Server. It should also run fine on variants
of BSD such as FreeBSD and OpenBSD. We have not tested Dendrite on AIX, Solaris, Plan 9 or z/OS —
your mileage may vary with these platforms.
It is difficult to state explicitly the amount of CPU, RAM or disk space that a Dendrite
installation will need, as this varies considerably based on a number of factors. In particular:
* The number of users using the server;
* The number of rooms that the server is joined to — federated rooms in particular will typically
use more resources than rooms with only local users;
* The complexity of rooms that the server is joined to — rooms with more members coming and
going will typically be of a much higher complexity.
Some tasks are more expensive than others, such as joining rooms over federation, running state
resolution or sending messages into very large federated rooms with lots of remote users. Therefore
you should plan accordingly and ensure that you have enough resources available to endure spikes
in CPU or RAM usage, as these may be considerably higher than the idle resource usage.
At an absolute minimum, Dendrite will expect 1GB RAM. For a comfortable day-to-day deployment
which can participate in federated rooms for a number of local users, be prepared to assign 2-4
CPU cores and 8GB RAM — more if your user count increases.
If you are running PostgreSQL on the same machine, allow extra headroom for this too, as the
database engine will also have CPU and RAM requirements of its own. Running too many heavy
services on the same machine may result in resource starvation and processes may end up being
killed by the operating system if they try to use too much memory.
## Dependencies
In order to install Dendrite, you will need to satisfy the following dependencies.
### Go
At this time, Dendrite supports being built with Go 1.16 or later. We do not support building
Dendrite with older versions of Go than this. If you are installing Go using a package manager,
you should check (by running `go version`) that you are using a suitable version before you start.
### PostgreSQL
If using the PostgreSQL database engine, you should install PostgreSQL 12 or later.
### NATS Server
Monolith deployments come with a built-in [NATS Server](https://github.com/nats-io/nats-server) and
therefore do not need this to be manually installed. If you are planning a monolith installation, you
do not need to do anything.
Polylith deployments, however, currently need a standalone NATS Server installation with JetStream
enabled.
To do so, follow the [NATS Server installation instructions](https://docs.nats.io/running-a-nats-service/introduction/installation) and then [start your NATS deployment](https://docs.nats.io/running-a-nats-service/introduction/running). JetStream must be enabled, either by passing the `-js` flag to `nats-server`,
or by specifying the `store_dir` option in the the `jetstream` configuration.
### Reverse proxy (polylith deployments)
Polylith deployments require a reverse proxy, such as [NGINX](https://www.nginx.com) or
[HAProxy](http://www.haproxy.org). Configuring those is not covered in this documentation,
although a [sample configuration for NGINX](https://github.com/matrix-org/dendrite/blob/main/docs/nginx/polylith-sample.conf)
is provided.
### Windows
Finally, if you want to build Dendrite on Windows, you will need need `gcc` in the path. The best
way to achieve this is by installing and building Dendrite under [MinGW-w64](https://www.mingw-w64.org/).

View file

@ -0,0 +1,93 @@
---
title: Setting up the domain
parent: Installation
nav_order: 2
permalink: /installation/domainname
---
# Setting up the domain
Every Matrix server deployment requires a server name which uniquely identifies it. For
example, if you are using the server name `example.com`, then your users will have usernames
that take the format `@user:example.com`.
For federation to work, the server name must be resolvable by other homeservers on the internet
— that is, the domain must be registered and properly configured with the relevant DNS records.
Matrix servers discover each other when federating using the following methods:
1. If a well-known delegation exists on `example.com`, use the path server from the
well-known file to connect to the remote homeserver;
2. If a DNS SRV delegation exists on `example.com`, use the hostname and port from the DNS SRV
record to connect to the remote homeserver;
3. If neither well-known or DNS SRV delegation are configured, attempt to connect to the remote
homeserver by connecting to `example.com` port TCP/8448 using HTTPS.
## TLS certificates
Matrix federation requires that valid TLS certificates are present on the domain. You must
obtain certificates from a publicly accepted Certificate Authority (CA). [LetsEncrypt](https://letsencrypt.org)
is an example of such a CA that can be used. Self-signed certificates are not suitable for
federation and will typically not be accepted by other homeservers.
A common practice to help ease the management of certificates is to install a reverse proxy in
front of Dendrite which manages the TLS certificates and HTTPS proxying itself. Software such as
[NGINX](https://www.nginx.com) and [HAProxy](http://www.haproxy.org) can be used for the task.
Although the finer details of configuring these are not described here, you must reverse proxy
all `/_matrix` paths to your Dendrite server.
It is possible for the reverse proxy to listen on the standard HTTPS port TCP/443 so long as your
domain delegation is configured to point to port TCP/443.
## Delegation
Delegation allows you to specify the server name and port that your Dendrite installation is
reachable at, or to host the Dendrite server at a different server name to the domain that
is being delegated.
For example, if your Dendrite installation is actually reachable at `matrix.example.com` port 8448,
you will be able to delegate from `example.com` to `matrix.example.com` so that your users will have
`@user:example.com` user names instead of `@user:matrix.example.com` usernames.
Delegation can be performed in one of two ways:
* **Well-known delegation**: A well-known text file is served over HTTPS on the domain name
that you want to use, pointing to your server on `matrix.example.com` port 8448;
* **DNS SRV delegation**: A DNS SRV record is created on the domain name that you want to
use, pointing to your server on `matrix.example.com` port TCP/8448.
If you are using a reverse proxy to forward `/_matrix` to Dendrite, your well-known or DNS SRV
delegation must refer to the hostname and port that the reverse proxy is listening on instead.
Well-known delegation is typically easier to set up and usually preferred. However, you can use
either or both methods to delegate. If you configure both methods of delegation, it is important
that they both agree and refer to the same hostname and port.
## Well-known delegation
Using well-known delegation requires that you are running a web server at `example.com` which
is listening on the standard HTTPS port TCP/443.
Assuming that your Dendrite installation is listening for HTTPS connections at `matrix.example.com`
on port 8448, the delegation file must be served at `https://example.com/.well-known/matrix/server`
and contain the following JSON document:
```json
{
"m.server": "https://matrix.example.com:8448"
}
```
## DNS SRV delegation
Using DNS SRV delegation requires creating DNS SRV records on the `example.com` zone which
refer to your Dendrite installation.
Assuming that your Dendrite installation is listening for HTTPS connections at `matrix.example.com`
port 8448, the DNS SRV record must have the following fields:
* Name: `@` (or whichever term your DNS provider uses to signal the root)
* Service: `_matrix`
* Protocol: `_tcp`
* Port: `8448`
* Target: `matrix.example.com`

View file

@ -0,0 +1,106 @@
---
title: Preparing database storage
parent: Installation
nav_order: 3
permalink: /installation/database
---
# Preparing database storage
Dendrite uses SQL databases to store data. Depending on the database engine being used, you
may need to perform some manual steps outlined below.
## SQLite
SQLite deployments do not require manual database creation. Simply configure the database
filenames in the Dendrite configuration file and start Dendrite. The databases will be created
and populated automatically.
Note that Dendrite **cannot share a single SQLite database across multiple components**. Each
component must be configured with its own SQLite database filename.
### Connection strings
Connection strings for SQLite databases take the following forms:
* Current working directory path: `file:dendrite_component.db`
* Full specified path: `file:///path/to/dendrite_component.db`
## PostgreSQL
Dendrite can automatically populate the database with the relevant tables and indexes, but
it is not capable of creating the databases themselves. You will need to create the databases
manually.
At this point, you can choose to either use a single database for all Dendrite components,
or you can run each component with its own separate database:
* **Single database**: You will need to create a single PostgreSQL database. Monolith deployments
can use a single global connection pool, which makes updating the configuration file much easier.
Only one database connection string to manage and likely simpler to back up the database. All
components will be sharing the same database resources (CPU, RAM, storage).
* **Separate databases**: You will need to create a separate PostgreSQL database for each
component. You will need to configure each component that has storage in the Dendrite
configuration file with its own connection parameters. Allows running a different database engine
for each component on a different machine if needs be, each with their own CPU, RAM and storage —
almost certainly overkill unless you are running a very large Dendrite deployment.
For either configuration, you will want to:
1. Configure a role (with a username and password) which Dendrite can use to connect to the
database;
2. Create the database(s) themselves, ensuring that the Dendrite role has privileges over them.
As Dendrite will create and manage the database tables, indexes and sequences by itself, the
Dendrite role must have suitable privileges over the database.
### Connection strings
The format of connection strings for PostgreSQL databases is described in the [PostgreSQL libpq manual](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING). Note that Dendrite only
supports the "Connection URIs" format and **will not** work with the "Keyword/Value Connection
string" format.
Example supported connection strings take the format:
* `postgresql://user:pass@hostname/database?options=...`
* `postgres://user:pass@hostname/database?options=...`
If you need to disable SSL/TLS on the database connection, you may need to append `?sslmode=disable` to the end of the connection string.
### Role creation
Create a role which Dendrite can use to connect to the database, choosing a new password when
prompted. On macOS, you may need to omit the `sudo -u postgres` from the below instructions.
```bash
sudo -u postgres createuser -P dendrite
```
### Single database creation
Create the database itself, using the `dendrite` role from above:
```bash
sudo -u postgres createdb -O dendrite dendrite
```
### Multiple database creation
The following eight components require a database. In this example they will be named:
| Appservice API | `dendrite_appservice` |
| Federation API | `dendrite_federationapi` |
| Media API | `dendrite_mediaapi` |
| MSCs | `dendrite_mscs` |
| Roomserver | `dendrite_roomserver` |
| Sync API | `dendrite_syncapi` |
| Key server | `dendrite_keyserver` |
| User API | `dendrite_userapi` |
... therefore you will need to create eight different databases:
```bash
for i in appservice federationapi mediaapi mscs roomserver syncapi keyserver userapi; do
sudo -u postgres createdb -O dendrite dendrite_$i
done
```

View file

@ -0,0 +1,79 @@
---
title: Generating signing keys
parent: Installation
nav_order: 4
permalink: /installation/signingkeys
---
# Generating signing keys
All Matrix homeservers require a signing private key, which will be used to authenticate
federation requests and events.
The `generate-keys` utility can be used to generate a private key. Assuming that Dendrite was
built using `build.sh`, you should find the `generate-keys` utility in the `bin` folder.
To generate a Matrix signing private key:
```bash
./bin/generate-keys --private-key matrix_key.pem
```
The generated `matrix_key.pem` file is your new signing key.
## Important warning
You must treat this key as if it is highly sensitive and private, so **never share it with
anyone**. No one should ever ask you for this key for any reason, even to debug a problematic
Dendrite server.
Make sure take a safe backup of this key. You will likely need it if you want to reinstall
Dendrite, or any other Matrix homeserver, on the same domain name in the future. If you lose
this key, you may have trouble joining federated rooms.
## Old signing keys
If you already have old signing keys from a previous Matrix installation on the same domain
name, you can reuse those instead, as long as they have not been previously marked as expired —
a key that has been marked as expired in the past is unusable.
Old keys from a previous Dendrite installation can be reused as-is without any further
configuration required. Simply use that key file in the Dendrite configuration.
If you have server keys from an older Synapse instance, you can convert them to Dendrite's PEM
format and configure them as `old_private_keys` in your config.
## Key format
Dendrite stores the server signing key in the PEM format with the following structure.
```
-----BEGIN MATRIX PRIVATE KEY-----
Key-ID: ed25519:<Key ID>
<Base64 Encoded Key Data>
-----END MATRIX PRIVATE KEY-----
```
## Converting Synapse keys
If you have signing keys from a previous Synapse installation, you should ideally configure them
as `old_private_keys` in your Dendrite config file. Synapse stores signing keys in the following
format:
```
ed25519 <Key ID> <Base64 Encoded Key Data>
```
To convert this key to Dendrite's PEM format, use the following template. You must copy the Key ID
exactly without modifying it. **It is important to include the trailing equals sign on the Base64
Encoded Key Data** if it is not already present in the original key, as the key data needs to be
padded to exactly 32 bytes:
```
-----BEGIN MATRIX PRIVATE KEY-----
Key-ID: ed25519:<Key ID>
<Base64 Encoded Key Data>=
-----END MATRIX PRIVATE KEY-----
```

View file

@ -0,0 +1,21 @@
---
title: Installing as a monolith
parent: Installation
has_toc: true
nav_order: 5
permalink: /installation/install/monolith
---
# Installing as a monolith
You can install the Dendrite monolith binary into `$GOPATH/bin` by using `go install`:
```sh
go install ./cmd/dendrite-monolith-server
```
Alternatively, you can specify a custom path for the binary to be written to using `go build`:
```sh
go build -o /usr/local/bin/ ./cmd/dendrite-monolith-server
```

View file

@ -0,0 +1,33 @@
---
title: Installing as a polylith
parent: Installation
has_toc: true
nav_order: 6
permalink: /installation/install/polylith
---
# Installing as a polylith
You can install the Dendrite polylith binary into `$GOPATH/bin` by using `go install`:
```sh
go install ./cmd/dendrite-polylith-multi
```
Alternatively, you can specify a custom path for the binary to be written to using `go build`:
```sh
go build -o /usr/local/bin/ ./cmd/dendrite-polylith-multi
```
The `dendrite-polylith-multi` binary is a "multi-personality" binary which can run as
any of the components depending on the supplied command line parameters.
## Reverse proxy
Polylith deployments require a reverse proxy in order to ensure that requests are
sent to the correct endpoint. You must ensure that a suitable reverse proxy is installed
and configured.
A [sample configuration file](https://github.com/matrix-org/dendrite/blob/main/docs/nginx/polylith-sample.conf)
is provided for [NGINX](https://www.nginx.com).

View file

@ -0,0 +1,145 @@
---
title: Populate the configuration
parent: Installation
nav_order: 7
permalink: /installation/configuration
---
# Populate the configuration
The configuration file is used to configure Dendrite. A sample configuration file,
called [`dendrite-config.yaml`](https://github.com/matrix-org/dendrite/blob/main/dendrite-config.yaml),
is present in the top level of the Dendrite repository.
You will need to duplicate this file, calling it `dendrite.yaml` for example, and then
tailor it to your installation. At a minimum, you will need to populate the following
sections:
## Server name
First of all, you will need to configure the server name of your Matrix homeserver.
This must match the domain name that you have selected whilst [configuring the domain
name delegation](domainname).
In the `global` section, set the `server_name` to your delegated domain name:
```yaml
global:
# ...
server_name: example.com
```
## Server signing keys
Next, you should tell Dendrite where to find your [server signing keys](signingkeys).
In the `global` section, set the `private_key` to the path to your server signing key:
```yaml
global:
# ...
private_key: /path/to/matrix_key.pem
```
## JetStream configuration
Monolith deployments can use the built-in NATS Server rather than running a standalone
server. If you are building a polylith deployment, or you want to use a standalone NATS
Server anyway, you can also configure that too.
### Built-in NATS Server (monolith only)
In the `global` section, under the `jetstream` key, ensure that no server addresses are
configured and set a `storage_path` to a persistent folder on the filesystem:
```yaml
global:
# ...
jetstream:
in_memory: false
storage_path: /path/to/storage/folder
topic_prefix: Dendrite
```
### Standalone NATS Server (monolith and polylith)
To use a standalone NATS Server instance, you will need to configure `addresses` field
to point to the port that your NATS Server is listening on:
```yaml
global:
# ...
jetstream:
addresses:
- localhost:4222
topic_prefix: Dendrite
```
You do not need to configure the `storage_path` when using a standalone NATS Server instance.
In the case that you are connecting to a multi-node NATS cluster, you can configure more than
one address in the `addresses` field.
## Database connections
Configuring database connections varies based on the [database configuration](database)
that you chose.
### Global connection pool (monolith with a single PostgreSQL database only)
If you are running a monolith deployment and want to use a single connection pool to a
single PostgreSQL database, then you must uncomment and configure the `database` section
within the `global` section:
```yaml
global:
# ...
database:
connection_string: postgres://user:pass@hostname/database?sslmode=disable
max_open_conns: 100
max_idle_conns: 5
conn_max_lifetime: -1
```
**You must then remove or comment out** the `database` sections from other areas of the
configuration file, e.g. under the `app_service_api`, `federation_api`, `key_server`,
`media_api`, `mscs`, `room_server`, `sync_api` and `user_api` blocks, otherwise these will
override the `global` database configuration.
### Per-component connections (all other configurations)
If you are building a polylith deployment, are using SQLite databases or separate PostgreSQL
databases per component, then you must instead configure the `database` sections under each
of the component blocks ,e.g. under the `app_service_api`, `federation_api`, `key_server`,
`media_api`, `mscs`, `room_server`, `sync_api` and `user_api` blocks.
For example, with PostgreSQL:
```yaml
room_server:
# ...
database:
connection_string: postgres://user:pass@hostname/dendrite_component?sslmode=disable
max_open_conns: 10
max_idle_conns: 2
conn_max_lifetime: -1
```
... or with SQLite:
```yaml
room_server:
# ...
database:
connection_string: file:roomserver.db
max_open_conns: 10
max_idle_conns: 2
conn_max_lifetime: -1
```
## Other sections
There are other options which may be useful so review them all. In particular, if you are
trying to federate from your Dendrite instance into public rooms then configuring the
`key_perspectives` (like `matrix.org` in the sample) can help to improve reliability
considerably by allowing your homeserver to fetch public keys for dead homeservers from
another living server.

View file

@ -0,0 +1,41 @@
---
title: Starting the monolith
parent: Installation
has_toc: true
nav_order: 9
permalink: /installation/start/monolith
---
# Starting the monolith
Once you have completed all of the preparation and installation steps,
you can start your Dendrite monolith deployment by starting the `dendrite-monolith-server`:
```bash
./dendrite-monolith-server -config /path/to/dendrite.yaml
```
If you want to change the addresses or ports that Dendrite listens on, you
can use the `-http-bind-address` and `-https-bind-address` command line arguments:
```bash
./dendrite-monolith-server -config /path/to/dendrite.yaml \
-http-bind-address 1.2.3.4:12345 \
-https-bind-address 1.2.3.4:54321
```
## Running under systemd
A common deployment pattern is to run the monolith under systemd. For this, you
will need to create a service unit file. An example service unit file is available
in the [GitHub repository](https://github.com/matrix-org/dendrite/blob/main/docs/systemd/monolith-example.service).
Once you have installed the service unit, you can notify systemd, enable and start
the service:
```bash
systemctl daemon-reload
systemctl enable dendrite
systemctl start dendrite
journalctl -fu dendrite
```

View file

@ -0,0 +1,73 @@
---
title: Starting the polylith
parent: Installation
has_toc: true
nav_order: 9
permalink: /installation/start/polylith
---
# Starting the polylith
Once you have completed all of the preparation and installation steps,
you can start your Dendrite polylith deployment by starting the various components
using the `dendrite-polylith-multi` personalities.
## Start the reverse proxy
Ensure that your reverse proxy is started and is proxying the correct
endpoints to the correct components. Software such as [NGINX](https://www.nginx.com) or
[HAProxy](http://www.haproxy.org) can be used for this purpose. A [sample configuration
for NGINX](https://github.com/matrix-org/dendrite/blob/main/docs/nginx/polylith-sample.conf)
is provided.
## Starting the components
Each component must be started individually:
### Client API
```bash
./dendrite-polylith-multi -config /path/to/dendrite.yaml clientapi
```
### Sync API
```bash
./dendrite-polylith-multi -config /path/to/dendrite.yaml syncapi
```
### Media API
```bash
./dendrite-polylith-multi -config /path/to/dendrite.yaml mediaapi
```
### Federation API
```bash
./dendrite-polylith-multi -config /path/to/dendrite.yaml federationapi
```
### Roomserver
```bash
./dendrite-polylith-multi -config /path/to/dendrite.yaml roomserver
```
### Appservice API
```bash
./dendrite-polylith-multi -config /path/to/dendrite.yaml appservice
```
### User API
```bash
./dendrite-polylith-multi -config /path/to/dendrite.yaml userapi
```
### Key server
```bash
./dendrite-polylith-multi -config /path/to/dendrite.yaml keyserver
```