Merge 4cef8cbb9f39dbb4be251c047394b4d69be66fba into ca8c1cf2b5416924c4379d43d5c943928377747d

Hopefully fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3544

CHANGELOG.md

@@ -1,3 +1,45 @@
+# 2024-11-23
+
+## (Backward Compatibility Break) The playbook now defaults to Valkey, instead of KeyDB
+
+**TLDR**: if the playbook installed KeyDB (or Redis) as a dependency for you before, it will now replace it with [Valkey](https://valkey.io/) (a drop-in alternative). We [previously switched from Redis to KeyDB](#backward-compatibility-break-the-playbook-now-defaults-to-keydb-instead-of-redis), but Valkey is a better alternative, so we're switching again.
+
+The playbook used to install Redis or KeyDB if services have a need for a Redis-compatible implementation ([enabling worker support for Synapse](docs/configuring-playbook-synapse.md#load-balancing-with-workers), [enabling Hookshot encryption](docs/configuring-playbook-bridge-hookshot.md#end-to-bridge-encryption), etc.).
+
+Earlier this year, we switched from Redis to KeyDB - see [(Backward Compatibility Break) The playbook now defaults to KeyDB, instead of Redis](#backward-compatibility-break-the-playbook-now-defaults-to-keydb-instead-of-redis).
+
+Because Valkey seems to be a better successor to Redis (than KeyDB) and likely doesn't suffer from [issues like this one](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3544), we now replace KeyDB with Valkey.
+
+Valkey (like KeyDB and Redis in the past) is an implicitly enabled dependency - you don't need custom configuration in `vars.yml` to enable it.
+
+Next time your run the playbook (via the `setup-all` tag), **KeyDB will be automatically uninstalled and replaced with Valkey**. Some Synapse downtime may occur while the switch happens.
+
+Users on `arm32` should be aware that there's **neither a prebuilt `arm32` container image for Valkey**, nor the Valkey role supports self-building yet. Users on this architecture likely don't run Synapse with workers, etc., so they're likely in no need of Valkey (or Redis/KeyDB). If Redis is necessary in an `arm32` deployment, disabling Valkey and making the playbook fall back to Redis is possible (see below).
+
+**The playbook still supports Redis** and you can keep using Redis (for now) if you'd like, by adding this additional configuration to your `vars.yml` file:
+
+```yml
+# Explicitly disable both Valkey and KeyDB.
+#
+# Redis will be auto-enabled if necessary,
+# because there's no other Redis-compatible implementation being enabled.
+valkey_enabled: false
+keydb_enabled: false
+```
+
+**The playbook still supports KeyDB** and you can keep using KeyDB (for now) if you'd like, by adding this additional configuration to your `vars.yml` file:
+
+```yml
+# Explicitly disable Valkey  enable KeyDB.
+#
+# Redis will not be auto-enabled beandcause a Redis-compatible implementation (KeyDB) is enabled.
+valkey_enabled: false
+keydb_enabled: true
+```
+
+At some point in time in the future, we'll remove both KeyDB and Redis from the playbook, so we recommend that you migrate to Valkey earlier anyway.
+
+
 # 2024-11-14
 
 ## HTTP-compression support for Traefik-based setups

README.md

@@ -27,6 +27,8 @@ While the [list of supported services](#-supported-services) and documentation i
 - Starting with the basics. You can always add/remove or tweak services later on.
 - Following our guided installation, starting with the [Prerequisites](./docs/prerequisites.md) documentation page
 
+If you have never configured Matrix services, follow the [**quick start**](./docs/quick-start.md) guide to set up minimum core services on your server.
+
 ## ✔ Supported services
 
 Using this playbook, you can get the following list of services configured on your server. Basically, this playbook aims to get you up-and-running with all the necessities around Matrix, without you having to do anything else.

docs/README.md

@@ -2,6 +2,8 @@
 
 - [FAQ](faq.md) - lots of questions and answers. Jump to [Prerequisites](prerequisites.md) to avoid reading too much and to just start a guided installation.
 
+- [Quick start](quick-start.md) - follow the guide to set up minimum core services on your server
+
 - [Prerequisites](prerequisites.md) - go here to a guided installation using this Ansible playbook
 
 - [Configuring your DNS settings](configuring-dns.md)

docs/configuring-dns.md

@@ -1,6 +1,6 @@
 # Configuring your DNS settings
 
-<sup>⚡️[Quick start](README.md) | [Prerequisites](prerequisites.md) > Configuring your DNS settings > [Getting the playbook](getting-the-playbook.md) > [Configuring the playbook](configuring-playbook.md) > [Installing](installing.md)</sup>
+<sup>⚡️[Quick start](quick-start.md) | [Prerequisites](prerequisites.md) > Configuring your DNS settings > [Getting the playbook](getting-the-playbook.md) > [Configuring the playbook](configuring-playbook.md) > [Installing](installing.md)</sup>
 
 To set up Matrix on your domain, you'd need to do some DNS configuration.
 
@@ -36,7 +36,7 @@ The `element.example.com` subdomain is necessary, because this playbook installs
 
 Be mindful as to how long it will take for the DNS records to propagate.
 
-If you are using Cloudflare DNS, make sure to disable the proxy and set all records to `DNS only`. Otherwise, fetching certificates will fail.
+If you are using Cloudflare DNS, make sure to disable the proxy and set all records to "DNS only". Otherwise, fetching certificates will fail.
 
 ## DNS settings for optional services/features
 

docs/configuring-playbook-prometheus-grafana.md

@@ -16,7 +16,7 @@ prometheus_node_exporter_enabled: true
 prometheus_postgres_exporter_enabled: true
 
 # You can remove this, if unnecessary.
-matrix_prometheus_nginxlog_exporter_enabled: true
+matrix_prometheus_traefik_exporter_enabled: true
 
 grafana_enabled: true
 
@@ -64,7 +64,7 @@ Name | Description
 `prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about.
 `prometheus_node_exporter_enabled`|[Node Exporter](https://prometheus.io/docs/guides/node-exporter/) is an addon of sorts to Prometheus that collects generic system information such as CPU, memory, filesystem, and even system temperatures
 `prometheus_postgres_exporter_enabled`|[Postgres Exporter](configuring-playbook-prometheus-postgres.md) is an addon of sorts to expose Postgres database metrics to Prometheus.
-`matrix_prometheus_nginxlog_exporter_enabled`|[NGINX Log Exporter](configuring-playbook-prometheus-nginxlog.md) is an addon of sorts to expose NGINX logs to Prometheus.
+`matrix_prometheus_traefik_exporter_enabled`|Export [Traefik metrics](https://doc.traefik.io/traefik/observability/metrics/prometheus) to Prometheus and add Grafana dashboard for them.
 `grafana_enabled`|[Grafana](https://grafana.com/) is the visual component. It shows (on the `stats.example.com` subdomain) the dashboards with the graphs that we're interested in
 `grafana_anonymous_access`|By default you need to log in to see graphs. If you want to publicly share your graphs (e.g. when asking for help in [`#synapse:matrix.org`](https://matrix.to/#/#synapse:matrix.org?via=matrix.org&via=privacytools.io&via=mozilla.org)) you'll want to enable this option.
 `grafana_default_admin_user`<br>`grafana_default_admin_password`|By default Grafana creates a user with `admin` as the username and password. If you feel this is insecure and you want to change it beforehand, you can do that here
@@ -100,6 +100,7 @@ Name | Description
 `prometheus_node_exporter_container_labels_traefik_enabled`|Set this to `true` to expose the node (general system stats) metrics on `https://matrix.example.com/metrics/node-exporter`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above.
 `prometheus_postgres_exporter_enabled`|Set this to `true` to enable the [Postgres exporter](configuring-playbook-prometheus-postgres.md) (locally, on the container network)
 `prometheus_postgres_exporter_container_labels_traefik_enabled`|Set this to `true` to expose the [Postgres exporter](configuring-playbook-prometheus-postgres.md) metrics on `https://matrix.example.com/metrics/postgres-exporter`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above.
+`matrix_prometheus_traefik_exporter_enabled`|Set this to `true` to export [Traefik metrics](https://doc.traefik.io/traefik/observability/metrics/prometheus) (locally, on the container network) and add Grafana dashboard for them.
 `matrix_prometheus_nginxlog_exporter_enabled`|Set this to `true` to enable the [NGINX Log exporter](configuring-playbook-prometheus-nginxlog.md) (locally, on the container network)
 `matrix_sliding_sync_metrics_enabled`|Set this to `true` to make [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) expose metrics (locally, on the container network)
 `matrix_sliding_sync_metrics_proxying_enabled`|Set this to `true` to expose the [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) metrics on `https://matrix.example.com/metrics/sliding-sync`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above.

docs/configuring-playbook.md

@@ -1,6 +1,6 @@
 # Configuring the playbook
 
-<sup>⚡️[Quick start](README.md) | [Prerequisites](prerequisites.md) > [Configuring your DNS settings](configuring-dns.md) > [Getting the playbook](getting-the-playbook.md) > Configuring the playbook > [Installing](installing.md)</sup>
+<sup>⚡️[Quick start](quick-start.md) | [Prerequisites](prerequisites.md) > [Configuring your DNS settings](configuring-dns.md) > [Getting the playbook](getting-the-playbook.md) > Configuring the playbook > [Installing](installing.md)</sup>
 
 If you've configured your DNS records and retrieved the playbook's source code to your computer, you can start configuring the playbook. To do so, follow these steps inside the playbook directory:
 

docs/faq.md

@@ -80,6 +80,16 @@ Alternatively, you can download Ansible and the playbook itself directly on the
 
 To learn more, see our [dedicated Ansible documentation page](ansible.md).
 
+### What is `just`?
+
+[`just`](https://github.com/casey/just) is a modern command-runner alternative to [make](https://www.gnu.org/software/make/). It can be used to invoke commands with less typing.
+
+The `just` utility executes shortcut commands (called "recipes"), which invoke `ansible-playbook`, `ansible-galaxy` or [`agru`](https://github.com/etkecc/agru) (depending on what is available in your system). The targets of the recipes are defined in [`justfile`](../justfile).
+
+For details about `just` commands, take a look at: [Running `just` commands](just.md).
+
+The playbook also contains a `Makefile` for the `make` tool, but most of the just recipes are not available as targets in the `Makefile`.
+
 ### Why use this playbook and not install Synapse and other things manually?
 
 There are various guides telling you how easy it is to install [Synapse](https://github.com/element-hq/synapse).

docs/getting-the-playbook.md

@@ -1,6 +1,6 @@
 # Getting the playbook
 
-<sup>⚡️[Quick start](README.md) | [Prerequisites](prerequisites.md) > [Configuring your DNS settings](configuring-dns.md) > Getting the playbook > [Configuring the playbook](configuring-playbook.md) > [Installing](installing.md)</sup>
+<sup>⚡️[Quick start](quick-start.md) | [Prerequisites](prerequisites.md) > [Configuring your DNS settings](configuring-dns.md) > Getting the playbook > [Configuring the playbook](configuring-playbook.md) > [Installing](installing.md)</sup>
 
 This Ansible playbook is meant to be executed on your own computer (not the Matrix server).
 

docs/installing.md

@@ -1,16 +1,21 @@
 # Installing
 
-<sup>⚡️[Quick start](README.md) | [Prerequisites](prerequisites.md) > [Configuring your DNS settings](configuring-dns.md) > [Getting the playbook](getting-the-playbook.md) > [Configuring the playbook](configuring-playbook.md) > Installing</sup>
+<sup>⚡️[Quick start](quick-start.md) | [Prerequisites](prerequisites.md) > [Configuring your DNS settings](configuring-dns.md) > [Getting the playbook](getting-the-playbook.md) > [Configuring the playbook](configuring-playbook.md) > Installing</sup>
 
 If you've configured your DNS records and the playbook, you can start the installation procedure.
 
 ## Update Ansible roles
 
-Before installing, you need to update the Ansible roles in this playbook by running `just roles`.
+Before installing, you need to update the Ansible roles that this playbook uses and fetches from outside.
 
-`just roles` is a shortcut (a `roles` target defined in [`justfile`](../justfile) and executed by the [`just`](https://github.com/casey/just) utility) which ultimately runs [agru](https://github.com/etkecc/agru) or [ansible-galaxy](https://docs.ansible.com/ansible/latest/cli/ansible-galaxy.html) (depending on what is available in your system) to download Ansible roles. If you don't have `just`, you can also manually run the `roles` commands seen in the `justfile`.
+To update your playbook directory and all upstream Ansible roles (defined in the `requirements.yml` file), run:
 
-There's another shortcut (`just update`) which updates the playbook (`git pull`) and updates roles (`just roles`) at the same time.
+- either: `just update`
+- or: a combination of `git pull` and `just roles` (or `make roles` if you have `make` program on your computer instead of `just`)
+
+If you don't have either `just` tool or `make` program, you can run the `ansible-galaxy` tool directly: `rm -rf roles/galaxy; ansible-galaxy install -r requirements.yml -p roles/galaxy/ --force`
+
+For details about `just` commands, take a look at: [Running `just` commands](just.md).
 
 ## Install Matrix server and services
 
@@ -48,7 +53,7 @@ To do the installation **without** starting services, run `ansible-playbook` wit
 ansible-playbook -i inventory/hosts setup.yml --tags=install-all
 ```
 
-**Note**: do not run the just "recipe" `just install-all` instead, because it automatically starts services at the end of execution.
+**Note**: do not run the just "recipe" `just install-all` instead, because it automatically starts services at the end of execution. See: [Difference between playbook tags and shortcuts](just.md#difference-between-playbook-tags-and-shortcuts)
 
 When this command completes, services won't be running yet.
 
@@ -74,18 +79,21 @@ As you have configured your brand new server and the client, you need to **creat
 
 After creating the user account, you can log in to it with [Element Web](configuring-playbook-client-element-web.md) that this playbook has installed for you at this URL: `https://element.example.com/`.
 
-To register a user via this Ansible playbook, run the command below on your local computer.
+To create your user account (as an administrator of the server) via this Ansible playbook, run the command below on your local computer.
 
 **Notes**:
-- Before running it, make sure to edit `YOUR_USERNAME_HERE` and `YOUR_PASSWORD_HERE`
-- In the command below, `YOUR_USERNAME_HERE` is just a plain username (like `john`), not your full `@user:example.com` identifier
+- Make sure to adjust `YOUR_USERNAME_HERE` and `YOUR_PASSWORD_HERE`
+- For `YOUR_USERNAME_HERE`, use a plain username like `john`, not your full identifier (`@user:example.com`)
+- Use `admin=yes` to make your user account an administrator of the Matrix server
 
 ```sh
-ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=YOUR_USERNAME_HERE password=YOUR_PASSWORD_HERE admin=<yes|no>' --tags=register-user
+ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=YOUR_USERNAME_HERE password=YOUR_PASSWORD_HERE admin=yes' --tags=register-user
 
-# Example: `ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=john password=secret-password admin=yes' --tags=register-user`
+# Example: ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=john password=secret-password admin=yes' --tags=register-user
 ```
 
+Feel free to create as many accounts (for friends, family, etc.) as you want. Still, perhaps you should grant full administrative access to your account only (with `admin=yes`), and others should be created with `admin=no`.
+
 For more information, see the documentation for [registering users](registering-users.md).
 
 ## Finalize the installation
@@ -94,20 +102,21 @@ Now you've configured Matrix services and your user account, you need to **final
 
 This is required for federation to work! Without a proper configuration, your server will effectively not be part of the Matrix network.
 
-If you need the base domain for anything else such as hosting a website, you have to configure it manually, following the procedure described on the linked documentation.
+To configure the delegation, you have these two options. Choose one of them according to your situation.
 
-However, if you do not need the base domain for anything else, the easiest way of configuring it is to [serve the base domain](configuring-playbook-base-domain-serving.md) from the integrated web server. It will enable you to use a Matrix user identifier like `@<username>:example.com` while hosting services on a subdomain like `matrix.example.com`.
+- If you can afford to point the base domain at the Matrix server, follow the instructions below which guide you into [serving the base domain](configuring-playbook-base-domain-serving.md) from the integrated web server. It will enable you to use a Matrix user identifier like `@<username>:example.com` while hosting services on a subdomain like `matrix.example.com`.
+- Alternatively, if you're using the base domain for other purposes and cannot point it to the Matrix server (and thus cannot "serve the base domain" from it), you most likely need to [manually install well-known files on the base domain's server](configuring-well-known.md#manually-installing-well-known-files-on-the-base-domains-server), but feel free to familiarize yourself with all [server delegation (redirection) options](howto-server-delegation.md).
 
-To configure server delegation in this way, add the following configuration to your `inventory/host_vars/matrix.example.com/vars.yml` file:
+To have the base domain served from the integrated web server, add the following configuration to your `inventory/host_vars/matrix.example.com/vars.yml` file:
 
 ```yaml
 matrix_static_files_container_labels_base_domain_enabled: true
 ```
 
-After configuring the playbook, run the installation command:
+After configuring the playbook, run the command below:
 
 ```sh
-ansible-playbook -i inventory/hosts setup.yml --tags=install-all,start
+ansible-playbook -i inventory/hosts setup.yml --tags=install-matrix-static-files,start
 ```
 
 ## Things to do next
@@ -125,8 +134,12 @@ After finilizing the installation, you can:
 
 ### Maintaining your setup in the future
 
-Feel free to **re-run the setup command any time** you think something is off with the server configuration. Ansible will take your configuration and update your server to match. To update the playbook and the Ansible roles in the playbook, simply run `just roles`.
+While this playbook helps you to set up Matrix services and maintain them, it will **not** automatically run the maintenance task for you. You will need to update the playbook and re-run it **manually**.
 
-Note that if you remove components from `vars.yml`, or if we switch some component from being installed by default to not being installed by default anymore, you'd need to run the setup command with `--tags=setup-all` instead of `--tags=install-all`. See [this page on the playbook tags](playbook-tags.md) for more information.
+The upstream projects, which this playbook makes use of, occasionally if not often suffer from security vulnerabilities.
 
-A way to invoke these `ansible-playbook` commands with less typing in the future is to use [just](https://github.com/casey/just) to run the "recipe": `just install-all` or `just setup-all`. See [our `justfile`](../justfile) for more information.
+Since it is unsafe to keep outdated services running on the server connected to the internet, please consider to update the playbook and re-run it periodically, in order to keep the services up-to-date.
+
+For more information about upgrading or maintaining services with the playbook, take at look at this page: [Upgrading the Matrix services](maintenance-upgrading-services.md)
+
+Feel free to **re-run the setup command any time** you think something is off with the server configuration. Ansible will take your configuration and update your server to match.

docs/just.md

@@ -0,0 +1,38 @@
+# Running `just` commands
+
+We have previously used [make](https://www.gnu.org/software/make/) for easily running some playbook commands (e.g. `make roles` which triggers [`ansible-galaxy`](https://docs.ansible.com/ansible/latest/cli/ansible-galaxy.html)). Our [`Makefile`](../Makefile) is still around, and you can still run these commands.
+
+In addition, we have added support for running commands via [`just`](https://github.com/casey/just) - a more modern command-runner alternative to `make`. It can be used to invoke `ansible-playbook` commands with less typing.
+
+The `just` utility executes shortcut commands (called as "recipes"), which invoke `ansible-playbook`, `ansible-galaxy` or [`agru`](https://github.com/etkecc/agru) (depending on what is available in your system). The targets of the recipes are defined in [`justfile`](../justfile). Most of the just recipes have no corresponding `Makefile` targets.
+
+For some recipes such as `just update`, our `justfile` recommends installing [`agru`](https://github.com/etkecc/agru) (a faster alternative to `ansible-galaxy`) to speed up the process.
+
+Here are some examples of shortcuts:
+
+| Shortcut                                      | Result                                                                                                         |
+|-----------------------------------------------|----------------------------------------------------------------------------------------------------------------|
+| `just roles`                                  | Install the necessary Ansible roles pinned in [`requirements.yml`](../requirements.yml)                        |
+| `just update`                                 | Run `git pull` (to update the playbook) and install the Ansible roles                                          |
+| `just install-all`                            | Run `ansible-playbook -i inventory/hosts setup.yml --tags=install-all,ensure-matrix-users-created,start`       |
+| `just setup-all`                              | Run `ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start`         |
+| `just install-all --ask-vault-pass`           | Run commands with additional arguments (`--ask-vault-pass` will be appended to the above installation command) |
+| `just run-tags install-mautrix-slack,start`   | Run specific playbook tags (here `install-mautrix-slack` and `start`)                                          |
+| `just install-service mautrix-slack`          | Run `just run-tags install-mautrix-slack,start` with even less typing                                          |
+| `just start-all`                              | (Re-)starts all services                                                                                       |
+| `just stop-group postgres`                    | Stop only the Postgres service                                                                                 |
+| `just register-user john secret-password yes` | Registers a `john` user with the `secret-password` password and admin access (admin = `yes`)                   |
+
+While [our documentation on prerequisites](prerequisites.md) lists `just` as one of the requirements for installation, using `just` is optional. If you find it difficult to install it, do not find it useful, or want to prefer raw `ansible-playbook` commands for some reason, feel free to run all commands manually. For example, you can run `ansible-galaxy` directly to install the Ansible roles: `rm -rf roles/galaxy; ansible-galaxy install -r requirements.yml -p roles/galaxy/ --force`.
+
+## Difference between playbook tags and shortcuts
+
+It is worth noting that `just` "recipes" are different from [playbook tags](playbook-tags.md). The recipes are shortcuts of commands defined in `justfile` and can be executed by the `just` program only, while the playbook tags are available for the raw `ansible-playbook` commands as well. Please be careful not to confuse them.
+
+For example, these two commands are different:
+- `just install-all`
+- `ansible-playbook -i inventory/hosts setup.yml --tags=install-all`
+
+The just recipe runs `ensure-matrix-users-created` and `start` tags after `install-all`, while the latter runs only `install-all` tag. The correct shortcut of the latter is `just run-tags install-all`.
+
+Such kind of difference sometimes matters. For example, when you install a Matrix server into which you will import old data (see [here](installing.md#installing-a-server-into-which-youll-import-old-data)), you are not supposed to run `just install-all` or `just setup-all`, because these commands start services immediately after installing components which may prevent your from importing old data.

docs/maintenance-checking-services.md

@@ -10,4 +10,4 @@ ansible-playbook -i inventory/hosts setup.yml --tags=self-check
 
 If it's all green, everything is probably running correctly.
 
-Besides this self-check, you can also check your server using the [Federation Tester](https://federationtester.matrix.org/).
+Besides this self-check, you can also check whether your server federates with the Matrix network by using the [Federation Tester](https://federationtester.matrix.org/) against your base domain (`example.com`), not the `matrix.example.com` subdomain.

docs/maintenance-upgrading-services.md

@@ -2,17 +2,39 @@
 
 This playbook not only installs the various Matrix services for you, but can also upgrade them as new versions are made available.
 
+While this playbook helps you to set up Matrix services and maintain them, it will **not** automatically run the maintenance task for you. You will need to update the playbook and re-run it **manually**.
+
+The upstream projects, which this playbook makes use of, occasionally if not often suffer from security vulnerabilities (for example, see [here](https://github.com/element-hq/element-web/security) for known ones on Element Web).
+
+Since it is unsafe to keep outdated services running on the server connected to the internet, please consider to update the playbook and re-run it periodically, in order to keep the services up-to-date.
+
+The developers of this playbook strive to maintain the playbook updated, so that you can re-run the playbook to address such vulnerabilities. It is **your responsibility** to keep your server and the services on it up-to-date.
+
 If you want to be notified when new versions of Synapse are released, you should join the Synapse Homeowners room: [#homeowners:matrix.org](https://matrix.to/#/#homeowners:matrix.org).
 
-To upgrade services:
+## Steps to upgrade the Matrix services
+
+Before updating the playbook and the Ansible roles in the playbook, take a look at [the changelog](../CHANGELOG.md) to see if there have been any backward-incompatible changes that you need to take care of.
+
+If it looks good to you, go to the `matrix-docker-ansible-deploy` directory, then:
 
 - update your playbook directory and all upstream Ansible roles (defined in the `requirements.yml` file) using:
 
   - either: `just update`
-  - or: a combination of `git pull` and `just roles` (or `make roles`)
+  - or: a combination of `git pull` and `just roles` (or `make roles` if you have `make` program on your computer instead of `just`)
 
-- take a look at [the changelog](../CHANGELOG.md) to see if there have been any backward-incompatible changes that you need to take care of
+  If you don't have either `just` tool or `make` program, you can run the `ansible-galaxy` tool directly: `rm -rf roles/galaxy; ansible-galaxy install -r requirements.yml -p roles/galaxy/ --force`
 
-- re-run the [playbook setup](installing.md) and restart all services: `just install-all` or `just setup-all`
+  For details about `just` commands, take a look at: [Running `just` commands](just.md).
+
+- re-run the [playbook setup](installing.md#maintaining-your-setup-in-the-future) and restart all services:
+
+  ```sh
+  ansible-playbook -i inventory/hosts setup.yml --tags=install-all,start
+  ```
+
+Note that if you remove components from `vars.yml`, or if we switch some component from being installed by default to not being installed by default anymore, you'd need to run the setup command with `--tags=setup-all` instead of `--tags=install-all`. See [this page on the playbook tags](playbook-tags.md) for more information.
+
+A way to invoke these `ansible-playbook` commands with less typing is to run the `just` "recipe": `just install-all` or `just setup-all`.
 
 **Note**: major version upgrades to the internal PostgreSQL database are not done automatically. To upgrade it, refer to the [upgrading PostgreSQL guide](maintenance-postgres.md#upgrading-postgresql).

docs/playbook-tags.md

@@ -20,4 +20,6 @@ Here are some playbook tags that you should be familiar with:
 
 - `ensure-matrix-users-created` - a special tag which ensures that all special users needed by the playbook (for bots, etc.) are created
 
-`setup-*` tags and `install-*` tags **do not start services** automatically, because you may wish to do things before starting services, such as importing a database dump, restoring data from another server, etc.
+**Notes**:
+- `setup-*` tags and `install-*` tags **do not start services** automatically, because you may wish to do things before starting services, such as importing a database dump, restoring data from another server, etc.
+- Please be careful not to confuse the playbook tags with the `just` shortcut commands ("recipes"). For details about `just` commands, see: [Running `just` commands](just.md)

docs/prerequisites.md

@@ -1,9 +1,11 @@
 # Prerequisites
 
-<sup>⚡️[Quick start](README.md) | Prerequisites > [Configuring your DNS settings](configuring-dns.md) > [Getting the playbook](getting-the-playbook.md) > [Configuring the playbook](configuring-playbook.md) > [Installing](installing.md)</sup>
+<sup>⚡️[Quick start](quick-start.md) | Prerequisites > [Configuring your DNS settings](configuring-dns.md) > [Getting the playbook](getting-the-playbook.md) > [Configuring the playbook](configuring-playbook.md) > [Installing](installing.md)</sup>
 
 To install Matrix services using this Ansible playbook, you need to prepare several requirements both on your local computer (where you will run the playbook to configure the server) and the server (where the playbook will install the Matrix services for you). **These requirements need to be set up manually** before proceeding to the next step.
 
+We will be using `example.com` as the domain in the following instruction. Please remember to replace it with your own domain before running any commands.
+
 ## Your local computer
 
 - [Ansible](http://ansible.com/) program. It's used to run this playbook and configures your server for you. Take a look at [our guide about Ansible](ansible.md) for more information, as well as [version requirements](ansible.md#supported-ansible-versions) and alternative ways to run Ansible.
@@ -12,7 +14,7 @@ To install Matrix services using this Ansible playbook, you need to prepare seve
 
 - [`git`](https://git-scm.com/) as the recommended way to download the playbook. `git` may also be required on the server if you will be [self-building](self-building.md) components.
 
-- [`just`](https://github.com/casey/just) for running `just roles`, `just update`, etc. (see [`justfile`](../justfile)), although you can also run these commands manually
+- [`just`](https://github.com/casey/just) for running `just roles`, `just update`, etc. (see [`justfile`](../justfile)), although you can also run these commands manually. Take at look at this documentation for more information: [Running `just` commands](just.md).
 
 - Strong password (random strings) generator. The playbook often requires you to create a strong password and use it for settings on `vars.yml`, components, etc. As any tools should be fine, this playbook has adopted [`pwgen`](https://linux.die.net/man/1/pwgen) (running `pwgen -s 64 1`). [Password Tech](https://pwgen-win.sourceforge.io/), formerly known as "PWGen for Windows", is available as free and open source password generator for Windows. Generally, using a random generator available on the internet is not recommended.
 

docs/quick-start.md

@@ -0,0 +1,202 @@
+# Quick start
+
+<!--
+NOTE:
+- Let's keep it as tidy and simple as possible.
+- Because this documentation is intended to be referred by those who have not configured a Matrix server and services by using the playbook, from the educational point of view it intentionally avoids instructions based on just program's "recipes" in favor of ansible-playbook commands in most cases.
+-->
+
+This page explains how to use this Ansible playbook to install Matrix services on your server with a minimal set of core services.
+
+We will be using `example.com` as the "base domain" in the following instruction.
+
+By following the instruction on this page, you will set up:
+
+- **your own Matrix server** on a `matrix.example.com` server, which is configured to present itself as `example.com`
+- **your user account** like `@user:example.com` on the server
+- a **self-hosted Matrix client**, [Element Web](configuring-playbook-client-element-web.md) with the default subdomain at `element.example.com`
+- Matrix delegation, so that your `matrix.example.com` server (presenting itself as `example.com`) can join the Matrix Federation and communicate with any other server in the Matrix network
+
+Please remember to replace `example.com` with your own domain before running any commands.
+
+## Prerequisites
+
+<sup>This section is optimized for this quick-start guide and is derived from the following full-documentation page: [Prerequisites](prerequisites.md)</sup>
+
+At first, **check prerequisites** and prepare for installation by setting up programs [on your own computer](prerequisites.md#your-local-computer) and [your server](prerequisites.md#server). You also need `root` access on your server (a user that could elevate to `root` via `sudo` also works).
+
+<!--
+TODO: Add one liners (or instructions, a script, etc.) for easy and consistent installation of required software. See: https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3757
+-->
+
+If you encounter an error during installation, please make sure that you have installed and configured programs correctly.
+
+One of the main reasons of basic errors is using an incompatible version of required software such as Ansible. Take a look at [our guide about Ansible](ansible.md) for more information. In short: installing the latest available version is recommended.
+
+## Configure your DNS settings
+
+<sup>This section is optimized for this quick-start guide and is derived from the following full-documentation page: [Configuring your DNS settings](configuring-dns.md)</sup>
+
+After installing and configuring prerequisites, you will need to **configure DNS records**.
+
+To configure Matrix services in the default settings, go to your DNS service provider, and adjust DNS records as below.
+
+| Type  | Host                         | Priority | Weight | Port | Target               |
+| ----- | ---------------------------- | -------- | ------ | ---- | ---------------------|
+| A     | `matrix`                     | -        | -      | -    | `matrix-server-IP`   |
+| CNAME | `element`                    | -        | -      | -    | `matrix.example.com` |
+
+As the table illustrates, you need to create 2 subdomains (`matrix.example.com` and `element.example.com`) and point both of them to your server's IP address (DNS `A` record or `CNAME` record is fine).
+
+It might take some time for the DNS records to propagate after creation.
+
+**💡 Note**: if you are using Cloudflare DNS, make sure to disable the proxy and set all records to "DNS only"
+
+## Get the playbook
+
+<sup>This section is optimized for this quick-start guide and is derived from the following full-documentation page: [Getting the playbook](getting-the-playbook.md)</sup>
+
+Next, let's **get the playbook's source code**.
+
+We recommend to do so with [git](https://git-scm.com/) as it enables you to keep it up to date with the latest source code. While it is possible to download the playbook as a ZIP archive, it is not recommended.
+
+To get the playbook with git, install git on your computer, go to a directory, and run the command:
+
+```sh
+git clone https://github.com/spantaleev/matrix-docker-ansible-deploy.git
+```
+
+It will fetch the playbook to a new `matrix-docker-ansible-deploy` directory underneath the directory you are currently in.
+
+## Configure the playbook
+
+<sup>This section is optimized for this quick-start guide and is derived from the following full-documentation page: [Configuring the playbook](configuring-playbook.md)</sup>
+
+Now that the playbook was fetched, it is time to **configure** it per your needs.
+
+To install Matrix services with this playbook, you would at least need 2 configuration files.
+
+For your convenience, we have prepared example files of them ([`vars.yml`](../examples/vars.yml) and [`hosts`](../examples/hosts)).
+
+To start quickly based on these example files, go into the `matrix-docker-ansible-deploy` directory and follow the instructions below:
+
+1. Create a directory to hold your configuration: `mkdir -p inventory/host_vars/matrix.example.com` where `example.com` is your "base domain"
+2. Copy the sample configuration file: `cp examples/vars.yml inventory/host_vars/matrix.example.com/vars.yml`
+3. Copy the sample inventory hosts file: `cp examples/hosts inventory/hosts`
+4. Edit the configuration file (`inventory/host_vars/matrix.example.com/vars.yml`)
+5. Edit the inventory hosts file (`inventory/hosts`)
+
+Before editing these 2 files, make sure to read explanations on them to understand what needs to be configured.
+
+**💡 Notes:**
+- If you are not in control of anything on the base domain, you would need to set additional configuration on `vars.yml`. For more information, see [How do I install on matrix.example.com without involving the base domain?](faq.md#how-do-i-install-on-matrix-example-com-without-involving-the-base-domain) on our FAQ.
+- Certain configuration decisions (like the base domain configured in `matrix_domain` and homeserver implementation configured in `matrix_homeserver_implementation`) are final. If you make the wrong choice and wish to change it, you'll have to run the Uninstalling step and start over.
+- Instead of configuring a lot of things all at once, we recommend starting with the basic (default) settings in order to get yourself familiar with how the playbook works. After making sure that everything works as expected, you can add (and remove) advanced settings / features and run the playbook as many times as you wish.
+
+## Install
+
+<sup>This section is optimized for this quick-start guide and is derived from the following full-documentation page: [Installing](installing.md)</sup>
+
+After editing `vars.yml` and `hosts` files, let's start the **installation** procedure.
+
+### Update Ansible roles
+
+Before installing, you need to update the Ansible roles that this playbook uses and fetches from outside.
+
+To update your playbook directory and all upstream Ansible roles, run:
+
+- either: `just update`
+- or: a combination of `git pull` and `just roles` (or `make roles` if you have `make` program on your computer instead of `just`)
+
+If you don't have either `just` tool or `make` program, you can run the `ansible-galaxy` tool directly: `rm -rf roles/galaxy; ansible-galaxy install -r requirements.yml -p roles/galaxy/ --force`
+
+### Run installation command
+
+Then, run the command below to start installation:
+
+````sh
+ansible-playbook -i inventory/hosts setup.yml --tags=install-all,ensure-matrix-users-created,start
+````
+
+If you **don't** use SSH keys for authentication, but rather a regular password, you may need to add `--ask-pass` to the command.
+
+If you **do** use SSH keys for authentication, **and** use a non-root user to *become* root (sudo), you may need to add `-K` (`--ask-become-pass`) to the command.
+
+Wait until the command completes. If it's all green, everything should be running properly.
+
+## Create your user account
+
+<sup>This section is optimized for this quick-start guide and is derived from the following full-documentation page: [Registering users](registering-users.md)</sup>
+
+As you have configured your brand new server and the client, you need to **create your user account** on your Matrix server.
+
+To create your user account (as an administrator of the server) via this Ansible playbook, run the command below on your local computer.
+
+**💡 Notes**:
+- Make sure to adjust `YOUR_USERNAME_HERE` and `YOUR_PASSWORD_HERE`
+- For `YOUR_USERNAME_HERE`, use a plain username like `john`, not your full identifier (`@user:example.com`)
+
+```sh
+ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=YOUR_USERNAME_HERE password=YOUR_PASSWORD_HERE admin=yes' --tags=register-user
+
+# Example: ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=john password=secret-password admin=yes' --tags=register-user
+```
+
+<!--
+NOTE: detailed instruction to add users can be found on docs/registering-users.md and installing.md, which include a note about usage of admin=yes and admin=no variables. In order to keep this guide as reasonably short as possible, let's not repeat the same instruction here.
+-->
+
+## Finalize server installation
+
+<sup>This section is optimized for this quick-start guide and is derived from the following full-documentation page: [Server Delegation](howto-server-delegation.md)</sup>
+
+Now that you've configured Matrix services and your user account, you need to **finalize the installation process** by [setting up Matrix delegation (redirection)](howto-server-delegation.md), so that your Matrix server (`matrix.example.com`) can present itself as the base domain (`example.com`) in the Matrix network.
+
+**This is required for federation to work!** Without a proper configuration, your server will effectively not be part of the Matrix network.
+
+To configure the delegation, you have these two options. Choose one of them according to your situation.
+
+- If you can afford to point the base domain at the Matrix server, follow the instruction below which guides you into [serving the base domain](configuring-playbook-base-domain-serving.md) from the integrated web server.
+- Alternatively, if you're using the base domain for other purposes and cannot point it to the Matrix server (and thus cannot "serve the base domain" from it), you most likely need to [manually install well-known files on the base domain's server](configuring-well-known.md#manually-installing-well-known-files-on-the-base-domains-server).
+
+To have the base domain served from the integrated web server, add the following configuration to your `inventory/host_vars/matrix.example.com/vars.yml` file:
+
+```yaml
+matrix_static_files_container_labels_base_domain_enabled: true
+```
+
+After configuring the playbook, run the command below and wait until it finishes:
+
+```sh
+ansible-playbook -i inventory/hosts setup.yml --tags=install-matrix-static-files,start
+```
+
+💡 Running the `install-matrix-static-files` playbook tag (as done here) is an optimized version of running [the full setup command](#run-the-installation-command).
+
+After the command finishes, you can also check whether your server federates with the Matrix network by using the [Federation Tester](https://federationtester.matrix.org/) against your base domain (`example.com`), not the `matrix.example.com` subdomain.
+
+If you think something is off with the server configuration, feel free to [re-run the full setup command](#run-the-installation-command) any time.
+
+## Log in to your user account
+
+Finally, let's make sure that you can log in to the created account with the specified password.
+
+You should be able to log in to it with your own [Element Web](configuring-playbook-client-element-web.md) client which you have set up at `element.example.com` by running the playbook. Open the URL (`https://element.example.com`) in a web browser and enter your credentials to log in.
+
+**If you successfully logged in to your account, installing and configuring is complete**🎉
+
+Come say Hi👋 in our support room - [#matrix-docker-ansible-deploy:devture.com](https://matrix.to/#/#matrix-docker-ansible-deploy:devture.com). You might learn something or get to help someone else new to Matrix hosting.
+
+## Things to do next
+
+Once you get familiar with the playbook, you might probably want to set up additional services such as a bridge on your server.
+
+As this page intends to be a quick start guide which explains how to start the core Matrix services, it does not cover a topic like how to set them up. Take a look at the list of [things to do next](installing.md#things-to-do-next) to learn more.
+
+### ⚠️Keep the playbook and services up-to-date
+
+While this playbook helps you to set up Matrix services and maintain them, it will **not** automatically run the maintenance task for you. You will need to update the playbook and re-run it **manually**.
+
+Since it is unsafe to keep outdated services running on the server connected to the internet, please consider to update the playbook and re-run it periodically, in order to keep the services up-to-date.
+
+For more information about upgrading or maintaining services with the playbook, take at look at this page: [Upgrading the Matrix services](maintenance-upgrading-services.md)

docs/registering-users.md

@@ -1,6 +1,6 @@
 # Registering users
 
-This documentation page tells you how to create user account on your Matrix server.
+This documentation page tells you how to create user accounts on your Matrix server.
 
 Table of contents:
 
@@ -14,7 +14,10 @@ Table of contents:
 
 ## Registering users manually
 
-**Note**: in the commands below, `<your-username>` is just a plain username (like `john`), not your full `@<username>:example.com` identifier.
+**Notes**:
+- Make sure to adjust `USERNAME_HERE` and `PASSWORD_HERE`
+- For `USERNAME_HERE`, use a plain username like `john`, not a full identifier (`@user:example.com`)
+- Use `admin=yes` or `admin=no` depending on whether you wish to make the user an administrator of the Matrix server
 
 After registering a user (using one of the methods below), **you can log in with that user** via the [Element Web](configuring-playbook-client-element-web.md) service that this playbook has installed for you at a URL like this: `https://element.example.com/`.
 
@@ -22,10 +25,10 @@ After registering a user (using one of the methods below), **you can log in with
 
 It's best to register users via the Ansible playbook, because it works regardless of homeserver implementation (Synapse, Dendrite, etc) or usage of [Matrix Authentication Service](configuring-playbook-matrix-authentication-service.md) (MAS).
 
-To register a user via this Ansible playbook (make sure to edit the `<your-username>` and `<your-password>` part below):
+To register a user via this Ansible playbook:
 
 ```sh
-just register-user <your-username> <your-password> <admin access: yes or no>
+just register-user USERNAME_HERE PASSWORD_HERE <admin access: yes or no>
 
 # Example: `just register-user john secret-password yes`
 ```
@@ -33,39 +36,41 @@ just register-user <your-username> <your-password> <admin access: yes or no>
 **or** by invoking `ansible-playbook` manually:
 
 ```sh
-ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=<your-username> password=<your-password> admin=<yes|no>' --tags=register-user
+ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=USERNAME_HERE password=PASSWORD_HERE admin=<yes|no>' --tags=register-user
 
-# Example: `ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=john password=secret-password admin=yes' --tags=register-user`
+# Example: ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=john password=secret-password admin=yes' --tags=register-user
 ```
 
+Feel free to register as many users (for friends, family, etc.) as you want. Still, perhaps you should grant full administrative access to your user account only (with `admin=yes`), and others should be created with `admin=no`.
+
 ⚠️ **Warning**: If you're registering users against Matrix Authentication Service, do note that it [still insists](https://github.com/element-hq/matrix-authentication-service/issues/1505) on having a verified email address for each user. Upon a user's first login, they will be asked to confirm their email address. This requires that email sending is [configured](./configuring-playbook-email.md). You can also consult the [Working around email deliverability issues](./configuring-playbook-matrix-authentication-service.md#working-around-email-deliverability-issues) section for more information.
 
 ### Registering users manually for Synapse
 
-If you're using the [Synapse](configuring-playbook-synapse.md) homeserver implementation (which is the default), you can register users via the command-line after **SSH**-ing to your server (requires that [all services have been started](#starting-the-services)):
+If you're using the [Synapse](configuring-playbook-synapse.md) homeserver implementation (which is the default), you can register users via the command-line after **SSH**-ing to your server (requires that [all services have been started](installing.md#install-matrix-server-and-services)):
 
 ```sh
-/matrix/synapse/bin/register-user <your-username> <your-password> <admin access: 0 or 1>
+/matrix/synapse/bin/register-user USERNAME_HERE PASSWORD_HERE <admin access: 0 or 1>
 
 # Example: `/matrix/synapse/bin/register-user john secret-password 1`
 ```
 
 ### Registering users manually for Dendrite
 
-If you're using the [Dendrite](./configuring-playbook-dendrite.md) homeserver implementation, you can register users via the command-line after **SSH**-ing to your server (requires that [all services have been started](#starting-the-services)):
+If you're using the [Dendrite](./configuring-playbook-dendrite.md) homeserver implementation, you can register users via the command-line after **SSH**-ing to your server (requires that [all services have been started](installing.md#install-matrix-server-and-services)):
 
 ```sh
-/matrix/dendrite/bin/create-account <your-username> <your-password> <admin access: 0 or 1>
+/matrix/dendrite/bin/create-account USERNAME_HERE PASSWORD_HERE <admin access: 0 or 1>
 
 # Example: `/matrix/dendrite/bin/create-account john secret-password 1`
 ```
 
 ### Registering users manually for Matrix Authentication Service
 
-If you're using the [Matrix Authentication Service](./configuring-playbook-matrix-authentication-service.md) and your existing homeserver (most likely [Synapse](./configuring-playbook-synapse.md)) is delegating authentication to it, you can register users via the command-line after **SSH**-ing to your server (requires that [all services have been started](#starting-the-services)):
+If you're using the [Matrix Authentication Service](./configuring-playbook-matrix-authentication-service.md) and your existing homeserver (most likely [Synapse](./configuring-playbook-synapse.md)) is delegating authentication to it, you can register users via the command-line after **SSH**-ing to your server (requires that [all services have been started](installing.md#install-matrix-server-and-services)):
 
 ```sh
-/matrix/matrix-authentication-service/bin/register-user <your-username> <your-password> <admin access: 0 or 1>
+/matrix/matrix-authentication-service/bin/register-user USERNAME_HERE PASSWORD_HERE <admin access: 0 or 1>
 
 # Example: `/matrix/matrix-authentication-service/bin/register-user john secret-password 1`
 ```

docs/updating-users-passwords.md

@@ -2,14 +2,16 @@
 
 ## Option 1 (if you are using the integrated Postgres database):
 
-You can reset a user's password via the Ansible playbook (make sure to edit the `<your-username>` and `<your-password>` part below):
+**Notes**:
+- Make sure to adjust `USERNAME_HERE` and `PASSWORD_HERE`
+- For `USERNAME_HERE`, use a plain username like `john`, not a full identifier (`@user:example.com`)
+
+You can reset a user's password via the Ansible playbook:
 
 ```
-ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=<your-username> password=<your-password>' --tags=update-user-password
+ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=USERNAME_HERE password=PASSWORD_HERE' --tags=update-user-password
 ```
 
-**Note**: `<your-username>` is just a plain username (like `john`), not your full `@<username>:example.com` identifier.
-
 **You can then log in with that user** via Element Web that this playbook has created for you at a URL like this: `https://element.example.com/`.
 
 

group_vars/matrix_servers

@@ -436,6 +436,8 @@ devture_systemd_service_manager_services_list_auto: |
     +
     ([{'name': (keydb_identifier + '.service'), 'priority': 750, 'groups': ['matrix', 'keydb']}] if keydb_enabled else [])
     +
+    ([{'name': (valkey_identifier + '.service'), 'priority': 750, 'groups': ['matrix', 'valkey']}] if valkey_enabled else [])
+    +
     ([{'name': 'matrix-pantalaimon.service', 'priority': 4000, 'groups': ['matrix', 'pantalaimon']}] if matrix_pantalaimon_enabled else [])
     +
     ([{'name': 'matrix-registration.service', 'priority': 4000, 'groups': ['matrix', 'registration', 'matrix-registration']}] if matrix_registration_enabled else [])
@@ -2203,12 +2205,14 @@ matrix_hookshot_systemd_wanted_services_list: |
     ([(redis_identifier + '.service')] if redis_enabled and matrix_hookshot_cache_redis_host == redis_identifier else [])
     +
     ([(keydb_identifier + '.service')] if keydb_enabled and matrix_hookshot_cache_redis_host == keydb_identifier else [])
+    +
+    ([(valkey_identifier + '.service')] if valkey_enabled and matrix_hookshot_cache_redis_host == valkey_identifier else [])
   }}
 
 # Hookshot's experimental encryption feature (and possibly others) may benefit from Redis, if available.
 # We only connect to Redis if encryption is enabled (not for everyone who has Redis enabled),
 # because connectivity is still potentially troublesome and is to be investigated.
-matrix_hookshot_cache_redis_host: "{{ redis_identifier if redis_enabled and matrix_hookshot_experimental_encryption_enabled else (keydb_identifier if keydb_enabled and matrix_hookshot_experimental_encryption_enabled else '') }}"
+matrix_hookshot_cache_redis_host: "{{ valkey_identifier if valkey_enabled else (redis_identifier if redis_enabled else (keydb_identifier if keydb_enabled else '')) }}"
 
 matrix_hookshot_container_network: "{{ matrix_addons_container_network }}"
 
@@ -2221,6 +2225,8 @@ matrix_hookshot_container_additional_networks_auto: |
       +
       ([keydb_container_network] if keydb_enabled and matrix_hookshot_cache_redis_host == keydb_identifier else [])
       +
+      ([valkey_container_network] if valkey_enabled and matrix_hookshot_cache_redis_host == valkey_identifier else [])
+      +
       ([matrix_playbook_reverse_proxyable_services_additional_network] if matrix_playbook_reverse_proxyable_services_additional_network and matrix_hookshot_container_labels_traefik_enabled else [])
     ) | unique
   }}
@@ -4393,11 +4399,11 @@ ntfy_visitor_request_limit_exempt_hosts_hostnames_auto: |
 
 ######################################################################
 #
-# etke/redis
+# redis
 #
 ######################################################################
 
-redis_enabled: "{{ not keydb_enabled and (matrix_synapse_workers_enabled or (matrix_hookshot_enabled and matrix_hookshot_experimental_encryption_enabled)) }}"
+redis_enabled: "{{ not (keydb_enabled or valkey_enabled) and (matrix_synapse_workers_enabled or (matrix_hookshot_enabled and matrix_hookshot_experimental_encryption_enabled)) }}"
 
 redis_identifier: matrix-redis
 
@@ -4408,7 +4414,7 @@ redis_base_path: "{{ matrix_base_data_path }}/redis"
 
 ######################################################################
 #
-# /etke/redis
+# /redis
 #
 ######################################################################
 
@@ -4418,7 +4424,7 @@ redis_base_path: "{{ matrix_base_data_path }}/redis"
 #
 ######################################################################
 
-keydb_enabled: "{{ matrix_synapse_workers_enabled or (matrix_hookshot_enabled and matrix_hookshot_experimental_encryption_enabled) }}"
+keydb_enabled: false
 
 keydb_identifier: matrix-keydb
 
@@ -4442,6 +4448,31 @@ keydb_arch: |-
 #
 ######################################################################
 
+
+######################################################################
+#
+# valkey
+#
+######################################################################
+
+valkey_enabled: "{{ matrix_synapse_workers_enabled or (matrix_hookshot_enabled and matrix_hookshot_experimental_encryption_enabled) }}"
+
+valkey_identifier: matrix-valkey
+
+valkey_uid: "{{ matrix_user_uid }}"
+valkey_gid: "{{ matrix_user_gid }}"
+
+valkey_base_path: "{{ matrix_base_data_path }}/valkey"
+
+valkey_arch: "{{ matrix_architecture }}"
+
+######################################################################
+#
+# valkey
+#
+######################################################################
+
+
 ######################################################################
 #
 # matrix-client-element
@@ -4664,6 +4695,8 @@ matrix_synapse_container_additional_networks_auto: |
       +
       ([keydb_container_network] if matrix_synapse_redis_enabled and matrix_synapse_redis_host == keydb_identifier else [])
       +
+      ([valkey_container_network] if matrix_synapse_redis_enabled and matrix_synapse_redis_host == valkey_identifier else [])
+      +
       ([exim_relay_container_network] if (exim_relay_enabled and matrix_synapse_email_enabled and matrix_synapse_email_smtp_host == exim_relay_identifier and matrix_synapse_container_network != exim_relay_container_network) else [])
       +
       ([matrix_ma1sd_container_network] if (matrix_ma1sd_enabled and matrix_synapse_account_threepid_delegates_msisdn == matrix_synapse_account_threepid_delegates_msisdn_mas1sd_url and matrix_synapse_container_network != matrix_ma1sd_container_network) else [])
@@ -4751,6 +4784,8 @@ matrix_synapse_systemd_required_services_list_auto: |
     +
     ([keydb_identifier ~ '.service'] if matrix_synapse_redis_enabled and matrix_synapse_redis_host == keydb_identifier else [])
     +
+    ([valkey_identifier ~ '.service'] if matrix_synapse_redis_enabled and matrix_synapse_redis_host == valkey_identifier else [])
+    +
     (['matrix-goofys.service'] if matrix_s3_media_store_enabled else [])
     +
     (['matrix-authentication-service.service'] if (matrix_authentication_service_enabled and matrix_synapse_experimental_features_msc3861_enabled) else [])
@@ -4764,9 +4799,9 @@ matrix_synapse_systemd_wanted_services_list_auto: |
   }}
 
 # Synapse workers (used for parallel load-scaling) need Redis for IPC.
-matrix_synapse_redis_enabled: "{{ redis_enabled or keydb_enabled }}"
-matrix_synapse_redis_host: "{{ redis_identifier if redis_enabled else (keydb_identifier if keydb_enabled else '') }}"
-matrix_synapse_redis_password: "{{ redis_connection_password if redis_enabled else (keydb_connection_password if keydb_enabled else '') }}"
+matrix_synapse_redis_enabled: "{{ redis_enabled or keydb_enabled or valkey_enabled }}"
+matrix_synapse_redis_host: "{{ valkey_identifier if valkey_enabled else (redis_identifier if redis_enabled else (keydb_identifier if keydb_enabled else '')) }}"
+matrix_synapse_redis_password: "{{ valkey_connection_password if valkey_enabled else (redis_connection_password if redis_enabled else (keydb_connection_password if keydb_enabled else '')) }}"
 
 matrix_synapse_container_extra_arguments_auto: "{{ matrix_homeserver_container_extra_arguments_auto }}"
 matrix_synapse_app_service_config_files_auto: "{{ matrix_homeserver_app_service_config_files_auto }}"
@@ -5327,6 +5362,8 @@ prometheus_config_scrape_configs_auto: |
     (matrix_prometheus_services_connect_scraper_media_repo_scrape_configs if matrix_prometheus_services_connect_scraper_media_repo_enabled else [])
     +
     (matrix_prometheus_services_connect_scraper_synapse_usage_exporter_scrape_configs if matrix_prometheus_services_connect_scraper_synapse_usage_exporter_enabled else [])
+    +
+    (matrix_prometheus_services_connect_scraper_traefik_scrape_configs if matrix_prometheus_services_connect_scraper_traefik_enabled else [])
   }}
 
 ######################################################################
@@ -5362,6 +5399,9 @@ matrix_prometheus_services_connect_scraper_hookshot_static_configs_target: "{{ m
 matrix_prometheus_services_connect_scraper_nginxlog_enabled: "{{ matrix_prometheus_nginxlog_exporter_enabled }}"
 matrix_prometheus_services_connect_scraper_nginxlog_static_configs_target: "{{ matrix_prometheus_nginxlog_exporter_identifier }}:{{ matrix_prometheus_nginxlog_exporter_container_metrics_port | string }}"
 
+matrix_prometheus_services_connect_scraper_traefik_enabled: "{{ matrix_prometheus_traefik_exporter_enabled }}"
+matrix_prometheus_services_connect_scraper_traefik_static_configs_target: "{{ traefik_identifier }}:{{ traefik_config_entrypoint_metrics_port | string }}"
+
 matrix_prometheus_services_connect_scraper_media_repo_enabled: "{{ matrix_media_repo_enabled and matrix_media_repo_metrics_enabled }}"
 matrix_prometheus_services_connect_scraper_media_repo_static_configs_target: "{{ matrix_media_repo_identifier }}:{{ matrix_media_repo_metrics_port }}"
 
@@ -5436,6 +5476,8 @@ grafana_dashboard_download_urls: |
     (matrix_media_repo_dashboard_urls if matrix_media_repo_metrics_enabled else [])
     +
     (matrix_synapse_usage_exporter_dashboard_urls if matrix_synapse_usage_exporter_enabled else [])
+    +
+    (matrix_traefik_dashboard_download_urls if matrix_traefik_dashboard_download_enabled else [])
   }}
 
 grafana_provisioning_dashboard_template_files: |
@@ -5953,6 +5995,9 @@ devture_container_socket_proxy_api_containers_enabled: true
 
 traefik_enabled: "{{ matrix_playbook_reverse_proxy_type == 'playbook-managed-traefik' }}"
 
+# Controls whether Traefik prometheus exporter integration should be done
+matrix_prometheus_traefik_exporter_enabled: false
+
 traefik_identifier: matrix-traefik
 
 traefik_base_path: "{{ matrix_base_data_path }}/traefik"
@@ -5976,6 +6021,8 @@ traefik_config_providers_docker_endpoint: "{{ devture_container_socket_proxy_end
 traefik_container_additional_networks_auto: |
   {{
     ([devture_container_socket_proxy_container_network] if devture_container_socket_proxy_enabled else [])
+    +
+    ([matrix_monitoring_container_network] if matrix_prometheus_traefik_exporter_enabled else [])
   }}
 
 traefik_systemd_required_services_list: |
@@ -5985,6 +6032,14 @@ traefik_systemd_required_services_list: |
     ([devture_container_socket_proxy_identifier + '.service'] if devture_container_socket_proxy_enabled else [])
   }}
 
+traefik_config_metrics_prometheus_enabled: "{{ matrix_prometheus_traefik_exporter_enabled }}"
+traefik_container_metrics_host_bind_port: "{{ '' if matrix_prometheus_traefik_exporter_enabled else (traefik_config_entrypoint_metrics_port if traefik_config_entrypoint_metrics_enabled else '') }}"
+
+# Controls whether the Traefik grafana dashboard shall be downloaded
+matrix_traefik_dashboard_download_enabled: "{{ matrix_prometheus_traefik_exporter_enabled }}"
+matrix_traefik_dashboard_download_download_tag: master
+matrix_traefik_dashboard_download_urls: ["https://raw.githubusercontent.com/traefik/traefik/refs/heads/{{ matrix_traefik_dashboard_download_download_tag }}/contrib/grafana/traefik.json"]
+
 ########################################################################
 #                                                                      #
 # /traefik                                                             #

requirements.yml

@@ -75,3 +75,6 @@
 - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-traefik-certs-dumper.git
   version: v2.8.3-5
   name: traefik_certs_dumper
+- src: git+https://github.com/mother-of-all-self-hosting/ansible-role-valkey.git
+  version: v8.0.1-0
+  name: valkey

roles/custom/matrix-prometheus-services-connect/defaults/main.yml

@@ -143,6 +143,26 @@ matrix_prometheus_services_connect_scraper_nginxlog_scrape_configs: |
     }]
   }}
 
+# Controls whether Traefik shall be scraped
+matrix_prometheus_services_connect_scraper_traefik_enabled: false
+matrix_prometheus_services_connect_scraper_traefik_job_name: traefik
+matrix_prometheus_services_connect_scraper_traefik_metrics_path: /metrics
+matrix_prometheus_services_connect_scraper_traefik_scrape_interval: 15s
+matrix_prometheus_services_connect_scraper_traefik_scrape_timeout: 10s
+matrix_prometheus_services_connect_scraper_traefik_static_configs: "{{ [{'targets': [matrix_prometheus_services_connect_scraper_traefik_static_configs_target]}] }}"
+matrix_prometheus_services_connect_scraper_traefik_static_configs_target: ''
+# The final scrape config for the Traefik scraper
+matrix_prometheus_services_connect_scraper_traefik_scrape_configs: |
+  {{
+    [{
+      'job_name': matrix_prometheus_services_connect_scraper_traefik_job_name,
+      'metrics_path': matrix_prometheus_services_connect_scraper_traefik_metrics_path,
+      'scrape_interval': matrix_prometheus_services_connect_scraper_traefik_scrape_interval,
+      'scrape_timeout': matrix_prometheus_services_connect_scraper_traefik_scrape_timeout,
+      'static_configs': matrix_prometheus_services_connect_scraper_traefik_static_configs,
+    }]
+  }}
+
 # Controls whether media-repo shall be scraped
 matrix_prometheus_services_connect_scraper_media_repo_enabled: false
 matrix_prometheus_services_connect_scraper_media_repo_job_name: media-repo

setup.yml

@@ -49,6 +49,8 @@
 
     - galaxy/redis
     - galaxy/keydb
+    - galaxy/valkey
+
     - role: custom/matrix-authentication-service
     - custom/matrix-corporal
     - custom/matrix-appservice-draupnir-for-all