Commit Graph

1240 Commits

Author SHA1 Message Date
Slavi Pantaleev
a1179289a1 Split some homeserver _additional_networks variables into _auto and _custom 2024-01-26 12:55:01 +02:00
Slavi Pantaleev
dafeee92f4 Adjust matrix_nginx_proxy_container_labels_traefik_proxy_matrix_federation_hostname validation check message to mention matrix_static_files_file_matrix_server_property_m_server 2024-01-26 12:17:49 +02:00
Slavi Pantaleev
b48b06d2f8 Add missing bracket 2024-01-26 12:10:34 +02:00
Slavi Pantaleev
5ca4d6ebc5 Add validation check for matrix_nginx_proxy_container_labels_traefik_proxy_matrix_federation_hostname 2024-01-26 12:09:54 +02:00
Slavi Pantaleev
bc7ed6bd38
Merge pull request #3131 from Michael-Hollister/michael/synapse-add-extra-arguments
Added extra systemd service arguments to synapse workers and proxy companion
2024-01-25 07:46:50 +02:00
Slavi Pantaleev
ad9ba1e2bd
Fix variable name typo 2024-01-25 07:39:25 +02:00
Slavi Pantaleev
243d828e50 Fix mautrix-discord config Jinja2 syntax error
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3133

Regression since 4ed522e8fe
2024-01-25 07:35:16 +02:00
needo37
4ed522e8fe
Bring default config inline with upstream 2024-01-24 19:41:58 -06:00
Michael Hollister
bd027159b1 Added extra systemd service arguments to synapse workers and proxy companion 2024-01-24 13:14:34 -06:00
Catalan Lover
c4992ca018
Explicitly Declare Draupnir and Mjolnir Config and enter Bot Mode
This should resolve [#2296](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2296) by fixing the noted issue.

This also paves the way for in the future working on D4A mode but that would require a rework to how these variables are done.
2024-01-24 15:26:05 +01:00
renovate[bot]
e2a4f119f1
chore(deps): update joseluisq/static-web-server docker tag to v2.25.0 2024-01-23 01:50:56 +00:00
Slavi Pantaleev
ecb5591743 Upgrade sliding-sync (v0.99.14 -> v0.99.15) 2024-01-22 14:36:05 +02:00
Slavi Pantaleev
5c66485c99 Ensure matrix-bot-mjolnir container network is created
Most addons live in the same network by default (matrix-addons) right now,
so this network would have usually been created by some other addon.

Howevre, if this is the only addon someone uses, it may have remained
uncreated causing a problem.
2024-01-20 15:42:12 +02:00
Pierre 'McFly' Marty
f10bc264da
chore(deps): update Telegrambot config 2024-01-20 12:58:41 +01:00
Slavi Pantaleev
9fb2d53b54 Rework Synapse workers documentation
Related to: https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3100
2024-01-20 12:41:21 +02:00
Slavi Pantaleev
84446e52e9 Rename Synapse worker preset name (room-workers -> specialized-workers)
I believe `specialized-workers` is a better name than `room-workers`,
because when enabled, 4 different types of specialized workers are
created:

- Room workers
- Sync workers
- Client readers
- Federation readers

Only one of these is called room-workers.

In the future, more specialized workers may be added, making the
`room-workers` preset name an even poorer choice.

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3100
2024-01-20 12:40:55 +02:00
Slavi Pantaleev
7cb33da46a Add some clarification comment in matrix-synapse-reverse-proxy-companion/defaults/main.yml 2024-01-20 11:35:20 +02:00
Slavi Pantaleev
16ca50c6ef Add a few more comments in matrix-synapse-reverse-proxy-companion.conf.j2
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3100
2024-01-20 11:24:59 +02:00
Slavi Pantaleev
3c7f896246 Prevent generic workers being combined with any of the other types
Until now, the validation check would only get tripped up
if generic workers are used, combined with at least one EACH
other type of specialized workers.

This means that someone doing this:

```
matrix_synapse_workers_preset: one-of-each
matrix_synapse_workers_client_reader_workers_count: 5
```

.. would not have triggered this safety check.

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3100
2024-01-20 11:24:32 +02:00
Slavi Pantaleev
826f757fbb
Merge branch 'master' into cvwright/room-workers-v2 2024-01-20 10:35:56 +02:00
renovate[bot]
0823efe22e
Update vectorim/element-web Docker tag to v1.11.55 2024-01-19 15:31:02 +00:00
Slavi Pantaleev
90332f8c3d Fix problematic Hookshot redirect for /hookshot/widgetapi/v1/static
Hookshot wants a trailing slash for this route.

If we let Hookshot redirect, it goes to `/widgetapi/v1/static/`,
instead of `/hookshot/widgetapi/v1/static/`, so we take this matter into our
own hands.
2024-01-19 17:08:14 +02:00
Slavi Pantaleev
f953dd2cd6 Only strip /hookshot prefix for Hookshot widgetapi
Public URLs are like: `/hookshot/widgetapi/v1/static/`
.. which get translated to requests for: `/widgetapi/v1/static/`

Previously, we were stripping the whole `/hookshot/widgetapi` prefix,
which is wrong.
2024-01-19 17:02:16 +02:00
Slavi Pantaleev
db7ed0e830 Fix Traefik load balancer port for matrix-mx-puppet-slack 2024-01-19 12:13:22 +02:00
Slavi Pantaleev
dbebe7c598 Add variable for controlling force_disable in io.element.e2ee in /.well-known/matrix/client 2024-01-19 08:19:28 +02:00
Slavi Pantaleev
aed641e694 Disable addons communicating with the homeserver via Traefik if there is no Traefik at all 2024-01-18 12:12:41 +02:00
Slavi Pantaleev
775000883a Fix Jinja issue related to Synapse workers keepalive templating
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3114
2024-01-18 11:31:59 +02:00
Charles Wright
7d9eb56164 Add a validation step to fail when the user allocates generic workers together with all of the new worker types 2024-01-17 15:22:27 -06:00
Slavi Pantaleev
51df34e7ae Ensure each container labels file defines at least one service
Most of these files were defining a service, usually toward the end.
These lines have been moved upward.

Some components (mautrix-signal, mautrix-gmessages, etc.) were defining
a service conditionally (only if metrics are exposed, etc). This was
causing issues like these in the Traefik logs:

> level=error msg="service \"matrix-mautrix-twitter\" error: port is missing" providerName=docker container=matrix-mautrix-twitter-..
2024-01-17 17:56:45 +02:00
Slavi Pantaleev
474db10238 Reorder Ansible task module parameters to make ansible-lint happy 2024-01-17 17:27:31 +02:00
Slavi Pantaleev
f9e19e9623 Always uninstall matrix-nginx-proxy, if discovered
This changes the behavior of
`matrix_playbook_migration_matrix_nginx_proxy_uninstallation_enabled`
and is against what we initially described in the changelog entry,
but I've discovered some problems when the `matrix-nginx-proxy` service
and container remain running. They need to go.
2024-01-17 17:22:08 +02:00
Charles Wright
025a7e5c66
Merge branch 'spantaleev:master' into cvwright/room-workers-v2 2024-01-17 08:02:47 -06:00
Slavi Pantaleev
042c74f90c Remove some useless oidc variables and /_synapse/oidc route handling
After some checking, it seems like there's `/_synapse/client/oidc`,
but no such thing as `/_synapse/oidc`.

I'm not sure why we've been reverse-proxying these paths for so long
(even in as far back as the `matrix-nginx-proxy` days), but it's time we
put a stop to it.

The OIDC docs have been simplified. There's no need to ask people to
expose the useless `/_synapse/oidc` endpoint. OIDC requires
`/_synapse/client/oidc` and `/_synapse/client` is exposed by default
already.
2024-01-17 14:45:19 +02:00
Slavi Pantaleev
f3a9a2b35e Make post-start delay for matrix-conduit configurable 2024-01-17 12:26:28 +02:00
Slavi Pantaleev
4407403ab7 Make post-start delay for matrix-dendrite configurable 2024-01-17 12:25:31 +02:00
Slavi Pantaleev
cd06e04497 Make post-start delay for matrix-synapse configurable 2024-01-17 12:25:22 +02:00
Slavi Pantaleev
3ba0642bcf Increase delay after starting of matrix-synapse
10 seconds is a better default for slower (or overloaded) servers
2024-01-17 12:21:19 +02:00
Slavi Pantaleev
94378a7729 Make use of matrix_synapse_container_labels_matrix_related_labels_enabled
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3102
2024-01-17 10:13:15 +02:00
Slavi Pantaleev
ee0a8c4a81 Upgrade Synapse (v1.98.0 -> v1.99.0) 2024-01-17 08:40:48 +02:00
Slavi Pantaleev
aa0a85b094 Properly switch to element-hq-synapse and introduce variables for customizing that 2024-01-17 08:40:23 +02:00
Slavi Pantaleev
c0afcaa2e3 Replace (almost) all matrix-org/synapse references with element-hq/synapse
Issues and Pull Requests were not migrated to the new
organization/repository, so `matrix-org/synapse/pull` and
`matrix-org/synapse/issues` references were kept as-is.

`matrix-org/synapse-s3-storage-provider` references were also kept,
as that module still continues living under the `matrix-org` organization.

This patch mainly aims to change documentation-related things, not actual
usage in full yet. For polish that, another more comprehensive patch is coming later.
2024-01-17 08:02:47 +02:00
Catalan Lover
cb7f2eff3d make synapse support alternative containers via new variable 2024-01-17 07:28:08 +02:00
Slavi Pantaleev
fb64e86ba1
Merge pull request #3104 from spantaleev/renovate/dock.mau.dev-mautrix-discord-0.x
Update dock.mau.dev/mautrix/discord Docker tag to v0.6.5
2024-01-17 07:01:27 +02:00
Slavi Pantaleev
67f5640b3f
Merge pull request #3105 from spantaleev/renovate/dock.mau.dev-mautrix-gmessages-0.x
Update dock.mau.dev/mautrix/gmessages Docker tag to v0.2.4
2024-01-17 07:01:13 +02:00
Slavi Pantaleev
0aff4abcb0
Merge pull request #3109 from Michael-Hollister/michael/mmr-grafana-update-10-1-0
Updated Grafana dashboard for MMR
2024-01-17 06:56:27 +02:00
renovate[bot]
6ec2a89dcb
Update vectorim/element-web Docker tag to v1.11.54 2024-01-17 02:12:46 +00:00
Michael Hollister
c269eb5c49 Updated Grafana dashboard 2024-01-16 17:43:02 -06:00
Charles Wright
55604f73c5 Bugfix: Locations for new workers must go *after* the stream writers 2024-01-16 17:24:13 -06:00
Charles Wright
0dbdaf5b9f Enable HTTP resources for new worker types 2024-01-16 16:51:23 -06:00
Charles Wright
a1cbe7f39b Add overrides for locations that must go to the main Synapse process 2024-01-16 16:32:32 -06:00
renovate[bot]
fba9addb03
Update dock.mau.dev/mautrix/gmessages Docker tag to v0.2.4 2024-01-16 21:36:04 +00:00
renovate[bot]
f6c636b5e2
Update dock.mau.dev/mautrix/discord Docker tag to v0.6.5 2024-01-16 21:36:01 +00:00
Charles Wright
124524ea1f Typo: Send sync endpoints to sync workers, not room workers 2024-01-16 11:22:46 -06:00
Charles Wright
1379200e9d Add new worker types to the dynamic workers list 2024-01-16 11:13:51 -06:00
Charles Wright
5ca9a7269a Add the new worker types to the list of available worker types 2024-01-16 10:58:46 -06:00
Charles Wright
12a8d535e8 Move maps inside the if-workers block; Add Tom's map to extract access token from the URI arg 2024-01-16 10:53:20 -06:00
Charles Wright
0175a472d7 Typo: forgot closing }}'s 2024-01-16 10:02:36 -06:00
Charles Wright
db70230ae1 Add room-workers as a new preset, with new room workers, sync workers, client readers, and federation readers. Based on https://tcpipuk.github.io/synapse/index.html 2024-01-16 09:17:24 -06:00
Slavi Pantaleev
1036ae212f Update deprecation message for matrix_playbook_ssl_retrieval_method 2024-01-16 10:12:43 +02:00
Slavi Pantaleev
8f56166e6b Restore invocation of matrix-mailer migration tasks
Seems like calling these tasks got removed at some point
while merge the `bye-bye-nginx-proxy` branch.
2024-01-16 09:40:01 +02:00
Catalan Lover
95f989ae8b
Update Draupnir to 1.86.0 from 1.85.1 2024-01-16 01:56:41 +01:00
Slavi Pantaleev
a4bea66553 Remove references to other-nginx-non-container/other-on-same-host/other-on-another-host reverse proxy types 2024-01-15 16:14:12 +02:00
Slavi Pantaleev
92c3122b96 Add additional-networks support to matrix-dynamic-dns
Not that it seems necessary right now, but it makes it consistent with
all other roles.
2024-01-15 11:18:25 +02:00
Slavi Pantaleev
ad32953e0b Add additional-networks support to matrix-coturn
Not that it seems necessary right now, but it makes it consistent with
all other roles.
2024-01-15 11:18:09 +02:00
Slavi Pantaleev
fe13d7d010 Fix additional-networks connectivity for a few services 2024-01-15 11:13:47 +02:00
Slavi Pantaleev
e0aebe9b1e Fix incorrect ExecStart (+ docker create) definition in matrix-mautrix-googlechat.service 2024-01-15 11:09:25 +02:00
Slavi Pantaleev
a717509531 Fix DB migrations for mautrix-hangouts failing to reach the database container 2024-01-15 11:07:41 +02:00
Slavi Pantaleev
48a4afb114 Make Traefik labels files look better
This moves the comments from being just in Jinja,
to actually ending up in the generated `labels` file,
which makes inspection of the final result easier.

Also, some new lines were added here and there to make labels
more legible.

The generated file may still include weird new-lines due to
various `if` statements yielding content or not, but that's not so ugly
anymore - now that we have proper start/end sections that are visible in
the final `labels` file.
2024-01-15 10:41:15 +02:00
Slavi Pantaleev
b9148675db Remove extraneous endif in Conduit labels 2024-01-15 09:41:19 +02:00
Slavi Pantaleev
b91ad453be Adjust TLS variables for homeservers to follow devture_traefik_config_entrypoint_web_secure_enabled (via matrix_federation_traefik_entrypoint_tls) 2024-01-15 09:39:36 +02:00
Slavi Pantaleev
25697861d7 Fix some variable typos in matrix-prometheus-nginxlog-exporter 2024-01-14 21:32:02 +02:00
Slavi Pantaleev
142a307af9 Fix more variable name typos in mx-puppet-twitter
Like 4f9b7ba656.
Regression since 8e8c9cc03.
2024-01-14 21:26:22 +02:00
Slavi Pantaleev
fe38c616c3 Fix variable name typo in matrix-bridge-mx-puppet-twitter 2024-01-14 21:21:11 +02:00
Slavi Pantaleev
8f64262e31 Fix yamllint-reported errors 2024-01-14 18:52:18 +02:00
Slavi Pantaleev
f4f3d57520 Remove all traces of matrix-nginx-proxy, add validation & uninstallation tasks 2024-01-14 18:42:14 +02:00
Slavi Pantaleev
18211810ef Fix some default values in matrix-static-files 2024-01-14 18:34:39 +02:00
Slavi Pantaleev
aff57d67c0 Adjust Synapse OIDC variable wiring and docs
Auto-enabling the OIDC APIs is convenient for people
using the new `matrix_synapse_oidc_*` variables.
2024-01-14 12:34:25 +02:00
Slavi Pantaleev
038c63888a Remove definition of old variable (matrix_synapse_admin_nginx_proxy_integration_enabled) 2024-01-14 12:12:15 +02:00
Slavi Pantaleev
aeb1bde4ab Remove matrix-nginx-proxy reference from matrix-bridge-hookshot 2024-01-14 12:06:05 +02:00
Slavi Pantaleev
69ca30d1b1 Add support for the internal Traefik entrypoint to matrix-media-repo 2024-01-14 11:57:51 +02:00
Slavi Pantaleev
6b5f42fa81 Indirectly make use of matrix_homeserver_federation_enabled in matrix-media-repo and add some comments around Traefik labels 2024-01-14 11:54:02 +02:00
Slavi Pantaleev
c238978ac8 Add new global variable for controlling federation regardless of homeserver implementation
The old variables still work. The global lets us avoid
auto-detection logic like we're currently doing for
`matrix_nginx_proxy_proxy_matrix_federation_api_enabled`.

In the future, we'd just be able to reference
`matrix_homeserver_federation_enabled` and know the up-to-date value
regardless of homeserver.
2024-01-14 11:52:40 +02:00
Slavi Pantaleev
df5d8bfc04 Remove matrix-homeserver-proxy role in favor of the new internal Traefik entrypoint
This was meant to serve as an intermediary for services needing to reach
the homeserver. It was used like that for a while in this
`bye-bye-nginx-proxy` branch, but was never actually public.

It has recently been superseded by homeserver-like services injecting
themselves into a new internal Traefik entrypoint
(see `matrix_playbook_internal_matrix_client_api_traefik_entrypoint_*`),
so `matrix-homeserver-proxy` is no longer necessary.

---

This is probably a good moment to share some benchmarks and reasons
for going with the internal Traefik entrypoint as opposed to this nginx
service.

1. (1400 rps) Directly to Synapse (`ab -n 1000 -c 100 http://matrix-synapse:8008/_matrix/client/versions`
2. (~900 rps) Via `matrix-homeserver-proxy` (nginx) proxying to Synapse (`ab -n 1000 -c 100 http://matrix-homeserver-proxy:8008/_matrix/client/versions`)
3. (~1200 rps) Via the new internal entrypoint of Traefik (`matrix-internal-matrix-client-api`) proxying to Synapse (`ab -n 1000 -c 100 http://matrix-traefik:8008/_matrix/client/versions`)

Besides Traefik being quicker for some reason, there are also other
benefits to not having this `matrix-homeserver-proxy` component:

- we can reuse what we have in terms of labels. Services can register a few extra labels on the new Traefik entrypoint
- we don't need services (like `matrix-media-repo`) to inject custom nginx configs into `matrix-homeserver-proxy`. They just need to register labels, like they do already.
- Traefik seems faster than nginx on this benchmark for some reason, which is a nice bonus
- no need to run one extra container (`matrix-homeserver-proxy`) and execute one extra Ansible role
- no need to maintain a setup where some people run the `matrix-homeserver-proxy` component (because they have route-stealing services like `matrix-media-repo` enabled) and others run an optimized setup without this component and everything needs to be rewired to talk to the homeserver directly. Now, everyone can go through Traefik and we can all run an identical setup

Downsides of the new Traefik entrypoint setup are that:

- all addon services that need to talk to the homeserver now depend on Traefik
- people running their own Traefik setup will be inconvenienced - they
  need to manage one additional entrypoint
2024-01-14 10:53:14 +02:00
Slavi Pantaleev
17c9e3f168 Add support for the internal Traefik entrypoint to synapse-reverse-proxy-companion 2024-01-14 10:48:55 +02:00
Slavi Pantaleev
4d66c14fd5 Add support for the internal Traefik entrypoint to Conduit 2024-01-14 10:48:55 +02:00
Slavi Pantaleev
ee0eb59dc6 Add support for the internal Traefik entrypoint to Dendrite 2024-01-14 10:48:54 +02:00
Slavi Pantaleev
b2aeb8cde9 Rename label-related variables for homeservers
We'd be adding integration with an internal Traefik entrypoint
(`matrix_playbook_internal_matrix_client_api_traefik_entrypoint`),
so renaming helps disambiguate things.

There's no need for deperecation tasks, because the old names
have only been part of this `bye-bye-nginx-proxy` branch and not used by
anyone publicly.
2024-01-14 10:48:54 +02:00
Slavi Pantaleev
39bddefd39 Make addons communicate with the homeserver via a new internal Traefik entrypoint
This also adds labels for Synapse. Support for other homeservers and
components will be added later.
2024-01-14 10:48:54 +02:00
Slavi Pantaleev
533dc711ad Merge branch 'master' into bye-bye-nginx-proxy 2024-01-14 09:23:43 +02:00
Slavi Pantaleev
95e5a5c62e Deprecate direct usage of devture_traefik_additional_entrypoints_auto 2024-01-14 09:23:36 +02:00
Slavi Pantaleev
d48a70b052 Connect matrix-synapse-auto-compressor directly to Postgres network, if integrated Postgres is used
This saves us one container network in the ideal case.
2024-01-13 20:01:06 +02:00
Slavi Pantaleev
10777218e8 Fix yamllint-reported errors in matrix-email2matrix 2024-01-13 19:47:04 +02:00
Slavi Pantaleev
62c4e76634 Ensure matrix-nginx-proxy container network is created 2024-01-13 19:44:26 +02:00
Slavi Pantaleev
17d80cb9e8 Move wsproxy to the matrix-addons network and adjust its Postgres connectivity
This is a bit of a compatibility break.
The role was defaulting the Postgres password to `some-password` and we
auto-generate it now.

However, rebuilding both Postgres and this service should unify the
database credentials and the service configs to the new value.
2024-01-13 18:13:06 +02:00
Slavi Pantaleev
b9dfa87f9a Document difference between matrix_homeserver_container_url and matrix_addons_homeserver_client_api_url 2024-01-13 18:07:00 +02:00
Slavi Pantaleev
ed63068e22 Make maubot talk to the homeserver via matrix_addons_homeserver_client_api_url 2024-01-13 18:04:21 +02:00
Slavi Pantaleev
fa591ba278 Add missing matrix_bot_maubot_admins variable to defaults for matrix-bot-maubot 2024-01-13 18:00:19 +02:00
Slavi Pantaleev
c79f354dce Move Dimension to the addons network and connect to Homeserver via matrix_addons_homeserver_client_api_url 2024-01-13 17:58:41 +02:00
Slavi Pantaleev
49066d41a9 Deprecate matrix_docker_network 2024-01-13 17:49:38 +02:00
Slavi Pantaleev
07d0ec4217 Fix variable name typo in validation task 2024-01-13 17:48:39 +02:00