Commit Graph

85 Commits

Author SHA1 Message Date
Michael Hollister
55f869254b
Created role for synapse-usage-exporter (#3442)
* Created role for synapse-usage-exporter

* Apply suggestions from code review

Co-authored-by: Slavi Pantaleev <slavi@devture.com>

* Renaming docker variables and moving synapse stats config location

* Respect devture_systemd_docker_base_docker_service_name

---------

Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2024-07-25 20:19:08 +03:00
Slavi Pantaleev
34b91957f0 Update comment 2024-07-17 17:54:10 +03:00
Slavi Pantaleev
a213164cb1 Enable client & federation listeners for media repository workers
Related to c6d8a68e77

Related to https://github.com/element-hq/synapse/pull/17421

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3427
2024-07-17 17:52:21 +03:00
Slavi Pantaleev
4aa3345db0
Simplify auto_join_mxid_localpart population 2024-06-27 21:35:56 +03:00
hagene
ef90ee9495
Allow configuring synapse auto_join_mxid_localpart
`auto_join_mxid_localpart` defines the local part of the user id which is used to create auto-join rooms. The variable needs to be set to invite new users to any auto-join rooms which are set to invite-only.
2024-06-27 15:05:46 +02:00
Slavi Pantaleev
09d9db5617 Add variables for controlling the native auto-accept-invites Synapse feature
Related to https://github.com/element-hq/synapse/pull/17147
2024-06-18 15:46:39 +03:00
Slavi Pantaleev
7891268873 Do not hardcode https:// in all remaining places, refer to matrix_static_files_scheme
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3333
2024-05-25 16:14:26 +03:00
Aine
532e8b498b
add matrix_synapse_redis_dbid var 2024-04-01 23:24:40 +03:00
xvsun
905bdfc551
Add Synapse module auto accept invite to rooms and direct messages (#3195)
* feat: auto-accept-invite module and docs

* fix: name typos and some forgot to adjust variables

* fix: accept only direct messages should work now and better wording

* changed: only_direct_messages variable naming

* feat: add logger, add synapse workers config

* Fix typo and add details about synapse-auto-acccept-invite

* Add newline at end of file

* Fix alignment

* Fix logger name for synapse_auto_accept_invite

The name of the logger needs to match the name of the Python module.

Ref: d673c67678/synapse_auto_accept_invite/__init__.py (L20)

* Add missing document start YAML annotation

* Remove trailing spaces

---------

Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2024-02-21 18:04:14 +02:00
Slavi Pantaleev
e1363c9b9b Add lt-cred-mech authentication mechanism to Coturn
All homeserver implementations have been updated to support this as
well.

It's just Jitsi that possibly doesn't work with anything other than `auth-secret`.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3191
2024-02-18 09:52:00 +02:00
Slavi Pantaleev
3d73ec887a Ensure integer values are used for cache_autotuning settings in homeserver.yaml
We're casting everything it `int`, but since Jinja templates are
involved, these values end up as strings anyway.

Doing `| int | to_json` is good, but we should only cast numbers to
integer, not empty strings, as that (0) may be interpreted differently
by Synapse.

To turn of auto-tuning, one is possibly supposed to pass empty strings:

> This option defaults to off, enable it by providing values for the sub-options listed below.

It could be that `0` is also considered "no value provided", but I
haven't verified that.

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3017
2024-02-14 13:36:20 +02:00
Catalan Lover
9eab0292d4
Increase Synapse caches and enable cache-autotuning by default (#3017)
* Modify Synapse Cache Factor to use Auto Tune

Synapse has the ability to as it calls in its config auto tune caches.

This ability lets us set very high cache factors and then instead limit our resource use.

Defaults for this commit are 1/10th of what Element apparently runs for EMS stuff and matrix.org on Cache Factor and upstream documentation defaults for auto tune.

* Add vars to Synapse main.yml to control cache related config

This commit adds various cache related vars to main.yml for Synapse.

Some are auto tune and some are just adding explicit ways to control upstream vars.

* Updated Auto Tune figures

Autotuned figures have been bumped in consultation with other community members as to a reasonable level. Please note these defaults are more on the one of each workers side than they are on the monolith Side.

* Fix YML Error

The playbook is not happy with the previous state of this patch so this commit hopefully fixes it

* Add to_json to various Synapse tuning related configs

* Fix incorrect indication in homeserver.yaml.j2

* Minor cleanups

* Synapse Cache Autotuning Documentation

* Upgrade Synapse Cache Autotune to auto configure memory use

* Update Synapse Tuning docs to reflect automatic memory use configuration

* Fix Linting errors in synapses main.yml

* Rename variables for consistency (matrix_synapse_caches_autotuning_* -> matrix_synapse_cache_autotuning_*)

* Remove FIX ME comment about Synapse's `cache_autotuning`

`docs/maintenance-synapse.md` and `roles/custom/matrix-synapse/defaults/main.yml`
already contains documentation about these variables and the default values we set.

* Improve "Tuning caches and cache autotuning" documentation for Synapse

* Announce larger Synapse caches and cache auto-tuning

---------

Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2024-02-14 12:02:06 +02:00
Slavi Pantaleev
e01aa667e7 Fix some comments in worker-labels for Synapse
Related to 929aee3022 and
https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3154
2024-02-03 18:53:17 +02:00
Tiago Carrondo
5ca527066d
Fix s3-storage migrate and shell (#3136)
* Fix s3-storage migrate and shell: container needs attachment to postgres network also

* Connect to s3-storage-provider migrate to multiple networks in multiple steps

Multiple `--network` calls lead to:

> docker: Error response from daemon: Container cannot be connected to network endpoints: NETWORK_1 NETWORK_2.

* Connect to s3-storage-provider shell to multiple networks in multiple steps

---------

Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2024-01-27 18:03:37 +02:00
Michael Hollister
bd027159b1 Added extra systemd service arguments to synapse workers and proxy companion 2024-01-24 13:14:34 -06:00
Charles Wright
025a7e5c66
Merge branch 'spantaleev:master' into cvwright/room-workers-v2 2024-01-17 08:02:47 -06:00
Slavi Pantaleev
042c74f90c Remove some useless oidc variables and /_synapse/oidc route handling
After some checking, it seems like there's `/_synapse/client/oidc`,
but no such thing as `/_synapse/oidc`.

I'm not sure why we've been reverse-proxying these paths for so long
(even in as far back as the `matrix-nginx-proxy` days), but it's time we
put a stop to it.

The OIDC docs have been simplified. There's no need to ask people to
expose the useless `/_synapse/oidc` endpoint. OIDC requires
`/_synapse/client/oidc` and `/_synapse/client` is exposed by default
already.
2024-01-17 14:45:19 +02:00
Slavi Pantaleev
cd06e04497 Make post-start delay for matrix-synapse configurable 2024-01-17 12:25:22 +02:00
Slavi Pantaleev
3ba0642bcf Increase delay after starting of matrix-synapse
10 seconds is a better default for slower (or overloaded) servers
2024-01-17 12:21:19 +02:00
Slavi Pantaleev
94378a7729 Make use of matrix_synapse_container_labels_matrix_related_labels_enabled
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3102
2024-01-17 10:13:15 +02:00
Slavi Pantaleev
c0afcaa2e3 Replace (almost) all matrix-org/synapse references with element-hq/synapse
Issues and Pull Requests were not migrated to the new
organization/repository, so `matrix-org/synapse/pull` and
`matrix-org/synapse/issues` references were kept as-is.

`matrix-org/synapse-s3-storage-provider` references were also kept,
as that module still continues living under the `matrix-org` organization.

This patch mainly aims to change documentation-related things, not actual
usage in full yet. For polish that, another more comprehensive patch is coming later.
2024-01-17 08:02:47 +02:00
Charles Wright
0dbdaf5b9f Enable HTTP resources for new worker types 2024-01-16 16:51:23 -06:00
Slavi Pantaleev
48a4afb114 Make Traefik labels files look better
This moves the comments from being just in Jinja,
to actually ending up in the generated `labels` file,
which makes inspection of the final result easier.

Also, some new lines were added here and there to make labels
more legible.

The generated file may still include weird new-lines due to
various `if` statements yielding content or not, but that's not so ugly
anymore - now that we have proper start/end sections that are visible in
the final `labels` file.
2024-01-15 10:41:15 +02:00
Slavi Pantaleev
b2aeb8cde9 Rename label-related variables for homeservers
We'd be adding integration with an internal Traefik entrypoint
(`matrix_playbook_internal_matrix_client_api_traefik_entrypoint`),
so renaming helps disambiguate things.

There's no need for deperecation tasks, because the old names
have only been part of this `bye-bye-nginx-proxy` branch and not used by
anyone publicly.
2024-01-14 10:48:54 +02:00
Slavi Pantaleev
39bddefd39 Make addons communicate with the homeserver via a new internal Traefik entrypoint
This also adds labels for Synapse. Support for other homeservers and
components will be added later.
2024-01-14 10:48:54 +02:00
Slavi Pantaleev
74099383cd Adapt external_prometheus.yml.example.j2 to our new metrics exposure setup 2024-01-12 13:01:06 +02:00
Slavi Pantaleev
c0308307e2 Make homeserver services sleep after startup, instead of all dependencies sleeping separately
This is an attempt at optimizing service startup.

The effect is most pronounced when many services are restarted one by one.
The systemd service manager role sometimes does this - for example when `just install-service synapse` runs.
In such cases, a 5-second delay for each Synapse worker service
(or other bridge/bot service that waits on the homeserver) quickly adds up to a lot.

When services are all stopped fully and then started, the effect is not so pronounced, because
`matrix-synapse.service` starts first and pulls all worker services (defined as `Wants=` for it).
Later on, when the systemd service manager role "starts" these worker services, they're started already.
Even if they had a 5-second wait each, it would have happened in parallel.
2024-01-12 12:45:18 +02:00
Slavi Pantaleev
41a52945d6 Add support for exposing metrics for Synapse workers 2024-01-12 12:16:06 +02:00
Slavi Pantaleev
22f5f0ba75 Add support for exposing metrics for Synapse (without workers) 2024-01-12 12:15:57 +02:00
Slavi Pantaleev
3556dd77ef Use variables instead of hardcoding service port numbers in labels for matrix-synapse 2024-01-12 09:31:31 +02:00
Slavi Pantaleev
e5130372b9 Switch matrix_synapse_container_additional_volumes from using -v to --mount
Depending on the `options` that people provide, this may break
compatibility.
2024-01-11 12:12:44 +02:00
Slavi Pantaleev
f257cd9fbe Fix a few incorrect service names in labels for matrix-synapse/matrix-synapse-reverse-proxy-companion 2024-01-11 11:58:20 +02:00
Slavi Pantaleev
015acb6d08 Add native Traefik support to matrix-synapse 2024-01-04 19:00:23 +02:00
Michael Hollister
a66a2d2692 Added Synapse connection pool config variables 2023-12-16 19:16:05 -06:00
Slavi Pantaleev
2511b34a7c Stop containers gracefully, instead of outright killing them 2023-12-06 11:52:23 +02:00
Aine
b2dd4bd1a4
Fix matrix_synapse_oidc_providers parsing 2023-10-17 12:09:30 +03:00
Slavi Pantaleev
c0e56ac1c4
Make OIDC providers if check safer 2023-10-02 19:32:09 +03:00
Aine
9167a7734e
add matrix_synapse_oidc_enabled and matrix_synapse_oidc_providers vars 2023-10-02 18:40:15 +03:00
Slavi Pantaleev
217ddad2de Add support for configuring forgotten_room_retention_period
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2918

Related to https://github.com/matrix-org/synapse/pull/15488
2023-09-28 12:54:52 +03:00
Slavi Pantaleev
e033520ce2 Make Synapse worker systemd service files depend on matrix-synapse.service
Many of these do depend on the Synapse master process (`matrix-synapse.service`),
so it makes sense to do it.

Furthermore, we're doing it so that one can stop the `matrix-synapse.service`
service and have systemd cascade this into stopping all the workers as well.

This is useful for easily stopping all of Synapse, so that Postgres
upgrades (`--tags=upgrade-postgres`) can happen cleanly.
Postgres upgrades currently stop `devture_postgres_systemd_services_to_stop_for_maintenance_list` which
includes Synapse, but stopping just the master process and leaving workers running is not safe enough and sometimes leads to errors like:

> ERROR:  insert or update on table "event_forward_extremities" violates foreign key constraint "event_forward_extremities_event_id"

With this dependency in place, stopping `matrix-synapse.service` will stop all Synapse processes.
2023-09-16 11:37:24 +03:00
Slavi Pantaleev
d207edb304 Deprecate matrix_synapse_spam_checker in favor of matrix_synapse_modules 2023-08-26 20:09:37 +03:00
Alexis Yushin
ba1d665bd9 make smtp tls configurable / optional 2023-07-27 11:26:15 +02:00
Slavi Pantaleev
c09c1265e8 Stop using deprecated worker settings (worker_replication_host, worker_replication_http_port)
Related to:
- e4f545c452
- 2481b7dfa4

We've prepared for this by adding the `main` process to the `instance_map` a long time ago,
in 49cb8b7b11.
2023-07-11 17:20:21 +03:00
Zac
f026c7cee1
Apply suggestions from code review
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2023-06-13 14:48:44 -05:00
cbackas
0e701bbece add var to make the creds optional 2023-06-13 11:08:14 -05:00
Zac
ceda4c41cc
remove the offending lines 2023-06-13 10:22:37 -05:00
Luke D Iremadze
5c0279f0c9
Update roles/custom/matrix-synapse/templates/synapse/homeserver.yaml.j2
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2023-06-02 23:39:55 -06:00
Luke D Iremadze
2f4428bf5b
Update roles/custom/matrix-synapse/templates/synapse/homeserver.yaml.j2
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2023-06-02 23:39:17 -06:00
Luke D Iremadze
1f0da9f744 Create hooks for user search ability 2023-06-02 12:50:46 -06:00
Slavi Pantaleev
c55371e305 Add changelog entry for the forget_rooms_on_leave default change for Synapse
Related to:

- https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/2698
- https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/2700
2023-05-25 09:14:09 +03:00