Commit Graph

242 Commits

Author SHA1 Message Date
Michael Hollister
55f869254b
Created role for synapse-usage-exporter (#3442)
* Created role for synapse-usage-exporter

* Apply suggestions from code review

Co-authored-by: Slavi Pantaleev <slavi@devture.com>

* Renaming docker variables and moving synapse stats config location

* Respect devture_systemd_docker_base_docker_service_name

---------

Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2024-07-25 20:19:08 +03:00
Slavi Pantaleev
34b91957f0 Update comment 2024-07-17 17:54:10 +03:00
Slavi Pantaleev
a213164cb1 Enable client & federation listeners for media repository workers
Related to c6d8a68e77

Related to https://github.com/element-hq/synapse/pull/17421

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3427
2024-07-17 17:52:21 +03:00
Slavi Pantaleev
615952cbaf Upgrade Synapse (v1.110.0 -> v1.111.0) 2024-07-16 18:10:27 +03:00
Slavi Pantaleev
c6d8a68e77 Add additional media repository prefix paths to matrix_synapse_workers_media_repository_endpoints
Related to https://github.com/element-hq/synapse/pull/17421
2024-07-16 18:10:27 +03:00
renovate[bot]
2c4ac73685
chore(deps): update ghcr.io/element-hq/synapse docker tag to v1.110.0 2024-07-03 19:50:40 +00:00
Slavi Pantaleev
4aa3345db0
Simplify auto_join_mxid_localpart population 2024-06-27 21:35:56 +03:00
hagene
ef90ee9495
Allow configuring synapse auto_join_mxid_localpart
`auto_join_mxid_localpart` defines the local part of the user id which is used to create auto-join rooms. The variable needs to be set to invite new users to any auto-join rooms which are set to invite-only.
2024-06-27 15:05:46 +02:00
Slavi Pantaleev
09d9db5617 Add variables for controlling the native auto-accept-invites Synapse feature
Related to https://github.com/element-hq/synapse/pull/17147
2024-06-18 15:46:39 +03:00
Slavi Pantaleev
9af4b491fa Upgrade Synapse (v1.108.0 -> v1.109.0) 2024-06-18 15:19:22 +03:00
Jim Myhrberg
247daf962f
fix(synapse/goofys): resolve Jinja2 syntax error in systemd service template
Commit 4224741130 missed a endfor
statement in the goofys systemd service unit template. This adds it,
avoiding a Jinja2 syntax error when using goofys.
2024-06-10 22:14:29 +01:00
Slavi Pantaleev
4224741130 Remove a few remaining hardcoded docker.service references
Continuation of 9f2eff2ac7

Provoked by 7749048bf8
(https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3353)
2024-06-05 21:22:21 +03:00
Slavi Pantaleev
9f2eff2ac7 Respect devture_systemd_docker_base_docker_service_name
Related to 0241c71a4c

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3270#issuecomment-2143782962

With this change, it should be possible for people to adjust the Docker
dependency from `docker.service` to something else (e.g. `pkg-ContainerManager-dockerd.service`),
or to completely eliminate it by setting `devture_systemd_docker_base_docker_service_name` to an empty string.

This makes it easier for people to use the playbook against a Synology DSM server.
2024-06-04 13:14:34 +03:00
renovate[bot]
ac864d713d
chore(deps): update ghcr.io/element-hq/synapse docker tag to v1.108.0 2024-05-28 12:12:34 +00:00
Slavi Pantaleev
7891268873 Do not hardcode https:// in all remaining places, refer to matrix_static_files_scheme
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3333
2024-05-25 16:14:26 +03:00
aquila
ac40afefff Add missing configuration matrix_synapse_ext_synapse_auto_accept_invite_accept_invites_only_from_local_users to specifies whether only invites from local users will be auto accepted. 2024-05-17 11:08:12 +08:00
renovate[bot]
83f5d73bf9
chore(deps): update ghcr.io/element-hq/synapse docker tag to v1.107.0 2024-05-14 17:25:29 +00:00
renovate[bot]
6f4e207823
chore(deps): update ghcr.io/element-hq/synapse docker tag to v1.106.0 2024-04-30 14:01:31 +00:00
renovate[bot]
e5296c6023
chore(deps): update ghcr.io/element-hq/synapse docker tag to v1.105.1 2024-04-23 15:39:08 +00:00
renovate[bot]
2a546a1e07
chore(deps): update ghcr.io/element-hq/synapse docker tag to v1.105.0 2024-04-16 16:13:53 +00:00
cksit
88609a59b1 Fixed the docker cmd for generating Synapse config 2024-04-14 18:12:32 +10:00
renovate[bot]
348c8c25e0
chore(deps): update ghcr.io/element-hq/synapse docker tag to v1.104.0 2024-04-02 18:49:21 +00:00
Aine
532e8b498b
add matrix_synapse_redis_dbid var 2024-04-01 23:24:40 +03:00
Slavi Pantaleev
3758b0cfeb Squashed commit of the following:
commit cf8637efaca0a0be3609fd6add0dff893a0a9194
Author: Slavi Pantaleev <slavi@devture.com>
Date:   Sun Mar 24 19:14:57 2024 +0200

    Make devture_systemd_docker_base_ipv6_enabled automatically reconfigure geerlingguy/ansible-role-docker

    Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3218

commit dc7af3bc7d25f321bf409477d823e43ea8a05803
Author: Slavi Pantaleev <slavi@devture.com>
Date:   Sun Mar 24 19:10:31 2024 +0200

    Replace matrix_ipv6_enabled with devture_systemd_docker_base_ipv6_enabled

    Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3218

commit 07e900d6a2
Author: Slavi Pantaleev <slavi@devture.com>
Date:   Sun Mar 24 19:01:51 2024 +0200

    Improve matrix_ipv6_enabled comments

commit 3f03ca7f69
Author: Tilo Spannagel <development@tilosp.de>
Date:   Sat Mar 9 19:27:50 2024 +0000

    Add setting to enable ipv6
2024-03-24 19:15:43 +02:00
Slavi Pantaleev
ab008e20cf Upgrade Synapse (v1.102.0 -> v1.103.0) 2024-03-19 16:56:58 +02:00
Slavi Pantaleev
8f800472ca Upgrade Synapse (v1.101.0 -> v1.102.0) 2024-03-05 20:08:56 +02:00
xvsun
905bdfc551
Add Synapse module auto accept invite to rooms and direct messages (#3195)
* feat: auto-accept-invite module and docs

* fix: name typos and some forgot to adjust variables

* fix: accept only direct messages should work now and better wording

* changed: only_direct_messages variable naming

* feat: add logger, add synapse workers config

* Fix typo and add details about synapse-auto-acccept-invite

* Add newline at end of file

* Fix alignment

* Fix logger name for synapse_auto_accept_invite

The name of the logger needs to match the name of the Python module.

Ref: d673c67678/synapse_auto_accept_invite/__init__.py (L20)

* Add missing document start YAML annotation

* Remove trailing spaces

---------

Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2024-02-21 18:04:14 +02:00
Slavi Pantaleev
e1363c9b9b Add lt-cred-mech authentication mechanism to Coturn
All homeserver implementations have been updated to support this as
well.

It's just Jitsi that possibly doesn't work with anything other than `auth-secret`.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3191
2024-02-18 09:52:00 +02:00
Slavi Pantaleev
972fc6b914 Fix ansible-lint-reported error related to spaces before comments 2024-02-14 13:46:55 +02:00
Slavi Pantaleev
d0cda27c97 Fix Synapse cache auto-tuning variables to use bytes, not KB
Fixup for https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3017

This reverts 1cd82cf068 and also multiplies results by `1024`
so as to pass bytes to Synapse, not KB (as done before).

1cd82cf068 was correctly documenting what we were doing (passing KB values),
but that's incorrect.

Synapse's Config Conventions
(https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#config-conventions)
are supposed to clear it up, but they don't currently state what happens when you pass a plain number (without a unit suffix).

Thankfully, the source code tells us:
bc1db16086/synapse/config/_base.py (L181-L206)

> If an integer is provided it is treated as bytes and is unchanged.
>
> String byte sizes can have a suffix of ...
> No suffix is understood as a plain byte count.

We were previously passing strings, but that has been improved in 3d73ec887a.

Regardless, non-suffixed values seem to be treated as bytes by Synapse,
so this patch changes the variables to use bytes.

Moreover, we're moving from `matrix_synapse_memtotal_kb` to
`matrix_synapse_cache_size_calculations_memtotal_bytes` as working with
the base unit everywhere is preferrable.

Here, we also introduce 2 new variables to allow for the caps to be
tweaked:

- `matrix_synapse_cache_size_calculations_max_cache_memory_usage_cap_bytes`
- `matrix_synapse_cache_size_calculations_target_cache_memory_usage_cap_bytes`
2024-02-14 13:39:40 +02:00
Slavi Pantaleev
3d73ec887a Ensure integer values are used for cache_autotuning settings in homeserver.yaml
We're casting everything it `int`, but since Jinja templates are
involved, these values end up as strings anyway.

Doing `| int | to_json` is good, but we should only cast numbers to
integer, not empty strings, as that (0) may be interpreted differently
by Synapse.

To turn of auto-tuning, one is possibly supposed to pass empty strings:

> This option defaults to off, enable it by providing values for the sub-options listed below.

It could be that `0` is also considered "no value provided", but I
haven't verified that.

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3017
2024-02-14 13:36:20 +02:00
Slavi Pantaleev
8b0e25966e Ensure cache_autotuning.max_cache_memory_usage & cache_autotuning.target_cache_memory_usage have int values
Fixes Synapse failing to start with:

> ValueError: invalid literal for int() with base 10: '2027264.0

Related to: https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3017
2024-02-14 12:20:53 +02:00
Catalan Lover
9eab0292d4
Increase Synapse caches and enable cache-autotuning by default (#3017)
* Modify Synapse Cache Factor to use Auto Tune

Synapse has the ability to as it calls in its config auto tune caches.

This ability lets us set very high cache factors and then instead limit our resource use.

Defaults for this commit are 1/10th of what Element apparently runs for EMS stuff and matrix.org on Cache Factor and upstream documentation defaults for auto tune.

* Add vars to Synapse main.yml to control cache related config

This commit adds various cache related vars to main.yml for Synapse.

Some are auto tune and some are just adding explicit ways to control upstream vars.

* Updated Auto Tune figures

Autotuned figures have been bumped in consultation with other community members as to a reasonable level. Please note these defaults are more on the one of each workers side than they are on the monolith Side.

* Fix YML Error

The playbook is not happy with the previous state of this patch so this commit hopefully fixes it

* Add to_json to various Synapse tuning related configs

* Fix incorrect indication in homeserver.yaml.j2

* Minor cleanups

* Synapse Cache Autotuning Documentation

* Upgrade Synapse Cache Autotune to auto configure memory use

* Update Synapse Tuning docs to reflect automatic memory use configuration

* Fix Linting errors in synapses main.yml

* Rename variables for consistency (matrix_synapse_caches_autotuning_* -> matrix_synapse_cache_autotuning_*)

* Remove FIX ME comment about Synapse's `cache_autotuning`

`docs/maintenance-synapse.md` and `roles/custom/matrix-synapse/defaults/main.yml`
already contains documentation about these variables and the default values we set.

* Improve "Tuning caches and cache autotuning" documentation for Synapse

* Announce larger Synapse caches and cache auto-tuning

---------

Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2024-02-14 12:02:06 +02:00
Slavi Pantaleev
a381fa4b21 Upgrade Synapse (v1.100.0 -> v1.101.0) 2024-02-13 14:56:42 +02:00
Slavi Pantaleev
e9a2b91da6 Enable federation API labels if the federation port is enabled
`matrix_synapse_federation_port_enabled` is defined like this:

```
matrix_synapse_federation_port_enabled: "{{ matrix_synapse_federation_enabled or matrix_synapse_federation_port_openid_resource_required }}"
```

Previously, people that disabled federation, but needed the `openid`
listener were running without these federation-related labels.

In this patch, we're also dropping the `not matrix_synapse_workers_enabled` condition,
because.. none of the Matrix-related labels would be applied anyway when
workers are enabled, thanks to `matrix_synapse_container_labels_matrix_related_labels_enabled`.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3127
2024-02-08 12:42:59 +02:00
Slavi Pantaleev
f3c69562fa Use devture_postgres_container_network for all rust-synapse-compress-state tasks
Using `matrix_synapse_container_network` for this task may have worked
before, when everything was in the same `matrix` network, but not anymore.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3165
2024-02-08 11:46:59 +02:00
Slavi Pantaleev
e01aa667e7 Fix some comments in worker-labels for Synapse
Related to 929aee3022 and
https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3154
2024-02-03 18:53:17 +02:00
Slavi Pantaleev
929aee3022 Fix incorrect prefix for Synapse worker metrics
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3154
2024-02-03 18:52:26 +02:00
renovate[bot]
b167f48396
Update ghcr.io/element-hq/synapse Docker tag to v1.100.0 2024-01-30 18:32:14 +00:00
Tiago Carrondo
5ca527066d
Fix s3-storage migrate and shell (#3136)
* Fix s3-storage migrate and shell: container needs attachment to postgres network also

* Connect to s3-storage-provider migrate to multiple networks in multiple steps

Multiple `--network` calls lead to:

> docker: Error response from daemon: Container cannot be connected to network endpoints: NETWORK_1 NETWORK_2.

* Connect to s3-storage-provider shell to multiple networks in multiple steps

---------

Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2024-01-27 18:03:37 +02:00
Slavi Pantaleev
a1179289a1 Split some homeserver _additional_networks variables into _auto and _custom 2024-01-26 12:55:01 +02:00
Slavi Pantaleev
ad9ba1e2bd
Fix variable name typo 2024-01-25 07:39:25 +02:00
Michael Hollister
bd027159b1 Added extra systemd service arguments to synapse workers and proxy companion 2024-01-24 13:14:34 -06:00
Slavi Pantaleev
9fb2d53b54 Rework Synapse workers documentation
Related to: https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3100
2024-01-20 12:41:21 +02:00
Slavi Pantaleev
84446e52e9 Rename Synapse worker preset name (room-workers -> specialized-workers)
I believe `specialized-workers` is a better name than `room-workers`,
because when enabled, 4 different types of specialized workers are
created:

- Room workers
- Sync workers
- Client readers
- Federation readers

Only one of these is called room-workers.

In the future, more specialized workers may be added, making the
`room-workers` preset name an even poorer choice.

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3100
2024-01-20 12:40:55 +02:00
Slavi Pantaleev
3c7f896246 Prevent generic workers being combined with any of the other types
Until now, the validation check would only get tripped up
if generic workers are used, combined with at least one EACH
other type of specialized workers.

This means that someone doing this:

```
matrix_synapse_workers_preset: one-of-each
matrix_synapse_workers_client_reader_workers_count: 5
```

.. would not have triggered this safety check.

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3100
2024-01-20 11:24:32 +02:00
Charles Wright
7d9eb56164 Add a validation step to fail when the user allocates generic workers together with all of the new worker types 2024-01-17 15:22:27 -06:00
Charles Wright
025a7e5c66
Merge branch 'spantaleev:master' into cvwright/room-workers-v2 2024-01-17 08:02:47 -06:00
Slavi Pantaleev
042c74f90c Remove some useless oidc variables and /_synapse/oidc route handling
After some checking, it seems like there's `/_synapse/client/oidc`,
but no such thing as `/_synapse/oidc`.

I'm not sure why we've been reverse-proxying these paths for so long
(even in as far back as the `matrix-nginx-proxy` days), but it's time we
put a stop to it.

The OIDC docs have been simplified. There's no need to ask people to
expose the useless `/_synapse/oidc` endpoint. OIDC requires
`/_synapse/client/oidc` and `/_synapse/client` is exposed by default
already.
2024-01-17 14:45:19 +02:00
Slavi Pantaleev
cd06e04497 Make post-start delay for matrix-synapse configurable 2024-01-17 12:25:22 +02:00