Commit Graph

7426 Commits

Author SHA1 Message Date
Slavi Pantaleev
d0cda27c97 Fix Synapse cache auto-tuning variables to use bytes, not KB
Fixup for https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3017

This reverts 1cd82cf068 and also multiplies results by `1024`
so as to pass bytes to Synapse, not KB (as done before).

1cd82cf068 was correctly documenting what we were doing (passing KB values),
but that's incorrect.

Synapse's Config Conventions
(https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#config-conventions)
are supposed to clear it up, but they don't currently state what happens when you pass a plain number (without a unit suffix).

Thankfully, the source code tells us:
bc1db16086/synapse/config/_base.py (L181-L206)

> If an integer is provided it is treated as bytes and is unchanged.
>
> String byte sizes can have a suffix of ...
> No suffix is understood as a plain byte count.

We were previously passing strings, but that has been improved in 3d73ec887a.

Regardless, non-suffixed values seem to be treated as bytes by Synapse,
so this patch changes the variables to use bytes.

Moreover, we're moving from `matrix_synapse_memtotal_kb` to
`matrix_synapse_cache_size_calculations_memtotal_bytes` as working with
the base unit everywhere is preferrable.

Here, we also introduce 2 new variables to allow for the caps to be
tweaked:

- `matrix_synapse_cache_size_calculations_max_cache_memory_usage_cap_bytes`
- `matrix_synapse_cache_size_calculations_target_cache_memory_usage_cap_bytes`
2024-02-14 13:39:40 +02:00
Slavi Pantaleev
3d73ec887a Ensure integer values are used for cache_autotuning settings in homeserver.yaml
We're casting everything it `int`, but since Jinja templates are
involved, these values end up as strings anyway.

Doing `| int | to_json` is good, but we should only cast numbers to
integer, not empty strings, as that (0) may be interpreted differently
by Synapse.

To turn of auto-tuning, one is possibly supposed to pass empty strings:

> This option defaults to off, enable it by providing values for the sub-options listed below.

It could be that `0` is also considered "no value provided", but I
haven't verified that.

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3017
2024-02-14 13:36:20 +02:00
Slavi Pantaleev
1cd82cf068 Fix unit inaccuracy in documentation for cache-autotuning-related variables
Related to: https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3017
2024-02-14 12:25:34 +02:00
Slavi Pantaleev
8b0e25966e Ensure cache_autotuning.max_cache_memory_usage & cache_autotuning.target_cache_memory_usage have int values
Fixes Synapse failing to start with:

> ValueError: invalid literal for int() with base 10: '2027264.0

Related to: https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3017
2024-02-14 12:20:53 +02:00
Catalan Lover
9eab0292d4
Increase Synapse caches and enable cache-autotuning by default (#3017)
* Modify Synapse Cache Factor to use Auto Tune

Synapse has the ability to as it calls in its config auto tune caches.

This ability lets us set very high cache factors and then instead limit our resource use.

Defaults for this commit are 1/10th of what Element apparently runs for EMS stuff and matrix.org on Cache Factor and upstream documentation defaults for auto tune.

* Add vars to Synapse main.yml to control cache related config

This commit adds various cache related vars to main.yml for Synapse.

Some are auto tune and some are just adding explicit ways to control upstream vars.

* Updated Auto Tune figures

Autotuned figures have been bumped in consultation with other community members as to a reasonable level. Please note these defaults are more on the one of each workers side than they are on the monolith Side.

* Fix YML Error

The playbook is not happy with the previous state of this patch so this commit hopefully fixes it

* Add to_json to various Synapse tuning related configs

* Fix incorrect indication in homeserver.yaml.j2

* Minor cleanups

* Synapse Cache Autotuning Documentation

* Upgrade Synapse Cache Autotune to auto configure memory use

* Update Synapse Tuning docs to reflect automatic memory use configuration

* Fix Linting errors in synapses main.yml

* Rename variables for consistency (matrix_synapse_caches_autotuning_* -> matrix_synapse_cache_autotuning_*)

* Remove FIX ME comment about Synapse's `cache_autotuning`

`docs/maintenance-synapse.md` and `roles/custom/matrix-synapse/defaults/main.yml`
already contains documentation about these variables and the default values we set.

* Improve "Tuning caches and cache autotuning" documentation for Synapse

* Announce larger Synapse caches and cache auto-tuning

---------

Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2024-02-14 12:02:06 +02:00
Slavi Pantaleev
f999947dfe
Merge pull request #3185 from Tupsi/master
Update configuring-playbook-bot-maubot.md
2024-02-13 19:06:48 +02:00
Slavi Pantaleev
d9940bd807 Upgrade Element (v1.11.57 -> v1.11.58) 2024-02-13 19:06:14 +02:00
Tupsi
60fbcebd59
Update configuring-playbook-bot-maubot.md
works in encrypted rooms now, so I removed the notion that it does not.
2024-02-13 17:42:09 +01:00
Slavi Pantaleev
a381fa4b21 Upgrade Synapse (v1.100.0 -> v1.101.0) 2024-02-13 14:56:42 +02:00
Slavi Pantaleev
51cb2f2288
Merge pull request #3182 from spantaleev/renovate/traefik-2.x
Update dependency traefik to v2.11.0-0
2024-02-13 07:46:44 +02:00
Slavi Pantaleev
95e557dcba
Merge pull request #3183 from spantaleev/renovate/joseluisq-static-web-server-2.x
Update joseluisq/static-web-server Docker tag to v2.27.0
2024-02-13 06:21:16 +02:00
Slavi Pantaleev
5268a8edce
Merge pull request #3184 from array-in-a-matrix/patch-25
Add missing link to synapse config docs
2024-02-13 06:20:41 +02:00
Array in a Matrix
1e9f472077
Add missing link to synapse config docs 2024-02-12 23:10:50 -05:00
renovate[bot]
4242f4f7cd
Update joseluisq/static-web-server Docker tag to v2.27.0 2024-02-13 03:02:42 +00:00
renovate[bot]
2bc6dcf4f3
Update dependency traefik to v2.11.0-0 2024-02-12 18:56:15 +00:00
chagai95
a27464a546
Update CHANGELOG.md (#3181)
* Update CHANGELOG.md

* Update CHANGELOG.md
2024-02-12 17:35:48 +02:00
Slavi Pantaleev
bbbe89e596
Merge pull request #3178 from FSG-Cat/patch-3
Update container-images.md to mention Draupnir
2024-02-12 07:45:29 +02:00
Catalan Lover
1aafb58d00
Update container-images.md to mention Draupnir
Adds a Draupnir mention to the list and as for why we pull from Gnuxie its because that is the official source of docker images as Draupnir used to be Gnuxie/Draupnir before it moved to The Draupnir Project.
2024-02-11 23:28:45 +01:00
Slavi Pantaleev
90679b7dce
Merge pull request #3177 from spantaleev/renovate/registry.gitlab.com-etke.cc-postmoogle-0.x
Update registry.gitlab.com/etke.cc/postmoogle Docker tag to v0.9.17
2024-02-11 22:34:53 +02:00
renovate[bot]
cf9ca9e602
Update registry.gitlab.com/etke.cc/postmoogle Docker tag to v0.9.17 2024-02-11 19:45:35 +00:00
Slavi Pantaleev
ce9a8d3a2c Rename base domain root path redirect middleware to improve consistency 2024-02-11 09:07:32 +02:00
Slavi Pantaleev
cf9388c546 Make base domain root path redirect regex configurable 2024-02-11 09:04:30 +02:00
Slavi Pantaleev
52d4b5083d
Merge pull request #3176 from spantaleev/renovate/joseluisq-static-web-server-2.x
Update joseluisq/static-web-server Docker tag to v2.26.0
2024-02-11 06:35:10 +02:00
renovate[bot]
e2ab339634
Update joseluisq/static-web-server Docker tag to v2.26.0 2024-02-11 00:58:31 +00:00
Slavi Pantaleev
522e89708d
Merge pull request #3173 from sidewinder94/patch-1
Update SRV delegation docs
2024-02-10 13:48:45 +02:00
Antoine-Ali Zarrouk
05e1fa3546
Update SRV delegation docs
The path rule was not working because for federation fo work it needs several endpoints.

Two of them are not under /_matrix/federation : 

- /_matrix/key
- /_matrix/media
2024-02-10 10:18:46 +01:00
Slavi Pantaleev
dad0d24312
Merge pull request #3171 from spantaleev/renovate/gnuxie-draupnir-1.x
Update gnuxie/draupnir Docker tag to v1.86.2
2024-02-10 05:45:40 +02:00
Slavi Pantaleev
a71546c3bf
Merge pull request #3172 from spantaleev/renovate/turt2live-matrix-media-repo-1.x
Update turt2live/matrix-media-repo Docker tag to v1.3.4
2024-02-10 05:44:31 +02:00
renovate[bot]
2d4b96e0c5
Update turt2live/matrix-media-repo Docker tag to v1.3.4 2024-02-10 01:50:50 +00:00
renovate[bot]
89288cce0e
Update gnuxie/draupnir Docker tag to v1.86.2 2024-02-09 21:13:33 +00:00
Slavi Pantaleev
b91da76c6c
Merge pull request #3169 from kumarunster/master
allow to configure whatsapp polls via extev_polls parameter.
2024-02-09 16:44:08 +02:00
Slavi Pantaleev
1bfafa7004
Use to_json for matrix_mautrix_whatsapp_extev_polls 2024-02-09 16:42:48 +02:00
Nikolai Raitsev
68d4e04f4f allow to configure whatsapp polls via extev_polls parameter. 2024-02-09 14:17:16 +01:00
Slavi Pantaleev
9f2fdd4148
Merge pull request #3168 from etkecc/patch-325
fix buscarron old vars
2024-02-08 22:02:10 +02:00
Aine
2096d13bbd
fix buscarron old vars 2024-02-08 21:17:12 +02:00
Slavi Pantaleev
41ca1a1d96 Upgrade synapse-admin (0.9.0 -> 0.9.1) 2024-02-08 16:39:29 +02:00
Slavi Pantaleev
e9a2b91da6 Enable federation API labels if the federation port is enabled
`matrix_synapse_federation_port_enabled` is defined like this:

```
matrix_synapse_federation_port_enabled: "{{ matrix_synapse_federation_enabled or matrix_synapse_federation_port_openid_resource_required }}"
```

Previously, people that disabled federation, but needed the `openid`
listener were running without these federation-related labels.

In this patch, we're also dropping the `not matrix_synapse_workers_enabled` condition,
because.. none of the Matrix-related labels would be applied anyway when
workers are enabled, thanks to `matrix_synapse_container_labels_matrix_related_labels_enabled`.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3127
2024-02-08 12:42:59 +02:00
Slavi Pantaleev
f3c69562fa Use devture_postgres_container_network for all rust-synapse-compress-state tasks
Using `matrix_synapse_container_network` for this task may have worked
before, when everything was in the same `matrix` network, but not anymore.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/3165
2024-02-08 11:46:59 +02:00
Slavi Pantaleev
d59a6943a9
Merge pull request #3166 from needo37/patch-5
Update Signal config.yaml.j2
2024-02-08 11:40:40 +02:00
needo37
193d20013f
Update Signal config.yaml.j2
Not sure why but the endraw is not working.
2024-02-08 09:16:29 +00:00
needo37
8a9a700cfc
Bring config.yaml.j2 in line with upstream (#3163)
* Bring config.yaml.j2 in line with upstream

* Update config.yaml.j2
2024-02-08 08:15:17 +02:00
needo37
518615a979
Update signal config.yaml.j2 merging upstream changes (#3164)
* Update signal config.yaml.j2 merging upstream changes

* Add raw/endraw around displayname_template for mautrix-signal

---------

Co-authored-by: Slavi Pantaleev <slavi@devture.com>
2024-02-08 08:13:07 +02:00
Slavi Pantaleev
cce395a88a
Merge pull request #3162 from needo37/patch-3
Update configuring-playbook-bridge-mautrix-whatsapp.md
2024-02-08 06:13:31 +02:00
needo37
0667907832
Update configuring-playbook-bridge-mautrix-whatsapp.md
Backfilling is now supported. Updating documentation.
2024-02-08 03:44:38 +00:00
Slavi Pantaleev
6892d32bfc
Merge pull request #3158 from etkecc/patch-324
update honoroit (v0.9.19 -> v0.9.20)
2024-02-06 08:06:14 +02:00
Slavi Pantaleev
928b21acf4 Add variable-deprecation task for Buscarron
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3157
2024-02-06 07:23:56 +02:00
Slavi Pantaleev
1ceb393fc3
Merge pull request #3157 from etkecc/buscarron-140
buscarron v1.4.0
2024-02-06 07:22:06 +02:00
Aine
a07345a42e
update honoroit (v0.9.19 -> v0.9.20)
**Warning**: [CI pipeline is in progress](https://gitlab.com/etke.cc/honoroit/-/pipelines/1165360868)

changelog:

* safer reaction forwarding
* fix duplicated prefix and suffix on completed requests
* add missing `!ho help` entries
* add new `!ho count` command
* count requests by homeserver and by MXID
* add new `!ho config` command set - configure honoroit directly from the chat
* mautrix-go 0.15.x+ migration
* shared secret auth support
* account data encyption support

removed env vars (automatic migration):

* HONOROIT_TEXT_*
* HONOROIT_ALLOWEDUSERS
* HONOROIT_IGNOREDROOMS
* HONOROIT_IGNORENOTHREAD
* HONOROIT_NOENCRYPTION
2024-02-05 22:12:24 +02:00
Aine
2baea7ce7b
buscarron v1.4.0 2024-02-05 22:07:45 +02:00
Slavi Pantaleev
7f337fc9a6 Upgrade synapse-admin (0.8.7 -> 0.9.0) 2024-02-05 19:07:51 +02:00