Commit Graph

276 Commits

Author SHA1 Message Date
Marcel Partap
b6b95fe742 synapse workers-doc-to-yaml script: compatibility++ with non-gnu awk 2020-12-02 23:22:02 +01:00
Marcel Partap
3156d96619 synapse workers-doc-to-yaml.awk: escape slash for non-gnu awk versions 2020-12-02 00:29:20 +01:00
Marcel Partap
e892ac464f synapse workers: untangle config template and specify bind address
.. to mitigate log noise - WARNING:
Failed to listen on 0.0.0.0, continuing because listening on [::]
2020-12-01 23:49:23 +01:00
Marcel Partap
f201bca519 synapse workers: define and expose METRICS port for each worker
As seen on TV:
https://github.com/matrix-org/synapse/blob/master/docs/metrics-howto.md#monitoring-workers
2020-12-01 22:49:15 +01:00
Marcel Partap
af08f18779 synapse workers default config: disable user_dir worker for now
(until https://github.com/matrix-org/synapse/issues/8787 is resolved)
2020-12-01 22:22:04 +01:00
Marcel Partap
414b812a29 synapse role workers setup: make configs clean action remote compatible
Many people probably didn't even know this - that ansible can be
quite a bit picky about what it will be willing to work with remotely.

Thanks @maxklenk !
2020-12-01 22:20:27 +01:00
Marcel Partap
d5932ca393 synapse role workers setup: execute the endpoint extraction locally
Thanks @maxklenk !
2020-12-01 22:18:42 +01:00
Marcel Partap
b73ac965ac Merge remote-tracking branch 'origin/master' into synapse-workers 2020-12-01 21:24:26 +01:00
Slavi Pantaleev
be5263f397 Move self-building git repository URLs to variables (stop hardcoding) 2020-11-28 21:34:14 +02:00
Slavi Pantaleev
75f9fde7a4 Remove some more -v usage
Continuation of 1fca917ad1.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/722
2020-11-25 10:49:59 +02:00
Slavi Pantaleev
1fca917ad1 Replace some -v instances with --mount
`-v` magically creates the source destination as a directory,
if it doesn't exist already. We'd like to avoid this magic
and the potential breakage that it might cause.

We'd rather fail while Docker tries to find things to `--mount`
than have it automatically create directories and fail anyway,
while having contaminated the filesystem.

There's a lot more `-v` instances remaining to be fixed later on.
This is just some start.

Things like `matrix_synapse_container_additional_volumes` and
`matrix_nginx_proxy_container_additional_volumes` were not changed to
use `--mount`, as options for each one are passed differently
(`ro` is `ro`, but `rw` doesn't exist and `slave` is `bind-propagation=slave`).
To avoid breaking people's custom volume mounts, we keep it as it is for now.

A deficiency with `--mount` is that it lacks the `z` option (SELinux
ownership changes), and some of our `-v` instances use that. I'm not
sure how supported SELinux is for us right now, but it might be,
and breaking that would not be a good idea.
2020-11-24 10:26:05 +02:00
Slavi Pantaleev
b627d93cdc Update homeserver.yaml to keep up with Synapse v1.23.0
Related to #724 (Github Pull Request)
2020-11-18 16:57:50 +02:00
transcaffeine
c58a7e03c7
synapse: update to 1.23.0 2020-11-18 14:16:46 +01:00
Slavi Pantaleev
5eed874199 Improve self-building experience (avoid conflict with pullable images)
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/716

This patch makes us use more fully-qualified container image names
(either prefixed with docker.io/ or with localhost/).

The latter happens when self-building is enabled.

We've recently had issues where if an image was removed manually
and the service was restarted (making `docker run` fetch it from Docker Hub, etc.),
we'd end up with a pulled image, even though we're aiming for a self-built one.
Re-running the playbook would then not do a rebuild, because:
- the image with that name already exists (even though it's something
else)
- we sometimes had conditional logic where we'd build only if the git
repo changed

By explicitly changing the name of the images (prefixing with localhost/),
we avoid such confusion and the possibility that we'd automatically pul something
which is not what we expect.

Also, I've removed that condition where building would happen on git
changes only. We now always build (unless an image with that name
already exists). We just force-build when the git repo changes.
2020-11-14 23:00:49 +02:00
Marcel Partap
4678c5d7bd Merge remote-tracking branch 'origin/master' into synapse-workers
Also, replace vague FIXME by a proper NOTE on the complete
story of the user_dir endpoints..
2020-11-11 21:26:08 +01:00
Marcel Partap
f3d2797d9c synapse workers: make awk script invocation handle paths with spaces
(quoting ftw)
2020-11-10 22:40:48 +01:00
Marcel Partap
1e971312e8 synapse workers: handle auth fallback endpoint on main process only
(allegedly breaks with SSO enabled)
2020-11-10 21:23:19 +01:00
Slavi Pantaleev
5c91e56898 Upgrade Synapse (v1.22.0 -> v1.22.1) 2020-10-30 19:35:55 +02:00
Marcel Partap
cce90b187a synapse workers: fix undefined variable cases when removing workers 2020-10-28 23:09:21 +01:00
Marcel Partap
e078e29ef8 synapse workers: fix self name in workers-doc-to-yaml.awk script 2020-10-28 08:39:31 +01:00
Marcel Partap
2d1b9f2dbf synapse workers: reworkings + get endpoints from upstream docs via awk
(yes, a bit awkward and brittle… xD)
2020-10-28 07:13:19 +01:00
Slavi Pantaleev
70f0b97a0a Upgrade Synapse (v1.21.2 -> v1.22.0) 2020-10-27 14:24:02 +02:00
Marcel Partap
87bd64ce9e Merge remote-tracking branch 'origin/master' into synapse-workers 2020-10-23 23:45:07 +02:00
Marcel Partap
a4125d5446 synapse workers: polishing, cleansing and installation of jq dependency 2020-10-23 20:49:53 +02:00
Marcel Partap
501efee07e synapse workers: supply systemd with actual worker PIDs (requires jq)
also, worker.yaml.j2:
  - hone worker_name
  - remove worker_pid_file entry (would only be used if worker_daemonize
    set to true; also, synapse only knows about the container namespace
    and thus can not provide the required host-view PID)
2020-10-22 20:53:41 +02:00
Slavi Pantaleev
f7ecc7a2a5 Upgrade Synapse (v1.21.1 -> v1.21.2) 2020-10-15 17:42:52 +03:00
Slavi Pantaleev
5abd511368 Upgrade Synapse (v1.21.0 -> v1.21.1) 2020-10-13 13:08:25 +03:00
Aaron Raimist
78529cbd47
Upgrade Synapse (v1.20.1 -> v1.21.0) 2020-10-12 23:59:34 -05:00
Marcel Partap
d2e61af224 Add worker_name to synapse worker config template
& restrict federation listener; frontend_proxy / user_dir don't need it
2020-10-11 21:52:08 +02:00
Marcel Partap
36e9be6092 matrix_synapse_workers_{avail,enabled}_list: sort non-generic workers
.. alphabetically and put those not documented as multi-instance
capable on ports ending on zero.
2020-10-11 21:44:42 +02:00
Marcel Partap
e9241f5fb9 Improve synapse-workers systemd service template
Is the PID magic gonna work? or will it need an ExecStartPost hack..
2020-10-11 21:09:19 +02:00
Marcel Partap
40024e9b81 Prevent workers failing if their config doesn't exist
- cherry-pick "Ensure worker config exists in systemd service (#7528)"
  from synapse d74cdc1a42e8b487d74c214b1d0ca575429d546a:
  "check that the worker config file exists instead of silently failing."
2020-10-11 21:09:19 +02:00
Marcel Partap
93a8ea7e4a Merge remote-tracking branch 'master' into feature/add-worker-support 2020-10-11 20:59:05 +02:00
Slavi Pantaleev
e68450f094 Upgrade Synapse (v1.20.0 -> v1.20.1) 2020-09-24 18:43:54 +03:00
Slavi Pantaleev
dd217137b6 Upgrade Synapse (v1.19.3 -> v1.20.0) 2020-09-22 19:28:07 +03:00
Slavi Pantaleev
65e22a6888 Upgrade Synapse (v1.19.2 -> v1.19.3) 2020-09-18 17:37:04 +03:00
Slavi Pantaleev
e10e3e354d Upgrade Synapse (v1.19.1 -> v1.19.2) 2020-09-16 16:35:17 +03:00
Max Klenk
4fdfc0a34f
add missing ratelimiting options required for load testing 2020-09-11 09:46:20 +02:00
Max Klenk
a25a429a52
add redis support 2020-09-10 13:39:00 +02:00
Max Klenk
06bc430c7c
refactor to use new workers and routes they serve 2020-08-28 13:53:39 +02:00
Max Klenk
53ccc783b7
remove duplicated key 2020-08-27 15:26:46 +02:00
Max Klenk
567d0318b0
Merge branch 'synapse-workers' into feature/add-worker-support 2020-08-27 15:22:12 +02:00
Slavi Pantaleev
6e9600ffec Upgrade Synapse (v1.19.0 -> v1.19.1) 2020-08-27 12:59:11 +03:00
Slavi Pantaleev
daf13107a0 Add support for rust-synapse-compress-state 2020-08-21 13:53:39 +03:00
Slavi Pantaleev
9952ec6c16 Upgrade Synapse (v1.18.0 -> v1.19.0) 2020-08-17 17:02:40 +03:00
Slavi Pantaleev
f78a5d4ee8 Upgrade Synapse (v1.17.0 -> v1.18.0) 2020-07-30 14:21:44 +03:00
Slavi Pantaleev
c6ab1c6a90 Riot is now Element
Fixes #586 (Github Issue)
2020-07-17 11:31:20 +03:00
Slavi Pantaleev
200f912c04 Upgrade Synapse (v1.16.1 -> v1.17.0)
Fixes #579 (Github Issue).
2020-07-13 14:08:50 +03:00
Slavi Pantaleev
eff55e4d00 Upgrade Synapse (v1.16.0 -> v1.16.1) 2020-07-10 14:33:18 +03:00
Slavi Pantaleev
928982cffe Upgrade Synapse (v1.15.2 -> v1.16.0) 2020-07-08 14:08:46 +03:00