Commit Graph

2513 Commits

Author SHA1 Message Date
Slavi Pantaleev
26b287bd17 Upgrade certbot (1.10.1 -> 1.11.0) 2021-01-27 21:51:46 +02:00
Slavi Pantaleev
1cd2a218de Merge branch 'master' into synapse-workers 2021-01-27 21:41:54 +02:00
Slavi Pantaleev
c6feb0b99e Upgrade Synapse (v1.25.0 -> v1.26.0) 2021-01-27 21:41:47 +02:00
Slavi Pantaleev
39c2d72d17 Merge branch 'master' into synapse-workers 2021-01-27 17:12:16 +02:00
Slavi Pantaleev
008049f2a9 Fix mautrix-telegram registration file mistake
Regression since f6097fbba1
2021-01-27 17:11:46 +02:00
Slavi Pantaleev
a49dab76f8 Merge branch 'master' into synapse-workers 2021-01-27 15:49:16 +02:00
Slavi Pantaleev
e3290d8bcb Remove |to_json causing trouble
Fixes a regression introduced in f6097fbba1, which was cauing Synapse
to die with this error message:

> ValueError: sender_localpart needs characters which are not URL encoded.
2021-01-27 15:48:35 +02:00
Slavi Pantaleev
a31c9603fa Merge branch 'master' into synapse-workers 2021-01-27 15:43:56 +02:00
Slavi Pantaleev
f6097fbba1 E2BE not working for mautrix bridges
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/806
2021-01-27 15:43:33 +02:00
Slavi Pantaleev
07f1ea24ee Make it possible to override the welcome.html.j2 template used for Element 2021-01-27 12:36:57 +02:00
Slavi Pantaleev
d98a1ceadd Merge branch 'master' into synapse-workers 2021-01-27 10:27:17 +02:00
Slavi Pantaleev
512f42aa76 Do not report docker kill/rm attempts as errors
These are just defensive cleanup tasks that we run.
In the good case, there's nothing to kill or remove, so they trigger an
error like this:

> Error response from daemon: Cannot kill container: something: No such container: something

and:

> Error: No such container: something

People often ask us if this is a problem, so instead of always having to
answer with "no, this is to be expected", we'd rather eliminate it now
and make logs cleaner.

In the event that:
- a container is really stuck and needs cleanup using kill/rm
- and cleanup fails, and we fail to report it because of error
suppression (`2>/dev/null`)

.. we'd still get an error when launching ("container name already in use .."),
so it shouldn't be too hard to investigate.
2021-01-27 10:22:46 +02:00
Slavi Pantaleev
869727a402 Add comment to mautrix-facebook bridge regarding alembic migrations 2021-01-27 10:17:48 +02:00
Slavi Pantaleev
a9af36841d Merge branch 'master' into synapse-workers 2021-01-27 09:34:29 +02:00
Slavi Pantaleev
1aec5f9735
Merge pull request #831 from phirz/correct-faq-typos
Correct some typos in FAQ
2021-01-27 09:31:56 +02:00
phirz
deff7421fb Correct some typos in FAQ 2021-01-27 00:13:22 +01:00
Slavi Pantaleev
e443b376b9 Improve "things to do after installation" docs section 2021-01-26 17:20:34 +02:00
Slavi Pantaleev
346f8b3475
Fix typo 2021-01-26 10:13:08 +02:00
Slavi Pantaleev
26542308b3 Use |to_json in more places in matrix-appservice-discord config
I don't think this was causing an issue, but it might
if the bot token has a more special value in the future.

Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/828
2021-01-26 10:00:07 +02:00
Béla Becker
42f338016b Etherpad matrix-nginx-proxy configuration 2021-01-26 05:04:47 +01:00
Béla Becker
7bc9be95cb Add map directive to the base of nginx.conf
This needs to be added for WebSocket upgrades to work properly (see doc:
http://nginx.org/en/docs/http/websocket.html)
2021-01-26 05:04:47 +01:00
Béla Becker
38bf1eda70 Etherpad Jitsi integration 2021-01-26 05:04:47 +01:00
Béla Becker
4b451ff782 Etherpad role 2021-01-26 05:04:47 +01:00
Béla Becker
8355348aae Etherpad documentation 2021-01-26 05:04:47 +01:00
Slavi Pantaleev
d9bf2f59db Fix adminme.js usage for matrix-appservice-discord (still hacky)
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/795
and https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/822

A better fix will come later: https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/825
2021-01-25 15:36:08 +02:00
Slavi Pantaleev
a535226210 Stop/disable unnecessary worker services before deleting them 2021-01-25 15:20:37 +02:00
Slavi Pantaleev
dd24942c03
Use |to_json for mautrix-telegram config
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/824
2021-01-25 15:15:27 +02:00
Slavi Pantaleev
778b66876c Merge branch 'master' into synapse-workers 2021-01-25 14:56:55 +02:00
Slavi Pantaleev
70dcdd41a7 Simplify matrix-remove-all
We don't have instantiated services anymore, nor
/etc/systemd/system/matrix-synapse.service.wants/ stuff.
2021-01-25 14:02:30 +02:00
Slavi Pantaleev
d3ecc6f017 Fix bridges failing to upload media when Synapse workers are enabled 2021-01-25 13:55:08 +02:00
Slavi Pantaleev
66cdc7bf5a Clean up worker.yaml generation a bit and make it more flexible 2021-01-25 13:02:01 +02:00
Slavi Pantaleev
1462409b34 Fix worker listening addresses
Not specifying bind addresses for the worker resulted in this warning:

> synapse.app - 47 - WARNING - None - Failed to listen on 0.0.0.0, continuing because listening on [::]

Additionally, metrics listening only on 127.0.0.1 seems like a no-op.
Only having it accessible from within the container is likely not what
we intend. Changed that to all interfaces as well.

Whether it actually gets exposed or not depends on the systemd service
and `matrix_synapse_workers_container_host_bind_address`.
2021-01-25 12:29:47 +02:00
Slavi Pantaleev
01747c8cc4 Prevent Synapse warning about enabling metric listeners with enable_metrics: false
> synapse.app.generic_worker - 606 - WARNING - None - Metrics listener configured, but enable_metrics is not True!
2021-01-25 12:24:12 +02:00
Slavi Pantaleev
70796703d3 Run Synapse workers in their own containers
This switches the `docker exec` method of spawning
Synapse workers inside the `matrix-synapse` container with
dedicated containers for each worker.

We also have dedicated systemd services for each worker,
so this are now:
- more consistent with everything else (we don't use systemd
instantiated services anywhere)
- we don't need the "parse systemd instance name into worker name +
port" part
- we don't need to keep track of PIDs manually
- we don't need jq (less depenendencies)
- workers dying would be restarted by systemd correctly, like any other
service
- `docker ps` shows each worker separately and we can observe resource
usage
2021-01-25 12:14:46 +02:00
Slavi Pantaleev
6fc214480c
Fix Signal role using incorrect database string variable
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/823
2021-01-25 10:42:23 +02:00
Slavi Pantaleev
da50fb27a0 Whitelist /_matrix/key requests for going to generic workers on the federation port 2021-01-25 09:46:50 +02:00
Slavi Pantaleev
4d62a75f6f Get matrix-corporal to play nicely with a Synapse worker setup
We do this by creating one more layer of indirection.

First we reach some generic vhost handling matrix.DOMAIN.
A bunch of override rules are added there (capturing traffic to send to
ma1sd, etc). nginx-status and similar generic things also live there.

We then proxy to the homeserver on some other vhost (only Synapse being
available right now, but repointing this to Dendrite or other will be
possible in the future).
Then that homeserver-specific vhost does its thing to proxy to the
homeserver. It may or may not use workers, etc.

Without matrix-corporal, the flow is now:
1. matrix.DOMAIN (matrix-nginx-proxy/matrix-domain.conf)
2. matrix-nginx-proxy/matrix-synapse.conf
3. matrix-synapse

With matrix-corporal enabled, it becomes:
1. matrix.DOMAIN (matrix-nginx-proxy/matrix-domain.conf)
2. matrix-corporal
3. matrix-nginx-proxy/matrix-synapse.conf
4. matrix-synapse

(matrix-corporal gets injected at step 2).
2021-01-25 09:46:41 +02:00
Slavi Pantaleev
c05d3d09bd Disable systemd services while stopping them
This removes some `multi-target.wants` symlinks as well, etc.

But despite systemd saying:

> Removed symlink /etc/systemd/system/matrix-synapse.service.wants/matrix-synapse-worker@appservice:0.service

.. I still see such symlinks tehre for me for some reason, so keeping the
code (below) to find & delete them still seems like a good idea.
2021-01-25 08:58:23 +02:00
Slavi Pantaleev
63301b0ef1 Improvements around Synapse worker/metrics ports exposure
There was a `matrix_nginx_proxy_enabled|default(False)` check, but:
- it didn't seem to work reliably for some reason (hmm)
- referring to a `matrix_nginx_proxy_*` variable from within the
  `matrix-synapse` role is not ideal
- exposing always happened on `127.0.0.1`, which may not be good enough
  for some rarer setups (where the own webserver is external to the host)
2021-01-25 08:25:43 +02:00
Slavi Pantaleev
f66a6b066b Be more specific with the Redis version being used 2021-01-25 01:34:58 +02:00
Slavi Pantaleev
5ca68210cd Do not handle /_matrix/federation on client-server port, nor /_matrix/client stuff on federation port
I guess it didn't hurt to do it until now, but it's not great serving
federation APIs on the client-server API port, etc.

matrix-corporal doesn't work yet (still something to be solved in the
future), but its firewalling operations will also be sabotaged
by Client-Server APIs being served on the federation port (it's a way to get around its firewalling).
2021-01-24 22:22:57 +02:00
Slavi Pantaleev
cc5cf0d725 Load roles/matrix-synapse/vars/workers.yml earlier to not break --tags=setup-nginx-proxy
If we load it at runtime, during matrix-synapse role execution,
it's good enough for matrix-synapse and all roles after that,
but.. it breaks when someone uses `--tags=setup-nginx-proxy` alone.

The downside of including this vars file like this in `setup.yml`
is that the variables contained in it cannot be overriden by the user
(in their inventory's `vars.yml`).
... but it's not like overriding these variables was possible anyway
when including them at runtime.
2021-01-24 20:19:55 +02:00
Slavi Pantaleev
92ee3d78a0 Fix matrix-remove-all for when Synapse workers are enabled 2021-01-24 19:42:32 +02:00
Slavi Pantaleev
8fa913dca7 Fix Ansible warning 2021-01-24 19:11:35 +02:00
Slavi Pantaleev
42f60142c0
Merge pull request #819 from jolly-jump/patch-1
small fix to CHANGELOG.md
2021-01-24 17:24:32 +02:00
T. Küchel
67fab21d7e
Update CHANGELOG.md
propose explicit showing single quotes around the password, since I forgot to put them there.
2021-01-24 12:31:07 +00:00
Slavi Pantaleev
9b5daf54f0 Fix wording a bit 2021-01-24 10:08:11 +02:00
Slavi Pantaleev
07c7afb861 Make README more tidy 2021-01-24 10:04:30 +02:00
Marcel Partap
edc21f15e5 Restrict publishing worker (metrics) ports to localhost 2021-01-24 08:53:09 +01:00
Marcel Partap
183adec3d8 Merge remote-tracking branch 'origin/master' into synapse-workers 2021-01-23 15:04:11 +01:00