This gets us started on adding a Traefik role and hooking Traefik:
- directly to services which support Traefik - we only have a few of
these right now, but the list will grow
- to matrix-nginx-proxy for most services that integrate with
matrix-nginx-proxy right now
Traefik usage should be disabled by default for now and nothing should
change for people just yet.
Enabling these experiments requires additional configuration like this:
```yaml
devture_traefik_ssl_email_address: '.....'
matrix_playbook_traefik_role_enabled: true
matrix_playbook_traefik_labels_enabled: true
matrix_ssl_retrieval_method: none
matrix_nginx_proxy_https_enabled: false
matrix_nginx_proxy_container_http_host_bind_port: ''
matrix_nginx_proxy_container_federation_host_bind_port: ''
matrix_nginx_proxy_trust_forwarded_proto: true
matrix_nginx_proxy_x_forwarded_for: '$proxy_add_x_forwarded_for'
matrix_coturn_enabled: false
```
What currently works is:
reverse-proxying for all nginx-proxy based services **except** for the Matrix homeserver
(both Client-Server an Federation traffic for the homeserver don't work yet)
Switching from doing "post-start" loop hacks to running the container
in 3 steps: `create` + potentially connect to additional networks + `start`.
This way, the container would be connected to all its networks even at
the very beginning of its life.
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/2427
`metrics_enabled` should only expose the metrics locally, on the
container network, so that a local Prometheus can consume them.
Exposing them publicly should be done via a separate toggle (`metrics_proxying_enabled`).
This is how all other roles work, so this makes these mautrix roles consistent with the rest.
This helps large deployments which need to open up thousands of ports
(matrix_coturn_turn_udp_min_port, matrix_coturn_turn_udp_min_port)
On a test VM, opening 1k ports takes 17 seconds for Docker to "publish"
all of these ports (setting up forwarding rules with the firewall, etc),
so service startup and shutdown take a long amount of time.
If host-networking is used, there's no need to open any ports at all
and startup/shutdown can be quick.
This container needs a writable $HOME, and will fail at startup if
there isn't one.
Provide one by pointing HOME to a path under the mounted /data
directory.
* Allow the mautrix whatsapp relaybot to be enable with a variable
This allows a user to enable the relaybot by setting a variable in
`vars.yml` in the same way that the mautrix signal relaybot is
configured.
* Correct default values for mautrix whatsapp relaybot variables
* Add documentation for using the relaybot with mautrix whatsapp
* Adjust variable names to better reflect what they do
* Set default variables properly and use to_json in template
This extends the collection with support for seamless authentication at the Jitsi server using Matrix OpenID.
1. New role for installing the [Matrix User Verification Service](https://github.com/matrix-org/matrix-user-verification-service)
2. Changes to Jitsi role: Installing Jitsi Prosody Mods and configuring Jitsi Auth
3. Changes to Jitsi and nginx-proxy roles: Serving .well-known/element/jitsi from jitsi.DOMAIN
4. We updated the Jitsi documentation on authentication and added documentation for the user verification service.
Please note that This Mjolnir version bump technnically is missing some extra stuff that mjolnir claims we should do but it didnt work when i tried it and well my mjolnir deployment has been running this since release day almost and its fine. No errors in log that are unexpected. (Mjolnir throws errors in the log for anyone who wonders for various things that are fine. Like if a protection is off that is an error. Its due to how matrix-bot-lib works.)
Without these:
- `--tags=install-synapse` and `--tags=install-all` would be incomplete
and will not contain Synapse worker configuration
- `--tags=install-synapse-reverse-proxy-companion` and
`--tags=setup-synapse-reverse-proxy-companion` would not contain
Synapse worker configuration
* Changes to allow a user to set the max participants on a jitsi
conference
* changed var name from jitsi_max_participants to matrix_prosody_jitsi_max_participants
* add prometheus-nginxlog-exporter role
* Rename matrix_prometheus_nginxlog_exporter_container_url to matrix_prometheus_nginxlog_exporter_container_hostname
* avoid referencing variables from other roles, handover info using group_vars/matrix_servers
* fix: stop service when uninstalling
fix: typo
move available arch's into a var
fix: text
* fix: prometheus enabled condition
Co-authored-by: ikkemaniac <ikkemaniac@localhost>
Running with a user (like `matrix:matrix`) fails if Etherpad is enabled,
because `/matrix/etherpad` is owned by `matrix_etherpad_user_uid`/`matrix_etherpad_user_gid` (`5001:5001`).
The `matrix` user can't acccess the Etherpad directory for this reason
and Borgmatic fails when trying to make a backup.
There may be other things under `/matrix` which similarly use
non-`matrix:matrix` permissions.
Another workaround might have been to add `/matrix/etherpad` (and
potentially other things) to `matrix_backup_borg_location_exclude_patterns`, but:
- that means Etherpad won't be backed up - not great
- only excluding Etherpad may not be enough. There may be other files we
need to exclude as well
---
Running with `root` is still not enough though.
We need at least the `CAP_DAC_OVERRIDE` capability, or we won't be able to read the
`/etc/borgmatic.d/config.yaml` configuration file (owned by
`matrix:matrix` with `0640` permissions).
---
Additionally, it seems like the backup process tries to write to at least a few directories:
- `/root/.borgmatic`
- `/root/.ssh`
- `/root/.config`
> [Errno 30] Read-only file system: '/root/.borgmatic'
> Error while creating a backup.
> /etc/borgmatic.d/config.yaml: Error running configuration file
We either need to stop mounting the container filesystem as readonly
(remove `--read-only`) or to allow writing via a `tmpfs`.
I've gone the `tmpfs` route which seems to work.
In any case, the mounted source directories (`matrix_backup_borg_location_source_directories`)
are read-only regardless, so our actual source files are protected from unintentional changes.
Without this, it's a string and borg says:
> At 'hooks.postgresql_databases[INDEX_HERE].port': '5432' is not of type 'integer'
> /etc/borgmatic/config.yaml /etc/borgmatic.d /tmp/.config/borgmatic/config.yaml /tmp/.config/borgmatic.d: No valid configuration files found
.. and fails to do anything.
This role is usable on its own and it's not tied to Matrix, so
extracting it out into an independent role that we install via
ansible-galaxy makes sense.
This also fixes the confusion from the other day, where
`matrix_postgres_*` had to be renamed to `devture_postgres_*`
(unless it was about `matrix_postgres_backup_*`).
We now can safely say that ALL `matrix_postgres_*` variables need to be
renamed.
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2305
More details about the new key type can be found here:
https://eff-certbot.readthedocs.io/en/stable/using.html#rsa-and-ecdsa-keys
Existing RSA-based keys will continue to renew as RSA until manual
action is taken. Example from the documentation above:
> certbot renew --key-type ecdsa --cert-name example.com --force-renewal
In the future, we may add a command which does this automatically for
all domains.
* added dendrite captcha options
* added hcaptcha doc
* proper url
* Apply suggestions from code review
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Update main.yml
* renamed captcha vars to new naming scheme
* change vars to new format
* Rename back some incorrect renamed variables
These variables are either not just part of the `client_api` subsection,
or are not even part of that section at all. They shouldn't have been
renamed in baaef2ed616e2645550d9
* Fix up naming inconsistencies
Some of these variables had been renamed in one place,
but not in other places, so it couldn't have worked that way.
* Add validation/deprecation for renamed Dendrite variables
Related to 4097898f885cf4c73, baaef2ed616e2645550, 68f4418092fa8ad
and a0b4a0ae6b2f1f18
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
- forego removing Docker images - it's not effective anyway, because it
only removes the last version.. which is a drop in the bucket, usually
- do not reload systemd - it's none of our business. `--tags=start`,
etc., handle this
- combine all uninstall tasks under a single block, which only runs if
we detect traces (a leftover systemd .service file) of the component.
If no such .service is detected, we skip them all. This may lead to
incorect cleanup in rare cases, but is good enough for the most part.
* Exempt Matrix server from ntfy rate limit
Add the matrix fqdn and localhost to ntfy's exemption list.
Also allow all ntfy rate limits to be configured through Ansible
variables.
* Fix names and formatting
* fixes
* tabs not spaces
* Lint
* Use raw tags instead of bracket soup
We had checks to avoid stopping/deleting systemd services for workers
that used to exist and will continue to exist, but we were deleting
config files for workers each time.. Only to recreate them again later.
This lead to:
- too many misleading "changed" tasks
- too much unnecessary work
- potential failures during playbook execution possibly leaving the
system in a bad state (no worker config files)
We need this to control whether `('matrix-' + matrix_homeserver_implementation + '.service')`
would get injected into `devture_systemd_service_manager_services_list_auto`
This was useful when the order of these roles in relation to Synapse
mattered (when we were injecting stuff into Synapse variables during
runtime). This is no longer the case since 0ea7cb5d18, so all of
this can be removed.
These `init.yml` (now `inject_into_nginx_proxy.yml`) tasks do not need
to `always` run. They only need to run for `setup-all` and
`setup-nginx-proxy`. Unless we're dealing with these 2 tags, we can
spare ourselves a lot of work.
This patch also moves the `when` statement from `init.yml` into
`main.yml` in an effort to further optimize things by potentially
avoiding the extra file include.
These are not even caused by Archlinux, but by running buggy Ansible on old Ubuntu
while targeting modern servers (like Archlinux, but also others, ..).
We shouldn't employ ugly workarounds like this. We should tell people to
avoid running buggy Ansible or bad distros like Ubuntu, even.
* Enable location sharing in Element
* Update roles/custom/matrix-client-element/tasks/validate_config.yml
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Update roles/custom/matrix-client-element/tasks/setup_install.yml
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Rename location sharing vars to be consistent with other vars
* Rename style.json to map_style.json
* Add m.tile_server section to /.well-known/matrix/client
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Add task to configure a standalone JVB on a different server
* add missing file
* set nginx config
* update prosody file and expose port 5222
* change variable name to server id
* formatting change
* use server id of jvb-1 for the main server
* adding documentation
* adding more jvbs
* rename variable
* revert file
* fix yaml error
* minor doc fixes
* renaming tags and introducing a common tag
* remove duplicates
* add mapping for jvb to hostname/ip
* missed a jvb_server
* Update roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-jitsi.conf.j2
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* PR review comments and additional documentation
* iterate on dict items
* Update docs/configuring-playbook-jitsi.md
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Update docs/configuring-playbook-jitsi.md
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Update docs/configuring-playbook-jitsi.md
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Update docs/configuring-playbook-jitsi.md
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Update docs/configuring-playbook-jitsi.md
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Update docs/configuring-playbook-jitsi.md
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Update docs/configuring-playbook-jitsi.md
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* adding documentation around the xmpp setting
* add common after
* reduce the number of services during init of the additional jvb
* remove rogue i
* revert change to jitsi init as it's needed
* only run the jvb service on the additional jvb host
* updating docs
* reset default and add documentation about the websocket port
* fix issue rather merge with master
* add missing role introduced in master
* this role is required too
* Adding new jitsi jvb playbook, moving setup.yml to matrix.yml and creating soft link
* updating documentation
* revert accidental change to file
* add symlink back to roles to aid running of the jitsi playbook
* Remove extra space
* Delete useless playbooks/roles symlink
* Remove blank lines
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Jitsi control sentry dns using vars
* renaming variables
* Revert "renaming variables"
This reverts commit 4146c48f6a.
* set to connection string or 0 to disable
* Update comments
* Use empty string for default Sentry DSN variables
Both should work identically, but an empty string seems better
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
This fixes a regression since the change done in c1c152f7ac.
When another role (say `matrix-jitsi`) included `roles/custom/matrix-base/tasks/util/ensure_openssl_installed.yml`,
which then included `{{ role_path }}/tasks/util/ensure_openssl_installed_DISTRO.yml`,
that `role_path` variable would end up being the parent role
(`matrix-jitsi`) and not the `matrix-base` role, so we'd get a failure.
An alternative solution may have been to avoid using `role_path`, but
importing roles properly (like we've done in this patch) sounds like a better way.
Unfortunately, `import_role` fails if `tasks_from` is something like
`util/ensure_openssl_installed` (containing a `/`), so I had to move
these utils out of `util/`.
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2228
We no longer ask users to create Matrix user accounts for these bots:
- Postmoogle
- Honoroit
- Reminder Bot
Other bots and services (matrix-registration-bot, maubot, mjolnir,
Dimension, etc.) require an Access Token to run (not a password),
so this new role doesn't help for them.
It does help for the above bots though, and for defining your own
"initial user accounts" in the `matrix_user_creator_users_additional`
variable.
Dendrite uses a lot of databases, but a single (`dendrite`) role, which
leads to `matrix_postgres_import_roles_to_ignore` being something like
`['dendrite', 'dendrite', 'dendrite', ...]` needlessly.
This leads to weird regexes being generated for
`matrix_postgres_import_roles_ignore_regex`.
It's not that it hurts, but it just looks odd.
We were only reporting failures for when the async task didn't finish.
We also need to report a failure for when the task finished, but
returned a non-zero exit code.
