matrix-docker-ansible-deploy/docs/configuring-playbook-backup-borg.md

94 lines
5.7 KiB
Markdown
Raw Normal View History

<!--
SPDX-FileCopyrightText: 2024 MDAD Team and contributors
SPDX-License-Identifier: AGPL-3.0-or-later
-->
# Setting up BorgBackup (optional)
The playbook can install and configure [BorgBackup](https://www.borgbackup.org/) (short: Borg) with [borgmatic](https://torsion.org/borgmatic/) for you.
BorgBackup is a deduplicating backup program with optional compression and encryption. That means your daily incremental backups can be stored in a fraction of the space and is safe whether you store it at home or on a cloud service.
You will need a remote server where BorgBackup will store the backups. There are hosted, BorgBackup compatible solutions available, such as [BorgBase](https://www.borgbase.com).
2022-04-12 20:14:12 +02:00
The backup will run based on `backup_borg_schedule` var (systemd timer calendar), default: 4am every day.
By default, if you're using the integrated Postgres database server (as opposed to [an external Postgres server](configuring-playbook-external-postgres.md)), backups with BorgBackup will also include dumps of your Postgres database. An alternative solution for backing up the Postgres database is [postgres backup](configuring-playbook-postgres-backup.md). If you decide to go with another solution, you can disable Postgres-backup support for BorgBackup using the `backup_borg_postgresql_enabled` variable.
**Note**: the component is not managed by this repository but its [own repository](https://github.com/mother-of-all-self-hosting/ansible-role-backup_borg).
## Prerequisites
1. If you do not disable Postgres-backup support, make sure that the Postgres version of your homeserver's database is compatible with borgmatic.
2. Create a new SSH key:
```sh
ssh-keygen -t ed25519 -N '' -f matrix-borg-backup -C matrix
```
This can be done on any machine and you don't need to place the key in the `.ssh` folder. It will be added to the Ansible config later.
2022-04-12 20:14:12 +02:00
3. Add the **public** part of this SSH key (the `matrix-borg-backup.pub` file) to your BorgBackup provider/server:
2022-04-12 20:14:12 +02:00
If you plan to use a hosted solution, follow their instructions. If you have your own server, copy the key over:
```sh
# example to append the new PUBKEY contents, where:
# PUBKEY is path to the public key,
# USER is a ssh user on a provider / server
# HOST is a ssh host of a provider / server
cat PUBKEY | ssh USER@HOST 'dd of=.ssh/authorized_keys oflag=append conv=notrunc'
```
## Adjusting the playbook configuration
Minimal working configuration (`inventory/host_vars/matrix.example.com/vars.yml`) to enable BorgBackup:
```yaml
backup_borg_enabled: true
backup_borg_location_repositories:
- ssh://USER@HOST/./REPO
backup_borg_storage_encryption_passphrase: "PASSPHRASE"
backup_borg_ssh_key_private: |
-----BEGIN OPENSSH PRIVATE KEY-----
TG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVyIGFkaXBpc2NpbmcgZW
xpdCwgc2VkIGRvIGVpdXNtb2QgdGVtcG9yIGluY2lkaWR1bnQgdXQgbGFib3JlIGV0IGRv
bG9yZSBtYWduYSBhbGlxdWEuIFV0IGVuaW0gYWQgbWluaW0gdmVuaWFtLCBxdWlzIG5vc3
RydWQgZXhlcmNpdGF0aW9uIHVsbGFtY28gbGFib3JpcyBuaXNpIHV0IGFsaXF1aXAgZXgg
ZWEgY29tbW9kbyBjb25zZXF1YXQuIA==
-----END OPENSSH PRIVATE KEY-----
```
where:
2022-04-12 20:14:12 +02:00
* USER - SSH user of a provider/server
* HOST - SSH host of a provider/server
* REPO - BorgBackup repository name, it will be initialized on backup start, eg: `matrix`, regarding Syntax see [Remote repositories](https://borgbackup.readthedocs.io/en/stable/usage/general.html#repository-urls)
2022-04-12 20:14:12 +02:00
* PASSPHRASE - passphrase used for encrypting backups, you may generate it with `pwgen -s 64 1` or use any password manager
* PRIVATE KEY - the content of the **private** part of the SSH key you created before. The whole key (all of its belonging lines) under `backup_borg_ssh_key_private` needs to be indented with 2 spaces
2022-04-12 20:14:12 +02:00
To backup without encryption, add `backup_borg_encryption: 'none'` to your vars. This will also enable the `backup_borg_unknown_unencrypted_repo_access_is_ok` variable.
`backup_borg_location_source_directories` defines the list of directories to back up: it's set to `{{ matrix_base_data_path }}` by default, which is the base directory for every service's data, such as Synapse, Postgres and the bridges. You might want to exclude certain directories or file patterns from the backup using the `backup_borg_location_exclude_patterns` variable.
Check the [backup_borg role](https://github.com/mother-of-all-self-hosting/ansible-role-backup_borg)'s [defaults/main.yml](https://github.com/mother-of-all-self-hosting/ansible-role-backup_borg/blob/main/defaults/main.yml) file for the full list of available options.
## Installing
Edit descriptions about installation of components (#3842) * Replace installation command shortcut for the "just" program with the most conservative raw ansible-playbook command This commit replaces installation command shortcut ("recipe") for the "just" program with the raw ansible-playbook command, so that the shortcut will be added to it later. The command is so conservative that failure of the command will mean something is clearly broken. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add comments about using setup-all instead of install-all Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add description about shortcut command with the "just" program to the ansible-playbook command with "setup-all" and "start" tags It also explains difference between "just install-all" and "just setup-all" recipes. The explanation is based on docs/playbook-tags.md Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update raw ansible-playbook command to have it do what "just install-all" or "just setup-all" does Since "just install-all" or "just setup-all" invokes "ensure-matrix-users-created" as well, it needs adding to the raw ansible-playbook command. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Remove "ensure-matrix-users-created" from the raw ansible-playbook command which does not need it Also: update the "just" recipes accordingly. "just install-all" and "just setup-all" run "ensure-matrix-users-created" tag as well, therefore they need to be replaced with "run-tags" recipes to skip "ensure-matrix-users-created" Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-etherpad.md: add ensure-matrix-users-created to the raw ansible-playbook Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add description about "ensure-matrix-users-created" and create a list with description about shortcut commands with "just" This commit also fixes list item capitalization and punctuation. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add notes bullet lists Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-matrix-corporal.md and docs/configuring-playbook-email2matrix.md: adopt common instructions Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Replace "run the installation command" with "run the playbook with tags" Now that shortcut commands for the "just" program are displayed along with the existing "installation command", this commit replaces "run the installation command" with "run the playbook with tags" in order to prevent misunderstanding and confusion. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add notes about changing passwords of users specified on vars.yml Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-synapse-admin.md: add the playbook command and just recipes Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Remove redundant blank lines Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-alertmanager-receiver.md: remove the direction to proceed to Usage Such a kind of direction is not used on other documentation, so it should be fine to just remove it. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/importing-synapse-media-store.md: code block for ansible-playbook Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> --------- Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> Co-authored-by: Suguru Hirahara <acioustick@noreply.codeberg.org>
2024-12-01 08:42:30 +01:00
After configuring the playbook, run it with [playbook tags](playbook-tags.md) as below:
Edit descriptions about installation of components (#3842) * Replace installation command shortcut for the "just" program with the most conservative raw ansible-playbook command This commit replaces installation command shortcut ("recipe") for the "just" program with the raw ansible-playbook command, so that the shortcut will be added to it later. The command is so conservative that failure of the command will mean something is clearly broken. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add comments about using setup-all instead of install-all Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add description about shortcut command with the "just" program to the ansible-playbook command with "setup-all" and "start" tags It also explains difference between "just install-all" and "just setup-all" recipes. The explanation is based on docs/playbook-tags.md Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update raw ansible-playbook command to have it do what "just install-all" or "just setup-all" does Since "just install-all" or "just setup-all" invokes "ensure-matrix-users-created" as well, it needs adding to the raw ansible-playbook command. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Remove "ensure-matrix-users-created" from the raw ansible-playbook command which does not need it Also: update the "just" recipes accordingly. "just install-all" and "just setup-all" run "ensure-matrix-users-created" tag as well, therefore they need to be replaced with "run-tags" recipes to skip "ensure-matrix-users-created" Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-etherpad.md: add ensure-matrix-users-created to the raw ansible-playbook Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add description about "ensure-matrix-users-created" and create a list with description about shortcut commands with "just" This commit also fixes list item capitalization and punctuation. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add notes bullet lists Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-matrix-corporal.md and docs/configuring-playbook-email2matrix.md: adopt common instructions Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Replace "run the installation command" with "run the playbook with tags" Now that shortcut commands for the "just" program are displayed along with the existing "installation command", this commit replaces "run the installation command" with "run the playbook with tags" in order to prevent misunderstanding and confusion. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add notes about changing passwords of users specified on vars.yml Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-synapse-admin.md: add the playbook command and just recipes Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Remove redundant blank lines Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-alertmanager-receiver.md: remove the direction to proceed to Usage Such a kind of direction is not used on other documentation, so it should be fine to just remove it. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/importing-synapse-media-store.md: code block for ansible-playbook Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> --------- Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> Co-authored-by: Suguru Hirahara <acioustick@noreply.codeberg.org>
2024-12-01 08:42:30 +01:00
<!-- NOTE: let this conservative command run (instead of install-all) to make it clear that failure of the command means something is clearly broken. -->
```sh
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
```
The shortcut commands with the [`just` program](just.md) are also available: `just install-all` or `just setup-all`
Edit descriptions about installation of components (#3842) * Replace installation command shortcut for the "just" program with the most conservative raw ansible-playbook command This commit replaces installation command shortcut ("recipe") for the "just" program with the raw ansible-playbook command, so that the shortcut will be added to it later. The command is so conservative that failure of the command will mean something is clearly broken. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add comments about using setup-all instead of install-all Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add description about shortcut command with the "just" program to the ansible-playbook command with "setup-all" and "start" tags It also explains difference between "just install-all" and "just setup-all" recipes. The explanation is based on docs/playbook-tags.md Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update raw ansible-playbook command to have it do what "just install-all" or "just setup-all" does Since "just install-all" or "just setup-all" invokes "ensure-matrix-users-created" as well, it needs adding to the raw ansible-playbook command. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Remove "ensure-matrix-users-created" from the raw ansible-playbook command which does not need it Also: update the "just" recipes accordingly. "just install-all" and "just setup-all" run "ensure-matrix-users-created" tag as well, therefore they need to be replaced with "run-tags" recipes to skip "ensure-matrix-users-created" Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-etherpad.md: add ensure-matrix-users-created to the raw ansible-playbook Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add description about "ensure-matrix-users-created" and create a list with description about shortcut commands with "just" This commit also fixes list item capitalization and punctuation. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add notes bullet lists Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-matrix-corporal.md and docs/configuring-playbook-email2matrix.md: adopt common instructions Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Replace "run the installation command" with "run the playbook with tags" Now that shortcut commands for the "just" program are displayed along with the existing "installation command", this commit replaces "run the installation command" with "run the playbook with tags" in order to prevent misunderstanding and confusion. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add notes about changing passwords of users specified on vars.yml Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-synapse-admin.md: add the playbook command and just recipes Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Remove redundant blank lines Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-alertmanager-receiver.md: remove the direction to proceed to Usage Such a kind of direction is not used on other documentation, so it should be fine to just remove it. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/importing-synapse-media-store.md: code block for ansible-playbook Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> --------- Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> Co-authored-by: Suguru Hirahara <acioustick@noreply.codeberg.org>
2024-12-01 08:42:30 +01:00
`just install-all` is useful for maintaining your setup quickly ([2x-5x faster](../CHANGELOG.md#2x-5x-performance-improvements-in-playbook-runtime) than `just setup-all`) when its components remain unchanged. If you adjust your `vars.yml` to remove other components, you'd need to run `just setup-all`, or these components will still remain installed. Note these shortcuts run the `ensure-matrix-users-created` tag too.
Edit descriptions about installation of components (#3842) * Replace installation command shortcut for the "just" program with the most conservative raw ansible-playbook command This commit replaces installation command shortcut ("recipe") for the "just" program with the raw ansible-playbook command, so that the shortcut will be added to it later. The command is so conservative that failure of the command will mean something is clearly broken. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add comments about using setup-all instead of install-all Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add description about shortcut command with the "just" program to the ansible-playbook command with "setup-all" and "start" tags It also explains difference between "just install-all" and "just setup-all" recipes. The explanation is based on docs/playbook-tags.md Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update raw ansible-playbook command to have it do what "just install-all" or "just setup-all" does Since "just install-all" or "just setup-all" invokes "ensure-matrix-users-created" as well, it needs adding to the raw ansible-playbook command. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Remove "ensure-matrix-users-created" from the raw ansible-playbook command which does not need it Also: update the "just" recipes accordingly. "just install-all" and "just setup-all" run "ensure-matrix-users-created" tag as well, therefore they need to be replaced with "run-tags" recipes to skip "ensure-matrix-users-created" Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-etherpad.md: add ensure-matrix-users-created to the raw ansible-playbook Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add description about "ensure-matrix-users-created" and create a list with description about shortcut commands with "just" This commit also fixes list item capitalization and punctuation. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add notes bullet lists Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-matrix-corporal.md and docs/configuring-playbook-email2matrix.md: adopt common instructions Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Replace "run the installation command" with "run the playbook with tags" Now that shortcut commands for the "just" program are displayed along with the existing "installation command", this commit replaces "run the installation command" with "run the playbook with tags" in order to prevent misunderstanding and confusion. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Add notes about changing passwords of users specified on vars.yml Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-synapse-admin.md: add the playbook command and just recipes Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Remove redundant blank lines Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/configuring-playbook-alertmanager-receiver.md: remove the direction to proceed to Usage Such a kind of direction is not used on other documentation, so it should be fine to just remove it. Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> * Update docs/importing-synapse-media-store.md: code block for ansible-playbook Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> --------- Signed-off-by: Suguru Hirahara <acioustick@noreply.codeberg.org> Co-authored-by: Suguru Hirahara <acioustick@noreply.codeberg.org>
2024-12-01 08:42:30 +01:00
## Manually start a backup
For testing your setup it can be helpful to not wait until 4am. If you want to run the backup immediately, log onto the server and run `systemctl start matrix-backup-borg`. This will not return until the backup is done, so possibly a long time. Consider using [tmux](https://en.wikipedia.org/wiki/Tmux) if your SSH connection is unstable.