From 78c35136b2cff4a6f24d01077c2fc19c36f1a878 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Mon, 13 Feb 2023 10:53:11 +0200 Subject: [PATCH] Replace matrix-backup-borg with an external role --- docs/configuring-playbook-backup-borg.md | 18 +-- group_vars/matrix_servers | 38 ++++--- playbooks/matrix.yml | 2 +- requirements.yml | 3 + .../matrix-backup-borg/defaults/main.yml | 107 ------------------ .../custom/matrix-backup-borg/tasks/main.yml | 20 ---- .../tasks/setup_install.yml | 107 ------------------ .../tasks/setup_uninstall.yml | 25 ---- .../tasks/validate_config.yml | 16 --- .../templates/config.yaml.j2 | 43 ------- .../matrix-backup-borg/templates/passwd.j2 | 29 ----- .../matrix-backup-borg/templates/sshkey.j2 | 1 - .../systemd/matrix-backup-borg.service.j2 | 65 ----------- .../systemd/matrix-backup-borg.timer.j2 | 10 -- .../defaults/main.yml | 3 + .../tasks/validate_config.yml | 17 +++ 16 files changed, 57 insertions(+), 447 deletions(-) delete mode 100644 roles/custom/matrix-backup-borg/defaults/main.yml delete mode 100644 roles/custom/matrix-backup-borg/tasks/main.yml delete mode 100644 roles/custom/matrix-backup-borg/tasks/setup_install.yml delete mode 100644 roles/custom/matrix-backup-borg/tasks/setup_uninstall.yml delete mode 100644 roles/custom/matrix-backup-borg/tasks/validate_config.yml delete mode 100644 roles/custom/matrix-backup-borg/templates/config.yaml.j2 delete mode 100644 roles/custom/matrix-backup-borg/templates/passwd.j2 delete mode 100644 roles/custom/matrix-backup-borg/templates/sshkey.j2 delete mode 100644 roles/custom/matrix-backup-borg/templates/systemd/matrix-backup-borg.service.j2 delete mode 100644 roles/custom/matrix-backup-borg/templates/systemd/matrix-backup-borg.timer.j2 diff --git a/docs/configuring-playbook-backup-borg.md b/docs/configuring-playbook-backup-borg.md index b6b5cb096..2f6ab5ad3 100644 --- a/docs/configuring-playbook-backup-borg.md +++ b/docs/configuring-playbook-backup-borg.md @@ -6,9 +6,9 @@ That means your daily incremental backups can be stored in a fraction of the spa You will need a remote server where borg will store the backups. There are hosted, borg compatible solutions available, such as [BorgBase](https://www.borgbase.com). -The backup will run based on `matrix_backup_borg_schedule` var (systemd timer calendar), default: 4am every day. +The backup will run based on `backup_borg_schedule` var (systemd timer calendar), default: 4am every day. -By default, if you're using the integrated Postgres database server (as opposed to [an external Postgres server](configuring-playbook-external-postgres.md)), Borg backups will also include dumps of your Postgres database. An alternative solution for backing up the Postgres database is [postgres backup](configuring-playbook-postgres-backup.md). If you decide to go with another solution, you can disable Postgres-backup support for Borg using the `matrix_backup_borg_postgresql_enabled` variable. +By default, if you're using the integrated Postgres database server (as opposed to [an external Postgres server](configuring-playbook-external-postgres.md)), Borg backups will also include dumps of your Postgres database. An alternative solution for backing up the Postgres database is [postgres backup](configuring-playbook-postgres-backup.md). If you decide to go with another solution, you can disable Postgres-backup support for Borg using the `backup_borg_postgresql_enabled` variable. ## Prerequisites @@ -38,11 +38,11 @@ cat PUBKEY | ssh USER@HOST 'dd of=.ssh/authorized_keys oflag=append conv=notrunc Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable borg backup: ```yaml -matrix_backup_borg_enabled: true -matrix_backup_borg_location_repositories: +backup_borg_enabled: true +backup_borg_location_repositories: - ssh://USER@HOST/./REPO -matrix_backup_borg_storage_encryption_passphrase: "PASSPHRASE" -matrix_backup_borg_ssh_key_private: | +backup_borg_storage_encryption_passphrase: "PASSPHRASE" +backup_borg_ssh_key_private: | -----BEGIN OPENSSH PRIVATE KEY----- TG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVyIGFkaXBpc2NpbmcgZW xpdCwgc2VkIGRvIGVpdXNtb2QgdGVtcG9yIGluY2lkaWR1bnQgdXQgbGFib3JlIGV0IGRv @@ -58,11 +58,11 @@ where: * HOST - SSH host of a provider/server * REPO - borg repository name, it will be initialized on backup start, eg: `matrix`, regarding Syntax see [Remote repositories](https://borgbackup.readthedocs.io/en/stable/usage/general.html#repository-urls) * PASSPHRASE - passphrase used for encrypting backups, you may generate it with `pwgen -s 64 1` or use any password manager -* PRIVATE KEY - the content of the **private** part of the SSH key you created before. The whole key (all of its belonging lines) under `matrix_backup_borg_ssh_key_private` needs to be indented with 2 spaces +* PRIVATE KEY - the content of the **private** part of the SSH key you created before. The whole key (all of its belonging lines) under `backup_borg_ssh_key_private` needs to be indented with 2 spaces -To backup without encryption, add `matrix_backup_borg_encryption: 'none'` to your vars. This will also enable the `matrix_backup_borg_unknown_unencrypted_repo_access_is_ok` variable. +To backup without encryption, add `backup_borg_encryption: 'none'` to your vars. This will also enable the `backup_borg_unknown_unencrypted_repo_access_is_ok` variable. -`matrix_backup_borg_location_source_directories` defines the list of directories to back up: it's set to `{{ matrix_base_data_path }}` by default, which is the base directory for every service's data, such as Synapse, Postgres and the bridges. You might want to exclude certain directories or file patterns from the backup using the `matrix_backup_borg_location_exclude_patterns` variable. +`backup_borg_location_source_directories` defines the list of directories to back up: it's set to `{{ matrix_base_data_path }}` by default, which is the base directory for every service's data, such as Synapse, Postgres and the bridges. You might want to exclude certain directories or file patterns from the backup using the `backup_borg_location_exclude_patterns` variable. Check the `roles/custom/matrix-backup-borg/defaults/main.yml` file for the full list of available options. diff --git a/group_vars/matrix_servers b/group_vars/matrix_servers index 1b31bf00d..fbae03046 100755 --- a/group_vars/matrix_servers +++ b/group_vars/matrix_servers @@ -204,7 +204,7 @@ matrix_homeserver_app_service_config_files_auto: | # because it requires SSL certificates. If Traefik is used, then Postmoogle needs to start at least after Traefik (3500) and Traefik certs dumper (3500). devture_systemd_service_manager_services_list_auto: | {{ - ([{'name': 'matrix-backup-borg.timer', 'priority': 5000, 'groups': ['matrix', 'backup', 'borg']}] if matrix_backup_borg_enabled else []) + ([{'name': (backup_borg_identifier + '.timer'), 'priority': 5000, 'groups': ['matrix', 'backup', 'borg']}] if backup_borg_enabled else []) + ([{'name': 'matrix-bot-buscarron.service', 'priority': 2200, 'groups': ['matrix', 'bots', 'buscarron']}] if matrix_bot_buscarron_enabled else []) + @@ -1871,34 +1871,42 @@ matrix_bot_draupnir_systemd_required_services_list: | # ###################################################################### + ###################################################################### # -# matrix-backup-borg +# etke/backup_borg # ###################################################################### -matrix_backup_borg_enabled: false +backup_borg_enabled: false -matrix_backup_borg_container_image_self_build: "{{ matrix_architecture not in ['amd64', 'arm32', 'arm64'] }}" +backup_borg_identifier: matrix-backup-borg -matrix_backup_borg_postgresql_enabled: "{{ devture_postgres_enabled }}" -matrix_backup_borg_postgresql_databases_hostname: "{{ devture_postgres_connection_hostname if devture_postgres_enabled else '' }}" -matrix_backup_borg_postgresql_databases_username: "{{ devture_postgres_connection_username if devture_postgres_enabled else '' }}" -matrix_backup_borg_postgresql_databases_password: "{{ devture_postgres_connection_password if devture_postgres_enabled else '' }}" -matrix_backup_borg_postgresql_databases_port: "{{ devture_postgres_connection_port if devture_postgres_enabled else 5432 }}" -matrix_backup_borg_postgresql_databases: "{{ devture_postgres_managed_databases | map(attribute='name') if devture_postgres_enabled else [] }}" +backup_borg_uid: "{{ matrix_user_uid }}" +backup_borg_gid: "{{ matrix_user_gid }}" -matrix_backup_borg_location_source_directories: +backup_borg_container_network: "{{ devture_postgres_container_network if devture_postgres_enabled else backup_borg_identifier }}" + +backup_borg_container_image_self_build: "{{ matrix_architecture not in ['amd64', 'arm32', 'arm64'] }}" + +backup_borg_postgresql_enabled: "{{ devture_postgres_enabled }}" +backup_borg_postgresql_databases_hostname: "{{ devture_postgres_connection_hostname if devture_postgres_enabled else '' }}" +backup_borg_postgresql_databases_username: "{{ devture_postgres_connection_username if devture_postgres_enabled else '' }}" +backup_borg_postgresql_databases_password: "{{ devture_postgres_connection_password if devture_postgres_enabled else '' }}" +backup_borg_postgresql_databases_port: "{{ devture_postgres_connection_port if devture_postgres_enabled else 5432 }}" +backup_borg_postgresql_databases: "{{ devture_postgres_managed_databases | map(attribute='name') if devture_postgres_enabled else [] }}" + +backup_borg_location_source_directories: - "{{ matrix_base_data_path }}" -matrix_backup_borg_location_exclude_patterns: | +backup_borg_location_exclude_patterns: | {{ ([matrix_synapse_media_store_path + '/local_thumbnails', matrix_synapse_media_store_path + '/remote_thumbnail', matrix_synapse_media_store_path + '/url_cache', matrix_synapse_media_store_path + '/url_cache_thumbnails'] if matrix_homeserver_implementation == 'synapse' else []) + ([devture_postgres_data_path] if devture_postgres_enabled else []) }} -matrix_backup_borg_systemd_required_services_list: | +backup_borg_systemd_required_services_list: | {{ ['docker.service'] + @@ -1907,9 +1915,11 @@ matrix_backup_borg_systemd_required_services_list: | ###################################################################### # -# /matrix-backup-borg +# /etke/backup_borg # ###################################################################### + + ###################################################################### # # matrix-cactus-comments diff --git a/playbooks/matrix.yml b/playbooks/matrix.yml index 508adb4d7..d976a0126 100755 --- a/playbooks/matrix.yml +++ b/playbooks/matrix.yml @@ -112,7 +112,7 @@ - role: galaxy/com.devture.ansible.role.postgres_backup - - custom/matrix-backup-borg + - role: galaxy/backup_borg - custom/matrix-user-creator - custom/matrix-common-after diff --git a/requirements.yml b/requirements.yml index 1341bbbe2..754491244 100644 --- a/requirements.yml +++ b/requirements.yml @@ -36,6 +36,9 @@ - src: git+https://gitlab.com/etke.cc/roles/prometheus_postgres_exporter.git version: v0.11.1-2 +- src: git+https://gitlab.com/etke.cc/roles/backup_borg.git + version: v1.2.3-1.7.5-0 + - src: git+https://github.com/devture/com.devture.ansible.role.traefik.git version: fb09fd26f877372417d5586f1e79e83f983f0bd6 diff --git a/roles/custom/matrix-backup-borg/defaults/main.yml b/roles/custom/matrix-backup-borg/defaults/main.yml deleted file mode 100644 index ad5a02811..000000000 --- a/roles/custom/matrix-backup-borg/defaults/main.yml +++ /dev/null @@ -1,107 +0,0 @@ ---- -# Project source code URL: https://gitlab.com/etke.cc/borgmatic - -matrix_backup_borg_enabled: true - -matrix_backup_borg_base_path: "{{ matrix_base_data_path }}/backup-borg" -matrix_backup_borg_config_path: "{{ matrix_backup_borg_base_path }}/config" - -matrix_backup_borg_container_image_self_build: false -matrix_backup_borg_docker_repo: "https://gitlab.com/etke.cc/borgmatic" -matrix_backup_borg_docker_repo_version: main -matrix_backup_borg_docker_src_files_path: "{{ matrix_backup_borg_base_path }}/docker-src" - -# image and postgres version determined automatically, based on detected postgres server version (if enabled), otherwise latest is used -matrix_backup_borg_version: "" -matrix_backup_borg_postgres_version: "" -matrix_backup_borg_borg_version: 1.2.3 -matrix_backup_borg_borgmatic_version: 1.7.6 -matrix_backup_borg_docker_image: "{{ matrix_backup_borg_docker_image_name_prefix }}etke.cc/borgmatic:{{ matrix_backup_borg_version }}" -matrix_backup_borg_docker_image_name_prefix: "{{ 'localhost/' if matrix_backup_borg_container_image_self_build else 'registry.gitlab.com/' }}" -matrix_backup_borg_docker_image_force_pull: "{{ matrix_backup_borg_docker_image.endswith(':latest') or matrix_backup_borg_version | default('') == '' }}" - -# A list of extra arguments to pass to the container -matrix_backup_borg_container_extra_arguments: [] - -# List of systemd services that matrix-backup-borg.service depends on -matrix_backup_borg_systemd_required_services_list: ['docker.service'] - -# List of systemd services that matrix-backup-borg.service wants -matrix_backup_borg_systemd_wanted_services_list: [] - -# systemd calendar configuration for the backup job -# the actual job may run with a delay (see matrix_backup_borg_schedule_randomized_delay_sec) -matrix_backup_borg_schedule: "*-*-* 04:00:00" -# the delay with which the systemd timer may run in relation to the `matrix_backup_borg_schedule` schedule -matrix_backup_borg_schedule_randomized_delay_sec: 2h - -# what directories should be added to backup -matrix_backup_borg_location_source_directories: [] - -# postgres db backup -matrix_backup_borg_postgresql_enabled: true -matrix_backup_borg_supported_postgres_versions: ['12', '13', '14', '15'] -matrix_backup_borg_postgresql_databases: [] -matrix_backup_borg_postgresql_databases_hostname: '' -matrix_backup_borg_postgresql_databases_username: "matrix" -matrix_backup_borg_postgresql_databases_password: "" -matrix_backup_borg_postgresql_databases_port: 5432 - -# target repositories -matrix_backup_borg_location_repositories: [] - -# exclude following paths: -matrix_backup_borg_location_exclude_patterns: [] - -# borg encryption mode, only "repokey-*" and "none" are supported -matrix_backup_borg_encryption: repokey-blake2 - -# private ssh key used to connect to the borg repo -matrix_backup_borg_ssh_key_private: "" - -# allow unencrypted repo access -matrix_backup_borg_unknown_unencrypted_repo_access_is_ok: "{{ matrix_backup_borg_encryption == 'none' }}" - -# borg ssh command with ssh key -matrix_backup_borg_storage_ssh_command: ssh -o "StrictHostKeyChecking accept-new" -i /etc/borgmatic.d/sshkey - -# compression algorithm -matrix_backup_borg_storage_compression: lz4 - -# archive name format -matrix_backup_borg_storage_archive_name_format: matrix-{now:%Y-%m-%d-%H%M%S} - -# repository passphrase -matrix_backup_borg_storage_encryption_passphrase: "" - -# retention configuration -matrix_backup_borg_retention_keep_hourly: 0 -matrix_backup_borg_retention_keep_daily: 7 -matrix_backup_borg_retention_keep_weekly: 4 -matrix_backup_borg_retention_keep_monthly: 12 -matrix_backup_borg_retention_keep_yearly: 2 - -# retention prefix -matrix_backup_borg_retention_prefix: matrix- - -# Default borgmatic configuration template which covers the generic use case. -# You can customize it by controlling the various variables inside it. -# -# For a more advanced customization, you can extend the default (see `matrix_backup_borg_configuration_extension_yaml`) -# or completely replace this variable with your own template. -matrix_backup_borg_configuration_yaml: "{{ lookup('template', 'templates/config.yaml.j2') }}" - -matrix_backup_borg_configuration_extension_yaml: | - # Your custom YAML configuration for borgmatic goes here. - # This configuration extends the default starting configuration (`matrix_borg_configuration_yaml`). - # - # You can override individual variables from the default configuration, or introduce new ones. - # - # If you need something more special, you can take full control by - # completely redefining `matrix_backup_borg_configuration_yaml`. - -matrix_backup_borg_configuration_extension: "{{ matrix_backup_borg_configuration_extension_yaml | from_yaml if matrix_backup_borg_configuration_extension_yaml | from_yaml is mapping else {} }}" - -# Holds the final borgmatic configuration (a combination of the default and its extension). -# You most likely don't need to touch this variable. Instead, see `matrix_backup_borg_configuration_yaml`. -matrix_backup_borg_configuration: "{{ matrix_backup_borg_configuration_yaml | from_yaml | combine(matrix_backup_borg_configuration_extension, recursive=True) }}" diff --git a/roles/custom/matrix-backup-borg/tasks/main.yml b/roles/custom/matrix-backup-borg/tasks/main.yml deleted file mode 100644 index 88d120c04..000000000 --- a/roles/custom/matrix-backup-borg/tasks/main.yml +++ /dev/null @@ -1,20 +0,0 @@ ---- - -- block: - - ansible.builtin.include_tasks: "{{ role_path }}/tasks/validate_config.yml" - when: "run_setup | bool and matrix_backup_borg_enabled | bool" - - - ansible.builtin.include_tasks: "{{ role_path }}/tasks/setup_install.yml" - when: "run_setup | bool and matrix_backup_borg_enabled | bool" - tags: - - setup-all - - setup-backup-borg - - install-all - - install-backup-borg - -- block: - - ansible.builtin.include_tasks: "{{ role_path }}/tasks/setup_uninstall.yml" - when: "run_setup | bool and not matrix_backup_borg_enabled | bool" - tags: - - setup-all - - setup-backup-borg diff --git a/roles/custom/matrix-backup-borg/tasks/setup_install.yml b/roles/custom/matrix-backup-borg/tasks/setup_install.yml deleted file mode 100644 index 41d953716..000000000 --- a/roles/custom/matrix-backup-borg/tasks/setup_install.yml +++ /dev/null @@ -1,107 +0,0 @@ ---- - -- when: matrix_backup_borg_postgresql_enabled | bool and matrix_backup_borg_version == '' - block: - - name: Fail with matrix_backup_borg_version advice if Postgres not enabled - ansible.builtin.fail: - msg: >- - You are not running a built-in Postgres server (`devture_postgres_enabled: false`), so auto-detecting its version and setting `matrix_backup_borg_version` automatically based on that cannot happen. - Consider setting `matrix_backup_borg_version` to your Postgres version manually. - when: not devture_postgres_enabled - - - ansible.builtin.include_role: - name: galaxy/com.devture.ansible.role.postgres - tasks_from: detect_existing_postgres_version - - - name: Fail if detected Postgres version is unsupported - ansible.builtin.fail: - msg: "You cannot use borg backup with such an old version ({{ devture_postgres_detected_version }}) of Postgres. Consider upgrading - link to docs for upgrading Postgres: docs/maintenance-postgres.md#upgrading-postgresql" - when: "devture_postgres_detected_version not in matrix_backup_borg_supported_postgres_versions" - - - name: Set the correct borg backup version to use - ansible.builtin.set_fact: - matrix_backup_borg_version: "{{ devture_postgres_detected_version }}-{{ matrix_backup_borg_borg_version }}-{{ matrix_backup_borg_borgmatic_version }}" - -- name: Ensure borg paths exist - ansible.builtin.file: - path: "{{ item.path }}" - state: directory - mode: 0750 - owner: "{{ matrix_user_username }}" - group: "{{ matrix_user_groupname }}" - with_items: - - {path: "{{ matrix_backup_borg_config_path }}", when: true} - - {path: "{{ matrix_backup_borg_docker_src_files_path }}", when: true} - when: "item.when | bool" - -- name: Ensure borgmatic config is created - ansible.builtin.copy: - content: "{{ matrix_backup_borg_configuration | to_nice_yaml(indent=2, width=999999) }}" - dest: "{{ matrix_backup_borg_config_path }}/config.yaml" - owner: "{{ matrix_user_username }}" - group: "{{ matrix_user_groupname }}" - mode: 0640 - -- name: Ensure borg passwd is created - ansible.builtin.template: - src: "{{ role_path }}/templates/passwd.j2" - dest: "{{ matrix_backup_borg_config_path }}/passwd" - owner: "{{ matrix_user_username }}" - group: "{{ matrix_user_groupname }}" - mode: 0640 - -- name: Ensure borg ssh key is created - ansible.builtin.template: - src: "{{ role_path }}/templates/sshkey.j2" - dest: "{{ matrix_backup_borg_config_path }}/sshkey" - owner: "{{ matrix_user_username }}" - group: "{{ matrix_user_groupname }}" - mode: 0600 - -- name: Ensure borg image is pulled - community.docker.docker_image: - name: "{{ matrix_backup_borg_docker_image }}" - source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" - force_source: "{{ matrix_backup_borg_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" - force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_backup_borg_docker_image_force_pull }}" - when: "not matrix_backup_borg_container_image_self_build | bool" - register: result - retries: "{{ devture_playbook_help_container_retries_count }}" - delay: "{{ devture_playbook_help_container_retries_delay }}" - until: result is not failed - -- name: Ensure borg repository is present on self-build - ansible.builtin.git: - repo: "{{ matrix_backup_borg_docker_repo }}" - version: "{{ matrix_backup_borg_docker_repo_version }}" - dest: "{{ matrix_backup_borg_docker_src_files_path }}" - force: "yes" - become: true - become_user: "{{ matrix_user_username }}" - register: matrix_backup_borg_git_pull_results - when: "matrix_backup_borg_container_image_self_build | bool" - -- name: Ensure borg image is built - community.docker.docker_image: - name: "{{ matrix_backup_borg_docker_image }}" - source: build - force_source: "{{ matrix_backup_borg_git_pull_results.changed if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" - force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mailer_git_pull_results.changed }}" - build: - dockerfile: Dockerfile - path: "{{ matrix_backup_borg_docker_src_files_path }}" - pull: true - when: "matrix_backup_borg_container_image_self_build | bool" - -- name: Ensure matrix-backup-borg.service installed - ansible.builtin.template: - src: "{{ role_path }}/templates/systemd/matrix-backup-borg.service.j2" - dest: "{{ devture_systemd_docker_base_systemd_path }}/matrix-backup-borg.service" - mode: 0644 - register: matrix_backup_borg_systemd_service_result - -- name: Ensure matrix-backup-borg.timer installed - ansible.builtin.template: - src: "{{ role_path }}/templates/systemd/matrix-backup-borg.timer.j2" - dest: "{{ devture_systemd_docker_base_systemd_path }}/matrix-backup-borg.timer" - mode: 0644 diff --git a/roles/custom/matrix-backup-borg/tasks/setup_uninstall.yml b/roles/custom/matrix-backup-borg/tasks/setup_uninstall.yml deleted file mode 100644 index 3d8b08ef4..000000000 --- a/roles/custom/matrix-backup-borg/tasks/setup_uninstall.yml +++ /dev/null @@ -1,25 +0,0 @@ ---- - -- name: Check existence of matrix-backup-borg service - ansible.builtin.stat: - path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-backup-borg.service" - register: matrix_backup_borg_service_stat - -- when: matrix_backup_borg_service_stat.stat.exists | bool - block: - - name: Ensure matrix-backup-borg is stopped - ansible.builtin.service: - name: matrix-backup-borg - state: stopped - enabled: false - daemon_reload: true - - - name: Ensure matrix-backup-borg.service doesn't exist - ansible.builtin.file: - path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-backup-borg.service" - state: absent - - - name: Ensure matrix-backup-borg.timer doesn't exist - ansible.builtin.file: - path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-backup-borg.timer" - state: absent diff --git a/roles/custom/matrix-backup-borg/tasks/validate_config.yml b/roles/custom/matrix-backup-borg/tasks/validate_config.yml deleted file mode 100644 index bb6cd4151..000000000 --- a/roles/custom/matrix-backup-borg/tasks/validate_config.yml +++ /dev/null @@ -1,16 +0,0 @@ ---- -- name: Fail if required backup-borg settings not defined - ansible.builtin.fail: - msg: >- - You need to define a required configuration setting (`{{ item.name }}`). - when: "item.when | bool and vars[item.name] == ''" - with_items: - - {'name': 'matrix_backup_borg_ssh_key_private', when: true} - - {'name': 'matrix_backup_borg_location_repositories', when: true} - - {'name': 'matrix_backup_borg_postgresql_databases_hostname', when: "{{ matrix_backup_borg_postgresql_enabled }}"} - -- name: Fail if encryption passphrase is undefined unless repository is unencrypted - ansible.builtin.fail: - msg: >- - You need to define a required passphrase using the `matrix_backup_borg_storage_encryption_passphrase` variable. - when: "matrix_backup_borg_storage_encryption_passphrase == '' and matrix_backup_borg_encryption != 'none'" diff --git a/roles/custom/matrix-backup-borg/templates/config.yaml.j2 b/roles/custom/matrix-backup-borg/templates/config.yaml.j2 deleted file mode 100644 index 105c0ce1f..000000000 --- a/roles/custom/matrix-backup-borg/templates/config.yaml.j2 +++ /dev/null @@ -1,43 +0,0 @@ -#jinja2: lstrip_blocks: "True", trim_blocks: "True" - -location: - source_directories: {{ matrix_backup_borg_location_source_directories|to_json }} - repositories: {{ matrix_backup_borg_location_repositories|to_json }} - one_file_system: true - exclude_patterns: {{ matrix_backup_borg_location_exclude_patterns|to_json }} - -storage: - compression: {{ matrix_backup_borg_storage_compression|to_json }} - ssh_command: {{ matrix_backup_borg_storage_ssh_command|to_json }} - archive_name_format: {{ matrix_backup_borg_storage_archive_name_format|to_json }} - encryption_passphrase: {{ matrix_backup_borg_storage_encryption_passphrase|to_json }} - unknown_unencrypted_repo_access_is_ok: {{ matrix_backup_borg_unknown_unencrypted_repo_access_is_ok|to_json }} - -retention: - keep_hourly: {{ matrix_backup_borg_retention_keep_hourly|to_json }} - keep_daily: {{ matrix_backup_borg_retention_keep_daily|to_json }} - keep_weekly: {{ matrix_backup_borg_retention_keep_weekly|to_json }} - keep_monthly: {{ matrix_backup_borg_retention_keep_monthly|to_json }} - keep_yearly: {{ matrix_backup_borg_retention_keep_yearly|to_json }} - prefix: {{ matrix_backup_borg_retention_prefix|to_json }} - -consistency: - checks: - - repository - - archives - -hooks: -{% if matrix_backup_borg_postgresql_enabled and matrix_backup_borg_postgresql_databases|length > 0 %} - postgresql_databases: - {% for database in matrix_backup_borg_postgresql_databases %} - - name: {{ database|to_json }} - hostname: {{ matrix_backup_borg_postgresql_databases_hostname|to_json }} - username: {{ matrix_backup_borg_postgresql_databases_username|to_json }} - password: {{ matrix_backup_borg_postgresql_databases_password|to_json }} - port: {{ matrix_backup_borg_postgresql_databases_port | int | to_json }} - {% endfor %} -{% endif %} - after_backup: - - echo "Backup created." - on_error: - - echo "Error while creating a backup." diff --git a/roles/custom/matrix-backup-borg/templates/passwd.j2 b/roles/custom/matrix-backup-borg/templates/passwd.j2 deleted file mode 100644 index d3665cf4b..000000000 --- a/roles/custom/matrix-backup-borg/templates/passwd.j2 +++ /dev/null @@ -1,29 +0,0 @@ -{# the passwd file with correct username, UID and GID is mandatory to work with borg over ssh, otherwise ssh connections will fail #} -root:x:0:0:root:/root:/bin/ash -bin:x:1:1:bin:/bin:/sbin/nologin -daemon:x:2:2:daemon:/sbin:/sbin/nologin -adm:x:3:4:adm:/var/adm:/sbin/nologin -lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin -sync:x:5:0:sync:/sbin:/bin/sync -shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown -halt:x:7:0:halt:/sbin:/sbin/halt -mail:x:8:12:mail:/var/mail:/sbin/nologin -news:x:9:13:news:/usr/lib/news:/sbin/nologin -uucp:x:10:14:uucp:/var/spool/uucppublic:/sbin/nologin -operator:x:11:0:operator:/root:/sbin/nologin -man:x:13:15:man:/usr/man:/sbin/nologin -postmaster:x:14:12:postmaster:/var/mail:/sbin/nologin -cron:x:16:16:cron:/var/spool/cron:/sbin/nologin -ftp:x:21:21::/var/lib/ftp:/sbin/nologin -sshd:x:22:22:sshd:/dev/null:/sbin/nologin -at:x:25:25:at:/var/spool/cron/atjobs:/sbin/nologin -squid:x:31:31:Squid:/var/cache/squid:/sbin/nologin -xfs:x:33:33:X Font Server:/etc/X11/fs:/sbin/nologin -games:x:35:35:games:/usr/games:/sbin/nologin -cyrus:x:85:12::/usr/cyrus:/sbin/nologin -vpopmail:x:89:89::/var/vpopmail:/sbin/nologin -ntp:x:123:123:NTP:/var/empty:/sbin/nologin -smmsp:x:209:209:smmsp:/var/spool/mqueue:/sbin/nologin -guest:x:405:100:guest:/dev/null:/sbin/nologin -{{ matrix_user_username }}:x:{{ matrix_user_uid }}:{{ matrix_user_gid }}:Matrix:/tmp:/bin/ash -nobody:x:65534:65534:nobody:/:/sbin/nologin diff --git a/roles/custom/matrix-backup-borg/templates/sshkey.j2 b/roles/custom/matrix-backup-borg/templates/sshkey.j2 deleted file mode 100644 index 999cf38d1..000000000 --- a/roles/custom/matrix-backup-borg/templates/sshkey.j2 +++ /dev/null @@ -1 +0,0 @@ -{{ matrix_backup_borg_ssh_key_private }} diff --git a/roles/custom/matrix-backup-borg/templates/systemd/matrix-backup-borg.service.j2 b/roles/custom/matrix-backup-borg/templates/systemd/matrix-backup-borg.service.j2 deleted file mode 100644 index 900369b9d..000000000 --- a/roles/custom/matrix-backup-borg/templates/systemd/matrix-backup-borg.service.j2 +++ /dev/null @@ -1,65 +0,0 @@ -#jinja2: lstrip_blocks: "True" -[Unit] -Description=Matrix Borg Backup -{% for service in matrix_backup_borg_systemd_required_services_list %} -Requires={{ service }} -After={{ service }} -{% endfor %} -{% for service in matrix_backup_borg_systemd_wanted_services_list %} -Wants={{ service }} -{% endfor %} -DefaultDependencies=no - -[Service] -Type=oneshot -Environment="HOME={{ devture_systemd_docker_base_systemd_unit_home_path }}" -ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} kill matrix-backup-borg 2>/dev/null || true' -ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-backup-borg 2>/dev/null || true' -ExecStartPre=-{{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-backup-borg \ - --log-driver=none \ - --cap-drop=ALL \ - --read-only \ - --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ - --network={{ matrix_docker_network }} \ - --tmpfs=/tmp:rw,noexec,nosuid,size=100m \ - --mount type=bind,src={{ matrix_backup_borg_config_path }}/passwd,dst=/etc/passwd,ro \ - --mount type=bind,src={{ matrix_backup_borg_config_path }},dst=/etc/borgmatic.d,ro \ - {% for source in matrix_backup_borg_location_source_directories %} - --mount type=bind,src={{ source }},dst={{ source }},ro \ - {% endfor %} - {% for arg in matrix_backup_borg_container_extra_arguments %} - {{ arg }} \ - {% endfor %} - {{ matrix_backup_borg_docker_image }} \ - sh -c "borgmatic rcreate --encryption {{ matrix_backup_borg_encryption }}" - -# The `CAP_DAC_OVERRIDE` capability is required, so that `root` in the container -# can read the `/etc/borgmatic.d/config.yaml` (`{{ matrix_backup_borg_config_path }}/config.yaml`) file, -# owned by `matrix:matrix` on the filesystem. -# -# `/root` is mountes as temporary filesystem, because we're using `--read-only` and because -# Borgmatic tries to write to at least a few paths under `/root` (`.config`, `.ssh`, `.borgmatic`). -ExecStart={{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-backup-borg \ - --log-driver=none \ - --cap-drop=ALL \ - --cap-add=CAP_DAC_OVERRIDE \ - --read-only \ - --network={{ matrix_docker_network }} \ - --tmpfs=/root:rw,noexec,nosuid,size=100m \ - --tmpfs=/tmp:rw,noexec,nosuid,size=100m \ - --mount type=bind,src={{ matrix_backup_borg_config_path }}/passwd,dst=/etc/passwd,ro \ - --mount type=bind,src={{ matrix_backup_borg_config_path }},dst=/etc/borgmatic.d,ro \ - {% for source in matrix_backup_borg_location_source_directories %} - --mount type=bind,src={{ source }},dst={{ source }},ro \ - {% endfor %} - {% for arg in matrix_backup_borg_container_extra_arguments %} - {{ arg }} \ - {% endfor %} - {{ matrix_backup_borg_docker_image }} - -ExecStop=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} kill matrix-backup-borg 2>/dev/null || true' -ExecStop=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-backup-borg 2>/dev/null || true' -SyslogIdentifier=matrix-backup-borg - -[Install] -WantedBy=multi-user.target diff --git a/roles/custom/matrix-backup-borg/templates/systemd/matrix-backup-borg.timer.j2 b/roles/custom/matrix-backup-borg/templates/systemd/matrix-backup-borg.timer.j2 deleted file mode 100644 index fdafef766..000000000 --- a/roles/custom/matrix-backup-borg/templates/systemd/matrix-backup-borg.timer.j2 +++ /dev/null @@ -1,10 +0,0 @@ -[Unit] -Description=Matrix Borg Backup timer - -[Timer] -Unit=matrix-backup-borg.service -OnCalendar={{ matrix_backup_borg_schedule }} -RandomizedDelaySec={{ matrix_backup_borg_schedule_randomized_delay_sec }} - -[Install] -WantedBy=timers.target diff --git a/roles/custom/matrix_playbook_migration/defaults/main.yml b/roles/custom/matrix_playbook_migration/defaults/main.yml index 496c4c753..109ecd726 100644 --- a/roles/custom/matrix_playbook_migration/defaults/main.yml +++ b/roles/custom/matrix_playbook_migration/defaults/main.yml @@ -5,3 +5,6 @@ matrix_playbook_migration_matrix_prometheus_node_exporter_migration_validation_e # Controls if (`matrix_prometheus_postgres_exporter` -> `prometheus_postgres_exporter`) validation will run. matrix_playbook_migration_matrix_prometheus_postgres_exporter_migration_validation_enabled: true + +# Controls if (`matrix_backup_borg` -> `backup_borg`) validation will run. +matrix_playbook_migration_matrix_backup_borg_migration_validation_enabled: true diff --git a/roles/custom/matrix_playbook_migration/tasks/validate_config.yml b/roles/custom/matrix_playbook_migration/tasks/validate_config.yml index b04c9e0e9..4f3d79318 100644 --- a/roles/custom/matrix_playbook_migration/tasks/validate_config.yml +++ b/roles/custom/matrix_playbook_migration/tasks/validate_config.yml @@ -80,3 +80,20 @@ Please change your configuration (vars.yml) to rename all variables (`matrix_prometheus_postgres_exporter_` -> `prometheus_postgres_exporter_`). We found usage of the following variables: {{ matrix_playbook_migration_matrix_prometheus_postgres_exporter_migration_vars.keys() | join(', ') }} when: "matrix_playbook_migration_matrix_prometheus_postgres_exporter_migration_vars | length > 0" + +- when: matrix_playbook_migration_matrix_backup_borg_migration_validation_enabled | bool + block: + - ansible.builtin.set_fact: + matrix_playbook_migration_backup_borg_migration_vars: |- + {{ vars | dict2items | selectattr('key', 'match', 'matrix_backup_borg_.*') | list | items2dict }} + + - name: (Deprecation) Catch and report matrix_backup_borg variables + ansible.builtin.fail: + msg: >- + The matrix-backup-borg role that used to be part of this playbook has been replaced by https://gitlab.com/etke.cc/roles/backup_borg. + The new role is compatible with the old one, but uses different names for its variables. + Please change your configuration (vars.yml) to rename all variables (`matrix_backup_borg_` -> `backup_borg_`). + We found usage of the following variables: {{ matrix_playbook_migration_backup_borg_migration_vars.keys() | join(', ') }} + when: "matrix_playbook_migration_backup_borg_migration_vars | length > 0" + +