From 747e8ad589ada45496e4d85588a0fe354f5157b7 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 14:54:21 +0900 Subject: [PATCH 01/19] Update docs/configuring-playbook-prometheus-grafana.md: switch the position of "Grafana" and "Prometheus" in the first sentence Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index 477cb172e..253ad0fb4 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -1,6 +1,6 @@ # Enabling metrics and graphs (Prometheus, Grafana) for your Matrix server (optional) -The playbook can install [Grafana](https://grafana.com/) with [Prometheus](https://prometheus.io/) and configure performance metrics of your homeserver with graphs for you. +The playbook can install [Prometheus](https://prometheus.io/) with [Grafana](https://grafana.com/) and configure performance metrics of your homeserver with graphs for you. ## Adjusting DNS records From 5ca5613d95cb5ef4cd8297856606e5d5ce6890a8 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 16:48:59 +0900 Subject: [PATCH 02/19] Update docs/configuring-playbook-prometheus-postgres.md: add the copyright header before deleting the file Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-postgres.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/configuring-playbook-prometheus-postgres.md b/docs/configuring-playbook-prometheus-postgres.md index 74d3eda4f..59080ba4d 100644 --- a/docs/configuring-playbook-prometheus-postgres.md +++ b/docs/configuring-playbook-prometheus-postgres.md @@ -1,3 +1,12 @@ + + # Enabling metrics and graphs for Postgres (optional) Expanding on the metrics exposed by the [synapse exporter and the node exporter](configuring-playbook-prometheus-grafana.md), the playbook enables the [postgres exporter](https://github.com/prometheus-community/postgres_exporter) that exposes more detailed information about what's happening on your postgres database. From 07cca8f4b4fd9cc20d666f965a076a81dc8b3765 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 15:19:04 +0900 Subject: [PATCH 03/19] Merge the document for instruction about metrics for Postgres to docs/configuring-playbook-prometheus-grafana.md to improve maintainability Signed-off-by: Suguru Hirahara --- CHANGELOG.md | 2 +- ...configuring-playbook-prometheus-grafana.md | 24 ++++++++-- ...onfiguring-playbook-prometheus-postgres.md | 46 ------------------- 3 files changed, 22 insertions(+), 50 deletions(-) delete mode 100644 docs/configuring-playbook-prometheus-postgres.md diff --git a/CHANGELOG.md b/CHANGELOG.md index bbbc609d9..c04e39d90 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1824,7 +1824,7 @@ See our [Setting up the ntfy push notifications server](docs/configuring-playboo **If you're using node-exporter** (`matrix_prometheus_node_exporter_enabled: true`) and would like to collect its metrics from an external Prometheus server, see `matrix_prometheus_node_exporter_metrics_proxying_enabled` described in our [Collecting metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server) documentation. You will be able to collect its metrics from `https://matrix.example.com/metrics/node-exporter`. -**If you're using [postgres-exporter](docs/configuring-playbook-prometheus-postgres.md)** (`prometheus_postgres_exporter_enabled: true`) and would like to collect its metrics from an external Prometheus server, see `matrix_prometheus_services_proxy_connect_prometheus_postgres_exporter_metrics_proxying_enabled` described in our [Collecting metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server) documentation. You will be able to collect its metrics from `https://matrix.example.com/metrics/postgres-exporter`. +**If you're using [postgres-exporter](docs/configuring-playbook-prometheus-grafana.md#enabling-metrics-and-graphs-for-postgres-optional)** (`prometheus_postgres_exporter_enabled: true`) and would like to collect its metrics from an external Prometheus server, see `matrix_prometheus_services_proxy_connect_prometheus_postgres_exporter_metrics_proxying_enabled` described in our [Collecting metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server) documentation. You will be able to collect its metrics from `https://matrix.example.com/metrics/postgres-exporter`. **If you're using Synapse** and would like to collect its metrics from an external Prometheus server, you may find that: diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index 253ad0fb4..0fe941d60 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -33,11 +33,28 @@ Name | Description -----|---------- `prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about. `prometheus_node_exporter_enabled`|[Node Exporter](https://prometheus.io/docs/guides/node-exporter/) is an addon of sorts to Prometheus that collects generic system information such as CPU, memory, filesystem, and even system temperatures. -`prometheus_postgres_exporter_enabled`|[Postgres Exporter](configuring-playbook-prometheus-postgres.md) is an addon of sorts to expose Postgres database metrics to Prometheus. +`prometheus_postgres_exporter_enabled`|[Postgres Exporter](#enabling-metrics-and-graphs-for-postgres-optional) is an addon of sorts to expose Postgres database metrics to Prometheus. `matrix_prometheus_nginxlog_exporter_enabled`|[nginx Log Exporter](configuring-playbook-prometheus-nginxlog.md) is an addon of sorts to expose nginx logs to Prometheus. **Note**: the retention policy of Prometheus metrics is [15 days by default](https://prometheus.io/docs/prometheus/latest/storage/#operational-aspects). Older data gets deleted automatically. +#### Enabling metrics and graphs for Postgres (optional) + +Expanding on the metrics exposed by the [synapse exporter and the node exporter](configuring-playbook-prometheus-grafana.md), the playbook enables the [postgres exporter](https://github.com/prometheus-community/postgres_exporter) that exposes more detailed information about what's happening on your postgres database. + +To enable the postgres exporter, add the following configuration to your `inventory/host_vars/matrix.example.com/vars.yml` file: + +```yaml +prometheus_postgres_exporter_enabled: true +``` + +Name | Description +-----|---------- +`prometheus_postgres_exporter_enabled`|Enable the postgres prometheus exporter. This sets up the docker container, connects it to the database and adds a 'job' to the prometheus config which tells prometheus about this new exporter. The default is 'false' +`prometheus_postgres_exporter_database_username`| The 'username' for the user that the exporter uses to connect to the database. The default is 'matrix_prometheus_postgres_exporter' +`prometheus_postgres_exporter_database_password`| The 'password' for the user that the exporter uses to connect to the database. By default, this is auto-generated by the playbook +`prometheus_postgres_exporter_container_labels_traefik_enabled`|If set to `true`, exposes the Postgres exporter metrics on `https://matrix.example.com/metrics/postgres-exporter` for usage with an [external Prometheus server](configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server). To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` on that other documentation page. + #### Extending the configuration There are some additional things you may wish to configure about Prometheus. @@ -123,8 +140,8 @@ Name | Description `matrix_synapse_metrics_proxying_enabled`|Set this to `true` to expose Synapse's metrics on `https://matrix.example.com/metrics/synapse/main-process` and `https://matrix.example.com/metrics/synapse/worker/TYPE-ID`. Read [below](#collecting-synapse-worker-metrics-to-an-external-prometheus-server) if you're running a Synapse worker setup (`matrix_synapse_workers_enabled: true`). To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `prometheus_node_exporter_enabled`|Set this to `true` to enable the node (general system stats) exporter (locally, on the container network). `prometheus_node_exporter_container_labels_traefik_enabled`|Set this to `true` to expose the node (general system stats) metrics on `https://matrix.example.com/metrics/node-exporter`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. -`prometheus_postgres_exporter_enabled`|Set this to `true` to enable the [Postgres exporter](configuring-playbook-prometheus-postgres.md) (locally, on the container network). -`prometheus_postgres_exporter_container_labels_traefik_enabled`|Set this to `true` to expose the [Postgres exporter](configuring-playbook-prometheus-postgres.md) metrics on `https://matrix.example.com/metrics/postgres-exporter`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. +`prometheus_postgres_exporter_enabled`|Set this to `true` to enable the [Postgres exporter](#enabling-metrics-and-graphs-for-postgres-optional) (locally, on the container network). +`prometheus_postgres_exporter_container_labels_traefik_enabled`|Set this to `true` to expose the [Postgres exporter](#enabling-metrics-and-graphs-for-postgres-optional) metrics on `https://matrix.example.com/metrics/postgres-exporter`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `matrix_prometheus_nginxlog_exporter_enabled`|Set this to `true` to enable the [nginx Log exporter](configuring-playbook-prometheus-nginxlog.md) (locally, on the container network). `matrix_sliding_sync_metrics_enabled`|Set this to `true` to make [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) expose metrics (locally, on the container network). `matrix_sliding_sync_metrics_proxying_enabled`|Set this to `true` to expose the [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) metrics on `https://matrix.example.com/metrics/sliding-sync`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. @@ -178,3 +195,4 @@ As with all other services, you can find the logs in [systemd-journald](https:// - [The Prometheus scraping rules](https://github.com/element-hq/synapse/tree/master/contrib/prometheus) (we use v2) - [The Synapse Grafana dashboard](https://github.com/element-hq/synapse/tree/master/contrib/grafana) - [The Node Exporter dashboard](https://github.com/rfrail3/grafana-dashboards) (for generic non-synapse performance graphs) +- [The PostgresSQL dashboard](https://grafana.com/grafana/dashboards/9628) (generic postgres dashboard) diff --git a/docs/configuring-playbook-prometheus-postgres.md b/docs/configuring-playbook-prometheus-postgres.md deleted file mode 100644 index 59080ba4d..000000000 --- a/docs/configuring-playbook-prometheus-postgres.md +++ /dev/null @@ -1,46 +0,0 @@ - - -# Enabling metrics and graphs for Postgres (optional) - -Expanding on the metrics exposed by the [synapse exporter and the node exporter](configuring-playbook-prometheus-grafana.md), the playbook enables the [postgres exporter](https://github.com/prometheus-community/postgres_exporter) that exposes more detailed information about what's happening on your postgres database. - -## Adjusting the playbook configuration - -To enable the postgres exporter, add the following configuration to your `inventory/host_vars/matrix.example.com/vars.yml` file: - -```yaml -prometheus_postgres_exporter_enabled: true -``` - -## Installing - -After configuring the playbook, run it with [playbook tags](playbook-tags.md) as below: - - -```sh -ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start -``` - -The shortcut commands with the [`just` program](just.md) are also available: `just install-all` or `just setup-all` - -`just install-all` is useful for maintaining your setup quickly ([2x-5x faster](../CHANGELOG.md#2x-5x-performance-improvements-in-playbook-runtime) than `just setup-all`) when its components remain unchanged. If you adjust your `vars.yml` to remove other components, you'd need to run `just setup-all`, or these components will still remain installed. Note these shortcuts run the `ensure-matrix-users-created` tag too. - -## What does it do? - -Name | Description ------|---------- -`prometheus_postgres_exporter_enabled`|Enable the postgres prometheus exporter. This sets up the docker container, connects it to the database and adds a 'job' to the prometheus config which tells prometheus about this new exporter. The default is 'false' -`prometheus_postgres_exporter_database_username`| The 'username' for the user that the exporter uses to connect to the database. The default is 'matrix_prometheus_postgres_exporter' -`prometheus_postgres_exporter_database_password`| The 'password' for the user that the exporter uses to connect to the database. By default, this is auto-generated by the playbook -`prometheus_postgres_exporter_container_labels_traefik_enabled`|If set to `true`, exposes the Postgres exporter metrics on `https://matrix.example.com/metrics/postgres-exporter` for usage with an [external Prometheus server](configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server). To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` on that other documentation page. - -## More information - -- [The PostgresSQL dashboard](https://grafana.com/grafana/dashboards/9628) (generic postgres dashboard) From 505ff582920fca7e16134a7785bf073ce6430482 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 15:21:40 +0900 Subject: [PATCH 04/19] Update docs/configuring-playbook-prometheus-grafana.md: tidy the section for Postgres exporter Signed-off-by: Suguru Hirahara --- CHANGELOG.md | 2 +- docs/configuring-playbook-prometheus-grafana.md | 14 +++++--------- 2 files changed, 6 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c04e39d90..4dcd99ba6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1824,7 +1824,7 @@ See our [Setting up the ntfy push notifications server](docs/configuring-playboo **If you're using node-exporter** (`matrix_prometheus_node_exporter_enabled: true`) and would like to collect its metrics from an external Prometheus server, see `matrix_prometheus_node_exporter_metrics_proxying_enabled` described in our [Collecting metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server) documentation. You will be able to collect its metrics from `https://matrix.example.com/metrics/node-exporter`. -**If you're using [postgres-exporter](docs/configuring-playbook-prometheus-grafana.md#enabling-metrics-and-graphs-for-postgres-optional)** (`prometheus_postgres_exporter_enabled: true`) and would like to collect its metrics from an external Prometheus server, see `matrix_prometheus_services_proxy_connect_prometheus_postgres_exporter_metrics_proxying_enabled` described in our [Collecting metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server) documentation. You will be able to collect its metrics from `https://matrix.example.com/metrics/postgres-exporter`. +**If you're using [postgres-exporter](docs/configuring-playbook-prometheus-grafana.md#enable-metrics-and-graphs-for-postgres-optional)** (`prometheus_postgres_exporter_enabled: true`) and would like to collect its metrics from an external Prometheus server, see `matrix_prometheus_services_proxy_connect_prometheus_postgres_exporter_metrics_proxying_enabled` described in our [Collecting metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server) documentation. You will be able to collect its metrics from `https://matrix.example.com/metrics/postgres-exporter`. **If you're using Synapse** and would like to collect its metrics from an external Prometheus server, you may find that: diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index 0fe941d60..8f63f8533 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -22,9 +22,6 @@ prometheus_enabled: true # Uncomment to enable Node Exporter. # prometheus_node_exporter_enabled: true -# Uncomment to enable Postgres Exporter. -# prometheus_postgres_exporter_enabled: true - # Uncomment to enable nginx Log Exporter. # matrix_prometheus_nginxlog_exporter_enabled: true ``` @@ -33,16 +30,15 @@ Name | Description -----|---------- `prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about. `prometheus_node_exporter_enabled`|[Node Exporter](https://prometheus.io/docs/guides/node-exporter/) is an addon of sorts to Prometheus that collects generic system information such as CPU, memory, filesystem, and even system temperatures. -`prometheus_postgres_exporter_enabled`|[Postgres Exporter](#enabling-metrics-and-graphs-for-postgres-optional) is an addon of sorts to expose Postgres database metrics to Prometheus. `matrix_prometheus_nginxlog_exporter_enabled`|[nginx Log Exporter](configuring-playbook-prometheus-nginxlog.md) is an addon of sorts to expose nginx logs to Prometheus. **Note**: the retention policy of Prometheus metrics is [15 days by default](https://prometheus.io/docs/prometheus/latest/storage/#operational-aspects). Older data gets deleted automatically. -#### Enabling metrics and graphs for Postgres (optional) +#### Enable metrics and graphs for Postgres (optional) -Expanding on the metrics exposed by the [synapse exporter and the node exporter](configuring-playbook-prometheus-grafana.md), the playbook enables the [postgres exporter](https://github.com/prometheus-community/postgres_exporter) that exposes more detailed information about what's happening on your postgres database. +Expanding on the metrics exposed by the Synapse exporter and the Node exporter, the playbook can also install and configure the [postgres exporter](https://github.com/prometheus-community/postgres_exporter) that exposes more detailed information about what's happening on your postgres database. -To enable the postgres exporter, add the following configuration to your `inventory/host_vars/matrix.example.com/vars.yml` file: +To enable it, add the following configuration to your `vars.yml` file: ```yaml prometheus_postgres_exporter_enabled: true @@ -140,8 +136,8 @@ Name | Description `matrix_synapse_metrics_proxying_enabled`|Set this to `true` to expose Synapse's metrics on `https://matrix.example.com/metrics/synapse/main-process` and `https://matrix.example.com/metrics/synapse/worker/TYPE-ID`. Read [below](#collecting-synapse-worker-metrics-to-an-external-prometheus-server) if you're running a Synapse worker setup (`matrix_synapse_workers_enabled: true`). To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `prometheus_node_exporter_enabled`|Set this to `true` to enable the node (general system stats) exporter (locally, on the container network). `prometheus_node_exporter_container_labels_traefik_enabled`|Set this to `true` to expose the node (general system stats) metrics on `https://matrix.example.com/metrics/node-exporter`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. -`prometheus_postgres_exporter_enabled`|Set this to `true` to enable the [Postgres exporter](#enabling-metrics-and-graphs-for-postgres-optional) (locally, on the container network). -`prometheus_postgres_exporter_container_labels_traefik_enabled`|Set this to `true` to expose the [Postgres exporter](#enabling-metrics-and-graphs-for-postgres-optional) metrics on `https://matrix.example.com/metrics/postgres-exporter`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. +`prometheus_postgres_exporter_enabled`|Set this to `true` to enable the [Postgres exporter](#enable-metrics-and-graphs-for-postgres-optional) (locally, on the container network). +`prometheus_postgres_exporter_container_labels_traefik_enabled`|Set this to `true` to expose the [Postgres exporter](#enable-metrics-and-graphs-for-postgres-optional) metrics on `https://matrix.example.com/metrics/postgres-exporter`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `matrix_prometheus_nginxlog_exporter_enabled`|Set this to `true` to enable the [nginx Log exporter](configuring-playbook-prometheus-nginxlog.md) (locally, on the container network). `matrix_sliding_sync_metrics_enabled`|Set this to `true` to make [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) expose metrics (locally, on the container network). `matrix_sliding_sync_metrics_proxying_enabled`|Set this to `true` to expose the [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) metrics on `https://matrix.example.com/metrics/sliding-sync`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. From 8b7b7732efa6f87c54993578c985b16ff5ce7585 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 15:41:30 +0900 Subject: [PATCH 05/19] Update docs/configuring-playbook-prometheus-grafana.md: move variables for setting username and password of Postgres exporter to the YAML block above Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index 8f63f8533..f37613603 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -40,16 +40,24 @@ Expanding on the metrics exposed by the Synapse exporter and the Node exporter, To enable it, add the following configuration to your `vars.yml` file: +**Note**: `prometheus_postgres_exporter_database_username` has nothing to do with your Matrix user ID. It can be any string you'd like. + ```yaml prometheus_postgres_exporter_enabled: true + +# The username for the user that the exporter uses to connect to the database. +# Uncomment and adjust this part if you'd like to use a username different than the default. +# prometheus_postgres_exporter_database_username: "matrix_prometheus_postgres_exporter" + +# The password for the user that the exporter uses to connect to the database. By default, this is auto-generated by the playbook. +# Uncomment and adjust this part if you'd like to set the password by yourself. +# prometheus_postgres_exporter_database_password: "PASSWORD_HERE" ``` Name | Description -----|---------- `prometheus_postgres_exporter_enabled`|Enable the postgres prometheus exporter. This sets up the docker container, connects it to the database and adds a 'job' to the prometheus config which tells prometheus about this new exporter. The default is 'false' -`prometheus_postgres_exporter_database_username`| The 'username' for the user that the exporter uses to connect to the database. The default is 'matrix_prometheus_postgres_exporter' -`prometheus_postgres_exporter_database_password`| The 'password' for the user that the exporter uses to connect to the database. By default, this is auto-generated by the playbook -`prometheus_postgres_exporter_container_labels_traefik_enabled`|If set to `true`, exposes the Postgres exporter metrics on `https://matrix.example.com/metrics/postgres-exporter` for usage with an [external Prometheus server](configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server). To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` on that other documentation page. +`prometheus_postgres_exporter_container_labels_traefik_enabled`|If set to `true`, exposes the Postgres exporter metrics on `https://matrix.example.com/metrics/postgres-exporter` for usage with an [external Prometheus server](#collecting-metrics-to-an-external-prometheus-server). To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` below. #### Extending the configuration From 4ecf6c2ba325cb25648bbaa254c0384b33625730 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 15:43:38 +0900 Subject: [PATCH 06/19] Update docs/configuring-playbook-prometheus-grafana.md: fix capitalization Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index f37613603..d85b60232 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -36,7 +36,7 @@ Name | Description #### Enable metrics and graphs for Postgres (optional) -Expanding on the metrics exposed by the Synapse exporter and the Node exporter, the playbook can also install and configure the [postgres exporter](https://github.com/prometheus-community/postgres_exporter) that exposes more detailed information about what's happening on your postgres database. +Expanding on the metrics exposed by the Synapse exporter and the Node exporter, the playbook can also install and configure the [PostgreSQL Server Exporter](https://github.com/prometheus-community/postgres_exporter) that exposes more detailed information about what's happening on your Postgres database. To enable it, add the following configuration to your `vars.yml` file: @@ -56,7 +56,7 @@ prometheus_postgres_exporter_enabled: true Name | Description -----|---------- -`prometheus_postgres_exporter_enabled`|Enable the postgres prometheus exporter. This sets up the docker container, connects it to the database and adds a 'job' to the prometheus config which tells prometheus about this new exporter. The default is 'false' +`prometheus_postgres_exporter_enabled`|Enable the Postgres Prometheus exporter. This sets up the docker container, connects it to the database and adds a 'job' to the Prometheus config which tells Prometheus about this new exporter. The default is 'false' `prometheus_postgres_exporter_container_labels_traefik_enabled`|If set to `true`, exposes the Postgres exporter metrics on `https://matrix.example.com/metrics/postgres-exporter` for usage with an [external Prometheus server](#collecting-metrics-to-an-external-prometheus-server). To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` below. #### Extending the configuration @@ -199,4 +199,4 @@ As with all other services, you can find the logs in [systemd-journald](https:// - [The Prometheus scraping rules](https://github.com/element-hq/synapse/tree/master/contrib/prometheus) (we use v2) - [The Synapse Grafana dashboard](https://github.com/element-hq/synapse/tree/master/contrib/grafana) - [The Node Exporter dashboard](https://github.com/rfrail3/grafana-dashboards) (for generic non-synapse performance graphs) -- [The PostgresSQL dashboard](https://grafana.com/grafana/dashboards/9628) (generic postgres dashboard) +- [The PostgresSQL dashboard](https://grafana.com/grafana/dashboards/9628) (generic Postgres dashboard) From 152031bbae616f193d26ee4c083ceb86292e4768 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 16:01:24 +0900 Subject: [PATCH 07/19] Update docs/configuring-playbook-prometheus-grafana.md: move the security notice to the top Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index d85b60232..96ee9b694 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -2,6 +2,11 @@ The playbook can install [Prometheus](https://prometheus.io/) with [Grafana](https://grafana.com/) and configure performance metrics of your homeserver with graphs for you. +> [!WARNING] +> Metrics and resulting graphs can contain a lot of information. This includes system specs but also usage patterns. This applies especially to small personal/family scale homeservers. Someone might be able to figure out when you wake up and go to sleep by looking at the graphs over time. Think about this before enabling anonymous access. And you should really not forget to change your Grafana password. +> +> Most of our docker containers run with limited system access, but the `prometheus-node-exporter` has access to the host network stack and (readonly) root filesystem. This is required to report on them. If you don't like that, you can set `prometheus_node_exporter_enabled: false` (which is actually the default). You will still get Synapse metrics with this container disabled. Both of the dashboards will always be enabled, so you can still look at historical data after disabling either source. + ## Adjusting DNS records By default, this playbook installs Grafana web user-interface on the `stats.` subdomain (`stats.example.com`) and requires you to create a CNAME record for `stats`, which targets `matrix.example.com`. @@ -117,12 +122,6 @@ The shortcut commands with the [`just` program](just.md) are also available: `ju `just install-all` is useful for maintaining your setup quickly ([2x-5x faster](../CHANGELOG.md#2x-5x-performance-improvements-in-playbook-runtime) than `just setup-all`) when its components remain unchanged. If you adjust your `vars.yml` to remove other components, you'd need to run `just setup-all`, or these components will still remain installed. Note these shortcuts run the `ensure-matrix-users-created` tag too. -## Security and privacy - -Metrics and resulting graphs can contain a lot of information. This includes system specs but also usage patterns. This applies especially to small personal/family scale homeservers. Someone might be able to figure out when you wake up and go to sleep by looking at the graphs over time. Think about this before enabling anonymous access. And you should really not forget to change your Grafana password. - -Most of our docker containers run with limited system access, but the `prometheus-node-exporter` has access to the host network stack and (readonly) root filesystem. This is required to report on them. If you don't like that, you can set `prometheus_node_exporter_enabled: false` (which is actually the default). You will still get Synapse metrics with this container disabled. Both of the dashboards will always be enabled, so you can still look at historical data after disabling either source. - ## Collecting metrics to an external Prometheus server **If the integrated Prometheus server is enabled** (`prometheus_enabled: true`), metrics are collected by it from each service via communication that happens over the container network. Each service does not need to expose its metrics "publicly". From 00f3eda4dc0ab3272bc8c58c64c4652dbbc5c102 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 16:09:08 +0900 Subject: [PATCH 08/19] Update docs for Prometheus: adopt the common warning message Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 2 +- docs/configuring-playbook-prometheus-nginxlog.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index 96ee9b694..4c4346e88 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -3,7 +3,7 @@ The playbook can install [Prometheus](https://prometheus.io/) with [Grafana](https://grafana.com/) and configure performance metrics of your homeserver with graphs for you. > [!WARNING] -> Metrics and resulting graphs can contain a lot of information. This includes system specs but also usage patterns. This applies especially to small personal/family scale homeservers. Someone might be able to figure out when you wake up and go to sleep by looking at the graphs over time. Think about this before enabling anonymous access. And you should really not forget to change your Grafana password. +> Metrics and resulting graphs can contain a lot of information. This includes system specs but also usage patterns. This applies especially to small personal/family scale homeservers. Someone might be able to figure out when you wake up and go to sleep by looking at the graphs over time. Think about this before enabling (anonymous) access. And you should really not forget to change your Grafana password. > > Most of our docker containers run with limited system access, but the `prometheus-node-exporter` has access to the host network stack and (readonly) root filesystem. This is required to report on them. If you don't like that, you can set `prometheus_node_exporter_enabled: false` (which is actually the default). You will still get Synapse metrics with this container disabled. Both of the dashboards will always be enabled, so you can still look at historical data after disabling either source. diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index 80107b906..4d1c08a9a 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -25,7 +25,7 @@ matrix_prometheus_nginxlog_exporter_enabled: true ### Save metrics on an external Prometheus server (optional) > [!WARNING] -> Metrics and resulting graphs can contain a lot of information. nginx logs contain information like IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access. Please make sure you change the default Grafana password. +> Metrics and resulting graphs can contain a lot of information. nginx logs contain information like IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access. And you should really not forget to change your Grafana password. The playbook will automatically integrate the metrics into the [Prometheus](./configuring-playbook-prometheus-grafana.md) server provided with this playbook (if enabled). In such cases, the metrics endpoint is not exposed publicly — it's only available on the container network. From a691306614eef3613a1acd4ae02b368e2a855944 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 16:12:40 +0900 Subject: [PATCH 09/19] Update docs/configuring-playbook-prometheus-grafana.md: create sections for adjusting configurations of Prometheus and Grafanato decrease the section level Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index 4c4346e88..c9de18383 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -15,9 +15,7 @@ When setting, replace `example.com` with your own. **Note**: It is possible to install Prometheus without installing Grafana. In this case it is not required to create the CNAME record. -## Adjusting the playbook configuration - -### Configure Prometheus +## Adjusting the playbook configuration — Prometheus Prometheus is an open-source systems monitoring and alerting toolkit. To enable it, add the following configuration to your `inventory/host_vars/matrix.example.com/vars.yml` file: @@ -39,7 +37,7 @@ Name | Description **Note**: the retention policy of Prometheus metrics is [15 days by default](https://prometheus.io/docs/prometheus/latest/storage/#operational-aspects). Older data gets deleted automatically. -#### Enable metrics and graphs for Postgres (optional) +### Enable metrics and graphs for Postgres (optional) Expanding on the metrics exposed by the Synapse exporter and the Node exporter, the playbook can also install and configure the [PostgreSQL Server Exporter](https://github.com/prometheus-community/postgres_exporter) that exposes more detailed information about what's happening on your Postgres database. @@ -64,7 +62,7 @@ Name | Description `prometheus_postgres_exporter_enabled`|Enable the Postgres Prometheus exporter. This sets up the docker container, connects it to the database and adds a 'job' to the Prometheus config which tells Prometheus about this new exporter. The default is 'false' `prometheus_postgres_exporter_container_labels_traefik_enabled`|If set to `true`, exposes the Postgres exporter metrics on `https://matrix.example.com/metrics/postgres-exporter` for usage with an [external Prometheus server](#collecting-metrics-to-an-external-prometheus-server). To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` below. -#### Extending the configuration +### Extending the configuration There are some additional things you may wish to configure about Prometheus. @@ -72,7 +70,7 @@ Take a look at: - [Prometheus role](https://github.com/mother-of-all-self-hosting/ansible-role-prometheus)'s [`defaults/main.yml`](https://github.com/mother-of-all-self-hosting/ansible-role-prometheus/blob/main/defaults/main.yml) for some variables that you can customize via your `vars.yml` file. You can override settings (even those that don't have dedicated playbook variables) using the `prometheus_configuration_extension_yaml` variable -### Configure Grafana +## Adjusting the playbook configuration — Grafana Grafana is an open source visualization and analytics software. To enable it, add the following configuration to your `vars.yml` file. Make sure to replace `USERNAME_HERE` and `PASSWORD_HERE`. @@ -96,7 +94,7 @@ Name | Description `grafana_default_admin_user`
`grafana_default_admin_password`|By default Grafana creates a user with `admin` as the username and password. You are asked to change the credentials on first login. If you feel this is insecure and you want to change them beforehand, you can do that here. `grafana_anonymous_access`|By default you need to log in to see graphs. If you want to publicly share your graphs (e.g. when asking for help in [`#synapse:matrix.org`](https://matrix.to/#/#synapse:matrix.org?via=matrix.org&via=privacytools.io&via=mozilla.org)) you'll want to enable this option. -#### Adjusting the Grafana URL (optional) +### Adjusting the Grafana URL (optional) By tweaking the `grafana_hostname` variable, you can easily make the service available at a **different hostname** than the default one. From f4bc9c98ed1cd0b12d41df816c1e2e1b5e4cfcbe Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 16:23:26 +0900 Subject: [PATCH 10/19] Update docs/configuring-playbook-prometheus-grafana.md: remove the note about the component disabled by default It is not common to add a note about the component disabled by default. Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index c9de18383..ba8085869 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -59,7 +59,7 @@ prometheus_postgres_exporter_enabled: true Name | Description -----|---------- -`prometheus_postgres_exporter_enabled`|Enable the Postgres Prometheus exporter. This sets up the docker container, connects it to the database and adds a 'job' to the Prometheus config which tells Prometheus about this new exporter. The default is 'false' +`prometheus_postgres_exporter_enabled`|Enable the Postgres Prometheus exporter. This sets up the docker container, connects it to the database and adds a 'job' to the Prometheus config which tells Prometheus about this new exporter. `prometheus_postgres_exporter_container_labels_traefik_enabled`|If set to `true`, exposes the Postgres exporter metrics on `https://matrix.example.com/metrics/postgres-exporter` for usage with an [external Prometheus server](#collecting-metrics-to-an-external-prometheus-server). To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` below. ### Extending the configuration From a67745b1fc9e6b6ff0f7cbe9e4a611b6c174311a Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 16:29:15 +0900 Subject: [PATCH 11/19] Remove the table for variables about the Postgres Server Exporter As prometheus_postgres_exporter_container_labels_traefik_enabled is explained below, it is sensible to remove this duplicated entry in favor of it. Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index ba8085869..cd6514e80 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -41,6 +41,8 @@ Name | Description Expanding on the metrics exposed by the Synapse exporter and the Node exporter, the playbook can also install and configure the [PostgreSQL Server Exporter](https://github.com/prometheus-community/postgres_exporter) that exposes more detailed information about what's happening on your Postgres database. +Enabling the exporter sets up the docker container, connects it to the database and adds a 'job' to the Prometheus config which tells Prometheus about this new exporter. + To enable it, add the following configuration to your `vars.yml` file: **Note**: `prometheus_postgres_exporter_database_username` has nothing to do with your Matrix user ID. It can be any string you'd like. @@ -57,11 +59,6 @@ prometheus_postgres_exporter_enabled: true # prometheus_postgres_exporter_database_password: "PASSWORD_HERE" ``` -Name | Description ------|---------- -`prometheus_postgres_exporter_enabled`|Enable the Postgres Prometheus exporter. This sets up the docker container, connects it to the database and adds a 'job' to the Prometheus config which tells Prometheus about this new exporter. -`prometheus_postgres_exporter_container_labels_traefik_enabled`|If set to `true`, exposes the Postgres exporter metrics on `https://matrix.example.com/metrics/postgres-exporter` for usage with an [external Prometheus server](#collecting-metrics-to-an-external-prometheus-server). To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` below. - ### Extending the configuration There are some additional things you may wish to configure about Prometheus. From 2df8793a9cb0749441b529bc456d25127330b595 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 18:24:41 +0900 Subject: [PATCH 12/19] Update docs/configuring-playbook-prometheus-grafana.md: add two entries for journalctl (matrix-prometheus-node-exporter and matrix-prometheus-postgres-exporter) Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index cd6514e80..5733bd44d 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -184,6 +184,8 @@ scrape_configs: As with all other services, you can find the logs in [systemd-journald](https://www.freedesktop.org/software/systemd/man/systemd-journald.service.html) by logging in to the server with SSH and running the commands below: - `journalctl -fu matrix-prometheus` for Prometheus +- `journalctl -fu matrix-prometheus-node-exporter` for Node Exporter +- `journalctl -fu matrix-prometheus-postgres-exporter` for PostgreSQL Server Exporter - `journalctl -fu matrix-grafana` for Grafana ## More information From c5019b5a30165553ec8ae0e4e228e6f3e83e9b5c Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 22:52:19 +0900 Subject: [PATCH 13/19] Update docs/configuring-playbook-prometheus-nginxlog.md: add the copyright header Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-nginxlog.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index 4d1c08a9a..f09ff568d 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -1,3 +1,11 @@ + + # Enabling metrics and graphs for nginx logs (optional) The playbook can install and configure the [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) service for you, in order to make it possible to have some (visual) insight into [nginx](https://nginx.org/) logs. From e4d3b00927d0c6c4e80c0337719da49c5cdb26cd Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 22:18:06 +0900 Subject: [PATCH 14/19] Partially merge docs for prometheus-nginx-log-exporter to docs/configuring-playbook-prometheus-grafana.md Signed-off-by: Suguru Hirahara --- ...configuring-playbook-prometheus-grafana.md | 27 ++++++++++++++----- ...onfiguring-playbook-prometheus-nginxlog.md | 22 +-------------- 2 files changed, 22 insertions(+), 27 deletions(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index 5733bd44d..8a3c8dd60 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -24,16 +24,12 @@ prometheus_enabled: true # Uncomment to enable Node Exporter. # prometheus_node_exporter_enabled: true - -# Uncomment to enable nginx Log Exporter. -# matrix_prometheus_nginxlog_exporter_enabled: true ``` Name | Description -----|---------- `prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about. `prometheus_node_exporter_enabled`|[Node Exporter](https://prometheus.io/docs/guides/node-exporter/) is an addon of sorts to Prometheus that collects generic system information such as CPU, memory, filesystem, and even system temperatures. -`matrix_prometheus_nginxlog_exporter_enabled`|[nginx Log Exporter](configuring-playbook-prometheus-nginxlog.md) is an addon of sorts to expose nginx logs to Prometheus. **Note**: the retention policy of Prometheus metrics is [15 days by default](https://prometheus.io/docs/prometheus/latest/storage/#operational-aspects). Older data gets deleted automatically. @@ -59,6 +55,24 @@ prometheus_postgres_exporter_enabled: true # prometheus_postgres_exporter_database_password: "PASSWORD_HERE" ``` +### Enable metrics and graphs for nginx logs (optional) + +The playbook can also install and configure the [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) service for you. + +It is an addon of sorts to expose nginx logs to Prometheus. The exporter will collect access logs from various nginx reverse-proxies which may be used internally (e.g. `matrix-synapse-reverse-proxy-companion`, if Synapse workers are enabled) and will make them available at a Prometheus-compatible `/metrics` endpoint. + +See the project's [documentation](https://github.com/martin-helmich/prometheus-nginxlog-exporter/blob/master/README.adoc) to learn what it does and why it might be useful to you. + +To enable it, add the following configuration to your `vars.yml` file: + +```yaml +matrix_prometheus_nginxlog_exporter_enabled: true +``` + +If you enable Grafana, a dedicated `NGINX PROXY` Grafana dashboard will be created. + +**Note**: nginx is only used internally by this Ansible playbook. With Traefik being our default reverse-proxy, collecting nginx metrics is less relevant. + ### Extending the configuration There are some additional things you may wish to configure about Prometheus. @@ -123,7 +137,7 @@ The shortcut commands with the [`just` program](just.md) are also available: `ju When you'd like **to collect metrics from an external Prometheus server**, you need to expose service metrics outside of the container network. -The playbook provides a single endpoint (`https://matrix.example.com/metrics/*`), under which various services may expose their metrics (e.g. `/metrics/node-exporter`, `/metrics/postgres-exporter`, `/metrics/hookshot`, etc). To expose all services on this `/metrics/*` feature, use `matrix_metrics_exposure_enabled`. To protect access using [Basic Authentication](https://en.wikipedia.org/wiki/Basic_access_authentication), see `matrix_metrics_exposure_http_basic_auth_enabled` and `matrix_metrics_exposure_http_basic_auth_users` below. +The playbook provides a single endpoint (`https://matrix.example.com/metrics/*`), under which various services may expose their metrics (e.g. `/metrics/node-exporter`, `/metrics/postgres-exporter`, `/metrics/nginxlog`, `/metrics/hookshot`, etc). To expose all services on this `/metrics/*` feature, use `matrix_metrics_exposure_enabled`. To protect access using [Basic Authentication](https://en.wikipedia.org/wiki/Basic_access_authentication), see `matrix_metrics_exposure_http_basic_auth_enabled` and `matrix_metrics_exposure_http_basic_auth_users` below. When using `matrix_metrics_exposure_enabled`, you don't need to expose metrics for individual services one by one. @@ -140,7 +154,8 @@ Name | Description `prometheus_node_exporter_container_labels_traefik_enabled`|Set this to `true` to expose the node (general system stats) metrics on `https://matrix.example.com/metrics/node-exporter`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `prometheus_postgres_exporter_enabled`|Set this to `true` to enable the [Postgres exporter](#enable-metrics-and-graphs-for-postgres-optional) (locally, on the container network). `prometheus_postgres_exporter_container_labels_traefik_enabled`|Set this to `true` to expose the [Postgres exporter](#enable-metrics-and-graphs-for-postgres-optional) metrics on `https://matrix.example.com/metrics/postgres-exporter`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. -`matrix_prometheus_nginxlog_exporter_enabled`|Set this to `true` to enable the [nginx Log exporter](configuring-playbook-prometheus-nginxlog.md) (locally, on the container network). +`matrix_prometheus_nginxlog_exporter_enabled`|Set this to `true` to enable the [nginx Log exporter](#enable-metrics-and-graphs-for-nginx-logs-optional) (locally, on the container network). +`matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled`|Set this to `true` to expose the [nginx Log exporter](#enable-metrics-and-graphs-for-nginx-logs-optional) metrics on `https://matrix.example.com/metrics/nginxlog`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `matrix_sliding_sync_metrics_enabled`|Set this to `true` to make [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) expose metrics (locally, on the container network). `matrix_sliding_sync_metrics_proxying_enabled`|Set this to `true` to expose the [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) metrics on `https://matrix.example.com/metrics/sliding-sync`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `matrix_bridge_hookshot_metrics_enabled`|Set this to `true` to make [Hookshot](configuring-playbook-bridge-hookshot.md) expose metrics (locally, on the container network). diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index f09ff568d..244d124b7 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -8,28 +8,12 @@ SPDX-License-Identifier: AGPL-3.0-or-later # Enabling metrics and graphs for nginx logs (optional) -The playbook can install and configure the [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) service for you, in order to make it possible to have some (visual) insight into [nginx](https://nginx.org/) logs. - -It will collect access logs from various nginx reverse-proxies which may be used internally (e.g. `matrix-synapse-reverse-proxy-companion`, if Synapse workers are enabled) and will make them available at a Prometheus-compatible `/metrics` endpoint. - -See the project's [documentation](https://github.com/martin-helmich/prometheus-nginxlog-exporter/blob/master/README.adoc) to learn what it does and why it might be useful to you. - -**Note**: nginx is only used internally by this Ansible playbook. With Traefik being our default reverse-proxy, collecting nginx metrics is less relevant. - ## Prerequisite To make use of this, you need to install [Prometheus](./configuring-playbook-prometheus-grafana.md) either via the playbook or externally. When using an external Prometheus, configuration adjustments are necessary — see [Save metrics on an external Prometheus server](#save-metrics-on-an-external-prometheus-server). -If your setup includes [Grafana](./configuring-playbook-prometheus-grafana.md), a dedicated `NGINX PROXY` Grafana dashboard will be created. - ## Adjusting the playbook configuration -Add the following configuration to your `inventory/host_vars/matrix.example.com/vars.yml` file: - -```yaml -matrix_prometheus_nginxlog_exporter_enabled: true -``` - ### Save metrics on an external Prometheus server (optional) > [!WARNING] @@ -39,11 +23,7 @@ The playbook will automatically integrate the metrics into the [Prometheus](./co When using an external Prometheus server, you'll need to expose metrics publicly. See [Collecting metrics to an external Prometheus server](./configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server). -You can either use `matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled: true` to expose just this one service, or `matrix_metrics_exposure_enabled: true` to expose all services. - -Whichever way you go with, this service will expose its metrics endpoint **without password-protection** at `https://matrix.example.com/metrics/nginxlog` by default. - -For password-protection, use (`matrix_metrics_exposure_http_basic_auth_enabled` and `matrix_metrics_exposure_http_basic_auth_users`) or (`matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_enabled` and `matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_users`). +For password-protection, use or (`matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_enabled` and `matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_users`). ### Docker Image Compatibility (optional) From be02f12fed822a654b3a143f1a30c9267c532bcd Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 22:22:30 +0900 Subject: [PATCH 15/19] Update docs/configuring-playbook-prometheus-grafana.md: fix the variables name Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index 8a3c8dd60..f36be5ec3 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -158,8 +158,8 @@ Name | Description `matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled`|Set this to `true` to expose the [nginx Log exporter](#enable-metrics-and-graphs-for-nginx-logs-optional) metrics on `https://matrix.example.com/metrics/nginxlog`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `matrix_sliding_sync_metrics_enabled`|Set this to `true` to make [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) expose metrics (locally, on the container network). `matrix_sliding_sync_metrics_proxying_enabled`|Set this to `true` to expose the [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) metrics on `https://matrix.example.com/metrics/sliding-sync`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. -`matrix_bridge_hookshot_metrics_enabled`|Set this to `true` to make [Hookshot](configuring-playbook-bridge-hookshot.md) expose metrics (locally, on the container network). -`matrix_bridge_hookshot_metrics_proxying_enabled`|Set this to `true` to expose the [Hookshot](configuring-playbook-bridge-hookshot.md) metrics on `https://matrix.example.com/metrics/hookshot`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. +`matrix_hookshot_metrics_enabled`|Set this to `true` to make [Hookshot](configuring-playbook-bridge-hookshot.md) expose metrics (locally, on the container network). +`matrix_hookshot_metrics_proxying_enabled`|Set this to `true` to expose the [Hookshot](configuring-playbook-bridge-hookshot.md) metrics on `https://matrix.example.com/metrics/hookshot`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `matrix_SERVICE_metrics_proxying_enabled`|Various other services/roles may provide similar `_metrics_enabled` and `_metrics_proxying_enabled` variables for exposing their metrics. Refer to each role for details. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above or `matrix_SERVICE_container_labels_metrics_middleware_basic_auth_enabled`/`matrix_SERVICE_container_labels_metrics_middleware_basic_auth_users` variables provided by each role. `matrix_media_repo_metrics_enabled`|Set this to `true` to make media-repo expose metrics (locally, on the container network). From 7907a649ee9648f7b98e9da59705bf085918e977 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 22:25:00 +0900 Subject: [PATCH 16/19] Update docs/configuring-playbook-prometheus-grafana.md: remove variables for Sliding Sync proxy metrics The component has been deprecated and it is not really sensible to keep those entries. Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index f36be5ec3..7b97fb4ba 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -156,8 +156,6 @@ Name | Description `prometheus_postgres_exporter_container_labels_traefik_enabled`|Set this to `true` to expose the [Postgres exporter](#enable-metrics-and-graphs-for-postgres-optional) metrics on `https://matrix.example.com/metrics/postgres-exporter`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `matrix_prometheus_nginxlog_exporter_enabled`|Set this to `true` to enable the [nginx Log exporter](#enable-metrics-and-graphs-for-nginx-logs-optional) (locally, on the container network). `matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled`|Set this to `true` to expose the [nginx Log exporter](#enable-metrics-and-graphs-for-nginx-logs-optional) metrics on `https://matrix.example.com/metrics/nginxlog`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. -`matrix_sliding_sync_metrics_enabled`|Set this to `true` to make [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) expose metrics (locally, on the container network). -`matrix_sliding_sync_metrics_proxying_enabled`|Set this to `true` to expose the [Sliding Sync](configuring-playbook-sliding-sync-proxy.md) metrics on `https://matrix.example.com/metrics/sliding-sync`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `matrix_hookshot_metrics_enabled`|Set this to `true` to make [Hookshot](configuring-playbook-bridge-hookshot.md) expose metrics (locally, on the container network). `matrix_hookshot_metrics_proxying_enabled`|Set this to `true` to expose the [Hookshot](configuring-playbook-bridge-hookshot.md) metrics on `https://matrix.example.com/metrics/hookshot`. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above. `matrix_SERVICE_metrics_proxying_enabled`|Various other services/roles may provide similar `_metrics_enabled` and `_metrics_proxying_enabled` variables for exposing their metrics. Refer to each role for details. To password-protect the metrics, see `matrix_metrics_exposure_http_basic_auth_users` above or `matrix_SERVICE_container_labels_metrics_middleware_basic_auth_enabled`/`matrix_SERVICE_container_labels_metrics_middleware_basic_auth_users` variables provided by each role. From 2180c58e6299a90aee670675daea5140552d2fee Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 22:27:02 +0900 Subject: [PATCH 17/19] Update docs for Prometheus: move the description for Docker image compatibility Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 9 +++++++++ docs/configuring-playbook-prometheus-nginxlog.md | 9 --------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index 7b97fb4ba..e37175569 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -73,6 +73,15 @@ If you enable Grafana, a dedicated `NGINX PROXY` Grafana dashboard will be creat **Note**: nginx is only used internally by this Ansible playbook. With Traefik being our default reverse-proxy, collecting nginx metrics is less relevant. +#### Docker image compatibility (optional) + +At the moment of writing only images for `amd64` and `arm64` architectures are available. The playbook currently does not support [self-building](./self-building.md) a container image on other architectures. You can however use a custom-build image by setting: + +```yaml +matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: false +matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag +``` + ### Extending the configuration There are some additional things you may wish to configure about Prometheus. diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index 244d124b7..6689d0d26 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -25,15 +25,6 @@ When using an external Prometheus server, you'll need to expose metrics publicly For password-protection, use or (`matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_enabled` and `matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_users`). -### Docker Image Compatibility (optional) - -At the moment of writing only images for `amd64` and `arm64` architectures are available. The playbook currently does not support [self-building](./self-building.md) a container image on other architectures. You can however use a custom-build image by setting: - -```yaml -matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: false -matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag -``` - ### Extending the configuration There are some additional things you may wish to configure about the component. From 7c81c148cfd108b95852680b9578bdb9abcebeb4 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 22:30:03 +0900 Subject: [PATCH 18/19] Update docs for Prometheus: move other relevant instructions Signed-off-by: Suguru Hirahara --- docs/configuring-playbook-prometheus-grafana.md | 4 +++- docs/configuring-playbook-prometheus-nginxlog.md | 12 ------------ 2 files changed, 3 insertions(+), 13 deletions(-) diff --git a/docs/configuring-playbook-prometheus-grafana.md b/docs/configuring-playbook-prometheus-grafana.md index e37175569..8a82002f8 100644 --- a/docs/configuring-playbook-prometheus-grafana.md +++ b/docs/configuring-playbook-prometheus-grafana.md @@ -84,11 +84,12 @@ matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag ### Extending the configuration -There are some additional things you may wish to configure about Prometheus. +There are some additional things you may wish to configure about Prometheus and its add-on. Take a look at: - [Prometheus role](https://github.com/mother-of-all-self-hosting/ansible-role-prometheus)'s [`defaults/main.yml`](https://github.com/mother-of-all-self-hosting/ansible-role-prometheus/blob/main/defaults/main.yml) for some variables that you can customize via your `vars.yml` file. You can override settings (even those that don't have dedicated playbook variables) using the `prometheus_configuration_extension_yaml` variable +- `roles/custom/matrix-prometheus-nginxlog-exporter/defaults/main.yml` for some variables that you can customize via your `vars.yml` file ## Adjusting the playbook configuration — Grafana @@ -208,6 +209,7 @@ As with all other services, you can find the logs in [systemd-journald](https:// - `journalctl -fu matrix-prometheus` for Prometheus - `journalctl -fu matrix-prometheus-node-exporter` for Node Exporter - `journalctl -fu matrix-prometheus-postgres-exporter` for PostgreSQL Server Exporter +- `journalctl -fu matrix-prometheus-nginxlog-exporter` for prometheus-nginxlog-exporter - `journalctl -fu matrix-grafana` for Grafana ## More information diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index 6689d0d26..fedd17609 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -25,14 +25,6 @@ When using an external Prometheus server, you'll need to expose metrics publicly For password-protection, use or (`matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_enabled` and `matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_users`). -### Extending the configuration - -There are some additional things you may wish to configure about the component. - -Take a look at: - -- `roles/custom/matrix-prometheus-nginxlog-exporter/defaults/main.yml` for some variables that you can customize via your `vars.yml` file - ## Installing After configuring the playbook, run it with [playbook tags](playbook-tags.md) as below: @@ -45,7 +37,3 @@ ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start The shortcut commands with the [`just` program](just.md) are also available: `just install-all` or `just setup-all` `just install-all` is useful for maintaining your setup quickly ([2x-5x faster](../CHANGELOG.md#2x-5x-performance-improvements-in-playbook-runtime) than `just setup-all`) when its components remain unchanged. If you adjust your `vars.yml` to remove other components, you'd need to run `just setup-all`, or these components will still remain installed. Note these shortcuts run the `ensure-matrix-users-created` tag too. - -## Troubleshooting - -As with all other services, you can find the logs in [systemd-journald](https://www.freedesktop.org/software/systemd/man/systemd-journald.service.html) by logging in to the server with SSH and running `journalctl -fu matrix-prometheus-nginxlog-exporter`. From 66753136c07b412bbd445bf3d88dbdf284656f11 Mon Sep 17 00:00:00 2001 From: Suguru Hirahara Date: Sun, 9 Feb 2025 22:37:06 +0900 Subject: [PATCH 19/19] Delete docs/configuring-playbook-prometheus-nginxlog.md Signed-off-by: Suguru Hirahara --- README.md | 2 +- ...onfiguring-playbook-prometheus-nginxlog.md | 39 ------------------- docs/configuring-playbook.md | 2 - docs/container-images.md | 2 +- 4 files changed, 2 insertions(+), 43 deletions(-) delete mode 100644 docs/configuring-playbook-prometheus-nginxlog.md diff --git a/README.md b/README.md index 38db15f0b..4fad24e8c 100644 --- a/README.md +++ b/README.md @@ -162,7 +162,7 @@ Services that help you in administrating and monitoring your Matrix installation | [matrix-alertmanager-receiver](https://github.com/metio/matrix-alertmanager-receiver) | ❌ | Prometheus' [Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/) client | [Link](docs/configuring-playbook-alertmanager-receiver.md) | | [Matrix Authentication Service](https://github.com/element-hq/matrix-authentication-service/) | ❌ | OAuth 2.0 and OpenID Provider server | [Link](docs/configuring-playbook-matrix-authentication-service.md) | | [synapse-admin](https://github.com/etkecc/synapse-admin) | ❌ | Web UI tool for administrating users and rooms on your Matrix server | [Link](docs/configuring-playbook-synapse-admin.md) | -| Metrics and Graphs | ❌ | Consists of the [Prometheus](https://prometheus.io) time-series database server, the Prometheus [node-exporter](https://prometheus.io/docs/guides/node-exporter/) host metrics exporter, and the [Grafana](https://grafana.com/) web UI, with [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) being available too | [Link](docs/configuring-playbook-prometheus-grafana.md) (for [prometheus-nginxlog-exporter](docs/configuring-playbook-prometheus-nginxlog.md)) | +| Metrics and Graphs | ❌ | Consists of the [Prometheus](https://prometheus.io) time-series database server, the Prometheus [node-exporter](https://prometheus.io/docs/guides/node-exporter/) host metrics exporter, and the [Grafana](https://grafana.com/) web UI, with [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) being available too | [Link](docs/configuring-playbook-prometheus-grafana.md) (for [prometheus-nginxlog-exporter](docs/configuring-playbook-prometheus-grafana.md#enable-metrics-and-graphs-for-nginx-logs-optional)) | | [Borg](https://borgbackup.org) | ❌ | Backups | [Link](docs/configuring-playbook-backup-borg.md) | | [rageshake](https://github.com/matrix-org/rageshake) | ❌ | Bug report server | [Link](docs/configuring-playbook-rageshake.md) | | [synapse-usage-exporter](https://github.com/loelkes/synapse-usage-exporter) | ❌ | Export the usage statistics of a Synapse homeserver to be scraped by Prometheus. | [Link](docs/configuring-playbook-synapse-usage-exporter.md) | diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md deleted file mode 100644 index fedd17609..000000000 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ /dev/null @@ -1,39 +0,0 @@ - - -# Enabling metrics and graphs for nginx logs (optional) - -## Prerequisite - -To make use of this, you need to install [Prometheus](./configuring-playbook-prometheus-grafana.md) either via the playbook or externally. When using an external Prometheus, configuration adjustments are necessary — see [Save metrics on an external Prometheus server](#save-metrics-on-an-external-prometheus-server). - -## Adjusting the playbook configuration - -### Save metrics on an external Prometheus server (optional) - -> [!WARNING] -> Metrics and resulting graphs can contain a lot of information. nginx logs contain information like IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access. And you should really not forget to change your Grafana password. - -The playbook will automatically integrate the metrics into the [Prometheus](./configuring-playbook-prometheus-grafana.md) server provided with this playbook (if enabled). In such cases, the metrics endpoint is not exposed publicly — it's only available on the container network. - -When using an external Prometheus server, you'll need to expose metrics publicly. See [Collecting metrics to an external Prometheus server](./configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server). - -For password-protection, use or (`matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_enabled` and `matrix_prometheus_nginxlog_exporter_container_labels_metrics_middleware_basic_auth_users`). - -## Installing - -After configuring the playbook, run it with [playbook tags](playbook-tags.md) as below: - - -```sh -ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start -``` - -The shortcut commands with the [`just` program](just.md) are also available: `just install-all` or `just setup-all` - -`just install-all` is useful for maintaining your setup quickly ([2x-5x faster](../CHANGELOG.md#2x-5x-performance-improvements-in-playbook-runtime) than `just setup-all`) when its components remain unchanged. If you adjust your `vars.yml` to remove other components, you'd need to run `just setup-all`, or these components will still remain installed. Note these shortcuts run the `ensure-matrix-users-created` tag too. diff --git a/docs/configuring-playbook.md b/docs/configuring-playbook.md index 71a7426c4..30e8c808d 100644 --- a/docs/configuring-playbook.md +++ b/docs/configuring-playbook.md @@ -199,8 +199,6 @@ Services that help you in administrating and monitoring your Matrix installation - [Enabling metrics and graphs (Prometheus, Grafana) for your Matrix server](configuring-playbook-prometheus-grafana.md) -- [Enabling metrics and graphs for nginx logs](configuring-playbook-prometheus-nginxlog.md) - - [Setting up the rageshake bug report server](configuring-playbook-rageshake.md) - [Enabling synapse-usage-exporter for Synapse usage statistics](configuring-playbook-synapse-usage-exporter.md) diff --git a/docs/container-images.md b/docs/container-images.md index 23e2004f9..efed979c2 100644 --- a/docs/container-images.md +++ b/docs/container-images.md @@ -123,7 +123,7 @@ Services that help you in administrating and monitoring your Matrix installation | [Metrics and Graphs](configuring-playbook-prometheus-grafana.md) | [prom/prometheus](https://hub.docker.com/r/prom/prometheus/) | ❌ | [Prometheus](https://prometheus.io) time-series database server | | [Metrics and Graphs](configuring-playbook-prometheus-grafana.md) | [prom/node-exporter](https://hub.docker.com/r/prom/node-exporter/) | ❌ | Prometheus [node-exporter](https://prometheus.io/docs/guides/node-exporter/) host metrics exporter | | [Metrics and Graphs](configuring-playbook-prometheus-grafana.md) | [grafana/grafana](https://hub.docker.com/r/grafana/grafana/) | ❌ | Graphing tool that works well with the above two images. Our playbook also adds two dashboards for [Synapse](https://github.com/element-hq/synapse/tree/master/contrib/grafana) and [Node Exporter](https://github.com/rfrail3/grafana-dashboards) | -| [Metrics and Graphs](configuring-playbook-prometheus-nginxlog.md) | [martin-helmich/prometheus-nginxlog-exporter/exporter](https://ghcr.io/martin-helmich/prometheus-nginxlog-exporter/exporter) | ❌ | Addon for Prometheus that gathers access logs from various nginx reverse-proxies | +| [Metrics and Graphs](configuring-playbook-prometheus-grafana.md#enable-metrics-and-graphs-for-nginx-logs-optional) | [martin-helmich/prometheus-nginxlog-exporter/exporter](https://ghcr.io/martin-helmich/prometheus-nginxlog-exporter/exporter) | ❌ | Addon for Prometheus that gathers access logs from various nginx reverse-proxies | | [Borg](configuring-playbook-backup-borg.md) | (N/A) | ❌ | Backups | | [rageshake](configuring-playbook-rageshake.md) | [matrix-org/rageshake](https://ghcr.io/matrix-org/rageshake) | ❌ | Bug report server | | [synapse-usage-exporter](configuring-playbook-synapse-usage-exporter.md) | Self-building | ❌ | Export the usage statistics of a Synapse homeserver to be scraped by Prometheus. |