Add support for not taking over a server (no matrix-nginx-proxy) and disabling Riot

README.md

@@ -16,10 +16,12 @@ Using this playbook, you can get the following services configured on your serve
 
 - a [STUN/TURN server](https://github.com/coturn/coturn) for WebRTC audio/video calls
 
-- a [Riot](https://riot.im/) web UI, which is configured to connect to your own Matrix Synapse server by default
+- (optional default) a [Riot](https://riot.im/) web UI, which is configured to connect to your own Matrix Synapse server by default
 
 - free [Let's Encrypt](https://letsencrypt.org/) SSL certificate, which secures the connection to the Synapse server and the Riot web UI
 
+- (optional default) an [nginx](http://nginx.org/) web server, listening on ports 80 and 443 - standing in front of all the other services. Using your own webserver [is possible](#using-your-own-webserver-instead-of-this-playbooks-nginx-proxy-optional)
+
 Basically, this playbook aims to get you up-and-running with all the basic necessities around Matrix, without you having to do anything else.
 
 
@@ -33,13 +35,17 @@ This is similar to the [EMnify/matrix-synapse-auto-deploy](https://github.com/EM
 
 - works on both **CentOS** (7.0+) and Debian-based distributions (**Debian** 9/Stretch+, **Ubuntu** 16.04+)
 
+- this one keeps mostly everything in a single directory (`/matrix` by default) and **doesn't "contaminate" your server** with files all over the place
+
+- this one **doesn't necessarily take over** ports 80 and 443. By default, it sets up nginx for you there, but you can disable that and configure your own webserver (proxy)
+
 - this one **runs everything in Docker containers** (like [silviof/docker-matrix](https://hub.docker.com/r/silviof/docker-matrix/) and [silviof/matrix-riot-docker](https://hub.docker.com/r/silviof/matrix-riot-docker/)), so it's likely more predictable
 
 - this one retrieves and automatically renews free [Let's Encrypt](https://letsencrypt.org/) **SSL certificates** for you
 
 - this one optionally can store the `media_store` content repository files on [Amazon S3](https://aws.amazon.com/s3/) (but defaults to storing files on the server's filesystem)
 
-- this one optionally allows you to use an external PostgreSQL server for Matrix Synapse's database (but defaults to running one in a container)
+- this one optionally **allows you to use an external PostgreSQL server** for Matrix Synapse's database (but defaults to running one in a container)
 
 Special thanks goes to:
 
@@ -52,7 +58,7 @@ Special thanks goes to:
 
 ## Prerequisites
 
-- **CentOS** (7.0+), **Debian** (9/Stretch+) or **Ubuntu** (16.04+) server with no services running on port 80/443
+- **CentOS** (7.0+), **Debian** (9/Stretch+) or **Ubuntu** (16.04+) server. This playbook can take over your whole server or co-exist with other services that you have there.
 
 - the [Ansible](http://ansible.com/) program, which is used to run this playbook and configures everything for you
 
@@ -158,6 +164,33 @@ The database (as specified in `matrix_postgres_db_name`) must exist and be acces
 It must be empty or contain a valid Matrix Synapse database. If empty, Matrix Synapse would populate it the first time it runs.
 
 
+## Using your own webserver, instead of this playbook's nginx proxy (optional)
+
+By default, this playbook installs its own nginx webserver (in a Docker container) which listens on ports 80 and 443.
+If that's alright, you can skip ahead.
+
+If you don't want this playbook's nginx webserver to take over your server's 80/443 ports like that,
+and you'd like to use your own webserver (be it nginx, Apache, Varnish Cache, etc.), you can.
+
+All it takes is editing your configuration file (`inventory/matrix.<your-domain>/vars.yml`):
+
+```
+matrix_nginx_proxy_enabled: false
+```
+
+**Note**: even if you do this, in order [to install](#installing), this playbook still expects port 80 to be available. **Please manually stop your other webserver while installing**. You can start it back again afterwards.
+
+**If your own webserver is nginx**, you can most likely directly use the config files installed by this playbook at: `/matrix/nginx-proxy/conf.d`. Just include them in your `nginx.conf` like this: `include /matrix/nginx-proxy/conf.d/*.conf;`
+
+**If your own webserver is not nginx**, you can still take a look at the sample files in `/matrix/nginx-proxy/conf.d`, and:
+
+- ensure you set up (separate) vhosts that proxy for both Riot (`localhost:8765`) and Matrix Synapse (`localhost:8008`)
+
+- ensure that the `/.well-known/acme-challenge` location for each "port=80 vhost" is an alias to the `/matrix/ssl/run/acme-challenge` directory (for automated SSL renewal to work)
+
+- ensure that you restart/reload your webserver once in a while, so that renewed SSL certificates would take effect (once a month should be enough)
+
+
 ## Installing
 
 Once you have your server and you have [configured your DNS records](#configuring-dns), you can proceed with installing.
@@ -226,12 +259,31 @@ You can do it via this Ansible playbook (make sure to edit the `<your-username>`
 **Note**: `<your-username>` is just a plain username (like `john`), not your full `@<username>:<your-domain>` identifier.
 
 
+## Uninstalling
+
+**Note**: If you have some trouble with your installation configuration, you can just re-run the playbook and it will try to set things up again. You don't need to uninstall and install fresh.
+
+However, if you've installed this on some server where you have other stuff you wish to preserve, and now want get rid of Matrix, it's enough to do these:
+
+- ensure all Matrix services are stopped (`systemctl stop 'matrix*'`)
+
+- delete the Matrix-related systemd .service files (`rm -f /etc/systemd/system/matrix*`) and reload systemd (`systemctl daemon-reload`)
+
+- delete all Matrix-related cronjobs (`rm -f /etc/cron.d/matrix*'`)
+
+- delete some helper scripts (`rm -f /usr/local/bin/matrix*`)
+
+- delete some cached Docker images (or just delete them all: `docker rmi $(docker images -aq)`)
+
+- uninstall Docker itself, if necessary
+
+- delete the `/matrix` directory (`rm -rf /matrix`)
+
+
 ## Deficiencies
 
 This Ansible playbook can be improved in the following ways:
 
-- not expecting to run its own nginx server overtaking port 80/443, thus allowing own/custom proxying to be configured
-
 - setting up automatic backups to one or more storage providers
 
 - [importing an old SQLite database](#Restoring-an-existing-SQLite=database-from-another-installation) likely works because of a patch, but may be fragile until [this](https://github.com/matrix-org/synapse/issues/2287) is fixed

roles/matrix-server/defaults/main.yml

@@ -61,3 +61,14 @@ matrix_s3_media_store_enabled: false
 matrix_s3_media_store_bucket_name: "your-bucket-name"
 matrix_s3_media_store_aws_access_key: "your-aws-access-key"
 matrix_s3_media_store_aws_secret_key: "your-aws-secret-key"
+
+# By default, this playbook installs the Riot.IM web UI on the `hostname_riot` domain.
+# If you wish to connect to your Matrix server by other means,
+# you may wish to disable this.
+matrix_riot_web_enabled: true
+
+# By default, this playbook sets up its own nginx proxy server on port 80/443.
+# This is fine if you're dedicating the whole server to Matrix.
+# But in case that's not the case, you may wish to prevent that
+# and take care of proxying by yourself.
+matrix_nginx_proxy_enabled: true

roles/matrix-server/tasks/setup_nginx_proxy.yml

@@ -1,5 +1,15 @@
 ---
 
+#
+# Generic tasks that we always want to happen, regardless
+# if the user wants matrix-nginx-proxy or not.
+#
+# If the user would set up their own nginx proxy server,
+# the config files from matrix-nginx-proxy can be reused.
+#
+# It doesn't hurt to put them in place, even if they turn out
+# to be unnecessary.
+#
 - name: Ensure Matrix nginx-proxy paths exists
   file:
     path: "{{ item }}"
@@ -11,10 +21,6 @@
     - "{{ matrix_nginx_proxy_data_path }}"
     - "{{ matrix_nginx_proxy_confd_path }}"
 
-- name: Ensure nginx Docker image is pulled
-  docker_image:
-    name: "{{ docker_nginx_image }}"
-
 - name: Ensure Matrix Synapse proxy vhost configured
   template:
     src: "{{ role_path }}/templates/nginx-conf.d/{{ item }}.j2"
@@ -24,6 +30,14 @@
     - "matrix-synapse.conf"
     - "matrix-riot-web.conf"
 
+#
+# Tasks related to setting up matrix-nginx-proxy
+#
+- name: Ensure nginx Docker image is pulled
+  docker_image:
+    name: "{{ docker_nginx_image }}"
+  when: matrix_nginx_proxy_enabled
+
 - name: Allow access to nginx proxy ports in firewalld
   firewalld:
     service: "{{ item }}"
@@ -33,10 +47,30 @@
   with_items:
     - "http"
     - "https"
-  when: ansible_os_family == 'RedHat'
+  when: "ansible_os_family == 'RedHat' and matrix_nginx_proxy_enabled"
 
 - name: Ensure matrix-nginx-proxy.service installed
   template:
     src: "{{ role_path }}/templates/systemd/matrix-nginx-proxy.service.j2"
     dest: "/etc/systemd/system/matrix-nginx-proxy.service"
     mode: 0644
+  when: matrix_nginx_proxy_enabled
+
+#
+# Tasks related to getting rid of matrix-nginx-proxy (if it was previously enabled)
+#
+
+- name: Check existence of matrix-nginx-proxy service
+  stat: path="/etc/systemd/system/matrix-nginx-proxy.service"
+  register: matrix_nginx_proxy_service_stat
+
+- name: Ensure matrix-nginx-proxy is stopped
+  service: name=matrix-nginx-proxy state=stopped daemon_reload=yes
+  register: stopping_result
+  when: "not matrix_nginx_proxy_enabled and matrix_nginx_proxy_service_stat.stat.exists"
+
+- name: Ensure matrix-nginx-proxy.service doesn't exist
+  file:
+    path: "/etc/systemd/system/matrix-nginx-proxy.service"
+    state: absent
+  when: "not matrix_nginx_proxy_enabled and matrix_nginx_proxy_service_stat.stat.exists"

roles/matrix-server/tasks/setup_riot_web.yml

@@ -1,5 +1,9 @@
 ---
 
+#
+# Tasks related to setting up riot-web
+#
+
 - name: Ensure Matrix riot-web paths exists
   file:
     path: "{{ matrix_nginx_riot_web_data_path }}"
@@ -7,10 +11,12 @@
     mode: 0750
     owner: "{{ matrix_user_username }}"
     group: "{{ matrix_user_username }}"
+  when: matrix_riot_web_enabled
 
 - name: Ensure riot-web Docker image is pulled
   docker_image:
     name: "{{ docker_riot_image }}"
+  when: matrix_riot_web_enabled
 
 - name: Ensure Matrix riot-web configured
   template:
@@ -22,9 +28,42 @@
   with_items:
     - "riot.im.conf"
     - "config.json"
+  when: matrix_riot_web_enabled
 
 - name: Ensure matrix-riot-web.service installed
   template:
     src: "{{ role_path }}/templates/systemd/matrix-riot-web.service.j2"
     dest: "/etc/systemd/system/matrix-riot-web.service"
-    mode: 0644
+    mode: 0644
+  when: matrix_riot_web_enabled
+
+#
+# Tasks related to getting rid of riot-web (if it was previously enabled)
+#
+
+- name: Check existence of matrix-riot-web service
+  stat: path="/etc/systemd/system/matrix-riot-web.service"
+  register: matrix_riot_web_service_stat
+
+- name: Ensure matrix-riot-web is stopped
+  service: name=matrix-riot-web state=stopped daemon_reload=yes
+  register: stopping_result
+  when: "not matrix_riot_web_enabled and matrix_riot_web_service_stat.stat.exists"
+
+- name: Ensure matrix-riot-web.service doesn't exist
+  file:
+    path: "/etc/systemd/system/matrix-riot-web.service"
+    state: absent
+  when: "not matrix_riot_web_enabled and matrix_riot_web_service_stat.stat.exists"
+
+- name: Ensure Matrix riot-web paths doesn't exist
+  file:
+    path: "{{ matrix_nginx_riot_web_data_path }}"
+    state: absent
+  when: "not matrix_riot_web_enabled"
+
+- name: Ensure riot-web Docker image doesn't exist
+  docker_image:
+    name: "{{ docker_riot_image }}"
+    state: absent
+  when: "not matrix_riot_web_enabled"

roles/matrix-server/tasks/setup_ssl.yml

@@ -1,5 +1,14 @@
 ---
 
+- name: Determine domains to obtain certificates for (Matrix)
+  set_fact:
+    domains_to_obtain_certificate_for: "['{{ hostname_matrix }}']"
+
+- name: Determine domains to obtain certificates for (Riot)
+  set_fact:
+    domains_to_obtain_certificate_for: "{{ domains_to_obtain_certificate_for + [hostname_riot] }}"
+  when: matrix_riot_web_enabled
+
 - name: Allow access to HTTP/HTTPS in firewalld
   firewalld:
     service: "{{ item }}"
@@ -38,9 +47,7 @@
     -e ACME_EMAIL={{ matrix_ssl_support_email }}
     willwill/acme-docker
     acmetool want {{ item }} --xlog.severity=debug
-  with_items:
+  with_items: "{{ domains_to_obtain_certificate_for }}"
-    - "{{ hostname_matrix }}"
-    - "{{ hostname_riot }}"
 
 - name: Ensure matrix-nginx-proxy is started (if previously installed & started)
   service: name=matrix-nginx-proxy state=started

roles/matrix-server/tasks/start.yml

@@ -13,6 +13,8 @@
 
 - name: Ensure matrix-riot-web autoruns and is restarted
   service: name=matrix-riot-web enabled=yes state=restarted daemon_reload=yes
+  when: matrix_riot_web_enabled
 
 - name: Ensure matrix-nginx-proxy autoruns and is restarted
   service: name=matrix-nginx-proxy enabled=yes state=restarted daemon_reload=yes
+  when: matrix_nginx_proxy_enabled

roles/matrix-server/templates/cron.d/matrix-ssl-certificate-renewal.j2

@@ -11,11 +11,14 @@ MAILTO="{{ matrix_ssl_support_email }}"
 #
 # How renewal works?
 #
-# acmetool will fail to bind to port :80 (because matrix-nginx-proxy is running there),
+# acmetool will fail to bind to port :80 (because matrix-nginx-proxy or some other server is running there),
 # and will fall back to its "webroot" validation method.
 #
 # Thus, it would put validation files in `/var/run/acme/acme-challenge`.
 # These files can be retrieved via any vhost on port 80 of matrix-nginx-proxy,
 # because it aliases `/.well-known/acme-challenge` to that same directory.
+#
+# When a custom proxy server (not matrix-nginx-proxy provided by this playbook),
+# you'd need to make sure you alias these files corretly or SSL renewal would not work.
 
 15 4 */5 * * root /usr/bin/docker run --rm --name acmetool-host-grab --net=host -v {{ matrix_ssl_certs_path }}:/certs -v {{ matrix_ssl_certs_path }}/run:/var/run/acme -e ACME_EMAIL={{ matrix_ssl_support_email }} willwill/acme-docker acmetool --batch reconcile # --xlog.severity=debug

roles/matrix-server/templates/nginx-conf.d/matrix-riot-web.conf.j2

@@ -6,7 +6,7 @@ server {
 
 	location /.well-known/acme-challenge {
 		default_type "text/plain";
-		alias /acmetool-certs/run/acme-challenge;
+		alias {{ matrix_ssl_certs_path }}/run/acme-challenge;
 	}
 
 	location / {
@@ -24,14 +24,14 @@ server {
 	root /dev/null;
 
 	ssl on;
-	ssl_certificate /acmetool-certs/live/{{ hostname_riot }}/fullchain;
+	ssl_certificate {{ matrix_ssl_certs_path }}/live/{{ hostname_riot }}/fullchain;
-	ssl_certificate_key /acmetool-certs/live/{{ hostname_riot }}/privkey;
+	ssl_certificate_key {{ matrix_ssl_certs_path }}/live/{{ hostname_riot }}/privkey;
 	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
 	ssl_prefer_server_ciphers on;
 	ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
 
     location / {
-        proxy_pass http://riot:8765;
+        proxy_pass http://{{ 'riot' if matrix_nginx_proxy_enabled else 'localhost' }}:8765;
         proxy_set_header X-Forwarded-For $remote_addr;
     }
 }

roles/matrix-server/templates/nginx-conf.d/matrix-synapse.conf.j2

@@ -6,7 +6,7 @@ server {
 
 	location /.well-known/acme-challenge {
 		default_type "text/plain";
-		alias /acmetool-certs/run/acme-challenge;
+		alias {{ matrix_ssl_certs_path }}/run/acme-challenge;
 	}
 
 	location / {
@@ -24,14 +24,14 @@ server {
 	root /dev/null;
 
 	ssl on;
-	ssl_certificate /acmetool-certs/live/{{ hostname_matrix }}/fullchain;
+	ssl_certificate {{ matrix_ssl_certs_path }}/live/{{ hostname_matrix }}/fullchain;
-	ssl_certificate_key /acmetool-certs/live/{{ hostname_matrix }}/privkey;
+	ssl_certificate_key {{ matrix_ssl_certs_path }}/live/{{ hostname_matrix }}/privkey;
 	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
 	ssl_prefer_server_ciphers on;
 	ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
 
 	location /_matrix {
-		proxy_pass http://synapse:8008;
+		proxy_pass http://{{ 'synapse' if matrix_nginx_proxy_enabled else 'localhost' }}:8008;
 		proxy_set_header X-Forwarded-For $remote_addr;
 
 		client_body_buffer_size 25M;

roles/matrix-server/templates/systemd/matrix-nginx-proxy.service.j2

@@ -17,7 +17,7 @@ ExecStart=/usr/bin/docker run --rm --name matrix-nginx-proxy \
 			--link matrix-synapse:synapse \
 			--link matrix-riot-web:riot \
 			-v {{ matrix_nginx_proxy_confd_path }}:/etc/nginx/conf.d \
-			-v {{ matrix_ssl_certs_path }}:/acmetool-certs \
+			-v {{ matrix_ssl_certs_path }}:{{ matrix_ssl_certs_path }} \
 			{{ docker_nginx_image }}
 ExecStop=-/usr/bin/docker kill matrix-nginx-proxy
 ExecStop=-/usr/bin/docker rm matrix-nginx-proxy

roles/matrix-server/templates/systemd/matrix-riot-web.service.j2

@@ -9,6 +9,9 @@ ExecStartPre=-/usr/bin/docker kill matrix-riot-web
 ExecStartPre=-/usr/bin/docker rm matrix-riot-web
 ExecStart=/usr/bin/docker run --rm --name matrix-riot-web \
 			-v {{ matrix_nginx_riot_web_data_path }}:/data \
+			{% if not matrix_nginx_proxy_enabled %}
+			-p 127.0.0.1:8765:8765 \
+			{% endif %}
 			{{ docker_riot_image }}
 ExecStop=-/usr/bin/docker kill matrix-riot-web
 ExecStop=-/usr/bin/docker rm matrix-riot-web

roles/matrix-server/templates/systemd/matrix-synapse.service.j2

@@ -21,6 +21,9 @@ ExecStart=/usr/bin/docker run --rm --name matrix-synapse \
 			--link matrix-postgres:{{ matrix_postgres_connection_hostname }} \
 			{% endif %}
 			-p 8448:8448 \
+			{% if not matrix_nginx_proxy_enabled %}
+			-p 127.0.0.1:8008:8008 \
+			{% endif %}
 			-p 3478:3478 \
 			-p 3478:3478/udp \
 			-p {{ matrix_coturn_turn_udp_min_port }}-{{ matrix_coturn_turn_udp_max_port }}:{{ matrix_coturn_turn_udp_min_port }}-{{ matrix_coturn_turn_udp_max_port }}/udp \