Building a Private Matrix Chat Server with Synapse and Element

Building a Private Matrix Chat Server with Synapse and Element

Matrix is an open standard for decentralized and end-to-end encrypted communication. It federates using homeservers that communicate with each other over the internet. Homeservers store data about their users and the chat history of rooms created (including rooms originating from other homeservers). Due to the nature of decentralization, when the original homeserver that created a room goes offline, other homeservers can continue communication without issues. Because of this design, there is not a single point of failure..

Configure Prerequisites#

Before executing any Docker commands, your underlying infrastructure needs to be properly provisioned.

Hardware Requirements#

Operating System & Core Engine#

Networking & Domain Setup#

Step 1: Generating the Configuration#

The docker-compose file is an example, please comment/uncomment sections that are not suitable for your usecase.

Terminal window
version: '3'
services:
synapse:
build:
context: ../..
dockerfile: docker/Dockerfile
image: docker.io/matrixdotorg/synapse:latest
restart: unless-stopped
environment:
- SYNAPSE_CONFIG_PATH=/data/homeserver.yaml
volumes:
# Store all files locally
- ./files:/data
depends_on:
- db
# Expose Synapse directly without a reverse proxy
ports:
- 8008:8008/tcp
- 8448:8448/tcp
db:
image: docker.io/postgres:12-alpine
environment:
- POSTGRES_USER=synapse
- POSTGRES_PASSWORD=ChangeME
- POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=C --lc-ctype=C
volumes:
# Store the database tables locally
- ./schemas:/var/lib/postgresql/data

Specify a SYNAPSE_CONFIG_PATH, preferably to a persistent path, to use manual configuration.

To generate a fresh homeserver.yaml, you can use the generate command. (See the documentation for more information.) You will need to specify appropriate values for at least the SYNAPSE_SERVER_NAME and SYNAPSE_REPORT_STATS environment variables. For example:

Terminal window
docker-compose run --rm -e SYNAPSE_SERVER_NAME=my.matrix.host -e SYNAPSE_REPORT_STATS=yes synapse generate

Then, customize your configuration and run the server:

Terminal window
docker-compose up -d

(This wClient Well-Known URI for your Reverse Proxyill also generate necessary signing keys.)
Ref:https://github.com/matrix-org/synapse/tree/develop/contrib/docker?ct=1783206682372

Step 2: Tweaking homeserver.yaml for Production#

Before you start Synapse for the first time, have a look at the Synapse configuration manual to familiarise yourself with the various settings. These configuration options can be added to your homeserver.yaml file.

Example:

Terminal window
# Configuration file for Synapse.
#
# This is a YAML file: see [1] for a quick introduction. Note in particular
# that *indentation is important*: all the elements of a list or dictionary
# should have the same indentation.
#
# [1] https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html
#
# For more information on how to configure Synapse, including a complete accounting of
# each option, go to docs/usage/configuration/config_documentation.md or
# https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html
server_name: "ikaldashev.ddns.net"
pid_file: /data/homeserver.pid
listeners:
- port: 8008
tls: false
type: http
x_forwarded: true
resources:
- names: [client, federation]
compress: false
database:
name: sqlite3
args:
database: /data/homeserver.db
log_config: "/data/ikaldashev.ddns.net.log.config"
media_store_path: /data/media_store
registration_shared_secret: "PASSWORD"
report_stats: true
signing_key_path: "/data/ikaldashev.ddns.net.signing.key"
trusted_key_servers:
- server_name: "matrix.org"
experimental_features:
msc3266_enabled: true
msc4222_enabled: true
msc4140_enabled: true
# The maximum allowed duration by which sent events can be delayed, as per MSC4140.
max_event_delay_duration: 24h
rc_message:
# This needs to match at least e2ee key sharing frequency plus a bit of headroom
# Note key sharing events are bursty
per_second: 0.5
burst_count: 30
rc_delayed_event_mgmt:
# This needs to match at least the heart-beat frequency plus a bit of headroom
per_second: 1
burst_count: 20
turn_uris:
- "turn:ikaldashev.ddns.net:3478?transport=udp"
- "turn:ikaldashev.ddns.net:3478?transport=tcp"
# if using TLS/DTLS you can also add:
- "turns:ikaldashev.ddns.net:5349?transport=udp"
- "turns:ikaldashev.ddns.net:5349?transport=tcp"
turn_shared_secret: "PASSWORD" #VOIP
turn_user_lifetime: 8640000 # e.g., 24h in milliseconds
turn_allow_guests: true
root@lab2:/home/ikaldashev/synaose/files#

It also keeps the user directory private, with a placeholder for your registration_shared_secret, which you’ll use to create users.

Step 3: Deploy Synapse and users#

Once you have saved your edited configuration files to your server directory, use the compose daemon to apply your modifications:

Terminal window
docker-compose down && docker-compose up -d

Then, create your first user and follow the prompts:

Terminal window
docker exec -it {docker_ID} register_new_matrix_user http://localhost:8008 -c /data/homeserver.yaml

Now, choose a client or app to log in with.

I went with the Element app from the Android store. It let me log into my server using the credentials from above. Element classic has downloads available for most platforms.

Step 4: Client Well-Known URI for your Reverse Proxy#

Setting up the client Well-Known URI is optional but if you set it up, it will allow users to enter their full username (e.g. @user:<server_name>) into clients which support well-known lookup to automatically configure the homeserver and identity server URLs. This is useful so that users don’t have to memorize or think about the actual homeserver URL you are using.

The URL https://<server_name>/.well-known/matrix/client should return JSON in the following format.

Terminal window
{
"m.homeserver": {
"base_url": "https://<matrix.example.com>"
}
}

It can optionally contain identity server information as well.

Terminal window
{
"m.homeserver": {
"base_url": "https://<matrix.example.com>"
},
"m.identity_server": {
"base_url": "https://<identity.example.com>"
}
}

To work in browser based clients, the file must be served with the appropriate Cross-Origin Resource Sharing (CORS) headers. A recommended value would be Access-Control-Allow-Origin: * which would allow all browser based clients to view it.

In nginx this would be something like:

Terminal window
location /.well-known/matrix/client {
return 200 '{"m.homeserver": {"base_url": "https://<matrix.example.com>"}}';
default_type application/json;
add_header Access-Control-Allow-Origin *;
}

You should also ensure the public_baseurl option in homeserver.yaml is set correctly. public_baseurl should be set to the URL that clients will use to connect to your server. This is the same URL you put for the m.homeserver base_url above.

Terminal window
public_baseurl: "https://<matrix.example.com>"

Step 5: Configuring the Coturn STUN/TURN Server#

The synapse Matrix homeserver supports integration with TURN server via the TURN server REST API. This allows the homeserver to generate credentials that are valid for use on the TURN server through the use of a secret shared between the homeserver and the TURN server.

Requirements#

For TURN relaying to work, the TURN service must be hosted on a server/endpoint with a public IP.

Hosting TURN behind NAT requires port forwarding and for the NAT gateway to have a public IP. However, even with appropriate configuration, NAT is known to cause issues and to often not work.

Afterwards, the homeserver needs some further configuration.

Synapse setup#

Your homeserver.yaml configuration file needs the following extra keys:

  1. turn_uris
  2. turn_shared_secret
  3. turn_user_lifetime
  4. turn_allow_guests

As an example, here is the relevant section of the config file for matrix.org. The turn_uris are appropriate for TURN servers listening on the default ports, with no TLS.

Terminal window
turn_uris: [ "turn:turn.matrix.org?transport=udp", "turn:turn.matrix.org?transport=tcp" ]
turn_shared_secret: "PASSWORD"
turn_user_lifetime: 86400000
turn_allow_guests: true

To handle voice and video calls (VoIP) reliably through firewalls and NAT networks, Synapse relies on a media relay server called Coturn. If you are setting up a secondary or standalone Coturn service, you need to configure its base daemon environment.

Create or edit the main config file in /DOCKER/coturn/turnserver.conf. The relevant lines, with example configurations for your server, are:

Example:

Terminal window
# Enable TLS/DTLS
listening-port=3478
tls-listening-port=5349
#tls-listening-port=443
no-tcp-relay
# TLS certificate and private key
cert=/certs/fullchain1.pem
pkey=/certs/privkey1.pem
# (Optional) stronger ciphers
cipher-list="ECDH+AESGCM:ECDH+AES256:ECDH+AES128:!aNULL:!MD5:!DSS"
# Fingerprint and no-SSO
fingerprint
#no-sslv2
#no-sslv3
#no-tls
#no-dtls
# Use shared secret authentication
use-auth-secret
static-auth-secret=PASSWORD //same in homeserver.yaml
#lt-cred-mech
#user=user1:mvwmaxVksfeYmaA
realm=ikaldashev.ddns.net
# External and internal IP mapping
external-ip=ikaldashev.ddns.net/192.168.1.101
# Explicit listener and relay IPs
listening-ip=0.0.0.0
#relay-ip=10.140.72.30
verbose
log-file=stdout
user-quota=50
total-quota=1200
min-port=49152
max-port=49200
no-multicast-peers
denied-peer-ip=10.0.0.0-10.255.255.255
#denied-peer-ip=192.168.0.0-192.168.255.255
denied-peer-ip=172.16.0.0-172.31.255.255

If you want to run Coturn inside a separate stack or keep its orchestrations modular, you can deploy it using a streamlined docker-compose.yml block. This approach mounts your local configurations directly into the server filesystem.

Terminal window
services:
coturn:
image: coturn/coturn
container_name: coturn
network_mode: host
restart: unless-stopped
#command: ["turnserver", "-c", "/etc/coturn/turnserver.conf", "-v", "--log-file=stdout", "--pidfile=/var/lib/coturn/turnserver.pid"]
volumes:
- "./turnserver.conf:/etc/coturn/turnserver.conf:ro"
- "./certs:/certs:ro"
- "coturn-data:/var/lib/coturn"
- "coturn-logs:/var/log"
volumes:
coturn-data:
coturn-logs:

Troubleshooting#

The normal symptoms of a misconfigured TURN server are that calls between devices on different networks ring, but get stuck at “call connecting”. Unfortunately, troubleshooting this can be tricky.

Here are a few things to try:

Try removing any AAAA records for your TURN server, so that it is only reachable over IPv4.

Terminal window
verbose

or with eturnal with the shell command eturnalctl loglevel debug or in the configuration file (the service needs to reload for it to become effective):

Terminal window
## Logging configuration:
log_level: debug

(Understanding the output is beyond the scope of this document!)

u=((date+p=((`date +%s` + 3600)):test p=(echo -n uopenssldgsthmacu | openssl dgst -hmac secret -sha1 -binary | base64) echo -e “username: u\npassword:u\npassword: p”

*Temporarily configure coturn to accept a static username/password. To do this, comment out use-auth-secret and static-auth-secret and add the following:
```bash
lt-cred-mech
user=username:password
License

CC BY-NC-SA 4.0 This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Related Posts