TLS Acceleration with relayd

relayd(8) can provide secure TLS acceleration.

Features:

Load balancer
Application layer gateway
Transparent proxy
SSL/TLS gateway

Pros:

Simpler to configure than haproxy or stunnel
Tight integration with OpenBSD's packet filter
Secure

Introduction

Sometimes a server does not provide TLS encryption by default. For example, leafnode lacks TLS encryption. It serves all its content as plaintext, exposing users' data to eavesdropping and forgery. When this happens, you can use relayd to provide TLS. This will encrypt connections and protect your users.

Another time relayd is useful is when we have two servers that both need to listen on the same socket ? (the same IP address and port). For example, the znc web panel and openhttpd both normally serve HTTPS content on the same IP address and on the same port, 443. It is not possible, however, to have two servers bind to the same socket. To share the same socket, it is necessary to use relayd with TLS Server Name Indication (SNI).

Before You Begin

Configure DNS

Before you begin, you will want to make sure DNS records are configured properly. You will want to use host to test if your hostnames are resolving properly:

$ host service1.example.com
service1.example.com has address 192.168.1.1
service1.example.com has IPv6 address 2001:db8::

Check Server in Plaintext

Before using relayd, you must make sure your server can serve its content in plaintext. If the server doesn't respond to plaintext requests, relayd won't work, either.

For webhosting, please see the openhttpd hosting guide.
For public bouncers, please see the znc chroot guide.

Afterward, you will want to test the setup using the telnet or netcat troubleshooting guides. openhttpd usually listens on port 80, and on IRCNow, the convention is to use port 1338 for znc's web server.

httpd must NOT listen on port 443

WARNING: If you are using relayd for TLS acceleration for openhttpd, make sure openhttpd does not have a listener on port 443. Look for blocks like the ones below in /etc/httpd.conf and comment it out:

server "example.com" {
        listen on * tls port 443
        tls {
                certificate "/etc/ssl/example.com.fullchain.pem"
                key "/etc/ssl/private/example.com.key"
        }
        location "/pub/*" {
                directory auto index
        }
        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }
}

Notice the line listen on * tls port 443 and the tls {...} block. If httpd is listening on port 443 while relayd is running, the reverse proxy will fail to forward for android, iOS, and other devices!

In the configuration below, we assume you are following the openhttpd hosting guide and listening on port 80.

Request SSL Certs

You will need the SSL certs for the domains you want to provide TLS acceleration for. Request them using acme-client if you have not already.

By default, relayd searches /etc/ssl/name:port.crt and /etc/ssl/private/name:port.key for the public/private keypair. If those are not present, it uses /etc/ssl/name.crt and /etc/ssl/private/name.key.

If your public cert and private key have different names, you should update /etc/acme-client.conf. It's recommended to use /etc/ssl/name.crt and /etc/ssl/private/name.key, where name is replaced with your actual domain name.

Optional: If you used the template from /etc/examples/acme-client.conf, your public cert ends in .fullchain.pem instead of .crt. In that case, you can create symbolic links:

$ doas ln -s /etc/ssl/example.com.fullchain.pem /etc/ssl/example.com.crt

You will want to replace example.com with your real domain. This will allow relayd to detect your public key. However, it is still highly recommended that you change acme-client.conf to create public keys that end with the extension .crt.

Edit relayd.conf

Let's create /etc/relayd.conf. Here is what we will put, one block at a time:

ip4="192.168.1.1"
ip6="2001:db8::"
table <service1> { 127.0.0.1 }
table <service2> { 127.0.0.1 }
log connection

Replace 192.168.1.1 and 2001:db8:: with your real IPv4 and IPv6 address ?. Make sure the IPv4 is DDoS-filtered if you have that option.

Replace service1 and service2 with the names of your real services, such as bnc, www, and mail.

Do NOT replace 127.0.0.1. You want relayd to forward its requests to the web server listening on localhost.

http protocol https {
        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" \
            value "$SERVER_ADDR:$SERVER_PORT"
        match request header set "Connection" value "close"
        tcp { sack, backlog 128 }
        tls { keypair service1.example.com }
        tls { keypair service2.example.com }
        match request header "Host" value "service1.example.com" forward to <service1>
        match request header "Host" value "service2.example.com" forward to <service2>
}

Here we define how to handle the http protocol. We add X-Forwarded-For, X-Forwarded-By, and Connection headers to HTTP requests before forwarding it to openhttpd.

We turn on selective acknowledgments and set the maximum queue to 128 connections in the tcp block.

We define the keypair names. Below is a table which shows the order in which relayd searches for them. Replace service1.example.com and service2.example.com with your real hostnames.

The last two lines in relayd.conf forward to the proper service based on the Host HTTP header.

Hostname: service1.example.com
Priority	Public Cert	Private Key
1	/etc/ssl/service1.example.com:443.crt	/etc/ssl/private/service1.example.com:443.key
2	/etc/ssl/service1.example.com.crt	/etc/ssl/private/service1.example.com.key

Hostname: service2.example.com
Priority	Public Cert	Private Key
1	/etc/ssl/service2.example.com:443.crt	/etc/ssl/private/service2.example.com:443.key
2	/etc/ssl/service2.example.com.crt	/etc/ssl/private/service2.example.com.key

The next section in relayd.conf indicates what port relayd should listen on, and where to forward:

relay wwwtls {
        listen on $ip4 port 443 tls
        protocol https
        forward to <service1> port 80 check icmp
        forward to <service2> port 80 check icmp
}
relay www6tls {
        listen on $ip6 port 443 tls
        protocol https
        forward to <service1> port 80 check icmp
        forward to <service2> port 80 check icmp
}

Replace service1 and service2 with the names of your real services.

We create two relays, one for IPv4 and another for IPv6. Both of them listen on port 443 using TLS. They use the protocol template for https and forward to the proper service on port 80 (see the above openhttpd hosting guide). Both check ICMP to see if the service is available.

Complete relayd.conf

Here is the entire /etc/relayd.conf without commentary:

Attach:relayd.conf Δ

ip4="192.168.1.1"
ip6="2001:db8::"
table <service1> { 127.0.0.1 }
table <service2> { 127.0.0.1 }
log connection

http protocol https {
        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" \
            value "$SERVER_ADDR:$SERVER_PORT"
        match request header set "Connection" value "close"
        tcp { sack, backlog 128 }
        tls { keypair service1.example.com }
        tls { keypair service2.example.com }
        match request header "Host" value "service1.example.com" forward to <service1>
        match request header "Host" value "service2.example.com" forward to <service2>
}

relay wwwtls {
        listen on $ip4 port 443 tls
        protocol https
        forward to <service1> port 80 check icmp
        forward to <service2> port 80 check icmp
}
relay www6tls {
        listen on $ip6 port 443 tls
        protocol https
        forward to <service1> port 80 check icmp
        forward to <service2> port 80 check icmp
}

Login class permissions

If you have a large number of TLS certs, you will need to increase the maximum number of files that relayd can open. Add this to the bottom of /etc/login.conf:

relayd:\
        :openfiles=4096:\
        :stacksize-cur=96M:\
        :stacksize-max=96M:\
        :tc=daemon:

Make sure there is no login.conf.db database, which would prevent the changes in login.conf from being applied:

$ doas rm /etc/login.conf.db

Starting relayd

To start relayd, use rcctl:

$ doas rcctl enable relayd
$ doas rcctl start relayd

WARNING: Make sure that packet filter is enabled! relayd will not run if pf is disabled. You can enable it by typing:

$ doas pfctl -e

To test relayd, use openssl.

Syslogd

The setting log connection produces enormous logs which will quickly fill up /var/log/daemon. To avoid this, configure syslogd ? to send all relayd messages into its own file. To that, see here.

In addition to splitting relayd logs to its own file, you may wish to create a new entry in your /etc/newsyslog.conf to handle log rotation for your relayd.

Configuring syslog

All log messages from relayd should go to /var/log/relayd.log. To do this, insert these three lines starting at line 3 (at the top) into /etc/syslog.conf:

!!relayd
*.*                                                     /var/log/relayd.log
!*

This directs all logs from relayd to go straight to /var/log/relayd.log.

Next, create the file /var/log/relayd.log and restart syslogd:

$ doas touch /var/log/relayd.log
$ doas rcctl restart syslogd

Troubleshooting

If relayd fails to start, you will see this message:

relayd(failed)

First, check the conf file to see if there are any errors:

$ doas relayd -n

When properly configured, relayd will say configuration OK.

Sample Errors

/etc/relayd.conf:NN: cannot load keypair example.com for relay wwwtls
Check line NN. Your keypair may be missing, have the wrong permissions, or are not labeled correctly.
/etc/relayd.conf:NN: syntax error

Check line NN for syntax errors.

To turn on debugging, first stop any running instances of relayd, then run it in the foreground:

$ doas rcctl stop relayd
$ doas relayd -dvv

-d is for debug and -v is to increase verbosity.

Common Mistakes

Make sure httpd is not also listening on port 443. Only one daemon can bind to the same socket ? at any time.
Make sure DNS records are properly configured; test with host
Make sure znc is listening on port 1338; test with netcat or telnet
Make sure packet filter is turned on