March 19, 2019
The post written about rdist(1) on johan.huldtgren.com sparked us to write one as well. It’s a great, underappreciated, tool. And we wanted to show how we wrapped doas(1) around it.
There are two services in our infrastructure for which we were looking to keep the configuration in sync and to reload the process when the configuration had indeed changed. There is a pair of nsd(8)/unbound(8) hosts and a pair of hosts running relayd(8)/httpd(8) with carp(4) between them.
We didn’t have a requirement to go full configuration management with tools like Ansible or Salt Stack. And there wasn’t any interest in building additional logic on top of rsync or repositories.
Enter rdist(1), rdist is a program to maintain identical copies of files over multiple hosts. It preserves the owner, group, mode, and mtime of files if possible and can update programs that are executing.
The only tricky part with rdist(1) is that in order to copy files and restart services, owned by a privileged user, has to be done by root. Our solution to the problem was to wrap doas(1) around rdist(1).
We decided to create a separate user account for rdist(1) to operate with on the destination host, for example:
ns2# useradd -m rupdate
Create an ssh key on the source host where you want to copy from:
ns1# ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_rdist
Copy the public key to the destination host for the rupdate user in .ssh/authorized_keys.
In order to wrap doas(1) around rdistd(1) we have to rename the original file. It’s the only way we were able to do this.
Move rdistd to rdistd-orig on the destination host:
ns2# mv /usr/bin/rdistd /usr/bin/rdistd-orig
Create a new shell script rdistd with the following:
#!/bin/sh
/usr/bin/doas /usr/bin/rdistd-orig -S
Make it executable:
ns2# chmod 555 /usr/bin/rdistd
Add rupdate to doas.conf(5) like:
permit nopass rupdate as root cmd /usr/bin/rdistd
permit nopass rupdate as root cmd /usr/bin/rdistd-orig
Once that is all done we can create the files needed for rdist(1).
To copy the nsd(8) and unbound(8) configuration we created a distfile like:
HOSTS = ( rupdate@ns2.example.com )
FILES = ( /var/nsd )
EXCL = ( nsd.conf *.key *.pem )
${FILES} -> ${HOSTS}
install ;
except /var/nsd/db ;
except /var/nsd/etc/${EXCL} ;
except /var/nsd/run ;
special "logger rdist update: $REMFILE" ;
cmdspecial "rcctl reload nsd" ;
unbound:
/var/unbound/etc/unbound.conf -> ${HOSTS}
install ;
special "logger rdist update: $REMFILE" ;
cmdspecial "rcctl reload unbound" ;
The distfile describes the destination HOSTS, the FILES which need to be copied and need to be EXCLuded. When it runs it will copy the selected FILES to the destination HOSTS, except the directories listed.
The install command is used to copy out-of-date files and/or directories.
The except command is used to update all of the files in the source list except for the files listed in name list.
The special command is used to specify sh(1) commands that are to be executed on the remote host after the file in name list is updated or installed.
The cmdspecial command is similar to the special command, except it is executed only when the entire command is completed instead of after each file is updated.
In our case the unbound(8) config doesn’t change very often, so we used a label to only update this when needed. With:
ns1# rdist unbound
To keep our relayd(8)/httpd(8) in sync we did something like:
HOSTS = ( rupdate@relayd2.example.com )
FILES = ( /etc/acme /etc/ssl /etc/httpd.conf /etc/relayd.conf /etc/acme-client.conf )
${FILES} -> ${HOSTS}
install ;
special "logger rdist update: $REMFILE" ;
cmdspecial "rcctl restart relayd httpd" ;
If you want cron(8) to pick this via the system script daily(8) you can save the file as /etc/Distfile.
To make sure the correct username and key are used you can add this to your .ssh/config file:
Host ns2.example.com
User rupdate
IdentityFile ~/.ssh/id_ed25519_rdist
When you don’t store the distfile in /etc you can add the following to your .profile:
alias rdist='rdist -f ~/distfile'
Running rdist will result in the following type of logging on the destination host:
==> /var/log/daemon <==
Nov 13 09:59:15 name2 rdistd-orig[763]: ns2: startup for ns1.example.com
==> /var/log/messages <==
Nov 13 09:59:15 ns2 rupdate: rdist update: /var/nsd/zones/reverse/192.168.10.0
==> /var/log/daemon <==
Nov 13 09:59:16 ns2 nsd[164]: zone 10.168.192.in-addr.arpa read with success