We're running dedicated vmd(8) servers to host opinionated VMs.
Follow us on Twitter or Mastodon as other kind people already do.

The setup

For all the people who want to know what our setup looks like. Below is a write-up of our setup and configuration. There aren't any packages installed on the servers running the Virtual Machines.

The building blocks

As with most things OpenBSD you can build almost anything with a base install alone. Our setup is no different! The building blocks which OpenBSD Amsterdam is built on are:

The building blocks that are also needed are pf(4), bridge(4), tap(4) and in our case vlan(4) and trunk(4).

All the configuration files are generated from a single text file, managed by rcs(1), which has all the details needed to create a VM.

The configuation files which need to be generated are:

vmm(4)/vmd(8)

It all starts of course with vmm(4)/vmd(8) and vm.conf(5). Our vm.conf(5) looks something like:

socket owner :vmdusers

switch "uplink_vlan42" {
    interface bridge42
}

vm "<vm-name>" {
    disable
    owner <user>
    disk "/var/vmm/<vm-name>.img"
    interface tap {
        switch "uplink_vlan42"
        lladdr fe:e1:bb:d1:c8:af
    }
}
...
vm "<vm-name>" {
    disable
    owner <user>
    disk "/var/vmm/<vm-name>.img"
    interface tap {
        switch "uplink_vlan42"
        lladdr fe:e1:bb:d1:e5:1d
    }
}

socket owner is introduced in -current and will be part of OpenBSD 6.4. This provides the option to change the owner of vmd.sock.

This is useful when multiple users need to control their own VM without adding them to group :wheel, the default owner.

A statically assigned MAC address (lladdr) is generated when a new server is build. This to make the installation of a new VM easier with autoinstall(8) in combination with dhcpd(8).

In order to make sure we have enough tap(4) interfaces, by default there are 4 present, we run the following command on a new server:

cd /dev
for i in $(jot 50 4 50); do sh MAKEDEV tap$i; done

dhcpd(8)

Our dhcpd.conf(5) is per vmd(8) server and pf(4) prevents the local dhcpd(8) server to respond to outside requests.

option domain-name "openbsd.amsterdam";
option domain-name-servers <dns>;

subnet <ip-space> netmask 255.255.255.0 {
    option routers <default-gateway-ipv4>;
    server-name "server#.openbsd.amsterdam";
    range <start-ip end-ip>;

    host <vm-name> {
        hardware ethernet fe:e1:bb:d1:c8:af;
        fixed-address <assigned-ipv4>;
        filename "auto_install";
        option host-name "<vm-name>";
    }
    ...
    host <vm-name> {
        hardware ethernet fe:e1:bb:d1:e5:1d;
        fixed-address <assigned-ipv4>;
        filename "auto_install";
        option host-name "<vm-name>";
    }

}

autoinstall(8)

When a new VM is being created autoinstall(8) is used for the installation of the VM. Per VM a config file is created based on the statically assigned MAC address. For example:

/var/www/htdocs/<install>/install-fe:e1:bb:d1:c8:af.conf

# <user> install.conf
System hostname = <hostname>
Password for root = <pwd>
Which speed should com0 = 115200
Network interfaces = vio0
IPv4 address for vio0 = dhcp
IPv6 address for vio0 = <assigned-ipv6>
IPv6 default router = <default-gateway-ipv6>
Setup a user = <user>
Password for user = <pwd>
Public ssh key for user = ssh-ed25519 AAAA...TxlrE5 <comment> <pwd>
Which disk is the root disk = sd0
What timezone are you in = Europe/Amsterdam
Location of sets = http
Server = ftp.nluug.nl
Set name(s) = -x* +xb* +xf*

/var/www/htdocs/<install>/install-fe:e1:bb:d1:e5:1d.conf

# <user> install.conf
System hostname = <hostname>
Password for root = <pwd>
Which speed should com0 = 115200
Network interfaces = vio0
IPv4 address for vio0 = dhcp
IPv6 address for vio0 = <assigned-ipv6>
IPv6 default router = <default-gateway-ipv6>
Setup a user = <user>
Password for user = <pwd>
Public ssh key for user = ssh-ed25519 AAAA...XrEl5T<comment> <pwd>
Which disk is the root disk = sd0
What timezone are you in = Europe/Amsterdam
Location of sets = http
Server = ftp.nluug.nl
Set name(s) = -x* +xb* +xf*

httpd(8)

Since OpenBSD comes with httpd(8) in base we are using it to serve the install-<MAC address>.conf files needed for autoinstall(8) with a minimal config.

server "default" {
    listen on * port 80
    root "/htdocs/<install>"
}

perl(1)

The text file used to capture all the VM data is partly generated by a perl script when a new server is being build. Both the MAC address as well as the temporary password are generated by this script. Here are two of the routines used to do this:

sub mac {
    my $mac_prefix = shift;
    my $random_mac;
    for ($i = 0; $i < 2; $i++) {
        $random_mac .= sprintf("%02x",int(rand(255))).(($i < 1)?':':'');
    }
    $random_mac = $mac_prefix . $random_mac;
    return $random_mac;
} 

sub generate_random_password {
    my $length_of_randomstring = shift;                         
    my @chars = ('a'..'z','A'..'Z','0'..'9');
    my $random_string;

    foreach (1..$length_of_randomstring) {
        $random_string .= $chars[rand @chars];
    }
    return $random_string;
}

We assign a MAC prefix per server and the last four bits are generated.

sensorsd(8)

To keep track of the hardware, especially disks, we are using sensorsd(8). The configuration for the disks we are using in sensorsd.conf(5) is as follows.

    hw.sensors.mfi0:command=echo "Raid state: %t %2" | mail -s \
"Sensor %t changed" -r noreply@example.com admin@example.com

pf.conf(5)

To prevent our local dhcpd(8) server to respond to outside requests we added a couple of rules to pf.conf(5).

block in quick on vlan42 proto {udp,tcp} to port 67:68
block out quick on vlan42 proto {udp,tcp} to port 67:68