March 05, 2019
For OpenBSD Amsterdam we were looking for a lightweight method to keep track of, at least, traffic and CPU load generated by the vmm(4)/vmd(8) hosts.
We had some experience with Observium, which doesn’t run well on OpenBSD, and LibreNMS. For some reason we were unable to get LibreNMS working on 6.4 nor on -current (6.5), so we decided to look elsewhere.
Considering our needs and what is available on OpenBSD we decided to go back in time and have a look at MRTG again.
Getting MRTG working with OpenBSD and collecting traffic is not a very big deal, cfgmaker is your friend! Getting CPU load was more of a challenge.
First we had to figure out what the SNMP OID were for the CPU, as the default ones, in the MRTG documentation didn’t cover them. We also had to consider the multi-core machines we are running.
After some digging in the MIBS we found ‘hrProcessorLoad’ in /usr/local/share/mibs/HOST-RESOURCES-MIB.txt.
$ snmpctl snmp walk <host> community <string> oid hrProcessorLoad
hrProcessorLoad.1=24
hrProcessorLoad.2=57
hrProcessorLoad.3=33
hrProcessorLoad.4=26
hrProcessorLoad.5=21
hrProcessorLoad.6=25
hrProcessorLoad.7=77
hrProcessorLoad.8=68
hrProcessorLoad.9=61
hrProcessorLoad.10=54
hrProcessorLoad.11=24
hrProcessorLoad.12=50
hrProcessorLoad.13=0
hrProcessorLoad.14=0
hrProcessorLoad.15=0
hrProcessorLoad.16=0
hrProcessorLoad.17=0
hrProcessorLoad.18=0
hrProcessorLoad.19=0
hrProcessorLoad.20=0
hrProcessorLoad.21=0
hrProcessorLoad.22=0
hrProcessorLoad.23=0
hrProcessorLoad.24=0
With some Startpage/DuckDuckGo-fu we stumbled upon a script that pulled a specific OID and ran some calculations on the CPU load based on the total number of cores and the sum of the load across these cores.
Here is the heavily modified version of that script.
#!/bin/sh
test -n "$1" || exit 1
HOST="$1"
CPUINFO="/tmp/cpuinfo.${HOST}"
snmpctl walk ${HOST} oid hrProcessorLoad | cut -d= -f2 > ${CPUINFO}
CORES=$(grep -cv "^0$" ${CPUINFO})
CPU_LOAD_SUM=$(awk '{sum += $1} END {print sum}' ${CPUINFO})
CPU_LOAD=$(echo "scale=2; ${CPU_LOAD_SUM}/${CORES}" | bc -l)
echo "$CPU_LOAD"
echo "$CPU_LOAD"
It reads all the CPU information from the host and writes the load of each core in a temporary file in /tmp. The cores are counted and the sum is calculated. Since SMT / Hyper Threading is off by default, we excluded the cores which are not taking any load.
MRTG expects two values, it primairily operates in inbound and outbound traffic, we print the $CPU_LOAD twice. Job done! Not quite…
It also expects the uptime to be presented in a readable format as well as the hostname.
So… TimeTicks here we come!
To collect the uptime of an OpenBSD machine we need to query hrSystemUptime.
hrSystemUptime OBJECT-TYPE
SYNTAX TimeTicks
MAX-ACCESS read-only
STATUS current
DESCRIPTION
"The amount of time since this host was last
initialized. Note that this is different from
sysUpTime in the SNMPv2-MIB [RFC1907] because
sysUpTime is the uptime of the network management
portion of the system."
::= { hrSystem 1 }
The snmpctl command is:
$ snmpctl snmp get <host> community <string> oid hrSystemUptime.0
0=15259107187
In order to get anything that resembles time we can read there are a number of calculations that need to happen.
15259107187 / 8640000 = days (+remainder) = 176.6107855324074
0.6107855324074 * 24 = hours (+remainder) = 14.65885277777778
0.65885277777778 * 60 = minutes (+remainder) = 39.53116666666667
0.53116666666667 * 60 = seconds.milliseconds = 31.87
Together with Roman Zolotarev we came up with the following part of the script:
TICKS=$(snmpctl snmp get ${HOST} oid hrSystemUptime.0 | cut -d= -f2)
DAYS=$(echo "${TICKS}/8640000" | bc -l)
HOURS=$(echo "0.${DAYS##*.} * 24" | bc -l)
MINUTES=$(echo "0.${HOURS##*.} * 60" | bc -l)
SECS=$(echo "0.${MINUTES##*.} * 60" | bc -l)
test -n "$DAYS" && printf '%s days, ' "${DAYS%.*}"
printf '%02d:%02d:%02d\n' "${HOURS%.*}" "${MINUTES%.*}" "${SECS%.*}"
Which results in 176 days, 14:39:31
The last part which MRTG expects is the hostname. This can be collected with:
snmpctl snmp get ${HOST} oid sysName.0 | cut -d= -f2 | tr -d '"'
All done!
What MRTG gets from the script is something like:
3.50
3.50
138 days, 02:37:03
server1.openbsd.amsterdam
The complete script can be found in our Git Repository.