Scripting FreeBSD Updates

Posted on Wed 06 March 2019 in technology

A small set of nits that I stumbled upon while attempting to script freebsd-update.

Paging Dr. Obvious

The worst part, bar none, is that it demands interaction from the user. If you're doing manual upgrades, this is fine: you probably want to know what changed, when, and how. In a scripted environment where you are running freebsd-update on several thousand hosts and containers, this is...inefficient, especially when you are confident that the messages are informational only, and don't need any kind of user intervention [1].

It is not immediately obvious how to get past all the prompts. My usual approach of piping the output of yes to the program didn't work because, it turns out, freebsd-update displays its prompts using the system $PAGER. Obviously.

From the code:

# Set a pager if the user doesn't
if [ -z "$PAGER" ]; then
        PAGER=/usr/bin/more
fi

The fix here is to set, somewhere in the calling script, $PAGER to something that allows output, but doesn't wait for user input. I ended up using cat:

export PAGER=/bin/cat

Hooray: output, but no more prompts.

Go Away, It Worked For Me

The second sticking point: scripts seemed to work fine on my development machine, but then fail silently when deployed to the remote machine. Kind of a bummer if your stated goal is bulletproof upgrades.

I will save how debugging this problem revealed a desperate need for better logging in my systems for another day, but the root of the problem is that freebsd-update refuses to fetch updates if not running from a TTY:

if [ ! -t 0 -a $NOTTYOK -eq 0 ]; then
        echo -n "`basename $0` fetch should not "
        echo "be run non-interactively."
        echo "Run `basename $0` cron instead."
        exit 1
fi

This probably merits an explanation in fetch section of the freebsd-update man page, which currently reads:

 fetch     Based on the currently installed world and the configuration
           options set, fetch all available binary updates.

The solution (if not the explanation) to our problem turns out to be in the man page: call freebsd-update with the --not-running-from-cron option:

 --not-running-from-cron
                Force freebsd-update fetch to proceed when there is no
                controlling tty.  This is for use by automated scripts and
                orchestration tools.  Please do not run freebsd-update
                fetch from crontab or similar using this flag, see:
                freebsd-update cron

Using this option allowed fetch to proceed even when run remotely. Yay.

FreeBSD Updates Aren't Ready For Cluster Computing

Did you blink a couple of times at the --not-running-from-cron option?

I understand that freebsd-update wants to save the user from footgun scenarios -- but the default behavior is not consistent with how I suspect most people expect upgrades and updates to work on a modern operating system. We are well into the era of treating servers (where FreeBSD is supposed to excel) like cattle; having core functionality assume interactivity by default is weirdly anachronistic and actively hostile to cluster computing.

I am not sure what way forward is, but whatever path we choose surely should not involve shell hacks and inelegant command line options.

[1] Your mileage will definitely vary. In my case, systems are engineered to leave base intact, thus preventing merge conflicts and making automated updates a possibility.