> I very much prefer regular manual updates, as it allows me to read release notes, bug reports and manual user intervention instructions before update.

Do you have the time for this though?

> 198 packages need to be updated

But, not having the time to review 198 updates shouldn’t justify automatically permitting the updates.

I agree there isn’t time to review them, unless you have plenty of staff and time (no one does)... but the recent Solarwinds debacle was a supply chain attack, and automatic updates allowed an exploit to be propagated to many companies and agencies.

I would ask myself if I need that many packages. And raise the red flag with management that maintaining proper security is a challenge under such circumstances.

How would reading release notes help in the case of the Solarwinds attack?

Sure, for most (including myself) it's impossible to review every little update. But the bug tracker of your distro is a good indicator if some of the most recent updates created a bunch of new tickets, especially those that are considered to be critical. In case of Arch Linux it is strongly recommended to read the latest update news on archlinux.org, before update.

In addition to that you get some experience over time which of your package updates caused most trouble in the past and you can take additional care. For example, I know that on my desktop the combo of displaylink and xorg does break from time to time with updates, therefore I carefully lookout for issues regarding xorg updates for displaylink users, before I do the xorg update.

Your service should be able to cope with a failure. Just ensure your updates don’t run at the same time on all your services.

I have two proxy servers which auto update. One in the morning, one in the evening. If the morning update breaks, it flags up the problem, and we can then fix before the evening one updates.

Okay, I have a server running the database (postgres) that backs my user-facing apps. How exactly shall I run kernel/glibc/systemd/postgres updates without customer-facing outages?

EDIT: To be clear, I'm mostly arguing that you're wrong, but if you have a solution then that'd be great, because I dislike the difficulty in patching these things.

Been a while since I’ve needed to run a critical database, but the options were generally a master-slave with replication, or an active-active cluster

Yes, we're currently doing primary-replica but that still requires downtime while we do a failover.

I remember building a system about 15 years ago which automatically failed over in under a second - the application connected to the most recent one and set up replication to the other one. If the master went down then the slave was promoted to master and when the original master came back it was configured as slave.

Looks like https://www.citusdata.com/blog/2019/05/30/introducing-pg-aut... will sort out a Postgres master master for you.

I certainly don’t have auto updates on all my servers (wouldn’t be good if a tv program going out to millions of people suddenly vanished — there’s always a glitch when it fails over), I do have it on public facing servers though, so in your case I’d have it on the web servers but not on the database.

Some people only have one server for a particular use case.

And servers might not always break in predictable/alertable ways.

If your service is only provided by one machine then you have to accept it will occasionally be out of use.

If the only monitoring of the service you have are users telling you, then that’s not a problem you can avoid by not updating.

> If your service is only provided by one machine then you have to accept it will occasionally be out of use.

But, I can minimize possible downtimes, by learning from other downtime reports on a bug tracker.