>I'm curious, could you elaborate on the part about package updates and
>race conditions?
Updating a live service directory is difficult.
- You want updates to be atomic. If they're not, and for some
reason there's a failure mid-update, recovering may be very painful.
- Service directories are made of several files. Atomically updating
a directory is impossible if it hasn't been planned in advance
(i.e. you address the directory via a symlink, and actually
update the symlink atomically after creating an entire new directory,
just like s6-rc-update does). Service directories generally are not
built to be atomically updatable.
- Even if they were, this method would not work when the service
directory is watched by a scanner such as s6-svscan or runsvdir.
The scanner actually identifies the service directories it watches
via device number + inode, and creating a new directory changes the
inode. Without significant changes to the scanner, it is impossible
to transparently switch service directories while the scanner is
running.
- So updates to a service directory cannot be atomic. This is not
a real issue when the service is down - you can update one file
after another, keeping track of changes, and rolling back in case of
failure. But when the service is up, things are different : the
service may die at any time. So ./finish can be run at any time, and
stuff being notified on service death can be run at any time. You
don't know what files those scripts depend on. If the service dies
mid-update, things running on service death may see an inconsistent
state.
- Risking such failure, or preventing it from happening by downing
services first, during a package upgrade, is extremely ugly. Package
upgrade is a dangerous time where inconsistencies may appear, because
*no* package manager I know of does the safe thing; so you want to
make sure you don't make the machine more unstable than it needs to
be, you don't want to be restarting services if you can avoid it.
If you need to restart services so they can pick up new binaries and/or
new configs, it's best to defer it until the current batch of package
upgrades is done and you have cross-service consistency again.
- Keeping the "stock" service directories separate from the "live"
service directories is the simplest way to make sure live services do
not interfere when a package upgrade is underway. The stock service
directories are upgraded by the package manager; and there can be a
trigger updating the live service directories if necessary when the
package upgrade is done and stuff is (hopefully) consistent again.
This is one of the reasons why s6-rc keeps its livedir and its
service database separate: easy integration with a package manager's
workflow.
--
Laurent
Received on Sun Oct 29 2017 - 21:36:52 UTC