Learned once the hard way that it makes sense to use "flock" to prevent overlapping executions of frequently running jobs. Server started to slow down, my monitoring jobs started piling, causing server to slow down even more.
Sometimes certain containerized processes need to run according to a schedule, but maintainers also need a way to run them manually without the scheduled processing running or starting concurrently. A shared FS seems like the ”simplest thing that could possibly work” distribution method for locks intended for that purpose, but unfortunately not all cloud storage volumes are strongly consistent, even to the same user, and may take several ms for the lock to take hold.
Wouldn't a database give you better consistency guarantees in that case? NFS locking semantics are a lot more complicated than just a `SELECT .. FOR UPDATE`
Sure, but that would require a separate database for this one use case. Mixing infra concerns into an app db doesn’t sound kosher, either, and a shared volume is already available.
Seems easier to have a managed lockfile for each process, diligently checking that the lock has actually been acquired. Performance is not a concern anyway, as long as acquire takes just a few ms we’re golden.
If a file system implements lock/unlock functions precisely to the spec, it should be fully consistent for the file/directory that is being locked. Does not matter if the file system is local or remote.
In other words, it's not the author's problem. It's the problem of a particular storage that may decide to throw the spec out of the window. But even in an eventually consistent file system, the manufacturer is better off ensuring that the locking semantics is fully consistent as per the spec.
