Subject: Re: ??? on NFS file locking

From: [email-masked] (Stephen R. van den Berg)
Date: Fri, 15 Oct 1993 16:52:07 +0100

Roman Czyborra <[email-masked]> wrote:
>know is:  how exactly does file locking  work in a network file system
>like I have here (i.e. mainly SunOS4 and Solaris2, a few Ultrix DECs)?

>used and hopefully safe synchronisation strategy.  If you tell me it's
>worth it, I'll even read Stephen's code.   If I've just missed the one
>hot source of information please direct me to it.

Tutorials on locking methods are hard to come by.  As to reading my code,
well, for the kernel-locking methods it's not worth it (that part is littered
with #ifdefs anyway), the system man pages provide ample documentation to that
respect.  But, for the dotlocking (.lock) method, my way of doing that
is fairly unique and worth taking a look at.

A brief overview regarding locking methods would be:
dotlocking (*.lock)		If you want to lock "testfile", try and
				create a *new* "testfile.lock" in the same
				directory.  Repeat (with ample sleep) until
				this succeeds.  To unlock, simply remove the
				file.  This method is supported by almost
				all configurations.
fcntl()		(POSIX)		Usually implemented over the lockd program
				(thus NFS aware).
lockf()		(SysV)		Usually implemented over fcntl().
flock()		(BSD)		On SunOS this only locks locally, on other
				systems it might use the lockd.

All these locking methods are advisory, some can be changed into
mandatory methods, but I strongly discourage that, it will be more trouble
than it is worth.
The kernel locking methods always lock the real file, not the *.lock
file.

>offsets.  So my operating  system has to  provide some locking method.
>And indeed  there  is flock or  lockd.   man  procmail tells  me  that
>procmail uses  flock.  But is that  really  safe?  Doesn't  flock just
>produce an advisory lock local to one host, cf. man flock BUGS?

On SunOS this is indeed inadequate, but that's where the dotlocking comes
in.  Procmail uses as many locking methods as are *reliable* on your
system.  The lockd is known for its *many* bugs.  When procmail is
being installed the lockingtests will determine if your lockd is reliable
enough for procmail's needs.  In the worst case, only dotlocking
remains.  You should be able to check what locking methods are being
employed by procmail in the procmail man page at the end of the
MISCELLANEOUS section.

>local locks seem very unsafe to me.  I haven't had any garbled folders
>so far but was that just good luck or good lock?

If you are not using dotlocking (lockfiles in procmail), then it was
"good luck".  BTW, sendmail is not a contender for the lock, it's either
procmail or your mail reader.  They should both employ at least on
overlapping locking method, or things look bad.

As far as I know the following locking methods are being employed:

SunOS /bin/mail, ucb-mail, Mail		/usr/spool/mail/$LOGNAME.lock
					& flock() on /usr/spool/mail/$LOGNAME
Generic /bin/mail, mailx, elm		/usr/spool/mail/$LOGNAME.lock
					& fcntl() on /usr/spool/mail/$LOGNAME
elm (depending on configuration)	fcntl() on all other mailboxes?
Emacs					/usr/spool/mail/$LOGNAME.lock
					& fcntl() on /usr/spool/mail/$LOGNAME
mush					fcntl() and *.lock (dotlocking) on
					all mailboxes.

>ucb-mail,  elm, pine, or vm.el?  Berkeley's  and Elm seem to be pretty
>clever  about new mail  coming into currently  visited folders.

Quite right.  Although, currently, as far as I know, mush is the only one
that can be configured to lock on every folder (which would be advisable
if procmail can deliver to any one of them).  If anyone is on good terms
with the elm-sources or elm-author(s), could he/she please check or
make sure that elm can (and does) do the same?

>  Emacs
>however only issues a warning but  cannot incorporate the new messages
>and would overwrite them given the yes.

Emacs is a bit limited in that respect.  Although, theoretically, it
could be changed to support arriving mail in files which are already
in a buffer.

[...several questions which are answered above deleted...]

>And did  you remember to
>always lock the  $LOGFILE?

Procmail (willfully) does *not* lock the $LOGFILE.  The $LOGFILE is
there for diagnostics and statistics, it only has informational
value.  To try and stay out of each other's hair, procmail writes
to the $LOGFILE in a line-buffered fashion.  Locking the $LOGFILE can
potentially deprive you of a few important diagnostics (because contending
procmails don't let go of the lock or whatever).

>  Do  I ever  have to use  the lockfile  -ml
>option  if  /usr/spool/mail  is   an  ordinary  writable  (drwxrwsrwt)
>directory?

No.  It exists solely for the benefit of systems that have non-world-writeable
mail-spool directories.

>I've noticed the sample rmail scripts that came with the procmail.tar.
>However, one big global lock is not an option  for me because it would
>pretty much  nullify  the  multitasking.  And  the  problem  with that
>2rmail local  lockfile wrapper is  that you could still change folders
>from  within mail  with the  fi  command.  Procmail's manpages left me
>with the impression that through  procmail everything's taken care of.
>Yet the more I think about it the more skeptical I grow.

Procmail can do only so much as to ensure that it doesn't interfere with
itself, and to make use of all known-to-be-reliable locking mechanisms
on your system.

The other program responsible is the mail-reader.  It ideally employs
the same locking mechanisms as procmail does.  I have no way to make
sure that your mail-reader does so.  The sample scripts are sort of a
first-aid solution to the problem (if you're not quite sure about what
methods your mail-reader uses).

Feel free to use other (better suited) scripts, or do without the scripts
if you know for sure that your mailreader uses at least one locking mechanism
on all mailboxes that overlaps with one of procmail's.

-- 
Sincerely,                                  [email-masked]
           Stephen R. van den Berg (AKA BuGless).    [email-masked]

"And now for something *completely* different!"