Re: [Nmh-workers] Locking (specifically, sequences)

[Ken Hornstein]

>That's what I tried to address with a lock-and-read-again (below)
>approach.  Maybe it's the same as what your suggesting.  But it
>doesnn't require keeping the fd open.  I don't think we want to
>do that, esp. if locked.

Well, I think it's unavoidable.  But it doesn't have to be for
long.  My plan is slightly different than yours, in that there's
no "complain if the sequence file is different".  I think that's
poor behavior, especially since (I believe) we can do the "right"
think with relatively little effort.

>--with-spool-locking?  Alternatively, --with-state-locking?
>I agree that run-time config would be nice.

I'm thinking --with-locking should define the default locking for the
spool file.  For state locking, we should simply default to fcntl (but
allow it to be changed at runtime).

>> See, I don't think this is necessary.  A brief look at the relevant
>> programs suggests to me that with a few API tweaks we can actually
>> maintain consistent locking across sequence files and (in most
>> cases) the locking window will be rather small.
>
>Could you list out the operation sequences?

Sure.

- seq_read() gets a new argument, a flag to control whether the sequence
  file is write-locked.  If it is write-locked, the descriptor for the
  lock gets stored in the folder structure (struct msgs).

- So, let's take a look at "show" as an example (other programs are similar).
  The general sequence is:

  1) folder_read() (which calls seq_read(), but NOT write-locking it).
  2) Convert message names to number
  3) Set the SELECT_UNSEEN bit on selected messages.
  4) Removed selected messages from unseen sequence (seq_setunseen())
  5) Set the previous-sequence (seq_setprev())
  6) Set the current message, the "cur" sequence (seq_setcur())
  7) Write out sequence file (seq_save()).

  So, what I WAS thinking was that between 3) and 4) I'd insert a call
  to seq_read() that would re-read the sequence file, but then write-lock
  it (and leave the descriptor open).  seq_save() would then see the open
  descriptor and close it after it wrote it.

  But looking at this further ... I am thinking that's dumb to have to
  reread the sequence file.  Those operations take almost no time, so
  perhaps the correct solution is to add a new argument to folder_read()
  to indicate whether or not the sequence file should be locked upon return.

  However, I can think of cases where seq_read() should be called directly
  in application programs.  For example, "pick" calls folder_read()
  relatively early, but searching through messages could take a while.
  So it probably makes sense to there to call seq_read() to read/lock
  the sequence file after all the messages are searched; that will
  reload any changes to the sequence file & write-lock it.

  The only program I see at first glance that might be a problem is inc,
  because that can potentially run for a long time.  Although ... it looks
  like what it does is call set_unseen() on each message as it's read
  in (which sets the SELECT_UNSEEN bit on the message) but the sequences
  aren't modified until after all of the messages are written out.
  So I think we can make that work.

>Context files hold private sequences, so I think they should
>be managed the same as sequences files.

Hm.  You know ... I never really understood the point of private sequences.
Why are they used?  _Do_ people use them?

--Ken