Re: Subject header matching--once again

[Lars Henriksen]

On Mon, Nov 04, 2002 at 03:16:45PM -0800, Mel Hatzis wrote:
> On 11/04/2002 11:11 AM, Lars Henriksen wrote:
> >On Sun, Nov 03, 2002 at 09:17:08PM -0800, Mel Hatzis wrote:
(snip)
> >>
> >>Building on your proposal, I suggest that it'd be even better if an
> >>array of capture groups could be specified, each associated with a
> >>field name. This would allow for fields other than 'category' on the
> >>subject line.
> >>
> >>This could take the following form:
> >>
> >>subject-matching {
> >>    "\\<PR[ \t#/]?([0-9]+)[ \t]?:(.*)"
> >>    captured-fields {
> >>          "Number" "Synopsis"
> >>    }
> >>}
> >
> >
> >I like that, a nice, concise way of identifying the groups and their 
> >purpose,
> >The built-in default would then become
> >
> >   subject-matching {
> >       "\\<(PR[ \t#]?\\|([-\\w+.]+)/)([0-9]+)"
> >       captured-fields {
> >           "" "Category" "Number"
> >       }
> >   }

No one made any comments on this thread, and silence gives consent. Mel, I hope
that you are going to replace your pending patch for dbconfig configurable
Subject matching with one as discussed, at least in the reduced form with
Number and Category.

In the meantime the existing Subject match code should be fixed to reflect
the agreement reached a year back, see my first mail in this thread:
http://mail.gnu.org/pipermail/help-gnats/2002-November/003185.html

A patch follows that includes an update to the documentation. The feature
is mentioned a couple of times in passing in 'Keeping Track'. I think it
deserves a (sub)section of its own and have inserted one called 'Following up
via direct email' in the 'Editing existing Problem Reports' section of 'The
GNATS User Tools' chapter. I have also corrected a couple of minor errors
that I ran across.

The regular expression used for matching the Subject line appears in the code
as

     \\<(PR[ \t#/]?|([-A-Za-z0-9_+.]+)/)([0-9]+)

whereas the documentation has

     \<(PR[ \t#/]?|[-\w+.]+/)[0-9]+

I couldn't get the GNU match-word-constituent operator (\w) to work inside
the bracket expression and am uncertain as to whether it is allowed there.
Perl has it. The parentheses which are in the code, but missing from the
manual, do not affect the matching; they are there only to capture Category
and Number.

I haven't aligned the regular expression syntax with the rest of GNATS as
suggested by Milan. This is a non-issue as long as the regular expression
is hard-coded and not exposed for users to modify. The regex searching is also
case sensitive now.

The patch is in production use in the GNATS installation that I am responsible
for. I hope it can make it into GNATS 4.0-beta2?

Lars Henriksen

Index: file-pr.c
===================================================================
RCS file: /cvsroot/gnats/gnats/gnats/file-pr.c,v
retrieving revision 1.51
diff -u -r1.51 file-pr.c
--- file-pr.c   1 Nov 2002 11:37:51 -0000       1.51
+++ file-pr.c   1 Dec 2002 22:04:21 -0000
@@ -595,14 +595,13 @@
 static char *
 checkIfReply (PR *pr, ErrorDesc *err)
 {
-  char *prID = NULL;
+  char *prID;
+  char *prCat;
   AdmEntry *cat;
   const char *headerValue;
   struct re_pattern_buffer regex;
   struct re_registers regs;
-  int i, start, end, idstart;
-  char case_fold[256];
-  char *possiblePrNum;
+  int i, len;
   reg_syntax_t old_syntax;
 
   headerValue = header_value (pr, SUBJECT);
@@ -612,22 +611,15 @@
       return NULL;
     }
 
-  old_syntax = re_set_syntax (RE_NO_BK_PARENS);
+  old_syntax = re_set_syntax (RE_NO_BK_PARENS | RE_NO_BK_VBAR);
   memset ((void *) &regex, 0, sizeof (regex));
 
-  for (i=0; i<256; i++)
-    {
-      case_fold[i] = tolower(i);
-    }
-  regex.translate = case_fold;
-  
   {
-    const char *const PAT = "\\<((PR[ \t/])\\|([-a-z0-9_+.]+)/)([0-9]+)";
+    const char *const PAT = "\\<(PR[ \t#/]?|([-A-Za-z0-9_+.]+)/)([0-9]+)";
     re_compile_pattern (PAT, strlen (PAT), &regex);
   }
   i = re_search (&regex, headerValue, strlen (headerValue), 0,
                 strlen (headerValue), &regs);
-  regex.translate = NULL;
   regfree (&regex);
   re_set_syntax (old_syntax);
 
@@ -636,43 +628,39 @@
       return NULL;
     }
 
-  start = regs.start[0];
-  end = regs.end[0];
-  idstart = regs.start[4] - start;
-
-  free (regs.start);
-  free (regs.end);
-
-  possiblePrNum = xmalloc (end - start + 1);
-  memcpy (possiblePrNum, headerValue + start, end - start);
-  possiblePrNum[end - start] = '\0';
-
-  *(possiblePrNum + idstart - 1) = '\0';
-
-  /* See if the category exists: */
-  cat = get_adm_record (CATEGORY (pr->database), possiblePrNum);
-
-  /* If no such category, then this is not a reply to a valid
-   * problem report.  This situtation can arise, for example, when
-   * someone has the string "OS/2" in their Subject header.
-   */
-  /* Folks often send in "pr/1234" instead of a valid category */
-  if ((cat != NULL) || (strcasecmp (possiblePrNum, "pr") == 0))
+  /* Check the category if there is one. */
+  if (regs.start[2] > -1)
     {
-      /* We only needed res, never cat, so free cat. */
-      free_adm_entry (cat);
-      prID = xstrdup (possiblePrNum + idstart);
+      len = regs.end[2] - regs.start[2];
+      prCat = xmalloc (len + 1);
+      memcpy (prCat, headerValue + regs.start[2], len);
+      prCat[len] = '\0';
+
+      /* See if the category exists: */
+      cat = get_adm_record (CATEGORY (pr->database), prCat);
+      free_adm_entry(cat);
+      free (prCat);
+      if (cat == NULL)
+       {
+          free (regs.start);
+          free (regs.end);
+         return NULL;
+       }
     }
 
-  free (possiblePrNum);
+  /* Check the PR number. */
+  len = regs.end[3] - regs.start[3];
+  prID = xmalloc (len + 1);
+  memcpy (prID, headerValue + regs.start[3], len);
+  prID[len] = '\0';
+
+  free (regs.start);
+  free (regs.end);
 
-  if (prID != NULL)
+  if (! prExists (pr->database, prID, err))
     {
-      if (! prExists (pr->database, prID, err))
-       {
-         free (prID);
-         prID = NULL;
-       }
+      free (prID);
+      prID = NULL;
     }
 
   return prID;

Index: fields.texi
===================================================================
RCS file: /cvsroot/gnats/gnats/doc/fields.texi,v
retrieving revision 1.7
diff -u -r1.7 fields.texi
--- fields.texi 24 Oct 2002 20:30:54 -0000      1.7
+++ fields.texi 1 Dec 2002 22:07:19 -0000
@@ -196,8 +196,6 @@
 
 @c FIXME - this node is loooooooooooooooong...
 
address@hidden Field descriptions
-
 In a standard @sc{gnats} installation, certain fields will always be
 present in a Problem Report.  If a PR arrives without one or more of
 these fields, @sc{gnats} will add them, and if they have default
@@ -503,16 +501,15 @@
 The reason for the change.
 @end table
 
address@hidden follow-up via email
 @cindex subsequent mail
address@hidden other mail
address@hidden appending PRs
address@hidden saving related mail
 @cindex related mail
 @noindent
 The @code{Audit-Trail} field also contains any mail messages received by
 @sc{gnats} related to this PR, in the order received.  @sc{gnats} needs
-to find a @var{category}/@var{number} at the beginning of the Subject
-field of received e-mail in order to be able to file it correctly.
+to find a reference to the PR in the Subject field of received email in
+order to be able to file it correctly, see @ref{follow-up via email,,
+Following up via direct email}.
 
 @cindex @code{Unformatted} field
 @item Unformatted

Index: p-usage.texi
===================================================================
RCS file: /cvsroot/gnats/gnats/doc/p-usage.texi,v
retrieving revision 1.10
diff -u -r1.10 p-usage.texi
--- p-usage.texi        24 Oct 2002 11:38:25 -0000      1.10
+++ p-usage.texi        1 Dec 2002 22:08:07 -0000
@@ -116,8 +116,13 @@
 entries will also cause a copy of the new @samp{Audit-Trail} message to
 be sent.
 
+Mail received at the PR submission email address and recognized by
address@hidden as relating to an existing PR is also appended to the
address@hidden field, see @ref{follow-up via email}.
+
 @menu
 * edit-pr from the shell::  Invoking @code{edit-pr} from the shell
+* follow-up via email:: Following up via direct email
 @end menu
 
 @node edit-pr from the shell
@@ -184,6 +189,58 @@
 information.  When you exit the editor, @code{edit-pr} prompts you on
 standard input for a reason if you have changed a field that requires
 specifying a reason for the change.
+
address@hidden follow-up via email
address@hidden Following up via direct email
address@hidden follow-up via email
address@hidden subsequent mail
address@hidden related mail
+
+If you have some additional information for a PR and for some reason
+do not want to (or cannot) edit the PR directly, you may append
+the information to the Audit-Trail field by mailing it to the PR
+submission address.
+
+In order for GNATS to be able to recognize the mail as pertaining to an
+existing PR (as opposed to a new PR, see @ref{Submitting via e-mail,,}),
+the Subject mail header field must contain a reference to the PR.
+GNATS matches the Subject header against the regular expression
+
address@hidden
+\<(PR[ \t#/]?|[-\w+.]+/)[0-9]+
address@hidden smallexample
+
address@hidden
+to determine whether such a reference is present. Any text may precede
+or follow the reference in the Subject header. If more than one reference
+is present, the first is used and the rest ignored.
+
+A PR reference matching the regular expression above has two parts. The
+second is the PR number (one or more digits). The first is either the
+capital letters 'PR' optionally followed by a separator character (blank,
+tab, hash mark or forward slash) or the category name followed by a
+forward slash. Following are some examples which match the regular
+expression:
+
address@hidden
+PR 123 PR4567 PR#890 gnats/4711
address@hidden smallexample
+
+The PR number and the category (if present) are checked for existence,
+and if the outcome is positive, the mail is appended to the Audit-Trail
+field of the PR. Note that the PR need not belong to the category because
+PRs may move between categories.
+
+Outgoing emails sent by GNATS itself may be configured to have a Subject
+header field that refers to the PR in question:
+
address@hidden
+Subject: Re: PR @var{category}/@var{gnats-id}: @var{original message subject}
address@hidden smallexample
+
+This makes it extremely easy to follow up on a PR by replying to such an
+email, see @ref{dbconfig file,,The @code{dbconfig} file} and the sample,
+default @code{dbconfig} file installed by @code{mkdb}.
 
 @c ---------------------------------------------------------------
 @node query-pr

Index: s-usage.texi
===================================================================
RCS file: /cvsroot/gnats/gnats/doc/s-usage.texi,v
retrieving revision 1.7
diff -u -r1.7 s-usage.texi
--- s-usage.texi        24 Oct 2002 21:42:31 -0000      1.7
+++ s-usage.texi        1 Dec 2002 22:08:41 -0000
@@ -3,7 +3,7 @@
 submitting,,Submitting Problem Reports from Emacs}.)
 
 @menu
-* using send-pr::             Creating new Problem Reports
+* PR template::               The Problem Report template
 * send-pr in Emacs::          Using send-pr from within Emacs
 * send-pr from the shell::    Invoking send-pr from the shell
 * Submitting via e-mail::     Submitting a Problem Report via direct e-mail
@@ -98,7 +98,7 @@
 described in more detail in a separate section, @xref{Emacs,,The Emacs
 interface to @sc{gnats}}.
 
address@hidden using send-pr
address@hidden PR template
 @section The Problem Report template
 
 Invoking @code{send-pr} presents a PR @dfn{template} with a number of
@@ -247,24 +247,6 @@
 running @code{send-pr} from Emacs, the Problem Report is placed in the
 buffer @address@hidden; you can edit this file and then submit
 it with @kbd{C-c C-c}.
-
address@hidden subsequent mail
address@hidden other mail
address@hidden appending PRs
address@hidden saving related mail
address@hidden related mail
-Any further mail concerning this Problem Report should be carbon-copied
-to the @sc{gnats} mailing address as well, with the category and
-identification number in the @code{Subject} line of the message.
-
address@hidden
-Subject: Re: PR @var{category}/@var{gnats-id}: @var{original message subject}
address@hidden smallexample
-
address@hidden
-Messages which arrive with @code{Subject} lines of this form are
-automatically appended to the Problem Report in the @code{Audit-Trail}
-field in the order they are received.
 
 @c -------------------------------------------------------------------------
 @node Submitting via e-mail