[gnuastro-commits] master 719aaea: Fits: CHECKSUM automatically updated if any keywords change

[Mohammad Akhlaghi]

branch: master
commit 719aaeafe96136bbc530f088f130e31d33024faf
Author: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Commit: Mohammad Akhlaghi <mohammad@akhlaghi.org>

    Fits: CHECKSUM automatically updated if any keywords change
    
    Until now, when any of the keywords were changed with the relevant options
    of the Fits program and the HDU already had a 'CHECKSUM' keyword, the
    change would invalidate the 'CHECKSUM'. The only way to avoid this was to
    call '--write=checksum' along with other modifications (as described under
    '--write' in the manual). For example the command below to delete the 'GSL'
    keyword.
    
        astfits a.fits --delete=GSL --write=checksum
    
    However, this was only useful for those keyword modifying operators that
    have less precedence than '--write' (precedence is also shown as a numbered
    list in the manual). As a result, it was is necessary for Gnuastro's Fits
    program to check if the file has a 'CHECKSUM' keyword or not. If 'CHECKSUM'
    is present, then upon any change in the keywords (Fits doesn't change the
    data!), the 'CHECKSUM' keyword should also be updated in the end.
    
    With this commit, this task has been done. In the process, I also noticed
    that we don't have a simple function to check the existance of a keyword,
    so a new function called 'gal_fits_key_exists_fptr' has also been
    added. Because of this change, the library so-name version was also
    incremented (it was still the same version as the published Gnuastro 0.16).
    
    This task was suggested by Tamara Civera Lorenzo.
---
 NEWS                         | 13 ++++++++
 THANKS                       |  1 +
 bin/fits/keywords.c          | 75 +++++++++++++++++++++++++++++++++++---------
 bin/fits/main.h              |  1 +
 bin/table/table.c            |  4 +--
 configure.ac                 |  2 +-
 doc/announce-acknowledge.txt |  1 +
 doc/gnuastro.texi            | 38 ++++++++++++++--------
 lib/fits.c                   | 13 ++++++++
 lib/gnuastro/fits.h          |  3 ++
 10 files changed, 121 insertions(+), 30 deletions(-)

diff --git a/NEWS b/NEWS
index 5d2283c..4cee3f4 100644
--- a/NEWS
+++ b/NEWS
@@ -28,6 +28,15 @@ See the end of the file for license conditions.
      fixed pixel size, even when the mode of the central coordinate is in
      WCS. This was suggested by Jesús Varela.
 
+  Fits:
+
+   - If a 'CHECKSUM' keyword exists in a HDU and any of the keyword
+     modification keywords are called, Fits will automatically update the
+     checksum after all the changes are done. This is important to keep the
+     checksum valid after the change (recall that you can verify the FITS
+     checksums with the '--verify' option to Fits). This task was suggested
+     by Tamara Civera Lorenzo.
+
   Table:
    --catrowfile: File to concatenate (i.e., add or append) rows into the
      main input table. With this option, you can add the rows of another
@@ -55,6 +64,10 @@ See the end of the file for license conditions.
    --kdtreehdu: the HDU of the external k-d tree (if a FITS file was given
      to '--kdtree').
 
+  Library:
+   - gal_fits_key_exists_fptr: Check if a certain keyword exists in the
+     already-open FITS file pointer.
+
 ** Removed features
 
 ** Changed features
diff --git a/THANKS b/THANKS
index 158c509..dc381c9 100644
--- a/THANKS
+++ b/THANKS
@@ -34,6 +34,7 @@ support in Gnuastro. The list is ordered alphabetically (by 
family name).
     Rosa Calvi                           rcalvi@iac.es
     Mark Calabretta                      mcalabre@atnf.csiro.au
     Nushkia Chamba                       chamba@iac.es
+    Tamara Civera Lorenzo                tcivera@cefca.es
     Benjamin Clement                     benjamin.clement@univ-lyon1.fr
     Nima Dehdilani                       nimadehdilani@gmail.com
     Andrés Del Pino Molina               adelpino@cefca.es
diff --git a/bin/fits/keywords.c b/bin/fits/keywords.c
index aa53e51..8518c53 100644
--- a/bin/fits/keywords.c
+++ b/bin/fits/keywords.c
@@ -112,6 +112,7 @@ keywords_rename_keys(struct fitsparams *p, fitsfile **fptr, 
int *r)
       /* Rename the keyword */
       fits_modify_name(*fptr, from, to, &status);
       if(status) *r=fits_has_error(p, FITS_ACTION_RENAME, from, status);
+      else p->updatechecksum=1;
       status=0;
 
       /* Clean up the user's input string. Note that 'strtok' just changes
@@ -129,8 +130,8 @@ keywords_rename_keys(struct fitsparams *p, fitsfile **fptr, 
int *r)
 /* Special write options don't have any value and the value has to be found
    within the script. */
 static int
-keywords_write_set_value(struct fitsparams *p, fitsfile **fptr,
-                         gal_fits_list_key_t *keyll)
+keywords_write_special(struct fitsparams *p, fitsfile **fptr,
+                       gal_fits_list_key_t *keyll)
 {
   int status=0;
 
@@ -159,6 +160,7 @@ keywords_write_set_value(struct fitsparams *p, fitsfile 
**fptr,
   else if( keyll->keyname[0]=='/' )
     {
       gal_fits_key_write_title_in_ptr(keyll->value, *fptr);
+      p->updatechecksum=1;
       return 0;
     }
   else
@@ -194,7 +196,7 @@ keywords_write_update(struct fitsparams *p, fitsfile **fptr,
       /* Deal with special keywords. */
       continuewriting=1;
       if( keyll->value==NULL || keyll->keyname[0]=='/' )
-        continuewriting=keywords_write_set_value(p, fptr, keyll);
+        continuewriting=keywords_write_special(p, fptr, keyll);
 
       /* Write the information: */
       if(continuewriting)
@@ -247,6 +249,11 @@ keywords_write_update(struct fitsparams *p, fitsfile 
**fptr,
              && fits_write_key_unit(*fptr, keyll->keyname, keyll->unit,
                                     &status) )
             gal_fits_io_error(status, NULL);
+
+          /* By this stage, a keyword has been written or updated. So its
+             necessary to update the checksum in the end. This should be
+             under the 'if(continuewriting)' conditional (because */
+          p->updatechecksum=1;
         }
 
       /* Free the allocated spaces if necessary: */
@@ -370,18 +377,19 @@ static void
 keywords_copykeys(struct fitsparams *p, char *inkeys, size_t numinkeys)
 {
   size_t i;
-  int status=0;
   long initial;
   fitsfile *fptr;
+  int status=0, updatechecksum=0, checksumexists=0;
 
   /* Initial sanity check. Since 'numinkeys' includes 'END' (counting from
      1, as we do here), the first keyword must not be larger OR EQUAL to
      'numinkeys'. */
   if(p->copykeysrange[0]>=numinkeys)
-    error(EXIT_FAILURE, 0, "%s (hdu %s): first keyword number give to "
-          "'--copykeys' (%ld) is larger than the number of keywords in this "
-          "header (%zu, including the 'END' keyword)", p->input->v, p->cp.hdu,
-          p->copykeysrange[0], numinkeys);
+    error(EXIT_FAILURE, 0, "%s (hdu %s): first keyword number give "
+          "to '--copykeys' (%ld) is larger than the number of "
+          "keywords in this header (%zu, including the 'END' "
+          "keyword)", p->input->v, p->cp.hdu, p->copykeysrange[0],
+          numinkeys);
 
   /* If the user wanted to count from the end (by giving a negative value),
      then do that. */
@@ -402,18 +410,29 @@ keywords_copykeys(struct fitsparams *p, char *inkeys, 
size_t numinkeys)
   /* Final sanity check (on range limit). */
   if(p->copykeysrange[1]>=numinkeys)
     error(EXIT_FAILURE, 0, "%s (hdu %s): second keyword number give to "
-          "'--copykeys' (%ld) is larger than the number of keywords in this "
-          "header (%zu, including the 'END' keyword)", p->input->v, p->cp.hdu,
-          p->copykeysrange[1], numinkeys);
-
+          "'--copykeys' (%ld) is larger than the number of keywords in "
+          "this header (%zu, including the 'END' keyword)", p->input->v,
+          p->cp.hdu, p->copykeysrange[1], numinkeys);
 
   /* Open the output HDU. */
   fptr=gal_fits_hdu_open(p->cp.output, p->outhdu, READWRITE);
 
+  /* See if a 'CHECKSUM' key exists in the HDU or not (to update in case we
+     wrote anything). */
+  checksumexists=gal_fits_key_exists_fptr(fptr, "CHECKSUM");
 
   /* Copy the requested headers into the output. */
   for(i=p->copykeysrange[0]-1; i<=p->copykeysrange[1]-1; ++i)
-    if( fits_write_record(fptr, &inkeys[i*80], &status ) )
+    {
+      if( fits_write_record(fptr, &inkeys[i*80], &status ) )
+        gal_fits_io_error(status, NULL);
+      else updatechecksum=1;
+    }
+
+  /* If a checksum existed, and we made changes in the file, we should
+     upate the checksum. */
+  if(checksumexists && updatechecksum)
+    if( fits_write_chksum(fptr, &status) )
       gal_fits_io_error(status, NULL);
 
   /* Close the output FITS file. */
@@ -919,16 +938,17 @@ int
 keywords(struct fitsparams *p)
 {
   char *inkeys=NULL;
-  int r=EXIT_SUCCESS;
   fitsfile *fptr=NULL;
   gal_list_str_t *tstll;
   int status=0, numinkeys;
+  int r=EXIT_SUCCESS, checksumexists=0;
 
   /* Print the requested keywords. Note that this option isn't called with
      the rest. It is independent of them. */
   if(p->keyvalue)
     keywords_value(p);
 
+
   /* Delete the requested keywords. */
   if(p->delete)
     {
@@ -941,11 +961,25 @@ keywords(struct fitsparams *p)
           fits_delete_key(fptr, tstll->v, &status);
           if(status)
             r=fits_has_error(p, FITS_ACTION_DELETE, tstll->v, status);
+          else
+            p->updatechecksum=1;
           status=0;
         }
     }
 
 
+  /* If the checksum keyword still exists in the HDU (wasn't deleted in the
+     previous step), then activate the flag to recalculate it at the
+     end. Note that the distortion or coordinate system functions will do
+     this job separately themselves. */
+  if(p->rename || p->update || p->write || p->asis || p->history
+     || p->comment || p->date)
+    {
+      keywords_open(p, &fptr, READWRITE);
+      checksumexists=gal_fits_key_exists_fptr(fptr, "CHECKSUM");
+    }
+
+
   /* Rename the requested keywords. */
   if(p->rename)
     {
@@ -978,6 +1012,7 @@ keywords(struct fitsparams *p)
         {
           fits_write_record(fptr, tstll->v, &status);
           if(status) r=fits_has_error(p, FITS_ACTION_WRITE, tstll->v, status);
+          else p->updatechecksum=1;
           status=0;
         }
     }
@@ -992,6 +1027,7 @@ keywords(struct fitsparams *p)
           fits_write_history(fptr, tstll->v, &status);
           if(status)
             r=fits_has_error(p, FITS_ACTION_WRITE, "HISTORY", status);
+          else p->updatechecksum=1;
           status=0;
         }
     }
@@ -1006,6 +1042,7 @@ keywords(struct fitsparams *p)
           fits_write_comment(fptr, tstll->v, &status);
           if(status)
             r=fits_has_error(p, FITS_ACTION_WRITE, "COMMENT", status);
+          else p->updatechecksum=1;
           status=0;
         }
     }
@@ -1017,10 +1054,17 @@ keywords(struct fitsparams *p)
       keywords_open(p, &fptr, READWRITE);
       fits_write_date(fptr, &status);
       if(status) r=fits_has_error(p, FITS_ACTION_WRITE, "DATE", status);
+      else p->updatechecksum=1;
       status=0;
     }
 
 
+  /* Update the checksum (if necessary). */
+  if(checksumexists && p->updatechecksum)
+    if( fits_write_chksum(fptr, &status) )
+      gal_fits_io_error(status, NULL);
+
+
   /* Print all the keywords in the extension. */
   if(p->printallkeys)
     {
@@ -1056,6 +1100,7 @@ keywords(struct fitsparams *p)
       keywords_date_to_seconds(p, fptr);
     }
 
+
   /* List all keyword names. */
   if(p->printkeynames)
     {
@@ -1063,6 +1108,7 @@ keywords(struct fitsparams *p)
       keywords_list_key_names(p, fptr);
     }
 
+
   /* Close the FITS file */
   if(fptr && fits_close_file(fptr, &status))
     gal_fits_io_error(status, NULL);
@@ -1075,6 +1121,7 @@ keywords(struct fitsparams *p)
       free(inkeys);
     }
 
+
   /* Convert the input's distortion to the desired output distortion. */
   if(p->wcsdistortion || p->wcscoordsys)
     keywords_wcs_convert(p);
diff --git a/bin/fits/main.h b/bin/fits/main.h
index 59dd7c3..b1422b7 100644
--- a/bin/fits/main.h
+++ b/bin/fits/main.h
@@ -93,6 +93,7 @@ struct fitsparams
   int                         mode;  /* Operating on HDUs or keywords.  */
   int                   coordsysid;  /* ID of desired coordinate system.*/
   int                 distortionid;  /* ID of desired distortion.       */
+  int               updatechecksum;  /* If CHECKSUM should be update.   */
   long            copykeysrange[2];  /* Start and end of copy.          */
   gal_fits_list_key_t  *write_keys;  /* Keys to write in the header.    */
   gal_fits_list_key_t *update_keys;  /* Keys to update in the header.   */
diff --git a/bin/table/table.c b/bin/table/table.c
index e0cc27b..1b4d770 100644
--- a/bin/table/table.c
+++ b/bin/table/table.c
@@ -849,7 +849,7 @@ table_catrows_findhdu(char *filename, gal_list_str_t 
**hdull)
 static size_t
 table_catrows_prepare(struct tableparams *p)
 {
-  char *hdu;
+  char *hdu=NULL;
   int tableformat;
   gal_data_t *tmp, *out=NULL;
   size_t nrows=p->table->size;
@@ -896,7 +896,7 @@ table_catrows_prepare(struct tableparams *p)
 static void
 table_catrows(struct tableparams *p)
 {
-  char *hdu;
+  char *hdu=NULL;
   gal_data_t *new, *ttmp, *tmp;
   gal_list_str_t *filell, *hdull;
   size_t colcount, ncols, ncolstest, filledrows;
diff --git a/configure.ac b/configure.ac
index c0d8728..b8a357f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -50,7 +50,7 @@ AC_CONFIG_MACRO_DIRS([bootstrapped/m4])
 
 # Library version, see the GNU Libtool manual ("Library interface versions"
 # section for the exact definition of each) for
-GAL_CURRENT=14
+GAL_CURRENT=15
 GAL_REVISION=0
 GAL_AGE=0
 GAL_LT_VERSION="${GAL_CURRENT}:${GAL_REVISION}:${GAL_AGE}"
diff --git a/doc/announce-acknowledge.txt b/doc/announce-acknowledge.txt
index b67d639..c2e4290 100644
--- a/doc/announce-acknowledge.txt
+++ b/doc/announce-acknowledge.txt
@@ -1,5 +1,6 @@
 Alphabetically ordered list to acknowledge in the next release.
 
+Tamara Civera Lorenzo
 Andres Del Pino Molina
 Alessandro Ederoclite
 Sepideh Eskandarlou
diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index 32770de..48e7279 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -9423,12 +9423,18 @@ You can read this option as 
column-information-in-standard-output.
 
 Below we will discuss the options that can be used to manipulate keywords.
 To see the full list of keywords in a FITS HDU, you can use the 
@option{--printallkeys} option.
-If any of the keyword modification options below are requested (for example 
@option{--update}), the headers of the input file will be changed first, then 
printed.
+If any of the keyword modification options below are requested (for example 
@option{--update}), the headers of the input file/HDU will be changed first, 
then printed.
 Keyword modification is done within the input file.
 Therefore, if you want to keep the original FITS file or HDU intact, it is 
easiest to create a copy of the file/HDU first and then run Fits on that (for 
copying a HDU to another file, see @ref{HDU information and manipulation}.
 In the FITS standard, keywords are always uppercase.
 So case does not matter in the input or output keyword names you specify.
 
+@cartouche
+@noindent
+@strong{@code{CHECKSUM} automatically updated, when present:} the keyword 
modification options will change the contents of the HDU.
+Therefore, if a @code{CHECKSUM} is present in the HDU, after all the keyword 
modification options have been complete, Fits will also update @code{CHECKSUM} 
before closing the file.
+@end cartouche
+
 Most of the options can accept multiple instances in one command.
 For example you can add multiple keywords to delete by calling 
@option{--delete} multiple times, since repeated keywords are allowed, you can 
even delete the same keyword multiple times.
 The action of such options will start from the top most keyword.
@@ -9779,6 +9785,8 @@ For more on their difference and links for further 
reading about epochs in astro
 @cindex SIP WCS distortion
 @cindex TPV WCS distortion
 If the argument has a WCS distortion, the output (file given with the 
@option{--output} option) will have the distortion given to this option (for 
example @code{SIP}, @code{TPV}).
+The output will be a new file (with a copy of the image, and the new WCS), so 
if it already exists, the file will be delete (unless you use the 
@code{--dontdelete} option, see @ref{Input output options}).
+
 With this option, the FITS program will read the minimal set of keywords from 
the input HDU and the HDU data, it will then write them into the file given to 
the @option{--output} option but with a newly created set of WCS-related 
keywords corresponding to the desired distortion standard.
 
 If no @option{--output} file is specified, an automatically generated output 
name will be used which is composed of the input's name but with the 
@file{-DDD.fits} suffix, see @ref{Automatic output}.
@@ -26714,22 +26722,26 @@ for adding elements to the list.
 @example
 typedef struct gal_fits_list_key_t
 @{
-  int                        tfree;   /* ==1, free title string.   */
-  int                        kfree;   /* ==1, free keyword name.   */
-  int                        vfree;   /* ==1, free keyword value.  */
-  int                        cfree;   /* ==1, free comment.        */
-  int                        ufree;   /* ==1, free unit.           */
-  uint8_t                     type;   /* Keyword value type.       */
-  char                      *title;   /* !=NULL, only print title. */
-  char                    *keyname;   /* Keyword Name.             */
-  void                      *value;   /* Keyword value.            */
-  char                    *comment;   /* Keyword comment.          */
-  char                       *unit;   /* Keyword unit.             */
-  struct gal_fits_list_key_t *next;   /* Pointer next keyword.     */
+  int                        tfree; /* ==1, free title string.  */
+  int                        kfree; /* ==1, free keyword name.  */
+  int                        vfree; /* ==1, free keyword value. */
+  int                        cfree; /* ==1, free comment.       */
+  int                        ufree; /* ==1, free unit.          */
+  uint8_t                     type; /* Keyword value type.      */
+  char                      *title; /* !=NULL, only print title.*/
+  char                    *keyname; /* Keyword Name.            */
+  void                      *value; /* Keyword value.           */
+  char                    *comment; /* Keyword comment.         */
+  char                       *unit; /* Keyword unit.            */
+  struct gal_fits_list_key_t *next; /* Pointer next keyword.    */
 @} gal_fits_list_key_t;
 @end example
 @end deftp
 
+@deftypefun int gal_fits_key_exists_fptr (fitsfile @code{*fptr}, char 
@code{*keyname})
+Return 1 (true) if the opened FITS file pointer contains the requested keyword 
and 0 (false) otherwise.
+@end deftypefun
+
 @deftypefun {void *} gal_fits_key_img_blank (uint8_t @code{type})
 Returns a pointer to an allocated space containing the value to the FITS
 @code{BLANK} header keyword, when the input array has a type of
diff --git a/lib/fits.c b/lib/fits.c
index bdedfde..0b2ea7d 100644
--- a/lib/fits.c
+++ b/lib/fits.c
@@ -955,6 +955,19 @@ gal_fits_hdu_open_format(char *filename, char *hdu, int 
img0_tab1)
 /**************************************************************/
 /**********            Header keywords             ************/
 /**************************************************************/
+int
+gal_fits_key_exists_fptr(fitsfile *fptr, char *keyname)
+{
+  int status=0;
+  char card[FLEN_CARD];
+  fits_read_card(fptr, keyname, card, &status);
+  return status==0;
+}
+
+
+
+
+
 /* Return allocated pointer to the blank value to use in a FITS file header
    keyword. */
 void *
diff --git a/lib/gnuastro/fits.h b/lib/gnuastro/fits.h
index 5008e2a..a661883 100644
--- a/lib/gnuastro/fits.h
+++ b/lib/gnuastro/fits.h
@@ -183,6 +183,9 @@ gal_fits_hdu_open_format(char *filename, char *hdu, int 
img0_tab1);
 /**************************************************************/
 /**********            Header keywords             ************/
 /**************************************************************/
+int
+gal_fits_key_exists_fptr(fitsfile *fptr, char *keyname);
+
 void *
 gal_fits_key_img_blank(uint8_t type);