gnuastro-commits
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[gnuastro-commits] master 0d5776a 1/2: Pointer library: memory-mapped fi


From: Mohammad Akhlaghi
Subject: [gnuastro-commits] master 0d5776a 1/2: Pointer library: memory-mapped files only written in .gnuastro_mmap
Date: Sun, 15 Mar 2020 15:44:59 -0400 (EDT)

branch: master
commit 0d5776acd448de897ab08aaf1acc5041a9fbaf3f
Author: Mohammad Akhlaghi <address@hidden>
Commit: Mohammad Akhlaghi <address@hidden>

    Pointer library: memory-mapped files only written in .gnuastro_mmap
    
    Until now, when a file needed to be memory-mapped, Gnuastro's programs
    (through the `pointer.h' library) would first try writing the mmap files in
    the `.gnuastro' directory. When it failed it would attempt writing in the
    `.gnuastro_mmap' directory. However, `.gnuastro' is also used to store
    configuration files (which are hand-written and thus valuable). Mixing the
    two types of source (configuration files) and automatically generated
    (memory-mapped) files is very problematic.
    
    With this commit, it no longer attempts to write memory-mapped files under
    `.gnuastro'. It will only attempt to write them under the `.gnuastro_mmap'
    directory.
---
 NEWS              | 10 ++++++++++
 doc/gnuastro.texi | 13 +++++--------
 lib/pointer.c     | 22 ++++++++--------------
 3 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/NEWS b/NEWS
index c27be58..43c9e97 100644
--- a/NEWS
+++ b/NEWS
@@ -48,6 +48,16 @@ See the end of the file for license conditions.
 ** Changed features
 
   All programs and libraries:
+   --minmapsize: Gnuastro's programs no longer attempt to write
+     memory-mapped files under `.gnuastro'. They will only attempt to write
+     them under the `.gnuastro_mmap' directory. Until now, when an internal
+     array needed to be memory-mapped, Gnuastro's programs (through the
+     `pointer.h' library) would first try writing the mmap files in the
+     `.gnuastro' directory. When it failed it would attempt writing in the
+     `.gnuastro_mmap' directory. However, `.gnuastro' is also used to store
+     configuration files (which are hand-written and thus valuable). Mixing
+     the two types of source (configuration files) and automatically
+     generated (memory-mapped) files is very problematic.
    - FITS ASCII tables: When a column has a floating point type, but its
      ASCII string can't be parsed as a number, it will be read as a
      NaN. Until now, the corresponding program/library would abort,
diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index e6414dd..f388b9d 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -6225,9 +6225,8 @@ In such scenarios, without this option, the program will 
crash.
 A random filename is assigned to the array.
 This file will keep the contents of the array as long as it is necessary and 
the program will delete it as soon as its not necessary any more.
 
-If the @file{.gnuastro} directory exists and is writable, then the random file 
will be placed in there.
-Otherwise, the @file{.gnuastro_mmap} directory will be checked.
-If @file{.gnuastro_mmap} does not exist, or @file{.gnuastro} is not writable, 
the random file will be directly written in the current directory with the 
@file{.gnuastro_mmap_} prefix.
+If the @file{.gnuastro_mmap} directory exists and is writable, then the random 
file will be placed in there.
+Otherwise, the randomly named file will be directly written in the current 
directory with the @file{.gnuastro_mmap_} prefix.
 
 By default, the name of the created file, and its size (in bytes) is printed 
by the program when it is created and later, when its deleted/freed.
 These messages are useful to the user who has enough RAM, but has forgot to 
increase the value to @code{--minmapsize} (this is often the case).
@@ -19261,11 +19260,9 @@ value of this variable is @code{NULL}, then the 
contents of @code{array}
 are actually stored in RAM, not in a file on the HDD/SSD. See the
 description of @code{minmapsize} below for more.
 
-If a file is used, it will be kept in the hidden @file{.gnuastro}, or
-@file{.gnuastro_mmap} directories with a randomly selected name to allow
-multiple arrays to be kept there at the same time, see description of
-@option{--minmapsize} in @ref{Processing options}. When
-@code{gal_data_free} is called the randomly named file will be deleted.
+If a file is used, it will be kept in the hidden @file{.gnuastro_mmap} 
directory.
+Its name is randomly selected to allow multiple arrays at the same time, see 
description of @option{--minmapsize} in @ref{Processing options}.
+When @code{gal_data_free} is called the randomly named file will be deleted.
 
 @item size_t minmapsize
 The minimum size of an array (in bytes) to store the contents of
diff --git a/lib/pointer.c b/lib/pointer.c
index 58e6d70..e01a2a5 100644
--- a/lib/pointer.c
+++ b/lib/pointer.c
@@ -115,30 +115,24 @@ gal_pointer_allocate_mmap(uint8_t type, size_t size, int 
clear,
   size_t bsize=size*gal_type_sizeof(type);
 
 
-  /* Check if the .gnuastro folder exists, write the file there. If it
-     doesn't exist, then make the .gnuastro directory. If it can't be
-     built, we'll make a randomly named directory. */
-  gal_checkset_allocate_copy("./.gnuastro/", &dirname);
+  /* Check if the `.gnuastro_mmap' folder exists, write the file there. If
+     it doesn't exist, then make it. If it can't be built, we'll make a
+     randomly named file in the current directory. */
+  gal_checkset_allocate_copy("./.gnuastro_mmap/", &dirname);
   if( gal_checkset_mkdir(dirname) )
     {
-      /* Free the old name. */
+      /* The directory couldn't be built. Free the old name. */
       free(dirname);
 
-      /* Try `.gnuastro_mmap' (to avoid making a separate directory for
-         each memory mapping if possible). */
-      gal_checkset_allocate_copy("./.gnuastro_mmap/", &dirname);
-      if( gal_checkset_mkdir(dirname) )
-        {
-          free(dirname);
-          dirname=NULL;
-        }
+      /* Set `dirname' to NULL so it knows not to write in a directory. */
+      dirname=NULL;
     }
 
 
   /* Set the filename. If `dirname' couldn't be allocated, directly make
      the memory map file in the current directory (just as a hidden
      file). */
-  if( asprintf(filename, "%smmap_XXXXXX", dirname?dirname:"./.gnuastro_")<0 )
+  if( asprintf(filename, "%sXXXXXX", dirname?dirname:"./.gnuastro_mmap_")<0 )
     error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
   if(dirname) free(dirname);
 



reply via email to

[Prev in Thread] Current Thread [Next in Thread]