gnuastro-commits
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[gnuastro-commits] master 7278976e 1/2: Makefile extensions: accounting


From: Mohammad Akhlaghi
Subject: [gnuastro-commits] master 7278976e 1/2: Makefile extensions: accounting for more RAM than needed
Date: Wed, 17 Apr 2024 10:56:06 -0400 (EDT)

branch: master
commit 7278976ef86195f996b7a3f48c54b21a7fe4a624
Author: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Commit: Mohammad Akhlaghi <mohammad@akhlaghi.org>

    Makefile extensions: accounting for more RAM than needed
    
    Until now, the 'ast-text-prev-batch-by-ram' Makefile extension would return
    a "Floating point exception (core dumped)" error when its 'NEEDED_RAM_GB'
    argument was larger than the available RAM on the machine.
    
    With this commit, the problem has been fixed by adding a check for the
    number of elements in each batch: in case it is zero (as it happened here),
    it will become one (effectively no parallelization if used in a
    prerequisite list). Also, a sanity check has been added at the start of the
    worker function to catch such cases by other functions.
---
 doc/gnuastro.texi | 1 +
 lib/makeplugin.c  | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index 748d0893..0acaa258 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -36078,6 +36078,7 @@ Any other rule that is later added to this make file 
(as a prerequisite/parent o
 Similar to @code{ast-text-prev-batch}, but instead of taking the number of 
words/files in each batch, this function takes the maximum amount of RAM that 
is needed by one instance of the recipe.
 Through the @code{NEEDED_RAM_GB} argument, you should specify the amount of 
ram that a @emph{single} instance of the recipe in this rule needs.
 If any of the arguments are an empty string (or only contain space characters 
like `@key{SPACE}', `@key{TAB}', new-line and etc), this function will return 
an empty string (having no effect in Make).
+When the needed RAM is larger than the available RAM only one job will be done 
at a time (similar to @code{ast-text-prev}).
 
 The number of files in each batch is calculated internally by reading the 
available RAM on the system at the moment Make calls this function.
 Therefore this function is more generalizable to different computers (with 
very different RAM and/or CPU threads).
diff --git a/lib/makeplugin.c b/lib/makeplugin.c
index f0e37f93..fdbe3427 100644
--- a/lib/makeplugin.c
+++ b/lib/makeplugin.c
@@ -437,7 +437,8 @@ makeplugin_text_prev_batch_by_ram(const char *caller, 
unsigned int argc,
 
   /* Estimate the number of words in each batch (to be run in parallel if
      this function is used in targets list) and call the final function. */
-  num=(size_t)(ram_b/(needed_gb*1e9));
+  num=ram_b/((size_t)(needed_gb*1e9));
+  if(num==0) num=1; /* needed_gb > ram_b: no parallelization! */
   return makeplugin_text_prev_batch_work(target, num, list);
 }
 



reply via email to

[Prev in Thread] Current Thread [Next in Thread]