gnuastro-commits
[Top][All Lists]

## [gnuastro-commits] master 7ac8715 2/2: Book: redistributed installed scr

 From: Mohammad Akhlaghi Subject: [gnuastro-commits] master 7ac8715 2/2: Book: redistributed installed script and astscript-sort-by-night sections Date: Wed, 24 Apr 2019 07:20:48 -0400 (EDT)

branch: master
commit 7ac87152de5f1ca9f3b9cda60b91e5c015c542a2

Book: redistributed installed script and astscript-sort-by-night sections

Until now, we had a separate chapter for installed scripts. This chapter
was intended to house the installed scripts. However, in the manual, we
have separate chapters for different types of operations. Therefore having
the scripts in a separate chapter (independent of the operation they do)
was confusing.

With this commit, the general introduction to installed scripts has been
moved to the "Common program behavior" chapter and the "Sort FITS files by
night" section has been moved to the data containers chapter.
---
doc/gnuastro.texi | 1377 ++++++++++++++++++++++++++++-------------------------
1 file changed, 721 insertions(+), 656 deletions(-)

diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index 34346ac..d0f1396 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -205,7 +205,6 @@ sub-component to a title is present.
* Data analysis::               Analyze images.
* Modeling and fittings::       Make and fit models.
* High-level calculations::     Physical calculations.
-* Installed scripts::           Higher-level usage of (multiple) programs.
* Library::                     Gnuastro's library of useful functions.
* Developing::                  The development environment.
* Gnuastro programs list::      List and short summary of Gnuastro.
@@ -295,6 +294,7 @@ Common program behavior
* Command-line::                How to use the command-line.
* Configuration files::         Values for unspecified variables.
+* Installed scripts::           Installed Bash scripts, not compiled programs.
* Numeric data types::          Different types and how to specify them.
* Tables::                      Recognized table formats.
@@ -352,6 +352,7 @@ Recognized table formats
Data containers

* Fits::                        View and manipulate extensions and keywords.
+* Sort FITS files by night::    Installed script to sort FITS files by obs
night.
* ConvertType::                 Convert data to various formats.
* Table::                       Read and Write FITS tables to plain text.

@@ -364,6 +365,10 @@ Invoking Fits
* HDU manipulation::            Manipulate HDUs within a FITS file.
* Keyword manipulation::        Manipulate metadata keywords in a HDU

+Sort FITS files by night
+
+* Invoking astscript-sort-by-night::  Inputs and outputs to this script.
+
ConvertType

* Recognized file formats::     Recognized file formats
@@ -550,14 +555,6 @@ Invoking CosmicCalculator
* CosmicCalculator input options::  Options to specify input conditions.
* CosmicCalculator specific calculations::  Requesting specific outputs.

-Installed scripts
-
-* Sort FITS files by night::
-
-Sort FITS files by night
-
-* Invoking astscript-sort-by-night::  Inputs and outputs to this script.
-
Library

* Review of library fundamentals::  Guide on libraries and linking.
@@ -7229,6 +7226,7 @@ store some very useful information in the header that is
discussed in
* Command-line::                How to use the command-line.
* Configuration files::         Values for unspecified variables.
+* Installed scripts::           Installed Bash scripts, not compiled programs.
* Numeric data types::          Different types and how to specify them.
* Tables::                      Recognized table formats.
@@ -8464,7 +8462,12 @@ source by your self as in @ref{Quick start}.

Common program behavior
+
+
+
+
+
address@hidden Getting help, Installed scripts, Configuration files, Common
program behavior
@section Getting help

@cindex Help
@@ -8778,7 +8781,101 @@ list. We have other mailing lists and tools for those
purposes, see

Common program behavior
+
+
+
+
+
Common program behavior
+
+Gnuastro's programs (introduced in previous chapters) are designed to be
+highly modular and thus mainly contain lower-level operations on the
+data. However, in many contexts, higher-level operations (for example a
+sequence of calls to multiple Gnuastro programs, or a special way of
+running a program and using the outputs) are also very similar between
+various projects.
+
+To facilitate data analysis on these higher-level steps also, Gnuastro also
+installs some scripts on your system with the (@code{astscript-}) prefix
+(in contrast to the other programs that only have the @code{ast}
+prefix).
+
+Like all of Gnuastro's source code, these scripts are also heavily
+commented. They are written in GNU Bash, which doesn't need
+compilation. Therefore, if you open the installed scripts in a text editor,
+only starting with @code{ast}) aren't human-readable. They are written in C
+and are thus compiled (optimized in binary CPU instructions that will be
+given directly to your CPU). Because they don't need an interpretter like
+Bash on every run, they are much faster and more independent than
+scripts. To read the source code of the programs, look into the
+source}). If you would like to read more about why C was chosen for the
+programs, please see @ref{Why C}.}. Bash is the same language that is
+mainly used when typing on the command-line. Because of these factors, Bash
+is much more widely known and used than C (the language of other Gnuastro
+programs). Gnuastro's installed scripts also do higher-level operations, so
+customizing these scripts for a special project will be more common than
+the programs. You can always inspect them (to customize, check, or educate
+text editor):
+
+$emacs$(which astscript-NAME)
+
+These scripts also accept options and are in many ways similar to the
+programs (see @ref{Common options}) with some minor differences:
+
+Currently they don't accept configuration files themselves. However, the
+configuration files of the Gnuastro programs they call are indeed parsed
+and used by those programs.
+
+As a result, they don't have the following options: @option{--checkconfig},
+
+They don't directly allocate any memory, so there is no
+
+They don't have an independent @option{--usage} option: when called with
address@hidden, they just recommend running @option{--help}.
+
+The output of @option{--help} is not configurable like the programs (see
+
+The scripts will commonly use your installed Bash and other basic
+command-line tools (for example AWK or SED). Different systems have
+different versions and implementations of these basic tools (for example
+GNU/Linux systems use GNU AWK and GNU SED which are far more advanced and
+up to date then the minimalist AWK and SED of most other
+systems). Therefore, unexpected errors in these tools might come up when
+you run these scripts. We will try our best to write these scripts in a
+portable way. However, if you do confront such strange errors, please
+submit a bug report so we fix it (see @ref{Report a bug}).
+
+
+
+
+
+
+
+
+
+
+
scripts, Common program behavior

@pindex nproc
@@ -9885,6 +9982,7 @@ save them into another plain text or FITS table.

* Fits::                        View and manipulate extensions and keywords.
+* Sort FITS files by night::    Installed script to sort FITS files by obs
night.
* ConvertType::                 Convert data to various formats.
* Table::                       Read and Write FITS tables to plain text.
@@ -9893,7 +9991,7 @@ save them into another plain text or FITS table.

address@hidden Fits, ConvertType, Data containers, Data containers
address@hidden Fits, Sort FITS files by night, Data containers, Data containers
@section Fits

@cindex Vatican library
@@ -10538,138 +10636,387 @@ number of days in different months, or leap years,
and etc).

address@hidden ConvertType, Table, Fits, Data containers
-
-The FITS format used in astronomy was defined mainly for archiving,
-transmission, and processing. In other situations, the data might be useful
-in other formats. For example, when you are writing a paper or report, or
-if you are making slides for a talk, you can't use a FITS image. Other
-image formats should be used. In other cases you might want your pixel
-values in a table format as plain text for input to other programs that
-don't recognize FITS. ConvertType is created for such situations. The
-various types will increase with future updates and based on need.

-The conversion is not only one way (from FITS to other formats), but two
-ways (except the EPS and PDF address@hidden EPS and PDF are
-vector, not raster/pixelated formats}). So you can also convert a JPEG
-image or text file into a FITS image. Basically, other than EPS/PDF, you
-can use any of the recognized formats as different color channel inputs to
-get any of the recognized outputs. So before explaining the options and
-description of the recognized files types in @ref{Recognized file formats},
-followed a short introduction to digital color in @ref{Color}.

-* Recognized file formats::     Recognized file formats
-* Color::                       Some explanations on color.
-* Invoking astconvertt::        Options and arguments to ConvertType.

address@hidden Recognized file formats, Color, ConvertType, ConvertType

-The various standards and the file name extensions recognized by
-ConvertType are listed below. Currently Gnuastro uses the file name's
-suffix to identify the format.

-Astronomical data are commonly stored in the FITS format (or the older data
-IRAF @file{.imh} format), a list of file name suffixes which indicate that
-the file is in this format is given in @ref{Arguments}.

-Each image extension of a FITS file only has one value per
-pixel/element. Therefore, when used as input, each input FITS image
-contributes as one color channel. If you want multiple extensions in one
-FITS file for different color channels, you have to repeat the file name
-multiple times and use the @option{--hdu}, @option{--hdu2}, @option{--hdu3}
-or @option{--hdu4} options to specify the different extensions.
address@hidden Sort FITS files by night, ConvertType, Fits, Data containers
address@hidden Sort FITS files by night

-The JPEG standard was created by the Joint photographic experts
-group. It is currently one of the most commonly used image
-formats. Its major advantage is the compression algorithm that is
-defined by the standard. Like the FITS standard, this is a raster
-graphics format, which means that it is pixelated.
+FITS images usually contain (several) keywords for preserving important
+dates. In particular, for lower-level data, this is usually the observation
+date and time (for example, stored in the @code{DATE-OBS} keyword
+value). When analyzing observed datasets, many calibration steps (like the
+dark, bias or flat-field), are commonly calculated on a per-observing-night
+basis.

-A JPEG file can have 1 (for gray-scale), 3 (for RGB) and 4 (for CMYK)
-color channels. If you only want to convert one JPEG image into other
-formats, there is no problem, however, if you want to use it in
-combination with other input files, make sure that the final number of
-color channels does not exceed four. If it does, then ConvertType will
-abort and notify you.
+However, the FITS standard's date format (@code{YYYY-MM-DDThh:mm:ss.ddd})
+is based on the western (Gregorian) calendar. Dates that are stored in this
+format are complicated for automatic processing: a night starts in the
+final hours of one calendar day, and extends to the early hours of the next
+calendar day. As a result, to identify datasets from one night, we commonly
+need to search for two dates. However calendar peculiarities can make this
+identification very difficult. For example when an observation is done on
+the night separating two months (like the night starting on March 31st and
+going into April 1st), or two years (like the night starting on December
+31st 2018 and going into January 1st, 2019). To account for such
+situations, it is necessary to keep track of how many days are in a month,
+and leap years, and etc.

-The file name endings that are recognized as a JPEG file for input
-are: @file{.jpg}, @file{.JPG}, @file{.jpeg}, @file{.JPEG},
+Gnuastro's @file{astscript-sort-by-night} script is created to help in such
+important scenarios. It uses @ref{Fits} to convert the FITS date format
+into the Unix epoch time (number of seconds since 00:00:00 of January 1st,
+1970), using the @option{--datetosec} option. The Unix epoch time is a
+single number (integer, if not given in sub-second precision), enabling
+easy comparison and sorting of dates after January 1st, 1970.

-TIFF (or Tagged Image File Format) was originally designed as a common
-format for scanners in the early 90s and since then it has grown to become
-very general. In many aspects, the TIFF standard is similar to the FITS
-image standard: it can allow data of many types (see @ref{Numeric data
-types}), and also allows multiple images to be stored in a single file
-(each image in the file is called a directory' in the TIFF
-standard). However, unlike FITS, it can only store images, it has no
-constructs for tables. Another (inconvenient) difference with the FITS
-standard is that keyword names are stored as numbers, not human-readable
-text.
+You can use this script as a basis for making a much more highly customized
+sorting script. Here are some examples

-However, outside of astronomy, because of its support of different numeric
-data types, many fields use TIFF images for accurate (for example 16-bit
-integer or floating point for example) imaging data.
+If you need to copy the files, but only need a single extension (not the
+whole file), you can add a step just before the making of the symbolic
+links, or copies, and change it to only copy a certain extension of the
+FITS file using the Fits program's @option{--copy} option, see @ref{HDU
+manipulation}.

-Currently ConvertType can only read TIFF images, if you are interested in
-writing TIFF images, please get in touch with us.
+If you need to classify the files with finer detail (for example the
+purpose of the dataset), you can add a step just before the making of the
+symbolic links, or copies, to specify a file-name prefix based on other
+certain keyword values in the files. For example when the FITS files have a
+keyword to specify if the dataset is a science, bias, or flat-field
+image. You can read it and to add a @code{sci-}, @code{bias-}, or
address@hidden to the created file (after the @option{--prefix})
+automatically.

-The Encapsulated PostScript (EPS) format is essentially a one page
-PostScript file which has a specified size. PostScript also includes
-non-image data, for example lines and texts. It is a fully functional
-programming language to describe a document. Therefore in ConvertType,
-EPS is only an output format and cannot be used as input. Contrary to
-the FITS or JPEG formats, PostScript is not a raster format, but is
-categorized as vector graphics.
+For example, let's assume the observing mode is stored in the hypothetical
address@hidden keyword, which can have three values of @code{BIAS-IMAGE},
address@hidden and @code{FLAT-EXP}. With the step below, you can
+generate a mode-prefix, and add it to the generated link/copy names (just
+correct the filename and extension of the first line to the script's
+variables):

-The Portable Document Format (PDF) is currently the most common format
-for documents. Some believe that PDF has replaced PostScript and that
-PostScript is now obsolete. This view is wrong, a PostScript file is
-an actual plain text file that can be edited like any program source
-with any text editor. To be able to display its programmed content or
-print, it needs to pass through a processor or compiler. A PDF file
-can be thought of as the processed output of the compiler on an input
-PostScript file. PostScript, EPS and PDF were created and are
+modepref=$(astfits infile.fits -h1 \ + | sed -e"s/'/ /g" \ + | awk '$1=="MODE"@{ \
+                       if($3=="BIAS-IMAGE") print "bias-"; \ + else if($3=="SCIENCE-IMAGE") print "sci-"; \
+                       else if($3==FLAT-EXP) print "flat-"; \ + else print$3, "NOT recognized"; exit address@hidden')

-With these features in mind, you can see that when you are compiling a
-document with @TeX{} or @LaTeX{}, using an EPS file is much more low
-level than a JPEG and thus you have much greater control and therefore
-quality. Since it also includes vector graphic lines we also use such
+Here is a description of it. We first use @command{astfits} to print all
+the keywords in extension @code{1} of @file{infile.fits}. In the FITS
+standard, string values (that we are assuming here) are placed in single
+quotes (@key{'}) which are annoying in this context/use-case. Therefore, we
+pipe the output of @command{astfits} into @command{sed} to remove all such
+quotes (substituting them with a blank space). The result is then piped to
+AWK for giving us the final mode-prefix: with @code{$1=="MODE"}, we ask AWK +to only consider the line where the first column is @code{MODE}. There is +an equal sign between the key name and value, so the value is the third +column (@code{$3} in AWK). We thus use a simple @code{if-else} structure to
+look into this value and print our custom prefix based on it. The output of
+AWK is then stored in the @code{modepref} shell variable which you can add
+
+With the solution above, the increment of the file counter for each night
+will be independent of the mode. If you want the counter to be
+mode-dependent, you can add a different counter for each mode and use that
+counter instead of the generic counter for each night (based on the value
+of @code{modepref}). But we'll leave the implementation of this step to you
+as an exercise.
+
+
+* Invoking astscript-sort-by-night::  Inputs and outputs to this script.
+
address@hidden Invoking astscript-sort-by-night,  , Sort FITS files by night,
Sort FITS files by night
+
+This installed script will read a FITS date formatted value from the given
+keyword, and classify the input FITS files into individual nights. For more
+on installed scripts please see (see @ref{Installed scripts}). This script
+can be used with the following general template:
+
+$astscript-sort-by-night [OPTION...] FITS-files address@hidden example + address@hidden +One line examples: + address@hidden +## Use the DATE-OBS keyword +$ astscript-sort-by-night --key=DATE-OBS /path/to/data/*.fits
+
+## Make links to the input files with the img-' prefix
+$astscript-sort-by-night --link --prefix=img- /path/to/data/*.fits address@hidden example + +This script will look into a HDU/extension (@option{--hdu}) for a keyword +(@option{--key}) in the given FITS files and interpret the value as a +date. The inputs will be separated by "night"s (9:00a.m to next day's +8:59:59a.m, spanning two calendar days, exact hour can be set with address@hidden). + +The default output is a list of all the input files along with the +following two columns: night number and file number in that night (sorted +by time). With @option{--link} a symbolic link will be made (one for each +input) that contains the night number, and number of file in that night +(sorted by time), see the description of @option{--link} for more. When address@hidden is used instead of a link, a copy of the inputs will be +made instead of symbolic link. + +Below you can see one example where all the @file{target-*.fits} files in +the @file{data} directory should be separated by observing night according +to the @code{DATE-OBS} keyword value in their second extension (number address@hidden, recall that HDU counting starts from 0). You can see the output +after the @code{ls} command. + address@hidden +$ astscript-sort-by-night -pimg- -h1 -kDATE-OBS data/target-*.fits
+$ls +img-n1-1.fits img-n1-2.fits img-n2-1.fits ... address@hidden example + +The outputs can be placed in a different (already existing) directory by +including that directory's name in the @option{--prefix} value, for example address@hidden/img-} will put them all under the @file{sorted} +directory. + +This script can be configured like all Gnuastro's programs (through +command-line options, see @ref{Common options}), with some minor +differences that are described in @ref{Installed scripts}. The particular +options to this script are listed below: + address@hidden @option address@hidden -h STR address@hidden --hdu=STR +The HDU/extension to use in all the given FITS files. All of the given FITS +files must have this extension. + address@hidden -k STR address@hidden --key=STR +The keyword name that contains the FITS date format to classify/sort by. + address@hidden -H FLT address@hidden --hour=FLT +The hour that defines the next night''. By default, all times before +9:00a.m are considered to belong to the previous calendar night. If a +sub-hour value is necessary, it should be given in units of hours, for +example @option{--hour=9.5} corresponds to 9:30a.m. + address@hidden -l address@hidden --link +Create a symbolic link for each input FITS file. This option cannot be used +with @option{--copy}. The link will have a standard name in the following +format (variable parts are written in @code{CAPITAL} letters and described +after it): + address@hidden +PnN-I.fits address@hidden example + address@hidden @code address@hidden P +This is the value given to @option{--prefix}. By default, its value is address@hidden/} (to store the links in the directory this script was run in). See +the description of @code{--prefix} for more. address@hidden N +This is the night-counter: starting from 1. @code{N} is just incremented by +1 for the next night, no matter how many nights (without any dataset) there +are between two subsequent observing nights (its just an identifier for +each night which you can easily map to different calendar nights). address@hidden I +File counter in that night, sorted by time. address@hidden table + address@hidden -c address@hidden --copy +Make a copy of each input FITS file with the standard naming convention +described in @option{--link}. With this option, instead of making a link, a +copy is made. This option cannot be used with @option{--link}. + address@hidden -p STR address@hidden --prefix=STR +Prefix to append before the night-identifier of each newly created link or +copy. This option is thus only relevant with the @option{--copy} or address@hidden options. See the description of @option{--link} for how its +used. For example, with @option{--prefix=img-}, all the created file names +in the current directory will start with @code{img-}, making outputs like address@hidden or @file{img-n3-42.fits}. + address@hidden can also be used to store the links/copies in another +directory relative to the directory this script is being run (it must +already exist). For example @code{--prefix=/path/to/processing/img-} will +put all the links/copies in the @file{/path/to/processing} directory, and +the files (in that directory) will all start with @file{img-}. address@hidden table + + + + + + + + + + + + + + + + + + + + address@hidden ConvertType, Table, Sort FITS files by night, Data containers address@hidden ConvertType + address@hidden Data format conversion address@hidden Converting data formats address@hidden Image format conversion address@hidden Converting image formats address@hidden @r{ConvertType (address@hidden)} +The FITS format used in astronomy was defined mainly for archiving, +transmission, and processing. In other situations, the data might be useful +in other formats. For example, when you are writing a paper or report, or +if you are making slides for a talk, you can't use a FITS image. Other +image formats should be used. In other cases you might want your pixel +values in a table format as plain text for input to other programs that +don't recognize FITS. ConvertType is created for such situations. The +various types will increase with future updates and based on need. + +The conversion is not only one way (from FITS to other formats), but two +ways (except the EPS and PDF address@hidden EPS and PDF are +vector, not raster/pixelated formats}). So you can also convert a JPEG +image or text file into a FITS image. Basically, other than EPS/PDF, you +can use any of the recognized formats as different color channel inputs to +get any of the recognized outputs. So before explaining the options and +arguments (in @ref{Invoking astconvertt}), we'll start with a short +description of the recognized files types in @ref{Recognized file formats}, +followed a short introduction to digital color in @ref{Color}. + address@hidden +* Recognized file formats:: Recognized file formats +* Color:: Some explanations on color. +* Invoking astconvertt:: Options and arguments to ConvertType. address@hidden menu + address@hidden Recognized file formats, Color, ConvertType, ConvertType address@hidden Recognized file formats + +The various standards and the file name extensions recognized by +ConvertType are listed below. Currently Gnuastro uses the file name's +suffix to identify the format. + address@hidden @asis address@hidden FITS or IMH address@hidden IRAF address@hidden Astronomical data format +Astronomical data are commonly stored in the FITS format (or the older data +IRAF @file{.imh} format), a list of file name suffixes which indicate that +the file is in this format is given in @ref{Arguments}. + +Each image extension of a FITS file only has one value per +pixel/element. Therefore, when used as input, each input FITS image +contributes as one color channel. If you want multiple extensions in one +FITS file for different color channels, you have to repeat the file name +multiple times and use the @option{--hdu}, @option{--hdu2}, @option{--hdu3} +or @option{--hdu4} options to specify the different extensions. + address@hidden JPEG address@hidden JPEG format address@hidden Raster graphics address@hidden Pixelated graphics +The JPEG standard was created by the Joint photographic experts +group. It is currently one of the most commonly used image +formats. Its major advantage is the compression algorithm that is +defined by the standard. Like the FITS standard, this is a raster +graphics format, which means that it is pixelated. + +A JPEG file can have 1 (for gray-scale), 3 (for RGB) and 4 (for CMYK) +color channels. If you only want to convert one JPEG image into other +formats, there is no problem, however, if you want to use it in +combination with other input files, make sure that the final number of +color channels does not exceed four. If it does, then ConvertType will +abort and notify you. + address@hidden Suffixes, JPEG images +The file name endings that are recognized as a JPEG file for input +are: @file{.jpg}, @file{.JPG}, @file{.jpeg}, @file{.JPEG}, address@hidden, @file{.jif}, @file{.jfif} and @file{.jfi}. + address@hidden TIFF address@hidden TIFF format +TIFF (or Tagged Image File Format) was originally designed as a common +format for scanners in the early 90s and since then it has grown to become +very general. In many aspects, the TIFF standard is similar to the FITS +image standard: it can allow data of many types (see @ref{Numeric data +types}), and also allows multiple images to be stored in a single file +(each image in the file is called a directory' in the TIFF +standard). However, unlike FITS, it can only store images, it has no +constructs for tables. Another (inconvenient) difference with the FITS +standard is that keyword names are stored as numbers, not human-readable +text. + +However, outside of astronomy, because of its support of different numeric +data types, many fields use TIFF images for accurate (for example 16-bit +integer or floating point for example) imaging data. + +Currently ConvertType can only read TIFF images, if you are interested in +writing TIFF images, please get in touch with us. + address@hidden EPS address@hidden EPS address@hidden PostScript address@hidden Vector graphics address@hidden Encapsulated PostScript +The Encapsulated PostScript (EPS) format is essentially a one page +PostScript file which has a specified size. PostScript also includes +non-image data, for example lines and texts. It is a fully functional +programming language to describe a document. Therefore in ConvertType, +EPS is only an output format and cannot be used as input. Contrary to +the FITS or JPEG formats, PostScript is not a raster format, but is +categorized as vector graphics. + address@hidden PDF address@hidden Adobe systems address@hidden PostScript vs. PDF address@hidden Compiled PostScript address@hidden Portable Document format address@hidden Static document description format +The Portable Document Format (PDF) is currently the most common format +for documents. Some believe that PDF has replaced PostScript and that +PostScript is now obsolete. This view is wrong, a PostScript file is +an actual plain text file that can be edited like any program source +with any text editor. To be able to display its programmed content or +print, it needs to pass through a processor or compiler. A PDF file +can be thought of as the processed output of the compiler on an input +PostScript file. PostScript, EPS and PDF were created and are +registered by Adobe Systems. + address@hidden @TeX{} address@hidden @LaTeX{} +With these features in mind, you can see that when you are compiling a +document with @TeX{} or @LaTeX{}, using an EPS file is much more low +level than a JPEG and thus you have much greater control and therefore +quality. Since it also includes vector graphic lines we also use such lines to make a thin border around the image to make its appearance in the document much better. No matter the resolution of the display or printer, these lines will always be clear and not pixelated. In the @@ -21753,7 +22100,7 @@ were of integer types. address@hidden High-level calculations, Installed scripts, Modeling and fittings, Top address@hidden High-level calculations, Library, Modeling and fittings, Top @chapter High-level calculations After the reduction of raw data (for example with the programs in @ref{Data @@ -21999,607 +22346,313 @@ independent of time and thus doesn't move'. We call it the @emph{comoving distance} and display with @mymath{\chi} such that: @mymath{l(r,t)=\chi(r)a(t)}. We have therefore, shifted the @mymath{r} dependence of the proper distance we derived above for a static universe to -the comoving distance: - address@hidden(r)=\sin^{-1}(r)\quad(K>0),\quad\quad -\chi(r)=r\quad(K=0),\quad\quad \chi(r)=\sinh^{-1}(r)\quad(K<0).} - -Therefore, @mymath{\chi(r)} is the proper distance to an object at a -specific reference time: @mymath{t=t_r} (the @mymath{r} subscript signifies -reference'') when @mymath{a(t_r)=1}. At any arbitrary moment -(@mymath{t\neq{t_r}}) before or after @mymath{t_r}, the proper distance to -the object can be scaled with @mymath{a(t)}. - -Measuring the change of distance in a time-dependent (expanding) universe -only makes sense if we can add up space and address@hidden other words, -making our space-time consistent with Minkowski space-time geometry. In this -geometry, different observers at a given point (event) in space-time split -up space-time into space' and time' in different ways, just like people at -the same spatial position can make different choices of splitting up a map -into left--right' and up--down'. This model is well supported by -twentieth and twenty-first century observations.}. But we can only add bits -of space and time together if we measure them in the same units: with a -conversion constant (similar to how 1000 is used to convert a kilometer -into meters). Experimentally, we find strong support for the hypothesis -that this conversion constant is the speed of light (or gravitational address@hidden speed of gravitational waves was recently found to be -very similar to that of light in vacuum, see address@hidden://arxiv.org/abs/1710.05834, arXiv:1710.05834}.}) in a -vacuum. This speed is postulated to be address@hidden @emph{natural -units}, speed is measured in units of the speed of light in vacuum.} and is -almost always written as @mymath{c}. We can thus parameterize the change in -distance on an expanding 2D surface as - address@hidden(t)ds_s^2 = c^2dt^2-a^2(t)(d\chi^2+r^2d\phi^2).} - - address@hidden Extending distance concepts to 3D, Invoking astcosmiccal, Distance on a 2D curved space, CosmicCalculator address@hidden Extending distance concepts to 3D - -The concepts of @ref{Distance on a 2D curved space} are here extended to a -3D space that @emph{might} be curved. We can start with the generic -infinitesimal distance in a static 3D universe, but this time in spherical -coordinates instead of polar coordinates. @mymath{\theta} is shown in address@hidden, but here we are 3D beings, positioned on @mymath{O} -(the center of the sphere) and the point @mymath{O} is tangent to a -4D-sphere. In our 3D space, a generic infinitesimal displacement will -correspond to the following distance in spherical coordinates: - address@hidden(d\theta^2+\sin^2{\theta}d\phi^2).} - -Like the 2D creature before, we now have to assume an abstract dimension -which we cannot visualize easily. Let's call the fourth dimension address@hidden, then the general change in coordinates in the @emph{full} four -dimensional space will be: - address@hidden(d\theta^2+\sin^2{\theta}d\phi^2)+dw^2.} - address@hidden -But we can only work on a 3D curved space, so following exactly the same -steps and conventions as our 2D friend, we arrive at: - address@hidden 1-Kr^2}+r^2(d\theta^2+\sin^2{\theta}d\phi^2).} - address@hidden -In a non-static universe (with a scale factor a(t)), the distance can be -written as: - address@hidden(t)[d\chi^2+r^2(d\theta^2+\sin^2{\theta}d\phi^2)].} - - - address@hidden@dispmath{H(z){\equiv}\left(\dot{a}\over a\right)(z)=H_0E(z) } - address@hidden@dispmath{E(z)=[ \Omega_{\Lambda,0} + \Omega_{C,0}(1+z)^2 + address@hidden,0}(1+z)^3 + \Omega_{r,0}(1+z)^4 ]^{1/2}} - address@hidden Let's take @mymath{r} to be the radial coordinate of the emitting address@hidden source, which emitted its light at redshift$z$. Then the comoving address@hidden distance of this object would be: - address@hidden@dispmath{ \chi(r)={c\over H_0a_0}\int_0^z{dz'\over E(z')} } - address@hidden@noindent address@hidden So the proper distance at the current time to that object is: address@hidden @mymath{a_0\chi(r)}, therefore the angular diameter distance address@hidden (@mymath{d_A}) and luminosity distance (@mymath{d_L}) can be written address@hidden as: - address@hidden@dispmath{ d_A={a_0\chi(r)\over 1+z}, \quad d_L=a_0\chi(r)(1+z) } - - - - address@hidden Invoking astcosmiccal, , Extending distance concepts to 3D, CosmicCalculator address@hidden Invoking CosmicCalculator - -CosmicCalculator will calculate cosmological variables based on the -input parameters. The executable name is @file{astcosmiccal} with the -following general template - address@hidden -$ astcosmiccal [OPTION...] ...
-
-
-One line examples:
-
-## Print basic cosmological properties at redshift 2.5:
-$astcosmiccal -z2.5 - -## Only print Comoving volume over 4pi stradian to z (Mpc^3): -$ astcosmiccal --redshift=0.8 --volume
-
-## Print luminosity distance, angular diameter distance and age
-## of universe in one row at redshift 0.4
-$astcosmiccal -z0.4 -LAg - -## Assume Lambda and matter density of 0.7 and 0.3 and print -## basic cosmological parameters for redshift 2.1: -$ astcosmiccal -l0.7 -m0.3 -z2.1
-
-The input parameters (for example current matter density and etc) can be
-given as command-line options or in the configuration files, see
address@hidden files}. For a definition of the different parameters,
-please see the sections prior to this. If no redshift is given,
-CosmicCalculator will just print its input parameters and abort. For a full
-list of the input options, please see @ref{CosmicCalculator input options}.
-
-When only a redshift is given, CosmicCalculator will print all calculations
-(one per line) with some explanations before each. This can be good when
-you want a general feeling of the conditions at a specific
-redshift. Alternatively, if any specific calculations are requested, only
-the requested values will be calculated and printed with one character
-space between them. In this case, no description will be printed. See
address@hidden specific calculations} for the full list of these
-options along with some explanations how when/how they can be useful.
-
-* CosmicCalculator input options::  Options to specify input conditions.
-* CosmicCalculator specific calculations::  Requesting specific outputs.
-
address@hidden CosmicCalculator input options, CosmicCalculator specific
calculations, Invoking astcosmiccal, Invoking astcosmiccal
-
-The inputs to CosmicCalculator can be specified with the following options:
-
-The redshift of interest.
-
-
-Cosmological constant density divided by the critical density in the
-current Universe (@mymath{\Omega_{\Lambda,0}}).
-
-Matter (including massive neutrinos) density divided by the critical
-density in the current Universe (@mymath{\Omega_{m,0}}).
-
-Radiation density divided by the critical density in the current Universe
-(@mymath{\Omega_{r,0}}).
-
-
-
address@hidden CosmicCalculator specific calculations,  , CosmicCalculator
input options, Invoking astcosmiccal
-By default, when no specific calculations are requested, CosmicCalculator
-will print a complete set of all its calculators (one line for each
-calculation, see @ref{Invoking astcosmiccal}). The full list of
-calculations can be useful when you don't want any specific value, but just
-a general view. In other contexts (for example in a batch script or during
-a discussion), you know exactly what you want and don't want to be
-distracted by all the extra information.
-
-You can use any number of the options described below in any order. When
-any of these options are requested, CosmicCalculator's output will just be
-a single line with a single space between the (possibly) multiple
-values. In the example below, only the tangential distance along one
-arcsecond (in kpc), absolute magnitude conversion, and age of the universe
-at redshift 2 are printed (recall that you can merge short options
-together, see @ref{Options}).
-
-$astcosmiccal -z2 -sag -8.585046 44.819248 3.289979 address@hidden example - -Here is one example of using this feature in scripts: by adding the -following two lines in a script to keep/use the comoving volume with -varying redshifts: - address@hidden -z=3.12 -vol=$(astcosmiccal --redshift=$z --volume) address@hidden example - address@hidden GNU Grep address@hidden -In a script, this operation might be necessary for a large number of -objects (several of galaxies in a catalog for example). So the fact that -all the other default calculations are ignored will also help you get to -your result faster. - -If you are indeed dealing with many (for example thousands) of redshifts, -using CosmicCalculator is not the best/fastest solution. Because it has to -go through all the configuration files and preparations for each -invocation. To get the best efficiency (least overhead), we recommend using -Gnuastro's cosmology library (see @ref{Cosmology -library}). CosmicCalculator also calls the library functions defined there -for its calculations, so you get the same result with no overhead. Gnuastro -also has libraries for easily reading tables into a C program, see address@hidden input output}. Afterwards, you can easily build and run your C -program for the particular processing with @ref{BuildProgram}. - -If you just want to inspect the value of a variable visually, the -description (which comes with units) might be more useful. In such cases, -the following command might be better. The other calculations will also be -done, but they are so fast that you will not notice on modern computers -(the time it takes your eye to focus on the result is usually longer than -the processing: a fraction of a second). - address@hidden -$ astcosmiccal --redshift=0.832 | grep volume
-
-The full list of CosmicCalculator's specific calculations is present
-below. In case you have forgot the units, you can use the @option{--help}
-option which has the units along with a short description.
+the comoving distance:

-The current age of the universe (given the input parameters) in Ga (Giga
-annum, or billion years).
+Therefore, @mymath{\chi(r)} is the proper distance to an object at a
+specific reference time: @mymath{t=t_r} (the @mymath{r} subscript signifies
+reference'') when @mymath{a(t_r)=1}. At any arbitrary moment
+(@mymath{t\neq{t_r}}) before or after @mymath{t_r}, the proper distance to
+the object can be scaled with @mymath{a(t)}.

-The current critical density (given the input parameters) in grams per
-centimeter-cube (@mymath{g/cm^3}).
+Measuring the change of distance in a time-dependent (expanding) universe
+only makes sense if we can add up space and address@hidden other words,
+making our space-time consistent with Minkowski space-time geometry. In this
+geometry, different observers at a given point (event) in space-time split
+up space-time into space' and time' in different ways, just like people at
+the same spatial position can make different choices of splitting up a map
+into left--right' and up--down'. This model is well supported by
+twentieth and twenty-first century observations.}. But we can only add bits
+of space and time together if we measure them in the same units: with a
+conversion constant (similar to how 1000 is used to convert a kilometer
+into meters).  Experimentally, we find strong support for the hypothesis
+that this conversion constant is the speed of light (or gravitational
address@hidden speed of gravitational waves was recently found to be
+very similar to that of light in vacuum, see
+vacuum. This speed is postulated to be address@hidden @emph{natural
+units}, speed is measured in units of the speed of light in vacuum.} and is
+almost always written as @mymath{c}. We can thus parameterize the change in
+distance on an expanding 2D surface as

-The proper distance (at current time) to object at the given redshift in
-Megaparsecs (Mpc). See @ref{Distance on a 2D curved space} for a
-description of the proper distance.

-The angular diameter distance to object at given redshift in Megaparsecs
-(Mpc).

-The tangential distance covered by 1 arcseconds at the given redshift in
-kiloparsecs (Kpc). This can be useful when trying to estimate the
-resolution or pixel scale of an instrument (usually in units of arcseconds)
-at a given redshift.
address@hidden Extending distance concepts to 3D, Invoking astcosmiccal,
Distance on a 2D curved space, CosmicCalculator
address@hidden Extending distance concepts to 3D

-The luminosity distance to object at given redshift in Megaparsecs (Mpc).
+The concepts of @ref{Distance on a 2D curved space} are here extended to a
+3D space that @emph{might} be curved. We can start with the generic
+infinitesimal distance in a static 3D universe, but this time in spherical
+coordinates instead of polar coordinates.  @mymath{\theta} is shown in
address@hidden, but here we are 3D beings, positioned on @mymath{O}
+(the center of the sphere) and the point @mymath{O} is tangent to a
+4D-sphere. In our 3D space, a generic infinitesimal displacement will
+correspond to the following distance in spherical coordinates:

-The distance modulus at given redshift.

-The conversion factor (addition) to absolute magnitude. Note that this is
-practically the distance modulus added with @mymath{-2.5\log{(1+z)}} for
-the the desired redshift based on the input parameters. Once the apparent
-magnitude and redshift of an object is known, this value may be added with
-the apparent magnitude to give the object's absolute magnitude.
+Like the 2D creature before, we now have to assume an abstract dimension
+which we cannot visualize easily. Let's call the fourth dimension
address@hidden, then the general change in coordinates in the @emph{full} four
+dimensional space will be:

-Age of the universe at given redshift in Ga (Giga annum, or billion years).

-The look-back time to given redshift in Ga (Giga annum, or billion
-years). The look-back time at a given redshift is defined as the current
-age of the universe (@option{--agenow}) subtracted by the age of the
-universe at the given redshift.
+But we can only work on a 3D curved space, so following exactly the same
+steps and conventions as our 2D friend, we arrive at:

-The critical density at given redshift in grams per centimeter-cube
-(@mymath{g/cm^3}).

-The comoving volume in Megaparsecs cube (address@hidden) until the desired
-redshift based on the input parameters.
+In a non-static universe (with a scale factor a(t)), the distance can be
+written as:

address@hidden Let's take @mymath{r} to be the radial coordinate of the emitting
address@hidden source, which emitted its light at redshift $z$. Then the
comoving
address@hidden distance of this object would be:

address@hidden So the proper distance at the current time to that object is:
address@hidden @mymath{a_0\chi(r)}, therefore the angular diameter distance
address@hidden (@mymath{d_A}) and luminosity distance (@mymath{d_L}) can be
written

address@hidden Installed scripts, Library, High-level calculations, Top

-Gnuastro's programs (introduced in previous chapters) are designed to be
-highly modular and thus mainly contain lower-level operations on the
-data. However, in many contexts, higher-level operations (for example a
-sequence of calls to multiple Gnuastro programs, or a special way of
-running a program and using the outputs) are also very similar between
-various projects.

-To facilitate data analysis on these higher-level steps also, Gnuastro also
-installs some scripts on your system with the (@code{astscript-}) prefix
-(in contrast to the other programs that only have the @code{ast}
-prefix). In this chapter, these scripts and their usage details are
-described.
address@hidden Invoking astcosmiccal,  , Extending distance concepts to 3D,
CosmicCalculator

-Like all of Gnuastro's source code, these scripts are also heavily
-commented. They are written in GNU Bash, which doesn't need compilation (so
-they are actually human-readable), and is the same language that is mainly
-used when typing on the command-line. Because of these factors, Bash is
-much more widely known and used than C (the language of other Gnuastro
-programs). They also do higher-level operations, so customizing these
-scripts for a special project will be more common than the programs. You
-can always inspect them (to customize, check, or educate your self) with
-this command (just replace @code{emacs} with your favorite text editor):
+CosmicCalculator will calculate cosmological variables based on the
+input parameters. The executable name is @file{astcosmiccal} with the
+following general template

@example
-$emacs$(which astscript-NAME)
+$astcosmiccal [OPTION...] ... @end example -These scripts also accept options and are in many ways similar to the -programs (see @ref{Common options}) with some minor differences: address@hidden address@hidden -Currently they don't accept configuration files themselves. However, the -configuration files of the Gnuastro programs they call are indeed parsed -and used by those programs. address@hidden +One line examples: -As a result, they don't have the following options: @option{--checkconfig}, address@hidden, @option{--lastconfig}, @option{--onlyversion}, address@hidden, @option{--setdirconf} and @option{--setusrconf}. address@hidden +## Print basic cosmological properties at redshift 2.5: +$ astcosmiccal -z2.5

-They don't directly allocate any memory, so there is no
+## Only print Comoving volume over 4pi stradian to z (Mpc^3):
+$astcosmiccal --redshift=0.8 --volume address@hidden -They don't have an independent @option{--usage} option: when called with address@hidden, they just recommend running @option{--help}. +## Print luminosity distance, angular diameter distance and age +## of universe in one row at redshift 0.4 +$ astcosmiccal -z0.4 -LAg

-The output of @option{--help} is not configurable like the programs (see
+## Assume Lambda and matter density of 0.7 and 0.3 and print
+## basic cosmological parameters for redshift 2.1:
+$astcosmiccal -l0.7 -m0.3 -z2.1 address@hidden example address@hidden itemize +The input parameters (for example current matter density and etc) can be +given as command-line options or in the configuration files, see address@hidden files}. For a definition of the different parameters, +please see the sections prior to this. If no redshift is given, +CosmicCalculator will just print its input parameters and abort. For a full +list of the input options, please see @ref{CosmicCalculator input options}. + +When only a redshift is given, CosmicCalculator will print all calculations +(one per line) with some explanations before each. This can be good when +you want a general feeling of the conditions at a specific +redshift. Alternatively, if any specific calculations are requested, only +the requested values will be calculated and printed with one character +space between them. In this case, no description will be printed. See address@hidden specific calculations} for the full list of these +options along with some explanations how when/how they can be useful. @menu -* Sort FITS files by night:: +* CosmicCalculator input options:: Options to specify input conditions. +* CosmicCalculator specific calculations:: Requesting specific outputs. @end menu address@hidden Sort FITS files by night, , Installed scripts, Installed scripts address@hidden Sort FITS files by night - address@hidden Calendar -FITS images usually contain (several) keywords for preserving important -dates. In particular, for lower-level data, this is usually the observation -date and time (for example, stored in the @code{DATE-OBS} keyword -value). When analyzing observed datasets, many calibration steps (like the -dark, bias or flat-field), are commonly calculated on a per-observing-night -basis. - -However, the FITS standard's date format (@code{YYYY-MM-DDThh:mm:ss.ddd}) -is based on the western (Gregorian) calendar. Dates that are stored in this -format are complicated for automatic processing: a night starts in the -final hours of one calendar day, and extends to the early hours of the next -calendar day. As a result, to identify datasets from one night, we commonly -need to search for two dates. However calendar peculiarities can make this -identification very difficult. For example when an observation is done on -the night separating two months (like the night starting on March 31st and -going into April 1st), or two years (like the night starting on December -31st 2018 and going into January 1st, 2019). To account for such -situations, it is necessary to keep track of how many days are in a month, -and leap years, and etc. - address@hidden Unix epoch time address@hidden Time, Unix epoch address@hidden Epoch, Unix time -Gnuastro's @file{astscript-sort-by-night} script is created to help in such -important scenarios. It uses @ref{Fits} to convert the FITS date format -into the Unix epoch time (number of seconds since 00:00:00 of January 1st, -1970), using the @option{--datetosec} option. The Unix epoch time is a -single number (integer, if not given in sub-second precision), enabling -easy comparison and sorting of dates after January 1st, 1970. - -You can use this script as a basis for making a much more highly customized -sorting script. Here are some examples address@hidden CosmicCalculator input options, CosmicCalculator specific calculations, Invoking astcosmiccal, Invoking astcosmiccal address@hidden CosmicCalculator input options address@hidden address@hidden -If you need to copy the files, but only need a single extension (not the -whole file), you can add a step just before the making of the symbolic -links, or copies, and change it to only copy a certain extension of the -FITS file using the Fits program's @option{--copy} option, see @ref{HDU -manipulation}. +The inputs to CosmicCalculator can be specified with the following options: address@hidden @option address@hidden -If you need to classify the files with finer detail (for example the -purpose of the dataset), you can add a step just before the making of the -symbolic links, or copies, to specify a file-name prefix based on other -certain keyword values in the files. For example when the FITS files have a -keyword to specify if the dataset is a science, bias, or flat-field -image. You can read it and to add a @code{sci-}, @code{bias-}, or address@hidden to the created file (after the @option{--prefix}) -automatically. address@hidden -z FLT address@hidden --redshift=FLT +The redshift of interest. -For example, let's assume the observing mode is stored in the hypothetical address@hidden keyword, which can have three values of @code{BIAS-IMAGE}, address@hidden and @code{FLAT-EXP}. With the step below, you can -generate a mode-prefix, and add it to the generated link/copy names (just -correct the filename and extension of the first line to the script's -variables): address@hidden -H FLT address@hidden --H0=FLT +Current expansion rate (in km address@hidden address@hidden). address@hidden -modepref=$(astfits infile.fits -h1 \
-                   | sed -e"s/'/ /g" \
-                   | awk '$1=="MODE"@{ \ - if($3=="BIAS-IMAGE") print "bias-"; \
-                       else if($3=="SCIENCE-IMAGE") print "sci-"; \ - else if($3==FLAT-EXP) print "flat-"; \
-                       else print $3, "NOT recognized"; exit address@hidden') address@hidden example address@hidden -l FLT address@hidden --olambda=FLT +Cosmological constant density divided by the critical density in the +current Universe (@mymath{\Omega_{\Lambda,0}}). address@hidden GNU AWK address@hidden GNU Sed -Here is a description of it. We first use @command{astfits} to print all -the keywords in extension @code{1} of @file{infile.fits}. In the FITS -standard, string values (that we are assuming here) are placed in single -quotes (@key{'}) which are annoying in this context/use-case. Therefore, we -pipe the output of @command{astfits} into @command{sed} to remove all such -quotes (substituting them with a blank space). The result is then piped to -AWK for giving us the final mode-prefix: with @code{$1=="MODE"}, we ask AWK
-to only consider the line where the first column is @code{MODE}. There is
-an equal sign between the key name and value, so the value is the third
-column (@code{$3} in AWK). We thus use a simple @code{if-else} structure to -look into this value and print our custom prefix based on it. The output of -AWK is then stored in the @code{modepref} shell variable which you can add -to the link/copy name. address@hidden -m FLT address@hidden --omatter=FLT +Matter (including massive neutrinos) density divided by the critical +density in the current Universe (@mymath{\Omega_{m,0}}). -With the solution above, the increment of the file counter for each night -will be independent of the mode. If you want the counter to be -mode-dependent, you can add a different counter for each mode and use that -counter instead of the generic counter for each night (based on the value -of @code{modepref}). But we'll leave the implementation of this step to you -as an exercise. address@hidden -r FLT address@hidden --oradiation=FLT +Radiation density divided by the critical density in the current Universe +(@mymath{\Omega_{r,0}}). address@hidden table address@hidden itemize address@hidden -* Invoking astscript-sort-by-night:: Inputs and outputs to this script. address@hidden menu address@hidden Invoking astscript-sort-by-night, , Sort FITS files by night, Sort FITS files by night address@hidden Invoking astscript-sort-by-night address@hidden CosmicCalculator specific calculations, , CosmicCalculator input options, Invoking astcosmiccal address@hidden CosmicCalculator specific calculations +By default, when no specific calculations are requested, CosmicCalculator +will print a complete set of all its calculators (one line for each +calculation, see @ref{Invoking astcosmiccal}). The full list of +calculations can be useful when you don't want any specific value, but just +a general view. In other contexts (for example in a batch script or during +a discussion), you know exactly what you want and don't want to be +distracted by all the extra information. -This script will read a FITS date formatted value from the given keyword, -and classify the input FITS files into individual nights. This script can -be used with the following general template: +You can use any number of the options described below in any order. When +any of these options are requested, CosmicCalculator's output will just be +a single line with a single space between the (possibly) multiple +values. In the example below, only the tangential distance along one +arcsecond (in kpc), absolute magnitude conversion, and age of the universe +at redshift 2 are printed (recall that you can merge short options +together, see @ref{Options}). @example -$ astscript-sort-by-night [OPTION...] FITS-files
+$astcosmiccal -z2 -sag +8.585046 44.819248 3.289979 @end example address@hidden -One line examples: +Here is one example of using this feature in scripts: by adding the +following two lines in a script to keep/use the comoving volume with +varying redshifts: @example -## Use the DATE-OBS keyword -$ astscript-sort-by-night --key=DATE-OBS /path/to/data/*.fits
-
-## Make links to the input files with the img-' prefix
-$astscript-sort-by-night --link --prefix=img- /path/to/data/*.fits +z=3.12 +vol=$(astcosmiccal --redshift=$z --volume) @end example -This script will look into a HDU/extension (@option{--hdu}) for a keyword -(@option{--key}) in the given FITS files and interpret the value as a -date. The inputs will be separated by "night"s (9:00a.m to next day's -8:59:59a.m, spanning two calendar days, exact hour can be set with address@hidden). address@hidden GNU Grep address@hidden +In a script, this operation might be necessary for a large number of +objects (several of galaxies in a catalog for example). So the fact that +all the other default calculations are ignored will also help you get to +your result faster. -The default output is a list of all the input files along with the -following two columns: night number and file number in that night (sorted -by time). With @option{--link} a symbolic link will be made (one for each -input) that contains the night number, and number of file in that night -(sorted by time), see the description of @option{--link} for more. When address@hidden is used instead of a link, a copy of the inputs will be -made instead of symbolic link. +If you are indeed dealing with many (for example thousands) of redshifts, +using CosmicCalculator is not the best/fastest solution. Because it has to +go through all the configuration files and preparations for each +invocation. To get the best efficiency (least overhead), we recommend using +Gnuastro's cosmology library (see @ref{Cosmology +library}). CosmicCalculator also calls the library functions defined there +for its calculations, so you get the same result with no overhead. Gnuastro +also has libraries for easily reading tables into a C program, see address@hidden input output}. Afterwards, you can easily build and run your C +program for the particular processing with @ref{BuildProgram}. -Below you can see one example where all the @file{target-*.fits} files in -the @file{data} directory should be separated by observing night according -to the @code{DATE-OBS} keyword value in their second extension (number address@hidden, recall that HDU counting starts from 0). You can see the output -after the @code{ls} command. +If you just want to inspect the value of a variable visually, the +description (which comes with units) might be more useful. In such cases, +the following command might be better. The other calculations will also be +done, but they are so fast that you will not notice on modern computers +(the time it takes your eye to focus on the result is usually longer than +the processing: a fraction of a second). @example -$ astscript-sort-by-night -pimg- -h1 -kDATE-OBS data/target-*.fits
-$ls -img-n1-1.fits img-n1-2.fits img-n2-1.fits ... +$ astcosmiccal --redshift=0.832 | grep volume
@end example

-The outputs can be placed in a different (already existing) directory by
-including that directory's name in the @option{--prefix} value, for example
address@hidden/img-} will put them all under the @file{sorted}
-directory.
-
-This script can be configured like all Gnuastro's programs (through
-command-line options, see @ref{Common options}), with some minor
-differences that are described in @ref{Installed scripts}. The particular
-options to this script are listed below:
+The full list of CosmicCalculator's specific calculations is present
+below. In case you have forgot the units, you can use the @option{--help}
+option which has the units along with a short description.

@table @option
-The HDU/extension to use in all the given FITS files. All of the given FITS
-files must have this extension.

-The keyword name that contains the FITS date format to classify/sort by.
+The current age of the universe (given the input parameters) in Ga (Giga
+annum, or billion years).

-The hour that defines the next night''. By default, all times before
-9:00a.m are considered to belong to the previous calendar night. If a
-sub-hour value is necessary, it should be given in units of hours, for
-example @option{--hour=9.5} corresponds to 9:30a.m.
+The current critical density (given the input parameters) in grams per
+centimeter-cube (@mymath{g/cm^3}).

-Create a symbolic link for each input FITS file. This option cannot be used
-with @option{--copy}. The link will have a standard name in the following
-format (variable parts are written in @code{CAPITAL} letters and described
-after it):
+The proper distance (at current time) to object at the given redshift in
+Megaparsecs (Mpc). See @ref{Distance on a 2D curved space} for a
+description of the proper distance.

-PnN-I.fits
+The angular diameter distance to object at given redshift in Megaparsecs
+(Mpc).

-This is the value given to @option{--prefix}. By default, its value is
address@hidden/} (to store the links in the directory this script was run in).
See
-the description of @code{--prefix} for more.
-This is the night-counter: starting from 1. @code{N} is just incremented by
-1 for the next night, no matter how many nights (without any dataset) there
-are between two subsequent observing nights (its just an identifier for
-each night which you can easily map to different calendar nights).
-File counter in that night, sorted by time.
+The tangential distance covered by 1 arcseconds at the given redshift in
+kiloparsecs (Kpc). This can be useful when trying to estimate the
+resolution or pixel scale of an instrument (usually in units of arcseconds)
+at a given redshift.
+
+The luminosity distance to object at given redshift in Megaparsecs (Mpc).
+
+The distance modulus at given redshift.
+
+The conversion factor (addition) to absolute magnitude. Note that this is
+practically the distance modulus added with @mymath{-2.5\log{(1+z)}} for
+the the desired redshift based on the input parameters. Once the apparent
+magnitude and redshift of an object is known, this value may be added with
+the apparent magnitude to give the object's absolute magnitude.
+
+Age of the universe at given redshift in Ga (Giga annum, or billion years).
+
+The look-back time to given redshift in Ga (Giga annum, or billion
+years). The look-back time at a given redshift is defined as the current
+age of the universe (@option{--agenow}) subtracted by the age of the
+universe at the given redshift.

@item -c
-Make a copy of each input FITS file with the standard naming convention
+The critical density at given redshift in grams per centimeter-cube
+(@mymath{g/cm^3}).

-Prefix to append before the night-identifier of each newly created link or
-copy. This option is thus only relevant with the @option{--copy} or
-used. For example, with @option{--prefix=img-}, all the created file names
+The comoving volume in Megaparsecs cube (address@hidden) until the desired
+redshift based on the input parameters.

-directory relative to the directory this script is being run (it must
-already exist). For example @code{--prefix=/path/to/processing/img-} will
-put all the links/copies in the @file{/path/to/processing} directory, and
@end table

@@ -22611,7 +22664,19 @@ the files (in that directory) will all start with
@file{img-}.

address@hidden Library, Developing, Installed scripts, Top
+
+
+
+
+
+
+
+
+
+
+
+
address@hidden Library, Developing, High-level calculations, Top
@chapter Library

Each program in Gnuastro that was discussed in the prior chapters (or any

`